A closer look at “Sparks of Weight-Space Programming”
Source: LessWrong ·
GPT-6 Astra’s improvement of Qwen’s math is not “native weight editing” but activation steering with clever use of synthetic data Zhi Wang, Brandon Hsu, Tianhao Wang, Mikhail Belkin (UC San Diego). Code: GitHub . TL;DR Astra’s edit of Qwen is equivalent to a special case of context-dependent activation steering: it pushes the activations along a fixed direction constructed from data, with a strength that depends linearly on the current activations. Generalizing Astra’s edit, with a trained “sensing vector” that determines how strongly to push along the same fixed direction, can further improve
Frequently Asked Questions
What is A closer look at “Sparks of Weight-Space Programming”?
A closer look at “Sparks of Weight-Space Programming” is a article discovery curated from LessWrong. GPT-6 Astra’s improvement of Qwen’s math is not “native weight editing” but activation steering with clever use of synthetic data Zhi Wang, Brandon Hsu, Tianhao Wang, Mikhail…
How can I access or experience this?
You can experience this content directly on its original host (LessWrong) by clicking the "Open Original" button.
Where does this content come from?
Curated from LessWrong by the OnSuggest team to surface the most interesting and serendipitous content on the web.
More like this
All Culture & ideas discoveries →After Gilligan's Island, Jim Backus And Alan Hale Jr. Reunited On A Legendary Western Series
Jim Backus and Alan Hale Jr. worked together on Gilligan's Island, and after that show went off the air, they reunited on this Western.
Batman Forever's Director Nearly Rejected Jim Carrey As The Riddler Until He Saw This Movie
Joel Schumacher was not planning to cast Jim Carrey as the Riddler in Batman Forever, but when the director saw Carrey in a different film, he changed his mind.
James Bond: How Old Every 007 Actor Was In Their First And Last Movies
Robert Pattinson has been cast as James Bond at the age of 40. Here's how old the other 007 actors were in their first movies, and their last.
Christopher Nolan Helped Rescue Theaters — Now He Could Save Physical Media
With Best Buy returning to physical media sales with The Odyssey, it seems Christopher Nolan might help save yet another format long overshadowed by streaming.
Jennifer Lawrence Dropped A Throwback Post With The Hunger Games’ Peeta And Gale (And Her Caption Is A+)
The actress is taking her fans back in time.
Johnny Depp Didn’t Just Leave Hollywood. He Left The US
Jack Sparrow isn't the only one who appreciates a private island.