Skip to content
onsuggest
Log in
Article en

A closer look at “Sparks of Weight-Space Programming”

Source: LessWrong ·

GPT-6 Astra’s improvement of Qwen’s math is not “native weight editing” but activation steering with clever use of synthetic data Zhi Wang, Brandon Hsu, Tianhao Wang, Mikhail Belkin (UC San Diego). Code: GitHub . TL;DR Astra’s edit of Qwen is equivalent to a special case of context-dependent activation steering: it pushes the activations along a fixed direction constructed from data, with a strength that depends linearly on the current activations. Generalizing Astra’s edit, with a trained “sensing vector” that determines how strongly to push along the same fixed direction, can further improve

What is A closer look at “Sparks of Weight-Space Programming”?

A closer look at “Sparks of Weight-Space Programming” is a article discovery curated from LessWrong. GPT-6 Astra’s improvement of Qwen’s math is not “native weight editing” but activation steering with clever use of synthetic data Zhi Wang, Brandon Hsu, Tianhao Wang, Mikhail…

How can I access or experience this?

You can experience this content directly on its original host (LessWrong) by clicking the "Open Original" button.

Where does this content come from?

Curated from LessWrong by the OnSuggest team to surface the most interesting and serendipitous content on the web.

Keyboard shortcuts

Previous discovery
←P
Next discovery
→N
Like
L
Not for me
X
Save
S
Show this help
?
Close
Esc

Shortcuts work on the discovery page.