#lens

The J-lens intermediate swap acts partly through the answer directions

TL;DR When a language model answers a multi-hop prompt, does it first compute the intermediate concept internally, or does it jump straight to the answer through some shortcut? The global workspace paper investigates this question using the Jacobian lens, which maps activations at a given layer to the tokens that they are poised to verbalize, with the activations possessing a coordinate in each token’s lens direction. The authors intervene on the model through an intermediate swap that exchanges the activation’s J-Lens coordinates for the intermediate and a counterfactual intermediate. They sh

LessWrong · EN

Show HN: Review Lens – review pull requests inside your JetBrains IDE

Review pull requests in your JetBrains IDE: full files, changes in the gutter, your uncommitted work parked and restored. VS Code coming. - vatsarishabh22/review-lens

Show HN · EN

J-lens preferentially reads from coordinate-aligned subspaces

Note: I only test on Gemma-4-31B, which contains the spaces I've been studying in this earlier post . This is a shorter post, with just a few observations potentially relevant to those working on the J-lens work. The main observations here are: In Gemma-4, there are two relatively small subspaces that account the majority of the J-lens readout. These are both coordinate-aligned subspaces. That means we can take a hidden state, mask out all coordinates except both these ~600 coordinate groups, and it will capture ~80-95% of the J-lens readout. These spaces are not built to capture J-lens contri

LessWrong · EN