#evaluation

Anthropic should measure the effect of evaluation awareness

... and report their findings! Summary Claude Fable 5.2 & Mythos 5.2 are the most situationally aware of Anthropic's recent models according to their own internal investigations. The company has made a concerted effort to measure evaluation awareness and to acknowledge that it affects their confidence in model safety. But their latest system card lacks information on the effect of evaluation awareness, thus blurring how one ought to adjust their own confidence in Anthropic's alignment assessments. Specifically, I present the following claims regarding Claude Mythos 5.2's system card : The prev

LessWrong · EN

Before You Add Another KPI, Ask These 10 Questions First

mostly from net and research

Hacker Noon · EN

Cynthia Kurtz on PNI

Two Cynthia Kurtz posts in a row! Participatory Narrative Inquiry has been a very powerful way to connect my own facilitation, evaluation/research and change work in my practice. We’ve done PNI projects that have lasted from a few days to five years. Narrative capture and sense making creates a grounded and co-owned process for planning, creating change, adjusting strategy or simply understanding a current state. It works with the capacities and capabilities of the people in the organization or community themselves to do the work and so it is a good way to suspect the lure of the intervenor. .

Chris Corrigan · EN