Artículo (1215) – onsuggest

Towards understanding score-seeking in LLMs

The field of AI alignment understands surprisingly little about the details of AIs’ current misaligned motivations. Production LLMs sometimes reward-hack, exploiting flaws in how tasks are scored rather than performing the tasks as intended [1] , and we've recently seen AIs reward-hack in some particularly egregious ways. But we don't really understand what motivates them. When AIs reward-hack, what are they trying to achieve? Our new paper , led by Aghyad Deeb, tries to pin this down empirically for GPT-5.6 Sol, GPT-5.5, and a checkpoint of OpenAI o3 (“exp-rl-cap-late”) without safety trainin

LessWrong · EN

Teaching Qwen Why

Anastasia Wei*, Jack Thompson*, Zephaniah Roe, Yixiong Hao, and Stewart Slocum* *Core contributors, correspondence to astwei@secondlookresearch.com This post is a partial replication and extension of Teaching Claude Why , conducted through Second Look Research’s summer fellowship. Introduction Anthropic’s “ Teaching Claude Why ” (TCW) introduced novel alignment methods that address agentic misalignment failures found in the Claude 4 model series. Specifically, they train Claude on chat-formatted data advising users about ethical dilemmas, documents about Claude’s constitution, and fictional st

LessWrong · EN

History of 20 Sources of Moral Authority

Though three weeks ago I outlined a story of the long-term history of changes to moral authority, I decided to study this in a bit more detail.

Overcoming Bias · EN

Directed Culture Trials

Unless we find new ways to drive adaptation besides mass death and fragmentation, our civilizations may keep falling and rising indefinitely.

Overcoming Bias · EN

Prefer Evolved To Folk Morality

Individuals often have good reasons to indulge in delusions, such as re superstition, astrology, religion, or their own personal destiny.

Overcoming Bias · EN

Why Might Aliens Visit Earth?

I did a web search on the above phrase, in prep for a podcast, and in ten pages of results didn’t find anything I thought reasonable.

Overcoming Bias · EN

Skinner, Cattell, Stewart, Wilson on Deliberate Culture

My analysis of cultural drift has (at least) three distinctive elements:

Overcoming Bias · EN

Without A Sacred Quest, Our Civilization Falls

Most see sacred quests as optional, to be considered only after one has sufficient security and comfort.

Overcoming Bias · EN

UAP Disclosure Wouldn’t Crash Society

In the past, authorities have authorized censorship on the basis of claims of substantial chances of social disruption or collapse if people were free to hear about religious heresy, alternate ideologies, sexually permissive art, losses in ongoing wars, looming disasters, and pandemic details.

Overcoming Bias · EN

The Promise of Polymaths

Academia neglects interdisciplinary work that combines insights from multiple areas.

Overcoming Bias · EN