#alignment

Can alignment realistically be solved under Plan C?

In his recent post , Scott Alexander argues that plans allocating a short amount of time—perhaps a few months—to solving alignment should be taken seriously. He argues that people calling for an indefinite or long pause have insufficient grounds for concluding that alignment of ASI is very difficult and cannot simply be solved by delegating it to AI systems. There's a kafkatrap where if someone expresses hope that alignment is not doomed, they get told that they don't understand the subject, but if they quote widely-acknowledged experts who say alignment isn't doomed, they get accused of "defe

LessWrong · EN

CRUCIBLE: IARPA-Funded AI Alignment Research — Applications Open

TL;DR: AE Studio is running a new government-funded research program that pairs alignment researchers with our internal engineering team for two-week sprints. We will select six alignment researchers whose proposed work attempts to answer one of our six open questions. Promising sprints have the opportunity to expand to longer funded work. Researchers are asked to commit two hours a week, and in return AE will provide compute and 60+ hours of engineering from us. Apply here . Applications close 11/15 at 11:59 PM PST. What is CRUCIBLE? CRUCIBLE is an IARPA-funded research program executed by AE

LessWrong · EN

Yudkowsky's mind model for AI alignment research: experiment results

TLDR. Yudkowsky mind-model (sideload) is good, can produce research, but is not better in math than Opus 5.5. However, in my impression, EY-sideload has broader view, knows what is really important to AI safety and has human anchoring. In blind review, EY-sideload won only on honesty, but honesty is important. EY-sideload also produced a good summary of the shutdown problem’s difficulty: “training reduces paid influence over shutdown least where the stakes are highest”. You can find full description of results in Substack article which is AI written. Here is short human-made summary. I put on

LessWrong · EN

Import AI 478: Math for alignment; ProgramBench; and Mirendil shows what RSI means

What is “night” to a machine?

Import AI · EN