P(doom) is under-specified: EV(moral flourishing) & EV(utopia)

It always bugs me when people use “P(doom)” instead of something more clearly specified like “P(AI takeover)”. But “P(AI takeover)” doesn’t feel great to me either, because we could avoid takeover and have a future that is extremely mediocre or even actively bad if S risks occur. Holden Karnofsky famously held a P(misaligned AI) of between 10% and 90% [1] . Depending on definition of P(doom), I could see myself anywhere in that range, partly because I’m indecisive, but partly because I don’t really know what P(doom) means. Ambiguous Scenarios: Doom or Not-Doom? Here are a few examples illustra

• #doom