“Right now, no lab has demonstrated much of a willingness to make any costly tradeoffs to get safety right (we get lots of safety committees, yes, but those are pretty meaningless). By default, we’ll probably stumble into the intelligence explosion and have gone through a few OOMs before people even realize what we’ve gotten into.”
Leopold Aschenbrenner · June 2024
Title and organization are as of the date said.
In context
We’re not on track for superdefense, for an airgapped cluster or any of that; I’m not sure we would even realize if a model self-exfiltrated. “Right now, no lab has demonstrated much of a willingness to make any costly tradeoffs to get safety right (we get lots of safety committees, yes, but those are pretty meaningless). By default, we’ll probably stumble into the intelligence explosion and have gone through a few OOMs before people even realize what we’ve gotten into.” We’re counting way too much on luck here.
The speaker’s own caveats
I’m incredibly bullish on the technical tractability of the superalignment problem.
Source
- IIIc. Superalignment (Situational Awareness: The Decade Ahead) · situational-awareness.ai (article)
- Archived copy (Wayback Machine)
- Content captured 2026-11-16, SHA-256 3cfc79dc1f5b9d8100b9bd574513726cb2ecbf7c42fae3bbbe340a246d67524b
Cite this quote
Aschenbrenner, L. (2024, June). IIIc. Superalignment (Situational Awareness: The Decade Ahead). situational-awareness.ai. https://situational-awareness.ai/superalignment/
More from Leopold Aschenbrenner
“On the one end there are the doomers. They have been obsessing over AGI for many years; I give them a lot of credit for their prescience. But their thinking has become ossified, untethered from the empirical realities of deep learning, their proposals naive and unworkable, and they fail to engage with the very real authoritarian threat. Rabid claims of 99% odds of doom, calls to indefinitely pause AI—they are clearly not the way.”
Leopold Aschenbrenner · June 2024
“We need to not screw it up. Recognizing the power of superintelligence also means recognizing its peril. There are very real safety risks; very real risks this all goes awry—whether it be because mankind uses the destructive power brought forth for our mutual annihilation, or because, yes, the alien species we’re summoning is one we cannot yet fully control. These are manageable—but improvising won’t cut it.”
Leopold Aschenbrenner · June 2024
“The AGI race has begun. We are building machines that can think and reason. By 2025/26, these machines will outpace many college graduates. By the end of the decade, they will be smarter than you or I; we will have superintelligence, in the true sense of the word. Along the way, national security forces not seen in half a century will be unleashed, and before long, The Project will be on. If we’re lucky, we’ll be in an all-out race with the CCP; if we’re unlucky, an all-out war.”
Leopold Aschenbrenner · June 2024
Others on the same topic
“Because if we are just competing with the machines, we will probably lose.”
Jan Kulveit · October 2026
“A lot of people are talking about rogue AI hacking into data centers and taking over the world, but it is not in the mainstream consciousness that like AI could literally take over the world just by continuing to do what it's supposed to do, which is automating everything”
Jay Chooi, Founder and CEO, Robocurve · October 2026
“Alignment is the problem of preventing this, and is both difficult and unsolved. Our present understanding of how to train AI systems that deeply want what we want, is extremely rudimentary. Worse, we are not on track to solve alignment in time: frontier AI capabilities are improving much faster than our understanding of AI alignment.”
Bilal Chughtai, AGI safety and alignment research, Google DeepMind · September 14, 2026
All quotes on loss of control →All quotes on race dynamics →