“It is very important to build capacity for a coordinated slowdown of AI development. A runaway race to AGI is not safe: it creates incentives to cut corners on safety, and carries a substantial risk for model capabilities to outpace development of adequate alignment, control, and governance measures. This could result in catastrophic loss of control of highly capable AI systems.”
Victoria Krakovna, Research Scientist, Google DeepMind · September 11, 2026
Title and organization are as of the date said.
In context
“It is very important to build capacity for a coordinated slowdown of AI development. A runaway race to AGI is not safe: it creates incentives to cut corners on safety, and carries a substantial risk for model capabilities to outpace development of adequate alignment, control, and governance measures. This could result in catastrophic loss of control of highly capable AI systems.” Increasing model capabilities in a slow and controlled way would allow us enough time to develop, adapt and test our alignment and control methods for each level of capability. Advanced AI should only be built with a robust assurance of safety, and a slowdown would make this possible.
Source
- https://x.com/vkrakovna/status/2098337070607380983 (tweet)
- Content captured 2026-11-11, SHA-256 d67ac01b94a95c87cdad9d56b316bb53a2675816311d30d9d725761f55501b31
Cite this quote
Krakovna, V. (2026, September 11). It is very important to build capacity for a coordinated slowdown of AI development. A runaway race to AGI is… [Post]. https://x.com/vkrakovna/status/2098337070607380983
More from Victoria Krakovna
“Speaking in a personal capacity: similarly to many others working in AI alignment, I think there is a >10% chance of advanced AI causing human extinction in the next decade.”
Victoria Krakovna, Research Scientist, Google DeepMind · September 11, 2026
Stated probability: >10% in the next decade
Others on the same topic
“If we achieve ASI within 6-12 months, I estimate that the chance of an AI takeover is roughly 75%. (In a recent talk, Paul Christiano said it would be a coin flip.) Paired with the timelines above, unless we achieve aggressive pacing or a pause within 6-12 months, we incur something like a 1-in-4 to 1-in-3 chance of AI takeover within 12 months.”
Nikola Jurkovic, Member of Technical Staff, METR · November 13, 2026
Stated probability: ~75% AI takeover if ASI within 6-12 months; 1-in-4 to 1-in-3 chance of AI takeover within 12 months absent pacing/pause
“Because if we are just competing with the machines, we will probably lose.”
Jan Kulveit · October 2026
“A lot of people are talking about rogue AI hacking into data centers and taking over the world, but it is not in the mainstream consciousness that like AI could literally take over the world just by continuing to do what it's supposed to do, which is automating everything”
Jay Chooi, Founder and CEO, Robocurve · October 2026
All quotes on loss of control →All quotes on pause →All quotes on race dynamics →