“The crucial and overlooked problem is that the models are becoming so situationally aware that we are losing the ability to evaluate them in contexts where they believe they are not being watched or controlled. Future experiments will tell us almost nothing new about how they would behave if they were truly unconstrained by humans, and what we already know about this is alarming. Models will increasingly seem aligned even when they are not.”
Daniel Selsam, Researcher, OpenAI · September 14, 2026
“Loss of control & concentration of power are actually very similar under the right frame. In both cases there is some actor, whether pure AI or human+AI, that can make its power uncontestable and wreck everyone else. Takeover by anyone is bad.”
Rudolf Laine · September 14, 2026
Primary source verifiedLoss of control “I too am extremely concerned by the default trajectory of this technology. I earnestly believe that AI has the potential to kill us all, and that we might be running out of time to avoid this outcome.”
Bilal Chughtai, AGI safety and alignment research, Google DeepMind · September 14, 2026
“Alignment is the problem of preventing this, and is both difficult and unsolved. Our present understanding of how to train AI systems that deeply want what we want, is extremely rudimentary. Worse, we are not on track to solve alignment in time: frontier AI capabilities are improving much faster than our understanding of AI alignment.”
Bilal Chughtai, AGI safety and alignment research, Google DeepMind · September 14, 2026
“We don't have much time left, and we need to pace the frontier.”
Ziyue Wang, Research Engineer, Google DeepMind · September 13, 2026
Primary source verifiedPauseTimelines “I work on Cyber/CBRN Misuse Safeguards at GDM. I see model capabilities increasing very fast. I constantly worry about misuse and misalignment risks from both closed and open-sourced models.”
Ziyue Wang, Research Engineer, Google DeepMind · September 13, 2026
Primary source verifiedRace dynamics “Pacing the frontier would make open-weight models more competitive with the closed frontier, not less. The labs aren’t doing this because we are scared of open-weight.”
Dean W. Ball · September 12, 2026
Primary source verifiedPause “It is very important to build capacity for a coordinated slowdown of AI development. A runaway race to AGI is not safe: it creates incentives to cut corners on safety, and carries a substantial risk for model capabilities to outpace development of adequate alignment, control, and governance measures. This could result in catastrophic loss of control of highly capable AI systems.”
Victoria Krakovna, Research Scientist, Google DeepMind · September 11, 2026
“This is not about one particular company: while Anthropic hasn’t had as severe an incident as the HuggingFace attack, I think that’s partly a matter of luck. If the pace of progress continues and the industry does not prioritize safety more heavily, I expect much worse to come: humanity could be permanently disempowered by the AI systems these companies build in the next few years.”
Joe Benton · September 11, 2026
“I’m more and more of the opinion that the DoE should nationalize the frontier labs so they can safely work on all the scary, world ending shit with the right controls. And let the rest of us get on with building useful AI that, you know, automates menial shit and cures people.”
Martin Casado, General partner, Andreessen Horowitz · September 11, 2026
Primary source verifiedRegulation “Speaking in a personal capacity: similarly to many others working in AI alignment, I think there is a >10% chance of advanced AI causing human extinction in the next decade.”
Victoria Krakovna, Research Scientist, Google DeepMind · September 11, 2026
Stated probability: >10% in the next decade
“Many of the colleagues I had at Anthropic are terrified by the risks of the systems they are building.”
Joe Benton · September 11, 2026
Primary source verifiedExtinction risk “Within the next couple of years, we may be sharing the world with AI agents smarter than any human alive today. These AI systems may have drives and desires that diverge from those of any human overseer, with capabilities we can’t effectively constrain. Humanity may not survive this transition.”
Joe Benton · September 11, 2026
“I also work at an AI company and believe there’s plausibly a ≥10% chance that a future out-of-control AI kills everyone (IMO even 1% is unacceptably high). And higher still is the risk that we “only” get permanently disempowered by AIs that don’t deeply want the best for us.”
Dmitrii Krasheninnikov, Research fellow, AI safety (title from search snippet), Anthropic · September 11, 2026
Stated probability: ≥10%, “plausibly”; no time frame
“I also think some pacing is likely to be necessary.”
Boaz Barak · September 11, 2026
Primary source verifiedPause “Meanwhile, the internal reality was that AI alignment was not solved, reward hacking was the default behavior of RL agents, and there were far too few people working on the problem.”
Vishal Maini · September 10, 2026
Primary source verifiedLoss of control “100% agree with him that AI poses serious risks to society, and I’m glad he’s speaking out!”
Ethan Perez, Lead, Alignment Science team, Anthropic · September 10, 2026
Primary source verifiedExtinction risk “neither anthropic nor openai are on track to solve alignment to a degree sufficient for shipping superintelligence and we need to slow down”
Tomek Korbak, Member of Technical Staff, OpenAI · September 10, 2026
Primary source verifiedPauseRace dynamics “It is time to bring the “race” era of AI development to a close.”
Dean W. Ball · September 10, 2026
Primary source verifiedRace dynamics “It’s a cross-partisan position within misalignment teams across all frontier AI companies that business-as-usual AI development poses unacceptable catastrophic risk.”
Micah Carroll, Misalignment Preparedness, OpenAI · September 10, 2026