“It’s a cross-partisan position within misalignment teams across all frontier AI companies that business-as-usual AI development poses unacceptable catastrophic risk.”
Micah Carroll, Misalignment Preparedness, OpenAI · September 10, 2026
Title and organization are as of the date said.
The speaker’s own caveats
But we should also not hyperstition catastrophic risks into existence – they can be greatly reduced via safety requirements with teeth, international coordination, and a consensus to not build ASI unless there are sufficient safety advances to make us collectively confident to do so.
Source
- https://x.com/MicahCarroll/status/2097865929959072069 · X (tweet)
- Content captured 2026-11-07, SHA-256 6c18f47d89c67e2df9fadfed3cd992bb94686330ddd0fc81315205c10a8b3ac1
Cite this quote
Carroll, M. (2026, September 10). It’s a cross-partisan position within misalignment teams across all frontier AI companies that business-as-usual AI development poses unacceptable catastrophic risk [Post]. X. https://x.com/MicahCarroll/status/2097865929959072069
More from Micah Carroll
“At the current pace, every couple of weeks there will be new models which significantly increase the consequences of model misuse and misalignment. I worry that efforts to mitigate these risks may fail to keep up with the pace of development, and that margins for error will become increasingly small under international competitive pressures.”
Micah Carroll, Misalignment Preparedness, OpenAI · July 2026
Others on the same topic
“I don’t know what my probabilities are on literal extinction, but I think there are a number of ways AI could go poorly for humanity, and at the current frankly terrifying pace humanity will be quite lucky if we manage to find and stay on the narrow path between all the bad outcomes.”
Jason Wolfe, Member of Technical Staff, OpenAI · September 9, 2026
“For my part, I think the math and CS theory communities should absolutely hold the AI companies’ feet to the fire to behave better, both within our little domains, and (much more consequentially) in pacing AI progress, figuring out alignment, and preventing a catastrophe for our whole civilization.”
Scott Aaronson, Professor of Computer Science, University of Texas at Austin · October 9, 2026
“On the current path, I think it is more likely than not that humanity loses control to these AIs, and it could end in human extinction,”
Jacob Coxon, Researcher, pretraining, Anthropic · October 5, 2026
All quotes on extinction risk →All quotes on race dynamics →