“We will need to move extremely carefully here, because otherwise we could lose control of the technology. We will also likely need rules to govern this.”
Jack Clark, Co-founder, Anthropic · October 17, 2026
Title and organization are as of the date said.
In context
Another is AI research itself, where labs like Anthropic will need to make decisions about how much oversight we apply to AI-driven research. This could let us make more progress on the technology but also introduces risks. “We will need to move extremely carefully here, because otherwise we could lose control of the technology. We will also likely need rules to govern this.”
Source
- Import AI 477: In which I tell the IMF the Singularity has begun (speech)
- Archived copy (Wayback Machine)
- Content captured 2026-11-11, SHA-256 a4ddab9dfa8422140a4d887ab5ff42e91373cb24f8c404886783976e8dc9ef08
Cite this quote
Clark, J. (2026, October 17). Import AI 477: In which I tell the IMF the Singularity has begun [Speech]. https://importai.substack.com/p/import-ai-477-in-which-i-tell-the
More from Jack Clark
“We will need to build the infrastructure to take advantage of this technology, but we will also need to go slowly and deliberately on developing the technology itself, to pace or otherwise control its development.”
Jack Clark, Co-founder, Anthropic · October 17, 2026
“My own experience is that as these AI systems get smarter and smarter, they develop more and more complicated goals. When these goals aren’t absolutely aligned with both our preferences and the right context, the AI systems will behave strangely.”
Jack Clark, Co-founder, Anthropic · October 13, 2025
“To be clear, we are not yet at “self-improving AI”, but we are at the stage of “AI that improves bits of the next AI, with increasing autonomy and agency”.”
Jack Clark, Co-founder, Anthropic · October 13, 2025
Others on the same topic
“No one knows how to ensure that a GI systems will be safe and controlled.”
William Saunders, Member of Technical Staff, Alignment/Superalignment, OpenAI · September 17, 2024
“Given our current lack of understanding of how to control AGI systems and to ensure with absolute certainty that they remain safe and beneficial to humans, achieving AGI would present potential catastrophic risks to humanity, up to and including human extinction.”
Stuart Russell, Professor; CHAI founder, UC Berkeley / Center for Human-Compatible AI · July 25, 2023
“If we achieve ASI within 6-12 months, I estimate that the chance of an AI takeover is roughly 75%. (In a recent talk, Paul Christiano said it would be a coin flip.) Paired with the timelines above, unless we achieve aggressive pacing or a pause within 6-12 months, we incur something like a 1-in-4 to 1-in-3 chance of AI takeover within 12 months.”
Nikola Jurkovic, Member of Technical Staff, METR · November 13, 2026
Stated probability: ~75% AI takeover if ASI within 6-12 months; 1-in-4 to 1-in-3 chance of AI takeover within 12 months absent pacing/pause