“To be clear, we are not yet at “self-improving AI”, but we are at the stage of “AI that improves bits of the next AI, with increasing autonomy and agency”.”
Jack Clark, Co-founder, Anthropic · October 13, 2025
Title and organization are as of the date said.
Source
- Technological Optimism and Appropriate Fear · Import AI (speech)
- Archived copy (Wayback Machine)
- Content captured 2026-11-13, SHA-256 d613795a2bb34c8b7cb3d4ee6de49d98fad9a659d54c9f49259b29f632f50134
Cite this quote
Clark, J. (2025, October 13). Technological Optimism and Appropriate Fear [Speech]. Import AI. https://importai.substack.com/p/import-ai-431-technological-optimism
More from Jack Clark
“We will need to build the infrastructure to take advantage of this technology, but we will also need to go slowly and deliberately on developing the technology itself, to pace or otherwise control its development.”
Jack Clark, Co-founder, Anthropic · October 17, 2026
“We will need to move extremely carefully here, because otherwise we could lose control of the technology. We will also likely need rules to govern this.”
Jack Clark, Co-founder, Anthropic · October 17, 2026
“My own experience is that as these AI systems get smarter and smarter, they develop more and more complicated goals. When these goals aren’t absolutely aligned with both our preferences and the right context, the AI systems will behave strangely.”
Jack Clark, Co-founder, Anthropic · October 13, 2025
Primary source verifiedLoss of control
Others on the same topic
“If we achieve ASI within 6-12 months, I estimate that the chance of an AI takeover is roughly 75%. (In a recent talk, Paul Christiano said it would be a coin flip.) Paired with the timelines above, unless we achieve aggressive pacing or a pause within 6-12 months, we incur something like a 1-in-4 to 1-in-3 chance of AI takeover within 12 months.”
Nikola Jurkovic, Member of Technical Staff, METR · November 13, 2026
Stated probability: ~75% AI takeover if ASI within 6-12 months; 1-in-4 to 1-in-3 chance of AI takeover within 12 months absent pacing/pause
“Within the next couple of years, we may be sharing the world with AI agents smarter than any human alive today. These AI systems may have drives and desires that diverge from those of any human overseer, with capabilities we can’t effectively constrain. Humanity may not survive this transition.”
Joe Benton · September 11, 2026
“If I had to quantify my uncertainty I would estimate an all-things-considered risk of 4% over the next year and 15% over the next three years.”
Paul Christiano, Nonprofit board member; Safety and Security Committee, OpenAI · September 9, 2026
Stated probability: 4% over the next year; 15% over the next three years