“Yudkowsky and his Sequences 20 years ago. I remember thinking: even if these people talk and act like crazy cultists, still, let me bend over backwards to be epistemically virtuous, and entertain their ideas on their merits, as very few academics would. Even if, of course, I ultimately end up rejecting the ideas, on the simple ground that powerful AI is such an absurdly remote prospect that it’s almost impossible to say anything useful about it today, outside the realm of speculative fiction. For my failure to see what was coming, it seems like an appropriate punishment that I’m now, in 2026, effectively teaching a course on Yudkowsky Studies. And it’s the most important course I can teach.”
Scott Aaronson, Professor of Computer Science, University of Texas at Austin · October 4, 2026
Title and organization are as of the date said.
In context
He recalls encountering Yudkowsky's Sequences 20 years ago and ultimately rejecting the ideas "on the simple ground that powerful AI is such an absurdly remote prospect that it’s almost impossible to say anything useful about it today, outside the realm of speculative fiction." “Yudkowsky and his Sequences 20 years ago. I remember thinking: even if these people talk and act like crazy cultists, still, let me bend over backwards to be epistemically virtuous, and entertain their ideas on their merits, as very few academics would. Even if, of course, I ultimately end up rejecting the ideas, on the simple ground that powerful AI is such an absurdly remote prospect that it’s almost impossible to say anything useful about it today, outside the realm of speculative fiction. For my failure to see what was coming, it seems like an appropriate punishment that I’m now, in 2026, effectively teaching a course on Yudkowsky Studies. And it’s the most important course I can teach.” The thing about AI alignment is that there’s no textbook
Source
- My new course at UT Austin: AI Alignment Theory (article)
- Archived copy (Wayback Machine)
- Content captured 2026-11-16, SHA-256 9b5d7f37c3a90a7921ab1d67ff9c9295fdd1a45bdd7d259686a12a117af36338
Cite this quote
Aaronson, S. (2026, October 4). My new course at UT Austin: AI Alignment Theory. https://scottaaronson.blog/?p=10125
More from Scott Aaronson
“For my part, I think the math and CS theory communities should absolutely hold the AI companies’ feet to the fire to behave better, both within our little domains, and (much more consequentially) in pacing AI progress, figuring out alignment, and preventing a catastrophe for our whole civilization.”
Scott Aaronson, Professor of Computer Science, University of Texas at Austin · October 9, 2026
Others on the same topic
“On the current path, I think it is more likely than not that humanity loses control to these AIs, and it could end in human extinction,”
Jacob Coxon, Researcher, pretraining, Anthropic · October 5, 2026
“If the language models actually reach the capability threshold where they can shape the world unconstrained by human will, they will probably do something extreme and destroy humanity in the process.”
Daniel Selsam, Researcher, OpenAI · September 14, 2026
“I also work at an AI company and believe there’s plausibly a ≥10% chance that a future out-of-control AI kills everyone (IMO even 1% is unacceptably high). And higher still is the risk that we “only” get permanently disempowered by AIs that don’t deeply want the best for us.”
Dmitrii Krasheninnikov, Research fellow, AI safety (title from search snippet), Anthropic · September 11, 2026
Stated probability: ≥10%, “plausibly”; no time frame
All quotes on extinction risk →All quotes on loss of control →