AI security researcher Nate Soares: „I no longer save for retirement“
The president of the Machine Intelligence Research Institute considers it more likely than not that a super‑powerful artificial intelligence will wipe out humanity within a decade. He calls for a halt to development, and many experts view his assessment as exaggerated. Photo: Nate Soares
Nate Soares, president of the Machine Intelligence Research Institute (MIRI) in Berkeley, California, estimates the probability that artificial intelligence will wipe out humanity in less than ten years at over 50 percent. He said this in an interview with Nicholas Thompson, the editor of the US magazine The Atlantic. Business Insider reported on it.
How serious he is about it is shown by Soares„ own retirement planning. “I am not currently saving for retirement," he said. He still has a retirement account from his time at Google, but hasn't contributed to it for a decade.
The feared scenario
Soares has been working on AI safety for more than ten years. Together with MIRI founder Eliezer Yudkowsky he wrote the book „If Anyone Builds It, Everyone Dies“ („Wenn es jemand baut, sterben alle“).
His concern is not about today's chatbots, but about a future, generally applicable „superintelligence.“ Such a system could, in his view, escape human control, improve itself and build its own infrastructure. In the end it could reshape Earth so that human life is no longer possible. „The real question is not how it kills us, but how it becomes autonomous,“ Soares said. AI still depends on humans to keep power supplies and chip production running.
His demand: halt development
This outcome is not inevitable, emphasizes Soares: „We could still stop this race toward the very dangerous type of AI.“
Governments should halt the training of general, ever more powerful AI systems. AI for narrowly defined tasks, such as cancer treatment, should be allowed to continue development.
Classification
With his assessment, Soares is at the pessimistic end of the expert debate. Nobel laureate and AI pioneer Geoffrey Hinton estimated the risk of humanity’s extinction by AI at about ten percent. Other leading researchers consider such scenarios far off or unlikely and see the more urgent problems in misinformation, the labor market, and power concentration.
At the same time, Microsoft, Alphabet, Amazon and other corporations are investing three‑digit billions in ever larger AI models. The debate over whether and how this development should be regulated is therefore becoming louder rather than quieter.