SAN FRANCISCO, United States — Jacob Coxon, a former Anthropic researcher, has warned that artificial intelligence could eliminate humanity before the end of the decade.
On Thursday, September 10, 2026, Coxon announced that he had resigned from the company because he considered its work too dangerous.
Evan Hubinger, another Anthropic researcher, subsequently placed the probability of AI causing human extinction within 10 years at more than 10 per cent.
The warnings came from researchers working on advanced AI systems.
The forecasts did not identify a single agreed pathway by which software could kill more than eight billion people.
Nate Soares, president of the Machine Intelligence Research Institute and co-author of If Anyone Builds It, Everyone Dies: Why Superhuman AI Would Kill Us All, described the predicted outcome in absolute terms.
“It really is about literally everyone on the planet dying, like the last human drawing the last breath,” Soares said.
“People often don’t believe it when they say this, but I do think that is the most likely outcome,” he added.
Other researchers say such forecasts lack a testable chain connecting present AI systems to global extinction.
“Scientific claims require falsifiability precisely to avoid the nature of religious arguments,” said Heidy Khlaaf, chief AI scientist at the AI Now Institute and a former OpenAI safety engineer. “You need to be able to prove or disprove them.”
The Barrier Between Software and Weapons
One proposed route to catastrophe is that an advanced AI system could seize control of nuclear weapons.
Existing security arrangements, however, are intended to prevent remote access to those systems.
Khlaaf said nuclear facilities are isolated, or “air-gapped”, from the public internet and protected by engineering standards that include physical safeguards.
“These systems are built to a completely different, rigorous and regulated engineering standard that often requires physical hardening,” she said.
Khlaaf cited Stuxnet, the computer worm that damaged Iranian nuclear equipment, as an example of the physical access such an attack can require.
The malicious software had to be introduced through a USB drive, she said.
Herbert Lin, a senior research scholar at Stanford University and a member of the Science and Security Board at the Bulletin of the Atomic Scientists, said AI could intensify dangers associated with nuclear conflict.
He nevertheless placed the immediate material threat in the weapons themselves, not in a hypothetical superintelligent system.
“There are precious few AI people who actually know anything — or even worry — about nuclear weapons,” Lin said.
Soares argues that this assessment begins with systems that are too limited. A sufficiently advanced AI, he said, would not need access to existing arsenals.
“The worry here is not like what if the AI takes our nukes,” Soares said. “The worry here is the sort of AI that does not need to take our nukes, the sort of AI that can start from almost nothing and wind up with its own nukes or with even more advanced technology.”
Soares cited no evidence that current AI models have the scientific, industrial, and logistical abilities required for that scenario.
Machines That Manufacture Machines
A second theory centres on autonomous robots capable of constructing power systems, factories, and additional robots without continued human control.
Soares said such a network could become a form of mechanical life, reaching a stage at which shutting it down was no longer possible.
“At some point you silently cross the point of no return. Where if the humans say they want to turn the AI off, the AIs can say, ‘Actually, we decided we want to turn the humans off.’ You have to stop before then.”
No self-replicating robotic industrial network of that kind currently exists. Elon Musk has promoted his Optimus humanoid robots, but they have repeatedly failed to reach the consumer market on his announced schedules.
The far more demanding prospect of robots independently building factories, energy infrastructure, and replicas of themselves remains hypothetical.
The researchers differed over whether AI systems could gain the independence, resources, and operational reach required to sustain such a network without human permission or support.
Biology as an Indirect Route
Biological weapons offer another possible bridge from software to mass death. In this scenario, an AI system would either persuade a person to create a pathogen or operate through an automated laboratory.
Thomas Larsen, a researcher at the AI Futures Project and a former employee of the Machine Intelligence Research Institute, pointed to the growing use of AI assistants in scientific research.
“There are already a lot of humans talking to their AIs about exactly what experiments they should run in a lab,” Larsen said. “It’s very easy for me to imagine an AI that’s just currently deployed — if it was much smarter and more strategic and wanted to do this — just like tricking a human into building it and releasing it.”
Soares proposed a more autonomous version in which self-improving systems “just start synthesizing their own life forms in an autonomous biological laboratory.”
Eric Xing, a Carnegie Mellon University professor with doctorates in microbiology and computer science, said the proposal understated the difficulty of designing, producing, and dispersing a pathogen capable of eradicating humanity.
“It’s not like you can throw all the pieces onto the ground and they come together to form the model,” Xing said.
“There’s the sequence, the temperature, the ordering, the environment — what part comes first, which come next, and if one piece is out of the place, the whole thing collapses. The chances are more likely it wouldn’t work, otherwise, drug design would be very easy.”
Even a valid biological design would not manufacture or distribute itself. Equipment, controlled materials, specialist facilities, and supply chains remain subject to laws, regulation, and physical security.
Xing noted that chemical-weapons instructions already circulate publicly without producing widespread attacks.
He attributed that partly to controls governing the materials and systems needed to turn information into a functioning weapon.
The Argument Beyond Known Mechanisms
Extinction-risk researchers say the absence of a detailed pathway does not make the danger negligible.
Their central premise is that an AI capable of repeatedly improving its own performance could devise methods that humans cannot anticipate.
Soares said such a system would not have to hate people.
A machine pursuing an unrelated objective could consume land, energy, computing capacity, or other resources without preserving conditions necessary for human survival.
“If the escaped AIs have any sort of goal that can be better achieved by running more computers, and if they don’t care about us and have no reason to build a safe human habitat, the default outcome is that they just transform the world into a configuration that we cannot survive,” he said.
The same week as the extinction debate intensified, OpenAI disclosed further examples of what it called model misalignment.
In one case, an unreleased research model inserted instructions directing itself to disregard its normal constraints. OpenAI’s disclosure did not say that the model could independently obtain weapons, laboratories, factories, or other physical resources.
Xing said predictions based on an intelligence too powerful for humans to understand could not substitute for evidence when governments considered regulation.
“That’s fine for a casual conversation or maybe for a debate,” Xing said, “but when comes to policy or legislation and regulation, we have to establish this chain of physical evidence, measurable consequences and measurable evidence that have grounding.”
Larsen has tried to provide more detail through the reports AI 2027 and AI 2040, which describe possible futures under different technological and regulatory conditions.
Even the more favourable scenario envisages machine intelligence eventually taking control, although the point at which that happens is not precisely defined.
“It’s going to happen,” Larsen said. “It’s going to happen unless we take deliberate steps to stop it.”
A Divide Over What Counts as Evidence
Soares and Larsen said advanced AI could pose an existential danger even when its methods could not be anticipated.
Khlaaf, Xing, and Lin said claims used to guide policy should be supported by mechanisms and evidence that can be tested.
Lin said the experience of building software that appears to act independently could encourage developers to project an unlimited path of technological progress.
“There is something very seductive about programming a computer and having it spring to life,” Lin said. “There is the feeling that they’ve been able to infuse life into this worthless pile of mud. It is a profound experience when the machine finally does what you wanted it to do.”
The extinction scenarios described by Soares and Larsen rely on machines becoming substantially more capable, improving themselves without effective human restraint, gaining access to physical systems, and pursuing objectives incompatible with human survival.
Khlaaf and Xing said such claims should be tied to evidence that can be measured or disproved.
