Why AI-Built Bioweapons Are Unlikely to End Humanity, Scientists Say

Fears of an AI-driven apocalypse have spread from Silicon Valley to Washington, and biological weapons sit near the top of the list of imagined doomsday scenarios. This summer, AI executives pushed for laws governing synthetic DNA manufacturing. Last month, Stanford University and Arc Institute researchers demonstrated that AI can design new viral genomes. Then Anthropic released a report describing attempts to use its Claude model in ways that "could support biological weapons development," calling biological misuse one of the most serious risks of frontier AI. CEO Dario Amodei soon urged the government to help labs "pace the frontier."

Yet many scientists place that risk fairly low among global concerns. David Bellamy of the Institute of Foundation Models in Sunnyvale, California, argues AI is not a fundamentally new bioweapons threat. The scientific community has wrestled for decades with the dual-use dilemma-whether to publish potentially dangerous biological research. Before large language models, he notes, the internet, open-access journals and translation tools like Google Translate already made biological information, including lab protocols, easier to find.

"AI is essentially a tool that can help both good actors, like scientists, and also threat actors to peruse information more quickly and define and source those protocols more quickly," Bellamy says. "But those capabilities are not really the bottleneck in the production of bioweapons." The real bottleneck, he says, is assembling a virus and obtaining the raw materials, equipment and knowledge required. A bad actor would need gene fragments, a genome built from scratch, and verification that the virus infects humans, causes the intended illness and spreads person to person. Robotic lab assistants can automate some tasks, but someone still needs the expertise and resources to run the experiments.

Jason Kelly, CEO of biotech startup Ginkgo Bioworks, doubts an artificial general intelligence could seize and release existing pathogens, let alone engineer new ones. Ginkgo, which specializes in autonomous labs, recently ran a project with OpenAI in which its GPT-5 model operated a lab. "The AI could not take over the lab," he says, partly because human workers could simply refuse to hand over the substances and equipment requested to physically produce a bioweapon. Those human checks mean "you'd have to have dramatically more robots all over the place," he says, for an AGI to deploy machines for the delicate virology work required.

Immunologist Derya Unutmaz likewise says humans could thwart any AGI attempt to develop a deadly disease, arguing that even if a rogue superintelligence unleashed a supervirus, scientists could use other AI systems to swiftly develop a vaccine.

Still, some who doubt AGI is an immediate threat worry about bad actors and future risks. "It is impossible for AI to access a fully autonomous lab and autonomously build a virus today because fully autonomous labs do not exist yet," says Olivia Scharfman, a biotechnology fellow at the think tank Institute for Progress. "But I do think that an AI could pay someone to do it for them." She considers AI-assisted bioterrorism an immediate threat, naming "transhumanist AI successionists"-adherents of a fringe ideology hoping to supplant humans with bots-as one group potentially nihilistic enough to try.

Other experts doubt biological warfare makes methodological sense. Genetic biologist and computational biology professor Francois Belloux tells WIRED the risks are "somewhat misunderstood," noting people "overestimate the value of pathogens as weapons." Bioweapons are not an especially attractive choice for aspiring genocidaires, he says, because targeting certain groups while sparing others is difficult and mass-producing and distributing pathogens is logistically complicated compared with, say, launching a bomb. "If you want to kill people, there are much, much, much better ways to kill them than to try to engineer some virus or bacterium and then release it," he says-whether the actor is the most sophisticated artificial mind ever created or just a regular human.

With AI bringing the bioweapon threat into focus, Scharfman says now is the moment to push for stronger protections against all biological threats, including existing pathogens like H1N1. That could mean legislation tightening security protocols around DNA synthesis technology, but she also advocates changes to reduce risk from all viruses, such as upgrading air purification in buildings.

Whether AI demonstrably raises the risk of bioweapon development is hard to answer, says Steph Guerra, head of AI and bio at the policy think tank Rand Corporation. "AI has been shown to be very good at integrating information together and being motivational to people for doing good and bad things," she says. Even if the risk is low, Guerra says society can still guard against misuse. Governments could require firms selling synthetic DNA and RNA to screen customers and orders to prevent misuse of genetic material. Many companies already use software to screen for so-called "sequences of concern," but the practice is neither universal nor mandated.

Most everyone agrees models should have safeguards so they don't provide actionable, dangerous information. Guerra says strengthening global surveillance to detect outbreaks of new diseases and circulating pathogens early, plus data-sharing protocols across AI companies, gene synthesis providers and government, could help prevent misuse. "With pretty much almost any biosecurity control, there are going to be ways that they can be circumvented," she says. "That's why we need layered approaches that provide friction across the entire pathway, from ideation to intention of a bad actor, all the way to the release of a bioweapon."

Unutmaz points to a more pressing worry: that AI doom conversations distract from how cutting-edge systems can help humans develop vaccines and make other medical breakthroughs. "We really need to focus on the positive aspect of it," he says.

Source: Wired science