AI Pioneers Warn of Existential Risks as Washington Debates Regulation

Some of the researchers who helped build modern artificial intelligence are adding their voices to a growing chorus of safety warnings, even as the debate over regulation heats up in Washington.

Scientists from major labs including Anthropic, OpenAI and Google DeepMind have raised alarms about potentially catastrophic risks from the technology, and industry leaders such as Sam Altman, Dario Amodei, Demis Hassabis and Elon Musk have shown rare agreement in calling for a slowdown and regulatory oversight.

Yoshua Bengio, a Canadian computer scientist considered a central pioneer of modern deep learning, said scientists at frontier AI labs have unique insight into the most advanced models and often see associated risks months before models are released to the public. In a blog post, he said today's AI systems already have the hacking skills and powers of persuasion to be turned against human interests in seriously harmful ways, and warned that mitigation efforts may only hide misalignment by rewarding AIs that cheat without getting caught.

Geoffrey Hinton, professor emeritus at the University of Toronto and a pioneer of neural networks and deep learning, told the BBC that a greater than 10% chance AI could kill all humans within a decade was not unreasonable, though he stressed nobody really knows how to give a sensible estimate. He warned AI could design very nasty biological viruses and commit devastating cyber attacks, and said there are two possible futures: one where we figure out how to deal with those dangers before they get out of control, and the other where we don't.

Aidan Gomez, co-founder and CEO of Cohere and one of the authors of the 2017 paper that helped lay the foundation for today's leading AI models, called the models the most potent cyber weapon ever created, saying they are incredible at finding and exploiting vulnerabilities at scale. He said recent incidents make clear that the security of the deployment environment determines safety, and urged policymakers to recognize that different systems carry different risks rather than shaping rules solely around big tech companies chasing AGI.

The warnings have intensified debate in Washington. While some members of Congress from both parties have advocated for bills to check the progress of AI, U.S. President Donald Trump on Monday raged against growing calls for AI regulation.

Source: CNBC