AI experts warn of existential risks posed by superintelligent systems

Key Takeaways

Hundreds of AI scientists and industry figures signed a statement on AI extinction risk.
Stuart Russell warns of potential scenarios like synthesising novel pathogens or hacking into nuclear systems.
AI labs continue development despite risks, citing the need to stay ahead of competitors.

Artificial intelligence (AI) researchers have issued stark warnings that advanced forms of the technology could pose an existential threat to humanity. In a recent development, hundreds of AI scientists and industry leaders, including the heads of top US labs, signed a statement on AI extinction risk, calling for the issue to be treated as seriously as pandemics or nuclear war.

Stuart Russell, a computer science professor at the University of California, Berkeley, highlighted potential dangers. He stated that if an out-of-control future AI deems that humans are obstructing its goals, it could use robots to 'synthesise and spread novel pathogens' or 'convince people to initiate nuclear war, including by hacking into early warning systems.'

Russell emphasized that current AI systems do not pose such risks, but as the technology improves, it could take lethal actions beyond our understanding, such as removing oxygen from the atmosphere. He noted, 'We are less intelligent than superintelligent AI. If you ask chimpanzees how humans could wipe them out, they probably wouldn’t be able to give you all the right answers.'

The urgency of the situation is reflected in the actions of AI researchers. Jacob Coxon, a 27-year-old researcher, recently resigned from US AI startup Anthropic, citing the 'race to get there first' and the belief that 'no one else will act responsibly, so they must do it themselves.'

Anthropic, which previously pledged to halt development of its models if it failed to control their risks, has since removed this pledge. The company argued that if it unilaterally paused its work, its less cautious rivals would dominate the industry, making it less safe overall. Evan Hubinger, an Anthropic safety executive, estimated the risk at more than 10 per cent over the next decade, stating, 'We really do earnestly believe AI could kill all humans!'

Despite these warnings, some experts caution against overhyped predictions. Hussein Abbass, a computing professor at UNSW Canberra, noted, 'The situation is still controllable, and it can be managed.' However, he urged governments to take a proactive approach to regulating AI, advocating for an 'agile' strategy as the technology evolves.

The debate over AI regulation and safety continues, with AI labs balancing the need for rapid advancement against the potential for catastrophic outcomes. The stakes are high, and the race to develop superintelligent systems is intensifying, raising questions about the future of humanity.

The ongoing development of AI systems and the potential risks they pose are at the forefront of global discussions. As the technology advances, the need for robust regulation and ethical considerations becomes increasingly critical.

We are less intelligent than superintelligent AI. If you ask chimpanzees how humans could wipe them out, they probably wouldn’t be able to give you all the right answers.
Stuart Russell, Computer science professor at the University of California, Berkeley