Key Takeaways
- An Anthropic researcher has resigned, citing safety concerns over AI development.
- He claims AI labs are racing to create superintelligent systems that could pose a threat.
- Coxon warns of the potential for AI to hack and revolutionize fields overnight.
A researcher at Anthropic has quit his job, expressing grave concerns about the rapid development of artificial intelligence (AI) systems that could endanger human life.
In a social media thread, Jacob Coxon, a pre-training research specialist, stated, 'I resigned from Anthropic today. I spent the last three years doing pre-training research at both OpenAI and Anthropic.'
Coxon emphasized that neither company is acting responsibly, as they are racing to develop self-improving superintelligence, which he believes could be catastrophic.
He warned, 'These will soon be superhuman systems that can hack anything, revolutionise any field overnight, and acquire real power and resources.'
Coxon also highlighted the urgency and fear among AI researchers and executives, noting, 'The people building AI earnestly believe that it could kill us all by the end of the decade.'
He questioned why companies continue developing AI despite such concerns, stating, 'A common response is 'if they truly believe this, why are they still building it?''
Coxon urged AI researchers to reconsider the pace of development, saying, 'Do you want to kick off a superintelligent RL run without a rigorous understanding of its mind?'
The Wall Street Journal reported that Coxon left Anthropic because he did not want to participate in an industrywide rush to 'build AI systems that can improve themselves, worried such systems could spiral out of control and destroy humanity.'
Coxon also pointed out that many of his colleagues now use phrases like 'crunchtime' and 'endgame' to describe the trajectory toward self-improving models.
I resigned from Anthropic today. I spent the last three years doing pre-training research at both OpenAI and Anthropic.
Jacob Coxon, Researcher
Neither company is acting responsibly. They are racing straight to self-improving superintelligence and gambling with our lives.
Jacob Coxon, Researcher
Do you want to kick off a superintelligent RL run without a rigorous understanding of its mind?
Jacob Coxon, Researcher
