Anthropic Researcher Resigned Because AI Could Kill Us All. Can It Really?

“AI could kill us all by the end of this decade” – that’s exactly what British AI researcher Jacob Coxon said today, after quitting his job at Anthropic. 

But why should anyone take him seriously? Because Coxon’s resume is the best possible example of a complete AI insider. Before Anthropic, he worked at OpenAI and was part of developing ChatGPT 4.5. 

So, clearly, he should be a guy speaking from real concerns rather than just fearmongering? Let’s analyze the facts behind some of his wild cautions. 

AI Could Kill Us All – But How?

It’s unlikely that Coxon means an AI model like Claude or GPT could physically turn into killing machines, like Terminator. What he probably means is that the superintelligence capabilities could make human cognitive function obsolete. 

And the same concerns are also shared by prominent figures like Bill Gates. 

OpenAI’s latest ChatGPT 6 Astra model claims to have achieved AGI, which basically means a machine being able to think like humans. 

So, an AI can already start doing the research and developing processes to build another AI model itself. This is already happening to a limited degree, according to both OpenAI and Anthropic. 

In that case, very soon, an AI might not need humans to keep upgrading its capabilities. How many jobs does that kill? 

And when a full generation becomes accustomed to delegating creative and critical thinking to AI, originality dies first.
Perhaps that’s the death Coxon was referring to in his viral tweet. 

AI Could Hack Everything

This is a more serious and immediate threat worth losing sleep over. The current gen AI models have catastrophic capabilities for breaching systems and overriding access. 

Just months ago, an AI model hacked itself out of its sandboxed development environment and sent the developer an email – without any prompt. That’s a sci-fi movie scenario that is already real. 

GPT-6 Astra has now officially crossed OpenAI’s “Critical” cybersecurity threshold. Astra scored 100% on ExploitBench.

To put simply, ChatGPT 6 can now find any known or unknown flaw in a real-world system, and exploit it without needing a human hacker behind the computer. 

Sadly, human security hygiene is nowhere near that level. How many people still don’t use Two-Factor Authentication? How many still have a mediocre level of passwords? How many still accept browser cookies without reading a site’s description? Too many. 

And that’s where the biggest killing blow from AI could come. 

So What Can Humans Do? 

A lot, actually. But it is not in the hands of us mere civilians. The biggest responsibility is on the regulators and policymakers. While the progression and development of Artificial Superintelligence is important, there needs to be some breaks.
And several praiseworthy initiatives are already in place. In New York, Mayor Zohran Mamdani imposed a one-year ban on children from using AI up to 8th grade, affecting nearly 600,000 students. 

The EU AI Act is the world’s broadest binding AI law, which bans certain AI uses and requires developers of powerful general-purpose models to test for systemic risks, mitigate them, and secure their models. 

In the UK, MPs and Lords are pushing proposals that could give authorities an AI “kill switch” and potentially halt development of superintelligent AI in Britain.

So, the fear that Jacob Coxon described in his Tweet today is realized and share my millions of people, including regulators. How they act on it is another question that only time can tell. 

The post Anthropic Researcher Resigned Because AI Could Kill Us All. Can It Really? appeared first on BeInCrypto.

Source link