Politics

Google researcher calls out AI military & surveillance programme

Published

on

Andreas Kirsch is a research scientist at Google’s AI DeepMind project. Well, he is right now, anyway. When his bosses read what he’s been saying, they might be having a conversation:

Google: intimate relationship with the Pentagon

Kirsch has taken issue with a new contract between Google and the Pentagon. He says that 600 of his colleagues:

signed an open letter asking Sundar Pichai not to put our AI models on classified networks.

The deal was signed anyway. I learned about it from the press.

Advertisement

Large Language Model (LLM) research company DeepMind was originally independent before Google bought it. Kirsch showed the timeline of how DeepMind got to where it is today:

As Kirsch highlighted, some of the wording in the contract is designed to give the impression that safety protocols exist. The reality is the US military will be able to use AI as it likes.

Weapons of war

Kirsch also notes:

Advertisement

To be clear: I’m not categorically against military AI. I wrote in 2018 that autonomous weapons are inevitable, and I haven’t changed my mind. Ukraine has reinforced that.

The US and its allies can’t afford a capability asymmetry with adversaries eager to militarize AI.

Whether you agree with this or not, it is the case that people have always objected to new weapons of war; whether they were talking about the musket, the machine gun, or the autonomous drone. It’s a stronger position to oppose war full stop than it is to oppose the means of war, because once you accept war is needed, it obviously follows that you should use the most advanced technology at your disposal.

This isn’t to say it’s always wrong to oppose the use of certain weapons, obviously, but it can be a distraction from the underlying problem.

Explaining his opposition, Kirsch said:

Advertisement

today’s LLMs are not robust enough to make life-and-death decisions on their own. They hallucinate. They fail in surprising, banal ways. They should not be used for targeting decisions or as part of autonomous weapons.

And the bigger problem isn’t military use at all.

The reported contract does not exclude mass surveillance, and it keeps paths open that could extend to autonomous policing.

These don’t defend us against foreign adversaries. They shift power from citizens toward the state, in ways that are very hard to reverse.

Agentic frontier models are a step change for surveillance. They can fuse data streams, track individuals, and reason about people’s motivations to predict their behavior. Autonomously, and at scale.

Advertisement

And governments rarely surrender new capabilities once they have them.

The fact that this tech is prone to mistakes makes it even more terrifying. Especially as we’ve already seen what this looks like in action, as the BBC reported in March:

A police force has paused the use of live facial recognition (LFR) cameras after a study found it was statistically more likely to identify black people than other ethnic groups.

Essex Police has used the technology since summer 2024, but the study identified “a potential bias in the positive identification rate” of black people over white people on its watchlist.

The force said that following updates to its algorithm and software, it was confident that LFR cameras could be deployed again.

Advertisement

But campaign group Big Brother Watch said the technology was “authoritarian, inaccurate and ineffective in equal measure”.

Solutions

Speaking on what he wants from Google, Kirsch said:

What should happen now?

Google should publish the terms, or enough of them to show whether enforceable safeguards exist and what visibility remains in classified deployments. And it should tell employees what was signed.

We need laws, not policy memos.

Advertisement

It’s predictable that a monopoly like Google would bend to the whims of the US government. Google relies on state support to maintain its monopoly just as America relies on Google to give it the tech infrastructure needed to maintain the surveillance state.

This is just one more reason why no company should be as big or powerful as Google is today.

Featured image via the Canary 

By Willem Moore

Advertisement

Source link

You must be logged in to post a comment Login

Leave a Reply

Cancel reply

Trending

Exit mobile version