The House Article | Anonymous bots are stealing from British businesses

3 min read10 min

No contracts are signed. The warehouse owner gets nothing. Multiply that by every business in the country, and you have a picture of what is happening to almost every major website in the UK, where bots harvest content on an industrial scale.

Many gather data to fuel AI models, and others extract valuable information from online businesses. Bots now account for more than half of all internet traffic, yet website owners cannot identify or verify 99 per cent of the bots attempting to access their site. There is no information on who is operating the bot – whether it be an AI firm, a search engine, or someone who just wants to sell the data on – or what they’ll do with the content.

Stealth bot operators have created a black market for data already estimated at over $1bn globally, growing fast as AI expands. British businesses produce some of the world’s most valuable digital content and data and so are squarely in the crosshairs.

The result is a wild west, where value is extracted with no accountability

Advertisement

Many bots are run by third-party companies set up specifically for this purpose. Online retailers are a prime target, with bots stripping out pricing and product data so relentlessly that they crash sites and lock out real customers. In the travel sector, bots add flights to their cart without buying them, blocking real customers from booking. 

Other bots copy trusted journalism, feeding it straight into AI assistants the moment a user asks a question, so the user never visits the publisher’s site. If there’s no return on investment in trusted journalism, the stories AI can’t function without simply won’t get written.

Website owners often can’t see any of this. Bots disguise themselves, meaning attempts to block them are routinely circumvented. The result is a wild west, where value is extracted with no accountability.

A website owner should have the same basic ability to control who accesses their property as any high street retailer does. That is why I am introducing a Private Member’s Bill with a simple principle: if you run a bot that accesses and copies content from a website, you must tell the owner who you are and what you intend to do with it.

The principle at the heart of this is about being able to enforce property rights – a principle that underpins the efficient operation of market economies.

Once bots must identify themselves, an owner can see who is knocking on their door and decide whether to let them in, block them, or strike a deal on agreed terms, giving them real leverage to negotiate the value of what they’ve built.

This proposal is not anti-tech. It places no requirements on AI models. It doesn’t even tell the bots what to do. Many major search and AI firms already identify their bots responsibly. Honest operators stand to benefit as greater transparency separates them from the stealth operators. Right now, many website owners are pushed towards blocking all non-human traffic indiscriminately; my Bill flips that incentive, making transparency the best route to access.

There is an increasing awareness in other jurisdictions of the threat to publishers from anonymous bots. In the United States, members of the House of Representatives have introduced the Stealth Bot Prohibition Act, which would require bots to disclose their identity and purpose.

This is a complex problem with a simple solution, and I hope the government will support this proposal for a fairer, richer digital economy in the UK. The longer we wait, the more entrenched this wild west becomes, and the harder it will be to put things right.

Damian Hinds is Conservative MP for East Hampshire