Women paying the price for algorithms that ‘normalise violence and abuse for profit’ | News UK

Social media algorithms are ‘normalising’ violence against women and girls, with one in three subjected to ‘shocking levels of abuse’ every day.

A new report from Refuge, the UK’s largest domestic abuse charity, and the End Violence Against Women Coalition reveals 36% of women experienced some form of violence online.

That number rises to 47% for women aged between 16 and 24.

A fifth of those said they encountered the same misogynistic content pumped into the manosphere ‘to maximise ad revenue’ most of or every time they logged on.

The online harms are having significant real-world consequences, with victims suffering anxiety or panic attacks and being driven from the platforms completely.

Most worryingly, 7% of women who experienced online violence said they felt suicidal – a figure which more than doubles (15%) among those aged 16 to 24.

Refuge CEO Gemma Sherrington said their findings ‘are reflective of a system that has continued to prioritise profits over safety, leaving women to pay the price’.

Jodie’s story

In January 2021, Jodie* opened an anonymous email directing her to images of herself which had been shared online, along with details identifying her and where she lived.

Weeks later she discovered explicit ‘deepfakes’ of her on an online forum.

Her ‘shock and desperation’ turned to ‘complete fear, anger and disbelief’ when she found out the perpetrator was her male best friend.

This Is Not Right

On November 25, 2024 Metro launched This Is Not Right, a campaign to address the relentless epidemic of violence against women.

With the help of our partners at Women’s Aid, This Is Not Right aims to shine a light on the sheer scale of this national emergency.

Advertisement

You can find more articles here, and if you want to share your story with us, you can send us an email at vaw@metro.co.uk.

Read more:

Advertisement

Among the women polled by Refuge and EVAW, one in 50 reported they had personally experienced an AI-generated or ‘deepfake’ intimate image of them being shared without their consent.

Nearly three quarters (72%) identified the perpetrator as male, with current or former partners and friends or relatives each responsible in roughly a fifth of cases.

The most commonly identified perpetrator (53%) was a stranger.

When Jodie went to the police to report the abuse, she was told there was ‘nothing they could do’.

She had to track down the images and videos herself, contacting the online forum and porn site directly to get the material taken down. 

‘Gathering the evidence was incredibly distressing and retraumatising. It felt as though the responsibility for proving the harm had been placed entirely on me,’ she told Metro.

After going through that process and collecting the evidence, Jodie was told the creation and sharing of sexually explicit deepfakes had not broken any laws, so her abuser could not be prosecuted. 

‘Despite the threats and risks we were facing, we did not have a liaison officer, and we were told that we were unable to access an Independent Sexual Violence Adviser because the abuse was not considered sexually motivated,’ Jodie said.

‘Survivors should not have to repeatedly explain, document and prove their abuse while navigating systems that do not understand the nature of online harms.‘Evidence gathering and reporting need to be trauma-informed, coordinated and designed around the safety of the person experiencing the abuse.

‘The burden should not fall on survivors to become experts in criminal investigations, technology and platform reporting while they are trying to recover.’

Jodie successfully campaigned with EVAW for the creation of non-consensual sexually explicit deepfakes to become a crime, which came into effect last year.
 
Despite this, the report shows that 66% of women surveyed continue to feel concerned about the role of AI technologies in enabling violence against women and girls.

Jodie said: ‘The risks extend beyond deepfake images. We are also seeing concerns around AI chatbots providing detailed guidance on stalking, simulating rape or abuse through roleplay, and reinforcing misogynistic ideas.

‘These are serious risks from systems that have been deployed without sufficient safeguards, oversight or meaningful consideration of the people who may be harmed.

‘There are also emerging risks from technologies such as smart glasses and other wearable devices, which could make covert image capture, surveillance and violations of consent easier.’

The report also shows that chillingly, 1 in 7 women who have experienced digital VAWG have left the platform altogether, as a direct result of the abuse.

Almost a quarter reduced commenting or posting on social media following violence.

‘My relationship with social media changed significantly,’ says Jodie. ‘I became much more conscious of what I shared. Something that had previously felt ordinary, such as posting a photo with work colleagues, or in my home, began to feel like a potential safety risk. 

‘I have had to think about how visible I want to be and whether participating online is worth the potential risk. That is an exhausting position to be put in.
‘Women and girls should not have to make themselves smaller or disappear to stay safe.’

Threats of sexual violence

Anybody can be a victim – young girls and older women, social media users with relatively few contacts and celebrities with followers numbering the thousands or millions.

Ashley James, the DJ, broadcaster and bestselling author of Bimbo, told Metro that online misogyny and abuse have followed her ever since she entered the public eye and gained a following in 2012.

‘At that time, it was horrifically graphic sexual videos on Snapchat. Since them, it’s always [been] men reaching out with sexualised insults, comments about my appearance, threats and abuse,’ she said.

Recently, the threats have escalated to rape and are now being directed at her three-year-old daughter through DMs online.

‘I felt sick,’ she said of seeing those messages coming through.

‘When it comes to my children, I would die to protect them. Everything I speak about is for them, to hopefully make the world a better place.’

This harrowing message was not a one off. Since, Ashley has received multiple similar threats from different men. 

‘It’s devastation and guilt. Any kind of threat to them makes you feel like a failure as a parent,’ she said.

‘I frequently receive rape threats myself. The threat of sexual violence to myself and my daughter is 100% gendered.

‘I think often my content ends up on the wrong side of the algorithm, being pushed towards [people who support] the manosphere.’

Ashley says that despite all of this, she refuses to let ‘trolls’ drive her away from social media.

‘We shouldn’t have to choose between having a voice and being safe. No one should be bullied into silence,’ she said.

‘Misogyny didn’t start with social media, I just think social media has given it an algorithm and an audience, and kind of made it a business model. 

‘I’ll continue using my platform, because I don’t want my daughter to think that being a woman online means accepting sexual harassment, misogyny or graphic imagery as the price of participation.’

Real-life impact

Jodie said ‘the line between online and offline harm is continuing to collapse’, with the impact stretching ‘far beyond the internet’.

‘It affected my mental health, my sense of safety and my ability to trust people,’ she said.

‘I experienced PTSD symptoms linked to a previous assault, as well as depression, and needed weekly private therapy for months.

‘The abuse became something I had to carry into every part of my life, including my relationships, work and everyday interactions.’

The Dangerous by Design report shows that 40% of those who have experienced online VAWG experienced anxiety, while 15% of those between 16 and 24 reported feeling suicidal as a result. 

Dr Sophie Nightingale specialises in researching the psychological impacts of online VAWG, particularly image-based abuse.

‘We’re seeing this overlap between the online, and in-person physical threats of sexual violence. Whether these are threats people are going to carry out or not, this still impacts the victim,’ she said.

‘People will respond by withdrawing, feeling afraid and petrified to go out and leave their house.

‘It’s another way to oppress women, and keep them isolated.’

The psychologist explains that the risk of online VAWG leading to suicidal thoughts is particularly intensified by threats of sexual violence.

‘There can be these feelings of embarrassment, shame and guilt that should never actually be attributed to the victim.

‘You’re already scared because of the abuse itself, but then you also feel isolated, not knowing where to turn. 

‘Sometimes, people think that the only option is to end things, which can be heightened by the thought that because of the online element, the abuse will never go away. 

‘This is often the case with non-consensual sharing of intimate imagery, as it carries a digital footprint that people live in fear of. They question ‘When is it going to emerge again?”’

Dr Nightingale says growing threats to women online are part of a wider cultural regression.

‘At the moment, we are seeing the growth of misogynistic views and social, cultural norms,’ she said.

‘From a young age, children are growing up thinking this is normal behaviour, that it’s okay to do these things, and share an intimate image of a woman or girl without her consent, because that’s what people do and they get away with it.

‘If we don’t do something drastic soon to stop that, it’s going to snowball even further, and we’re going to take a huge step backwards in terms of women and girls’ rights.’

What needs to be done

The report shines light on how women are dealing with this widespread issue alone. 

One in five did not tell anyone about the abuse. Less than a third reported it to the platform where it occurred, meanwhile only 12% reported it to the police.

Over half of women who did report abuse to the platform itself were not satisfied by the handling of their report, demonstrating the urgency for tech companies to change their designs and approach.

Ms Sherrington said: ‘This report comes almost one year after Ofcom introduced its guidance for tech companies on how to tackle VAWG on online platforms.

‘Despite being a welcome step forward, Refuge has long-been clear that voluntary guidance lacks the force needed to address the shocking levels of abuse and violence faced by women and girls online every day.

‘Our report testifies to this, shining a light on the sheer scale of harm that women in this country are experiencing and the lack of confidence they have in tech companies, the criminal justice system, and policymakers to protect them.

‘These findings are reflective of a system that has continued to prioritise profits over safety, leaving women to pay the price. This cannot continue.

‘The government and Ofcom must go much further in holding tech companies legally responsible for combatting the normalisation of misogyny, holding perpetrators to account, and keeping all women and survivors safe.’

The report contains several recommendations, calling for the Online Safety Act to be updated, allowing Ofcom’s VAWG guidance to be upgraded to a legally enforceable code of practice. 

The report also demands safety by design, through the introduction of a Safety by Design code of practice.

EVAW’s interim director, Janaya Walker, said: ‘All women and girls deserve to live free from violence and abuse.

‘This report is a stark reminder of the ways in which online VAWG is a major feature of women’s everyday lives to devastating effect, particularly for young women.

‘The status quo is untenable and the government’s efforts to prevent VAWG needs to confront the business model of big tech, with a regulatory system which has the teeth to hold them to account.’

Earlier this year, Labour asked Ofcom to report on which social media companies are failing to implement measures to protect women and girls online.
Then-technology secretary Liz Kendall said companies ‘must go above and beyond’ to make their platforms safer.

She warned that the Government would be prepared to intervene if firms failed to use every tool at their disposal to protect women and girls from abuse and misogyny online.

‘This government has taken tough action to tackle intimate image abuse, deepfakes and the online harms women and girls face every day.

‘Now, tech companies must go above and beyond to use the tools readily available to them to make their platforms safer.’

As part of the Government’s VAWG online crackdown, cyberflashing was made a priority offence under the Online Safety Act earlier this year.

The change means that rather than reacting after such images have been received, platforms must take proactive steps to prevent such content appearing in the first place.

The Government also vowed to close legal loopholes which have allowed chatbots to create deepfake nude images and is planning further curbs on social media.

Get in touch with our news team by emailing us at webnews@metro.co.uk.

For more stories like this, check our news page.

MORE: OpenAI agent manages to hack into Australia’s healthcare portal

MORE: Prince Harry eerily cut off mid-speech on dangers of AI

MORE: Donald Trump wants to rename AI ‘super intelligence’ because ‘artificial makes it sound fake’