UL’s Prof Pepijn van de Ven discusses his research, which involves using simple AI models to benefit mental health interventions.

The topic of AI’s use in healthcare has been prevalent in the world of tech recently.

Last month, prominent generative AI companies OpenAI and Anthropic both launched dedicated healthcare-focused services for their respective chatbots.

While both features – ChatGPT Health and Claude for Healthcare – were developed to assist users with tasks such as understanding test results and preparing for appointments, some are looking at the potential of AI in more focused areas of the healthcare umbrella.

One such researcher is Prof Pepijn van de Ven, a professor in the Department of Electronic and Computer Engineering at University of Limerick (UL).

With a background in electronic engineering – and a PhD in artificial intelligence – van de Ven is currently the course leader of Ireland’s National Master’s in AI, delivered by UL in close collaboration with ICT Skillnet, as well as the founding director of UL’s D2iCE research centre, which conducts research into AI development and deployment with ethical, sustainable and trustworthy use of AI in society at its core.

Currently, van de Ven’s research focuses on the use of AI in mental health interventions.

“I’ve been very lucky and have had the opportunity to collaborate with some of the trailblazers in what we call internet interventions, which is any intervention delivered via the web,” he tells SiliconRepublic.com.

“In the last 15 years, I have contributed to research programmes which focused on the use of smart technologies in the delivery of mental health interventions with partners across Europe, Australia, North and South America, and of course also Ireland.”

He explains that the contributions he and his team have made to these projects revolve around using artificial intelligence to improve the delivery of said interventions.

“For example, we have shown that AI can do the time-consuming screening of patients that a clinician would otherwise have to do, thus freeing up that person for contact with patients,” he says. “Such screening interviews tend to use a battery of questionnaires that can be a real burden on patients. We do a lot of work around analysing the questionnaires typically used in mental health during screening to see if these can be shortened.”

‘We’ll need to think very carefully about the use of AI wherever we consider its use to prevent unintended consequences.’

Benefits and caution

Van de Ven considers his research important because of its potential to assist an area of healthcare that has long suffered from a lack of proper attention.

“Unfortunately, there is still a massive stigma on mental health and services tend to be under-resourced.  The well-considered use of AI has the potential to reduce thresholds to access in these services and can also make the provision of these services more efficient.

“As our population ages, the need for healthcare services, including, of course, mental healthcare services, will only increase. I think it’s a simple fact that the only way we can ensure high quality services for everybody is through the use of AI.”

One misconception he says people have about his work is the belief that “AI equates to generative technologies such as ChatGPT”.

“This misconception, given all the remarkable advances with generative AI, has led to a lot of hesitance around the use of AI,” he says. “The models that we use are really simple compared to ChatGPT.”

He explains that by using simple AI models within such a sensitive area, the risk of harm to patients is lessened – adding that he cautions against the use of generative AI and large language models to replace human staff in services such as counselling.

“We should be very careful,” he says. “I am a proponent of the careful use of AI to support healthcare providers in their roles and to allow them to spend more time with patients where possible.

“We’ve all heard the stories of people using generative models such as ChatGPT to discuss their mental health issues and really confiding in these AI models. And unfortunately, this has led to catastrophic outcomes in some cases.”

For instance, in December OpenAI was sued over claims that ChatGPT encouraged a man with mental illness to kill his mother and himself.

“As it stands, we cannot guarantee how a generative model will respond to a prompt and for this reason such use requires further research and careful testing before it can become mainstream.

“Although any AI model can cause harm just like most other technologies, the simple models we develop help with a very narrow task and often do so in a way that can be understood by a clinician,” he says. “As a result, their capability to do harm is limited and well understood.”

Personae

One project that van de Ven and his team is involved with – as the only non-Danish partner, he adds – is the Personae project, which aims to adapt a fully online mental health service already used in the Danish healthcare system to a “so-called stepped care model”, according to van de Ven.

He explains that this model presents support for patients across three different steps, or levels.

At the lowest level, patient engagement is self-directed, while the second level incorporates a blended approach where patients have access to self-directed treatment, while also being able to avail of a therapist in online sessions.

The last step or level is the “traditional approach”, he says, where patients see a therapist for every session, albeit through an online format.

“The expectation is that this stepped-care approach will result in more efficient use of healthcare resources and thus an opportunity to treat more people with the available resources,” he says. “Our role in this project is to create AI models that can predict what type of intervention a patient requires based on assessing the information people provide when they enter the service.

“Down the line, the hope is that our models can also inform what step in the stepped care model a patient should receive.”

In terms of current progress of Personae, van de Ven tells us that his project partners in Denmark have created a new intervention that is suitable for delivery on these three different levels, as well as a brand-new mobile platform to support delivery of the intervention.

“After two years of hard work, the trial was started recently and it’s going well. In the very near future we hope to receive lots of interesting data to improve the performance of our AI models further.”

Speaking of the future, what are van de Ven’s hopes for the long-term impact of his work?

“I’m hopeful that we can do right by mental health patients and their loved ones by improving the services provided to them,” he says. “Internet interventions and AI will play an important role in this process, but AI is very much a double-edged sword.

“We’ll need to think very carefully about the use of AI wherever we consider its use to prevent unintended consequences.”

Don’t miss out on the knowledge you need to succeed. Sign up for the Daily Brief, Silicon Republic’s digest of need-to-know sci-tech news.

The AgreeTo add-in for Outlook has been hijacked and turned into a phishing kit that stole more than 4,000 Microsoft account credentials.

Originally a legitimate meeting scheduling tool for Outlook users, the module was developed by an independent publisher and has been on the Microsoft Office Add-in Store since December 2022.

Office add-ins are just URLs pointing to content loaded into Microsoft products from the developer’s server. In the case of AgreeTo, the developer used a Vercel-hosted URL (outlook-one.vercel.app) but abandoned the project, despite the userbase it formed.

However, the add-in continued to be listed on Microsoft’s store, and a threat actor claimed its orphaned URL to plant a phishing kit.

According to researchers at supply-chain security company Koi say that the threat actor taking over the project deployed a fake Microsoft sign-in page, a password collection page, an exfiltration script, and a redirect.

It is worth noting that once an add-in is in the Microsoft store, there is no further verification process. When submitting a module, Microsoft reviews the manifest file and signs it for approval.

AgreeTo had already been reviewed and approved, and loaded all the resources – user interface and everything the user interacts with, from the developer’s server, now under the control of the threat actor.

Koi researchers discovered the compromise and accessed the attacker’s exfiltration channel. They found that over 4,000 Microsoft account credentials had been stolen, along with credit card numbers and banking security answers.

The add-in was present in the store until today, when Microsoft removed it. Koi researchers say that the threat actor was actively testing stolen credentials during their examination.

When users opened the  malicious AgreeTo add-in in Outlook, instead of the scheduling interface, they would see a fake Microsoft login page in the program’s sidebar, which can easily be mistaken for a legitimate login prompt.

Any account credentials entered there are exfiltrated via a Telegram bot API to the attackers, while victims are then redirected to the real Microsoft login page to reduce suspicion.

It is noted that the add-in retained ReadWriteItem permissions, enabling it to read and modify user emails, though no such activity was confirmed.

Koi Security found that the operator behind this attack runs at least a dozen additional phishing kits targeting internet service providers, banks, and webmail providers.

While malicious add-ins aren’t new, we have previously seen such tools promoted via spam forum comments, phishing emails, and malvertising. The case of AgreeTo stands out, though, as it is likely the first to be hosted on Microsoft’s Marketplace.

Koi Security researcher Oren Yomtov told BleepingComputer that this is the first malware found on the official Microsoft Marketplace and the first malicious Outlook add-in detected in the wild.

If you still have AgreeTo installed on Outlook, you are recommended to remove it immediately and reset your passwords. BleepingComputer has contacted Microsoft for a comment on Koi researchers’ findings, but we are still waiting for a response.

Modern IT infrastructure moves faster than manual workflows can handle.

In this new Tines guide, learn how your team can reduce hidden manual delays, improve reliability through automated response, and build and scale intelligent workflows on top of tools you already use.

Get the guide

OpenAI has started testing ads in ChatGPT for US users across both the Free and Go subscription tiers. However, Plus, Pro, Business, Enterprise, and Education tiers remain ad‑free.

Needless to say, it has rattled quite a number of people using the service.

To understand the change, consider OpenAI’s mission: ChatGPT serves hundreds of millions of users for learning, work, and daily decisions. Running Free and Go tiers requires heavy infrastructure and investment. Ads help fund that work, supporting broader access to AI features at lower cost.

However, OpenAI is keen to point out that ads do not influence answers. Responses remain independent and optimised for user needs, with sponsored content appearing clearly labelled and visually separated from organic answers. This design aims to ensure transparency and preserve trust in ChatGPT’s output.