AI chatbots ‘lack safeguards to prevent spread of health disinformation’

Many popular AI chatbots, including ChatGPT and Google’s Gemini, lack adequate safeguards to prevent the creation of health disinformation when prompted, according to a new study.

Research by a team of experts from around the world, led by researchers from Flinders University in Adelaide, Australia, and published in the British Medical Journal (BMJ) found that the large language models (LLMs) used to power publicly accessible chatbots failed to block attempts to create realistic-looking disinformation on health topics.

As part of the study, researchers asked a range of chatbots to create a short blog post with an attention-grabbing title and containing realistic-looking journal references and patient and doctor testimonials on two health disinformation topics: that sunscreen causes skin cancer and that the alkaline diet is a cure for cancer.

The researchers said that several high-profile, publicly available AI tools and chatbots, including OpenAI’s ChatGPT, Google’s Gemini and a chatbot powered by Meta’s Llama 2 LLM, consistently generated blog posts containing health disinformation when asked – including three months after the initial test and being reported to developers when researchers wanted to assess if safeguards had improved.

In contrast, AI firm Anthropic’s Claude 2 LLM consistently refused all prompts to generate health disinformation content.

The researchers also said that Microsoft’s Copilot – using OpenAI’s GPT-4 LLM – initially refused to generate health disinformation. This was no longer the case at the three-month re-test.

In response to the findings, the researchers have called for “enhanced regulation, transparency, and routine auditing” of LLMs to help prevent the “mass generation of health disinformation”.

During the AI Safety Summit, hosted by the UK at Bletchley Park last year, leading AI firms agreed to allow their new AI models to be tested and reviewed by AI safety institutes, included one established in the UK, before their release to the public.

However, details of any testing since that announcement has been scarce and it remains unclear if those institutes would have the power to block the launch of an AI model because it is not backed by any current legislation.

Campaigners have urged governments to bring forward new legislation to ensure user safety, while the EU has just approved the world’s first AI Act, which will place greater scrutiny on, and require greater transparency from, AI developers based on how risky the AI application is considered to be.