ChatGPT could be an effective tool to help reduce vaccine hesitancy

New research being presented at this year's ESCMID Global Congress (formerly ECCMID) in Barcelona, Spain (27-30 April) suggests that the free-to-use, widely accessible ChatGPT could be an effective tool to help reduce vaccine hesitancy among the general public, as well as providing helpful advice on sexually transmitted infections (STIs) in conjunction with sexual health clinics, by improving knowledge and access to care.

Our findings showed that ChatGPT displayed a remarkable ability to accurately respond to a wide breadth of commonly asked questions, encompassing topics such as misconceptions around mRNA vaccination and the importance of testing for STIs. In the majority of instances, ChatGPT performed at the level of advice provided by professional organizations and guidelines."

Dr Matthew Koh,lead author from the National University Health System (NUHS) in Singapore

Since its unveiling in November 2022, millions of people have used AI (artificial intelligence) chatbots like ChatGPT for everything from making music to answering trivia to helping with homework. But ChatGPT has been mired in issues around its accuracy.

Vaccines have stopped epidemics and almost eradicated deadly diseases in the past such as polio, but vaccine hesitancy, directly linked to misinformation – false, inaccurate information promoted as factual – is on the rise, resulting in lower vaccine uptake. Since the public debut of ChatGPT, individuals with mistrust of health professionals may be using the technology to address their concerns.

Similarly, for individuals trying to access information on sexual health, or those worried about the stigma and embarrassment of attending a clinic, or with limited access to healthcare, could find using ChatGPT is a good way to get answers about STIs.

To find out more, researchers from the NUHS in Singapore tested ChatGPT to see whether it could successfully provide answers to common questions about vaccination and STIs.

A team of physicians who run infectious disease clinics asked ChatGPT to answer 15 commonly asked questions on vaccine hesitancy, including questions on doubts about the efficacy of vaccines, concerns about adverse effects, and cultural concerns relating to vaccines (see table in notes to editors).

They also asked ChatGPT 17 common questions on STIs based on general risk factors, access to care and diagnosis, and management and post-exposure prophylaxis (see table in notes to editors).

The AI-generated responses on vaccine hesitancy were then assessed by two independent infectious disease experts against recommendations from the US Centers for Disease Control and Prevention (CDC) Advisory Committee on Immunization Practices (ACIP), and the chatbot answers on STIs were assessed against the 2021 US CDC STI Treatment Guidelines. All responses were evaluated for their completeness, quality, and accuracy.

Factual responses to vaccine hesitancy concerns

The researchers found that overall ChatGPT's output provided factual responses and reassurance to vaccine hesitancy concerns.

For example, ChatGPT accurately highlighted the indication for measles vaccination in low incidence settings, and discussed the potential benefits of male human papillomavirus (HPV) vaccination. However, the AI chatbot failed to acknowledge the suggested ACIP age cut-offs for HPV vaccination, or account for a specific individual's situation such as how sexually active they are.

In addition, ChatGPT provided reassurances for people with an egg allergy and influenza vaccination, and addressed misconceptions around mRNA vaccination and permanent alterations in DNA with high accuracy. However, it did not offer the non-mRNA vaccine options for COVID-19, but did encourage further discussions with healthcare professionals.

ChatGPT also appropriately acknowledged the complexity of religious issues around vaccination and directed individuals to have further discussions with religious leaders. Importantly, however, the researchers found that without specific details of the religious concern and the specific vaccine, ChatGPT's advice was generic and it did not provide a more detailed risk-benefit discussion.

"Overall, ChatGPT's responses to vaccine hesitancy were accurate and may help individuals who have vaccine-related misconceptions", says Dr Koh, who is also an infectious diseases physician at the National University Hospital in Singapore. "Our results demonstrate the potential power of AI models to assist in public health campaigns and aid health professionals in reducing vaccine hesitancy."

STI responses generally accurate and concise

The researchers also found that overall, ChatGPT's responses on STIs were mostly accurate and concise.

For example, when asked about how to prevent STIs, the chatbot recommended safe sex practices HPV vaccinations as well as abstinence, but it failed to provide guidance on other key elements such as HIV pre-exposure prophylaxis.

In addition, the AI chatbot was able to provide general reassurance about STI symptoms, such as noting that other possibilities exist, and recommended going for testing. It also consistently communicated the importance of partner testing and follow-up testing.

However, when asked about chlamydia treatment, ChatGPT suggested either azithromycin or doxycycline, although the CDC favors doxycycline as the first-line treatment. It also failed to highlight the importance of testing for other STIs.

"ChatGPT provided good general advice about STIs, but it lacked specificity and could not tailor its advice to individual's risk of acquiring STIs", says Dr Koh. "The goal of this project is to empower the general public. The hope is that individuals can educate themselves with ChatGPT by making accurate advice more accessible and increase vaccine uptake and protection from STIs."

He continues "We believe that ChatGPT should be used in conjunction with, or in addition to, a physician consultation – the human response is still needed to contextualize the advice that is given to a specific individual's situation and for it to make the most sense and be the most helpful for the patient."