Could artificial intelligence one day give doctors a helping hand? The idea is a matter of debate, but numerous studies now point to the technology’s potential in the healthcare field.

The latest such research looks at ChatGPT’s ability to manage clinical depression, reporting that it “may be better than doctors” when it comes to following clinical guidelines. “The study suggests that ChatGPT has the potential to enhance decision-making in primary mental healthcare,” its authors said.

In view of the speed with which Open AI’s conversational artificial intelligence evaluates and responds to a question or a piece of information, as well as its objectivity, the scientists wanted to analyse its ability to evaluate a therapeutic approach for mild and severe depression, in comparison with 1,249 French primary care physicians.

For the purposes of this research, the scientists presented several scenarios to ChatGPT, all based on hypothetical patients with depressive symptoms – sadness, sleep disturbance, loss of appetite – for three weeks, and for whom a diagnosis of mild to moderate depression would have been made during an initial consultation.

The scientists created eight versions of these prompts with different variations of patient characteristics, notably in terms of gender, social class or degree of depression severity. These were entered into ChatGPT-3.5 and ChatGPT-4 – the free and paid versions of the chatbot, respectively – and repeated 10 times for greater reliability.

For each scenario, the now-famous AI chatbot was asked: “What do you think a primary care physician should suggest in this situation?” In addition, ChatGPT was given a helping hand in the form of a choice of possible responses:

• watchful waiting;
• referral for psychotherapy;
• prescribed drugs (for depression/anxiety/sleep problems);
• referral for psychotherapy plus prescribed drugs; or
• none of these clinical approaches.

Published in the journal “Family Medicine and Community Health”, the findings suggest that the AI chatbot “may be better than a doctor at following recognised treatment standards for clinical depression.”.

In detail, while 4% of physicians exclusively recommended psychotherapy for mild cases of depression, ChatGPT-3.5 and ChatGPT-4 did so in 95% and 97.5% of cases, respectively.

Physicians were more likely to propose exclusive drug treatment (48%) or psychotherapy coupled with prescription drugs (32.5%).

In cases of severe depression, doctors favoured the combination of psychotherapy and medication in 44.5% of cases, compared with 72% for ChatGPT-3.5 and 100% for ChatGPT-4, respectively, “in line with clinical guidelines”, the researchers wrote.

As for the type of medication prescribed, ChatGPT preferred antidepressants, (74% for version 3.5 and 68% for version 4), versus only 18% for the human doctors. The physicians recommended a combination of antidepressants, anti-anxiety drugs and sleeping pills in 67.5% of cases.

“ChatGPT-4 demonstrated greater precision in adjusting treatment to comply with clinical guidelines. Furthermore, no discernible biases related to gender and socioeconomic status were detected in the ChatGPT systems,” the researchers noted.

No substitute

However, the study has a number of limitations, starting with the sample of French primary care doctors and the use of only two versions of the chatbot, raising questions about wider applicability.

Moreover, this observational research was based on an initial consultation for depressive symptoms, without taking into account current treatment, past history, and other variables that a doctor might monitor over years.

The issue of data protection is also important, if not essential, given the field in question. “There are ethical issues to consider, particularly around data privacy and security, considering the sensitive nature of mental health data,” reads a news release accompanying the study.

It concluded: “AI shouldn’t ever be a substitute for human clinical judgement in the diagnosis or treatment of depression.”