An Experiment to Improve Medical Consultation Education with Chatbots
Outsourcing my education to AI
The highlights:
Discover how AI could support medical training, enhancing doctor-patient interactions.
I developed chatbots that act as virtual patients, providing a practice environment for students.
These simulated interactions offer valuable realism and control to students, refining their clinical questioning methods.
Beyond education, these chatbots can also provide insightful feedback on consultation skills, elevating the learning experience.
Read time: 6 minutes.
Welcome to a new edition of Transforming Med.Tech, the newsletter in which I share my exploration of technical innovations that may change the world of medicine. Today’s topic: medical consultation education.
Medical consultation is essential to provide the best care to the patient, but also pivotal to the patient-doctor relationship. Wouldn't you prefer a doctor who digs deep, beyond your visible signs and complaints, to unearth your genuine concerns and expectations? For a patient it is just as important, if not more, to have an effective and satisfactory interaction with their doctor as it is for the medical process to run smoothly.
Although I do not practice medicine anymore, I found patient interactions one of the most satisfying aspects of medicine. That is why, next to my scientific work, I still love teaching medical consultation. The warmth of these interactions can bring comfort to a patient's deepest needs. I was curious to see if the recent developments in 'soft' AI (like natural language processing and chatbots) could contribute to medical education and help prepare future physicians for these patient interactions.
AI cannot replace me yet
At the university where I work, students practice medical consultations with trained volunteers who act as patients. Peer evaluations form the backbone of our training process where students and teachers review and provide feedback on video recordings of these interactions.
My first try to inject AI into this process was not very successful. I wanted to see if I could use a chatbot to provide feedback on one of such doctor-patient interactions. These bots pass medical exams, but can they also provide useful feedback on such soft skills? I used the Whisper neural network (which runs locally, yay for privacy!) to transcribe the video recording of a doctor-patient interaction. Unfortunately, the AI struggled with speech recognition and separating the two speakers in the recording, resulting in a transcript that I found unreadable. Here, AI is not replacing me yet…
Using AI to stop time
This failure prompted me to look for a more controlled setting to capture interactions between medical students and training patients. I decided to let go of the natural training environment with a live conversation and focus only on particular skills. One challenge for starting students is that - when they’re on the spot - they do not know what to ask, let alone how to ask it. Under the time pressure of a real conversation, they are distracted by what they should be asking next and forget to listen to the patient. In the beginning, these conversations easily hit a dead end and students often overlook the true needs of a patient.
So why not design a setting where the student determines the pace of the interaction? This is where chatbots step in. Their patience and programmability provide students with a unique opportunity to practice the flow of the conversation. The asynchronous interaction gives the student time to think about a logical next question and its best formulation.
A virtual patient to chat with
Using OpenAI's tools, I created a chatbot with the persona of a patient. I gave "it" a background, needs, and expectations. I also had to give it some additional instructions, for example: don't tell everything right away - the default chatbot is way too chatty! The student then can simply ask questions as if they were a doctor and see if they can discover what the actual needs of this virtual patient are. And it works surprisingly well! I tried different approaches, both good and bad, and noticed how the chatbot would respond differently too. I had all the time to come up with new questions - and if I was not satisfied, I could simply start over again.
I also asked a few real doctors to have a go. A friend who is a general practitioner sat down to show me how it’s done. She thought the interaction was realistic and fun, and could be very helpful to give students a first try at such conversations.
The use of chatbots comes with a host of benefits. They provide a no-stress practice environment with unlimited tries. They allow flexibility to tweak the patient's case and personality, dramatically diversifying the training options. While these bots can’t replace real-life training, they can simulate a variety of scenarios that can be a valuable supplement to traditional training methods and are much more scaleable. I think the biggest benefit may be to let students try out different questions and formulations in a variety of conditions, which helps them to build a repertoire of trusted techniques that they can fall back on when needed.
Not just chatting - but feedback too
And it doesn't end here: this chat-based interaction means we automatically obtain a high-quality transcript of the consultation. I created another chatbot, which I asked to act as a trainer and provide feedback on consultation skills. It provides remarkably useful feedback with clear examples.
However, at times the feedback-bot seems overly critical or fails to recognize that this was a training setting. For example, the training now focuses on the first phase of the interaction, in which (at least in our university’s framework) the doctor covers the patient's needs more than their precise complaints. The chatbot, however, grumbles that the doctor did not ask detailed questions about those complaints. The bot probably needs more finetuning to really incorporate preferred interaction styles. Rightfully so, my GP friend complained that the feedback-bot was a bit harsh on her...
Future AImplications
Can I outsource my education on medical consultation to AI? No - and I don’t want to, either. But I was surprised by the fun and usefulness of talking to such relatively simple chatbots, complementing an existing curriculum. (Next step: create a virtual reality training, wear immersive VR goggles, and incorporate nonverbal communication too?)
For now, feel free to try this out for yourself: I've created two virtual patient bots and a feedback bot that you can interact with. Challenge your consultation skills and get some feedback! I’ve developed English and Dutch versions.
These experiences have shown me the potential of AI for improving medical education. The ease with which I could develop this educational tool underscores the need for us to integrate such advancements into our curriculum. Moreover, the versatility of these chatbots indicates their potential application in real patient interactions, beyond education. They could assist in many aspects of healthcare:
mental health (do your mindfulness sessions with a bot or interact with your digital twin),
patient education (like I used a chatbot to explain a scientific model to me, it could explain disease or therapeutic procedures to a patient),
appointment planning (stay tuned for the next newsletter!),
and so much more.
The journey of exploring and integrating ‘soft’ AI into a traditionally intimate space has been eye-opening. Let's continue the conversation—how do you envision the role of AI in healthcare? Looking forward to your comments and replies!