Educational Scholarship

Primary Investigator: Tabor Flickinger, MD, MPH
Supported by the Anne L. Brodie Humanist Medical Education Award

This project aims to advance the training of clinicians equipped to deliver patient-centered, compassionate care. We are developing assessment and feedback methods for communication skills, grounded in the conceptual framework of Patient-Centered Communication (PCC), which includes understanding patient perspectives, responding to emotion, and expressing empathy. Our interdisciplinary team draws from the Schools of Medicine, Nursing, and Engineering with the common goal of improving patient-clinician communication. Rigorous, reliable methods are needed to assess communication skills, but strategies reliant on human coders can be too resource-intensive for real-world use. We will use innovative Large Language Models (LLMs) to create scalable strategies for timely feedback generation. Our LLM will be validated by human input and developed with the intent to support patient-clinician relationships. We will seek student, instructor, and patient input on a User Interface for LLM data that provides personalized, actionable feedback for trainees to improve their communication skills.

Primary Investigator: Nathan Sheffield, PhD

Research suggests students remember more when they have to explain an idea in their own words instead of picking an answer from a list. Despite this, most courses rely on multiple-choice quizzes and tests by default. The reason isn’t pedogogical, but practical: grading written answers just takes more time, so open-ended questions are not practical for low-stakes practice quizzing or scalable for large classes. autoTA is a web app that solves this. By using an AI to grade answers, we can now give students open-ended questions, even in low-stakes practice settings or for large classes. Students write open-ended responses from their own mind rather than selecting from a list of responses, and get feedback back right away from an AI. What makes autoTA different from just asking a chatbot is that the instructor writes a grading rubric, which grounds the AI in an explicit prompt about what a good answer covers, with a point rubric naming the specific ideas a complete answer has to hit. The AI grades against those points specifically, and tells the student exactly what the response is missing. This helps the AI stay on the course material rather than veering into areas not covered by the course, or, worse, making things up. Students can retry as many times as they want and nothing counts toward their grade, giving them practice with recall, rather than recognition. Ultimately, this approach provides a cheap and easy way to provide recall-based quizzing, which has potential to improve long-term retention due to the benefits of forcing students to recall knowledge, rather than merely recognize it. Our trial deployment of the software trials the approach in a computational biology course at UVA, transitioning from multiple-choice to open-ended questions.

Primary Investigator: Milad Memari, MD, MS, MSEd

Diagnostic error, often due to breakdowns in clinical reasoning, occurs in an estimated 10–15% of clinical encounters and is a leading cause of preventable patient harm. Because of these reasoning breakdowns, remediation requires high-volume practice with structured, step-specific feedback, which is resource-intensive, and many standardized-patient programs cannot provide this degree of support at scale. Individualized feedback on observable clinical skills, particularly at the start of medical school, is resource limited. This makes it challenging for coaches to offer individualized, structured feedback to learners. This project implements MedSimAI, a generative-AI platform pairing interactive simulated patient encounters with automated, rubric-aligned coaching feedback, and evaluates its application as a formative supplement to the Foundations of Clinical Medicine course at the University of Virginia School of Medicine. The study examines how AI-simulated encounters influence clinical reasoning development over the course of medical training, and explores learners’ and coaches’ perceptions of its impact on self-regulated learning and the development of learning goals. The work aims to evaluate several outcomes of scalable, AI-supported reasoning simulation in undergraduate medical education.