The medical journal The Lancet published a study on October 8, 2026, that its authors consider the first prospective, real-world evaluation of a patient-facing conversational artificial intelligence system in primary care. The work, by researchers at Boston’s Beth Israel Deaconess Medical Center (BIDMC) in collaboration with Google, analyzed how AMIE (Articulate Medical Intelligence Explorer), the diagnostic chatbot Google is developing as a research project, performed when talking to people waiting for an urgent medical appointment.
For Google, the milestone also carries symbolic weight. Mike Schaekermann, the project’s research lead, wrote on the company blog that it is Google’s first publication in the main journal of The Lancet. Until now, AMIE had shown striking results mostly in simulations, with actors playing patients. The big question was how it would behave with real people, real symptoms and a real clinical workflow.
The study design was cautious. It was a prospective, single-center, single-arm trial, registered on ClinicalTrials.gov under the identifier NCT06911398. Participants were English-speaking adults aged at least 18 who had booked an urgent primary care appointment for a single complaint. Up to five days before the visit, each patient chatted with AMIE from home through a secure chat interface. The system asked about symptoms, took a medical history, suggested possible diagnoses to discuss with the doctor and generated a summary.
Every conversation was monitored in real time by a board-certified internist acting as a safety supervisor. That supervisor could step in if the patient appeared to be at risk, experienced significant emotional distress or asked to end the session, or if they spotted a problem, such as the need to clarify symptoms, give emergency instructions or address concern about a serious diagnosis. Afterwards, the transcript and summary were sent to the primary care physician before the appointment.
Between April and November 2025, 114 patients enrolled, and 98 completed both the AMIE conversation and the in-person visit. The main result was clear: based on the predefined criteria, no conversation had to be stopped for safety reasons. Supervisors did identify one hallucination, meaning information made up by the system, and added clinical information in five of the interactions.
The quality of the conversations received good marks. Clinical evaluators rated AMIE’s conversations favorably in 87% to 100% of cases across the 17 criteria analyzed. Patients were somewhat more demanding: their favorable ratings ranged from 48% to 96% across 16 criteria. In addition, participants’ attitudes toward artificial intelligence improved after talking to the system and remained higher after their appointment with their doctor.
The results on usefulness for clinicians are more nuanced. Primary care physicians completed the follow-up survey in 60 of the 98 cases, and in 44 of those they had reviewed the AMIE transcript before the visit. In 33 of those 44 cases, about 75%, they said the material helped them prepare, and in 25, roughly 57%, they said it might have changed their approach. Google also highlighted that AMIE’s differential diagnoses matched the doctors’ final diagnoses 90% of the time.
In only one case did a physician describe the interaction as somewhat harmful: they worried that a patient may have become anxious after AMIE listed lymphoma among its possible diagnoses. It is a reminder that sharing diagnostic hypotheses with patients, even when they are technically sound, has emotional consequences that an automated system must handle with care.
Patients particularly valued the system’s ability to listen, explain information and put them at ease. But there were two areas of distrust: the confidentiality of what they shared with the chatbot and the honesty of the system itself. “Research suggests that patient trust in medical AI systems flows from relationships with their human providers,” said Adam Rodman, director of AI programs at BIDMC’s Carl J. Shapiro Institute.
Rodman stressed the purpose of the work: “Despite the theoretical performance of patient-facing AI in experimental settings, it is unclear how conversationally safe these encounters are in a real-world setting.” Before evaluating them as workload tools, he added, they must first be shown to operate safely and acceptably in real care workflows. Peter Brodeur, co-first author, said the study establishes a baseline for future trials on when less direct human oversight might be appropriate.
The authors themselves stress the limitations. The study was designed to assess feasibility, not to show that artificial intelligence improves health outcomes. It involved a single center, English-speaking patients and a physician watching every conversation, which would not be scalable in practice. Google acknowledges on its blog that larger clinical trials are needed to assess such tools at scale, and experts such as cardiologist Eric Topol, according to the conversation on X, praised the results while pointing to that same need.
The context explains the interest. An aging population, workforce changes and patients’ increasingly complex needs are straining primary care systems around the world. A tool that can take a medical history before the appointment could free up valuable minutes and let the visit focus on the clinical decision and the relationship with the patient, rather than on the initial questioning.
“We believe the future involves continued involvement of human clinicians in patient care, but see a future where the triad of patient, human clinician and AI helps to provide our patients with the very best evidence-based care possible,” said Marc L. Cohen, clinical chief of primary care at BIDMC and co-senior author. The study does not make AMIE a product ready for the clinic, but it delivers what was missing until now: safety data with real patients, published in one of the world’s most demanding medical journals.
