Google's 90% figure for its medical chatbot doesn't measure what you do when you ask a chatbot at home

You type “dry cough for over a week, no fever” into a chatbot and get back a list of possible causes. Google has just reported that, for its medical chatbot AMIE, a list like that contained the final diagnosis of the clinic visit within its first seven entries in 90% of cases. The number sounds reassuring enough to trust, but it measures something much narrower than what you do at home.

How the study worked

On 8 October 2026, Google and Beth Israel Deaconess Medical Center, a hospital in the United States, published a study in The Lancet on AMIE, which Google calls a diagnostic chatbot for research. Before a visit, patients chatted in text with AMIE about a new, non-urgent symptom. According to Google’s research page, 100 people chatted and 98 came to their appointment. The physician who saw them afterward received the transcript and a summary.

Each conversation was watched live over video by a physician, ready to step in under four stopping criteria set in advance. Google says no conversation had to be stopped.

What the 90% measures

The 90% is the share of cases in which the final diagnosis appeared among the first seven entries of AMIE’s list: 88 of 98 cases, according to the earlier arXiv version. Also according to Google’s research page, the share is 75% when you count only the first three entries, and 56% when you count only the top entry.

For a physician reading the list just before meeting the patient, a list with the right answer somewhere in the first seven is still useful, because they know how to rule entries out. Reading alone at home is different: nobody tells you which entry on the list is the right one, and for many cases the top entry is not. That is why 90% cannot answer the question “should I trust a chatbot when it talks about my symptoms?”

The figure 75% appears a second time in Google’s post, with a completely different meaning: physicians said the summary helped them prepare for the visit in 75% of cases. The two numbers coincide but measure different things, so read a number’s label before you quote it.

Conditions you do not have at home

The study had a physician monitoring every conversation, and a real appointment right afterward that established the final diagnosis. A chatbot you use on your own has neither.

The study was also small and narrow. It took place at a single site, had no control group, and the patients were younger than the clinic’s patients overall in that period. Google writes that, because the study has no controlled comparison, it does not support quantitative conclusions about effectiveness, and that larger trials are needed to evaluate a chatbot working directly with patients. AMIE reads text only, does not look at medical records and cannot examine anyone; Google’s page notes that physicians did better on how practical and how costly the management plan was.

Using a chatbot in a way that fits what the study tried

What the study tried was preparing for the visit, not replacing it. The closest thing at home is asking a chatbot to help you prepare to see a doctor: write down when the symptom started, how it has changed and which medicines you take, then ask the chatbot to suggest questions to ask. Treat the list it gives you as questions to bring along, not as a conclusion.

One way to check yourself: after the chat, can you name three things the doctor needs to know that you have not yet written down? If not, the chatbot has only given you more text and has not helped with the visit. And if there are severe signs, or signs that appear suddenly, such as difficulty breathing, severe chest pain, confusion, severe or worsening abdominal pain, or high fever, do not wait for a chatbot’s reply: get examined right away or call emergency services (115 in Vietnam). This article does not replace the advice of a health professional.

Sources

1. Google — Study in The Lancet suggests AI could improve patient-physician relationships — published 8 October 2026. States 98 patients, that no conversation had to be stopped, the 90% and 75% figures in Google’s post, and Google’s statement that larger trials are needed.
2. Google Research — Exploring the feasibility of conversational diagnostic AI in a real-world clinical study — states the single-site, uncontrolled design, the four stopping criteria, the 90%, 75% and 56% rates, and the limitations.
3. arXiv — A prospective clinical feasibility study of a conversational diagnostic AI in an ambulatory primary care clinic — the authors’ own preprint. It states plainly that the final diagnosis was within the first seven entries in 88 of 98 cases. The authors write that physicians did better than AMIE on how practical and how costly the management plan was. The Lancet paper could not be opened when this article was written, so the figures come from the three pages above.