Exclusive: Amazon's Health AI Bot Exposes a Flaw in Patient Data Access

In March, Amazon broadly launched a health AI bot to pull a patient’s medical history and answer questions about their health. The bot, launched through One Medical, is causing a kerfuffle among health tech leaders.

Members of Carequality, a framework that health institutions use to access patient medical records, are debating whether a bot should be able to access patient data as someone involved in the treatment of a patient, according to two people familiar with the matter. 

To be clear, Amazon’s primary care clinic, One Medical, is a legitimate care organization that employs doctors and nurses. The discussion is over whether AI should be able to request data for patient treatment. 

Carequality and Amazon did not return a request for comment. 

Why does that matter? The big fear is that Amazon’s health AI will inspire a wave of bot-driven apps posing as telehealth providers to fraudulently access lucrative patient data.

At the heart of the issue is a discrepancy over how patients and doctors get health data. 

• Doctors can hop on the Carequality network and access patient data pretty immediately.
• Patients must use certified apps. These app developers have to jump through a lot of hoops to prove they are requesting data on behalf of a verified patient. 

Reserve Your Spot for Upcoming Webinars!

Doctors or clinics need an easy-to-register-for national provider identifier. This bifurcation in policy incentivizes companies to say they’re accessing data for treatment, whether or not they actually are, S. Yin Ho, author of “Rushing Headlong: Health IT's Legacy and the Road to Responsible AI,” told me.

Amazon’s Health AI bot has added urgency to the need for health care leaders to resolve a long-simmering debate over what exactly treatment is.

Amazon is not the only company seizing on patients’ desire to have AI answer their burning health questions. Other health systems are experimenting with using health AI bots to route patients to care, too. Pure telehealth operators like Verily are also building such apps, and more are on the way. 

President Donald Trump is asking the private sector to build even more of these apps. His administration believes free or low-cost health apps can help patients better manage chronic diseases like obesity and diabetes. 

As more apps come online, experts warn that the existing data exchange framework is not set up to police data-hungry scam artists posing as doctors. There is a secondary concern that a surge in bot requests could overwhelm the data exchanges, slowing access to records for everyone. 

A new old problem: Since 2024, Epic Systems, the largest electronic medical record company in the U.S., has been pushing interoperability network overseers to investigate potential instances of data fraud on Carequality and the government backed Trusted Exchange Framework and Common Agreement.