What is AI for?

This week, there have been a couple of studies and a prominent New York Times story that raise questions about how AI should be used in health care. 

One study, published Thursday in Nature, illuminates how people are using general-use chatbots to guide their health. The study, conducted by Microsoft researchers, analyzes 500,000 conversations with the company’s large language model chatbot Copilot. The data reveal that 40% of users' health-related conversations were seeking health information, not medical advice. Symptom review and medical advice comprised a tenth of users’ queries. 

That said, researchers noted, “Nearly one in five conversations involve users describing their own symptoms, interpreting their own test results or managing their own conditions. These interactions exist on a spectrum: at one end, a user asking ‘what does high cholesterol mean’ is seeking general education; at the other, a user describing persistent headaches alongside their medication list is seeking information specific to their own circumstances.” This comment reflects the difficulty researchers had in determining whether queries were basic information seeking or an attempt to self-diagnose. 

Notably, queries about symptoms and health concerns rose at night, as did mental health and wellbeing conversations. People are also more likely to ask chatbots questions about their own symptoms from their phone, rather than a desktop. Still, the study highlights that people are predominantly using LLMs for general health inquiries.

Understanding how people are using LLMs for health is increasingly important as developers, doctors, and regulators consider how these technologies should be best used and regulated. 

While there have been several articles showing how chatbots have sussed out rare disease diagnoses when doctors couldn’t, LLMs are also highly fallible. A story in the New York Times this week highlights what goes wrong when AI gets it wrong. In this story, a man, convinced by an AI diagnosis, refuses life-saving cancer care. 

Past research suggests that general-use chatbots are not ready for direct patient care.

But regulators aren’t likely to get involved. The Food and Drug Administration under President Donald Trump has softened its stance on regulating AI and other technology that assists doctors with patient diagnosis and treatment, and isn’t interested in regulating consumer health and wellbeing tools.   

Commissioner Marty Makary has repeatedly voiced support for AI-enabled care and, as a perspective in JAMA published this week lays out, has let Utah use AI for drug prescription, even though it is within the agency’s remit to regulate this use case.    

But this abdication of responsibility is a disservice to health care, say the paper’s authors, Daniel G. Aaron, MD, JD, a professor of law at the University of Utah, and Christopher Robertson, professor of public health law at Boston University. They assert the health care industry should insist on regulatory guardrails. 

“The FDA fell behind in regulating flavored e-cigarettes, which are now ubiquitous and have contributed to a youth e-cigarette epidemic. The sheer scale of the unauthorized market and the subsequent legal tactics used by tobacco companies turned premarket requirements into a mere technicality. If AI prescribing becomes the industry standard before safety and liability frameworks are established, the power problem may render future regulation infeasible.”

The question all these AI conversations raise is: what should the guardrails be?

Now, onto the news of the week.

150 companies are selected for CMS ACCESS Model

A program from the Center for Medicare and Medicaid Innovation greenlit 150 digital health companies to participate in its ACCESS Model. The program is designed to expand digital care for chronic conditions, and will be paying participants set rates, provided the participants show their patients are showing tangible improvements through their technology. 

On the list of selected companies are wearable companies like Withings and Whoop; virtual care companies like Cadence, Doctronic, and Verily; healthcare navigation app Castlight; and mental and behavioral therapy app Headspace. There were also some doctor groups in the mix. Not all the companies that are in this provisional group will necessarily go live when the program starts in July.  

Medicare payments to insurers will go up 2.5% next year

The administration plans to increase payments to Medicare insurers by 2.48%, a significant increase compared to the initial proposal, which had the payment nearly flat. Big health insurance stocks jumped after the announcement: Humana was up by 12%, Elevance and CVS Health by 8%, and UnitedHealth by 9%. The increase will cost an estimated $13 billion in additional payments to insurers. 

The FDA is going to discuss bringing back compounded peptides

The Food and Drug Administration (FDA) announced it will bring together a panel of outside experts in July to consider whether or not to allow compounding pharmacies to make seven peptides that were removed from its list of authorized compounding products in 2023. This comes after RFK Jr. said he would like to see easier access to some peptides, which he took to treat injuries. The effectiveness and safety of the products aren’t proven, but the HHS secretary is a fan. 

House Energy and Commerce considers a healthier America 

The Health, Energy, and Commerce committee met this week to discuss a spate of bills aimed at improving Americans’ health. Among these was a bill from Troy Balderson (R-Ohio) the Digital Health Screener Act of 2026. The bill would carve out wellness devices from FDA authority. This includes technology that “measures, estimates, infers, or outputs one or more physiologic parameters and displays values, ranges, trends, baselines, or longitudinal summaries; or contextualizes such outputs in relation to sleep, activity, stress, recovery, or similar wellness domains.”

During the hearing, Congress members discussed the need for stronger data privacy rules to ensure that data collected in health wearables and apps be protected. Health data collected by health providers, insurers, and their vendors is guarded under federal rules. App and wearable data is not.

Screening algorithms could detect risks that the human eye misses. For instance, they could identify heart risk from a CT scan by identifying calcium residue in the coronary arteries. But who is going to pay for the follow-up that the findings will require? STAT’s Katie Palmer looks into how AI impacts the affordability of healthcare. 

Medicare navigation company Chapter raised $100 million in a Series E round led by Generation Investment Management. 

Jimini Health raised $17 million in seed funding. The company is gearing up to launch Sage, its mental health chatbot that interacts with patients under clinical supervision, in partnership with large behavioral health organizations. So far, the startup has raised a total of $25 million.

Virtual breastfeeding support startup SimpliFed raised $10.8 million in a Series A round to expand access to its maternal health and lactation support. 

Click Therapeutics raised $50 million in a Series D round investment, raising funds from Boehringer Ingelheim, which provided the company with additional funds to commercialize CT-155, a prescription digital therapeutic developed to treat schizophrenia symptoms. Immediately following the deal, Click Therapeutics laid off 27% of its workforce. 

Remission Medical, a virtual rheumatology company, closes Series A led by Blue Heron Capital. The company didn’t disclose the funding amount, but shared that it would be used to build partnerships across US health systems. 

Novo Nordisk partners with Open AI to accelerate drug discovery. The goal is for the drugmaker to employ AI to process large datasets and more quickly move from research to trials. The announcement sent Novo’s stock up by 4%.

Reproductive health company Maven Clinic announced a partnership with Wellthy, an app for employees to navigate family health and caregiving needs. The partnership would help coordinate complex needs, such as a pregnant person trying to organize childcare while also taking care of an older parent. 

Direct-to-consumer testing platform Function acquired Getlabs, a mobile healthcare company that delivers specialized health services to one’s location. The transaction amount was not disclosed.

This week on Lifers Christina Farr sits down with Sunita Mohanty, CEO and Co-founder of Ultralight, an AI-native operating system powering over 75 functional and longevity medicine clinics across the U.S., to talk about peptides.