Patient Input Can Improve AI Models in Health Care: Innovators

Doctors have recognized the importance of empowering patients and “humanization” of care. But interventions based on artificial intelligence, particularly risk assessments, are being developed through black-box computer programs, walled off from human review. Can patients have a say in the AI models that are applied to them?

Most AI would benefit from human review. And a few companies are undertaking large effects to involve patients in the development of AI.

This two-part series covers several companies—in particular, Ubie, Ada Health, Spire Health, and Healthvana—in the hope of encouraging other AI developers to take that big step toward transparency and accountability.

Ubie Incorporates Survey Input into AI Model Development

The AI Symptom Checker from Ubie, a Japanese firm, is a fairly typical application of AI in health. It combines medical research with clinical data and self-reported data from patients to turn up conditions that might be missed in a standard doctor visit. Ubie is currently investigating the diagnosis of a rare disease known as Cushing’s Syndrome, collaborating with the Cushing’s Support and Research Foundation on a symptom checker that patients showing symptoms of the disease can fill out.

Cushing’s is a good candidate for applying AI because it is rare and complex. Because it is rare, doctors likely haven’t encountered it and it doesn’t come naturally to mind when evaluating a patient. The medical literature on rare diseases is also sparser. Because its presentation of symptoms is complex, Cushing’s is also hard to distinguish from many other possible causes.

Ubie CEO and Cofounder Kota Kubo emphasizes that Ubie includes several layers of human input during the development of AI models. They employ more than 50 specialists in a clinical review process, and get input from the experts at hospitals and clinics they partner with. Now they are adding patient input as a third layer.

Ubie’s symptom checker can capture the typical ways that patients describe their symptoms, which is often different from the terms clinicians use. Often, rerunning the AI models on what patients say can improve their ability to predict Cushing’s. “The patient journey is unique,” Kubo says.

Topic-of-the-Month and Other Patient Comments at Ada

Ada Health offers patients a tool to evaluate symptoms and determine what level of care to seek. An independent study showed it to be 35% more accurate than other tools on the market, its safety being on par with the diagnostics of a general practitioner.

Dr. Tauseef Mehrali, Vice President of Medical Safety and Regulatory Affairs at Ada, says they employ about 50 physicians who assess the AI assessments and find ways to improve them. Many of their physicians are also software engineers.

The Ada tool encourages patient suggestions for improvement (Figure 1), which are included when the staff evaluate AI results. A “topic-of-the-month” project highlights some form of user input each month.

A screen asks whether the symptom assessment was useful.
Figure 1. Example of accepting patient feedback.

In response to a patient suggestion, the software engineer might create a new field in the input to AI, or change the weights of various factors. Ada uses this flexibility to adapt the models to different populations and geographies. For instance, they are recently conducted post-partum studies among women in South Africa.

Mehrali points out that population-level studies are valuable input, but “might drown out the voice of the individual.” At Ada, he says, “The voice of the user is given a platform.”

In addition to improving the accuracy of their product, Mehrali says, there are other benefits to working with patient input. It offers staff a level of fulfillment comparable to working in a health care system, and makes the models explainable.

Spire Health Employs Patient Interviews

Spire Health alerts clinicians to impending respiratory attacks and incidents by tracking patient breathing, pulse rate, and activity and feeding this data to patient-specific models. According to Founder and Chief Scientific Officer Neema Moraveji, Spire employs remote nurses to contact patients and verify the model’s predictions. This patient-reported data further augments automated predictions and can also be used to improve the models.

Healthvana Evaluates Patient Answers and Post-Visit Surveys

Healthvana has created a Conversational AI Navigator, described by Chief Product Officer Sam Warmuth as providing “nonjudgmental health information and supportive HIV/STI prevention” that includes “health education, personalized medical information, and administrative navigation.”

Each patient message, and its AI-generated response, is assessed by at least two Healthvana staff for “accuracy, comprehensiveness, and contextual sensitivity.” Warmuth says, “We use post-visit surveys as well as the message score to identify conversations that could be improved. Those conversations are added to an in-house automated testing framework, separate from the models themselves, that allows a specific combination of prompt, patient messages, AI messages, and additional context to be used as ‘replay.’ The replay is added to our replay test library that is run before any changes to the AI are released, to protect against regressions in behavior.”

They mitigate against bias through random selection of early users, demographic representations that mirror the races and regions of patients using the system, and broad geographic deployment.

In the second article of the series, we’ll look at some more companies, and at the ethics and challenges of using patient input.


Leave a Reply

Your email address will not be published. Required fields are marked *