Charting a Path to the Right Diagnosis with AI Copilots | Inside Precision Medicine

The rapid growth and widespread adoption of large language models (LLMs) across real-world applications offer a host of new opportunities for the healthcare sector. Hospitals and clinics are starting to rely on artificial intelligence (AI) assistants for daily administrative tasks, and the technology is expanding to support doctors in making more accurate diagnoses, which remains a cornerstone of effective medical care.

“AI can improve speed, consistency, and reach while keeping the doctor at the front and center,” said Marc Succi, MD, an attending radiologist at Massachusetts General Hospital, associate professor at Harvard Medical School, and founder and executive director of the Medically Engineered Solutions in Healthcare (MESH) Incubator at Mass General Brigham.

His research covers pioneering studies on the potential of using LLMs in clinical decision support and developing and commercializing AI tools across medical specialties. These tools include a real-time AI instructor that can help non-specialists in suburban or rural areas perform techniques like colonoscopies or ultrasounds, and an emergency stroke triage AI that helps doctors prioritize the most likely positive cases, effectively shortening the time to treatment for a condition in which every hour counts.

Rapid advances in LLMs also enable them to support doctors with differential diagnosis. This is an iterative process that combines information from the patient’s clinical history, physical examination, and test results to create a list of potential conditions, which physicians rely on to decide upon the next steps necessary for diagnosis and treatment.

“The differential [diagnosis] is probably one of the hardest things to do in medicine,” said Succi. A major challenge is having to work off very limited information while striking a balance between listing every rare but possible diagnosis and reducing the amount of testing needed to confirm the final diagnosis. In this context, Succi added, “AI is really good at taking a lot of information, synthesizing it, and finding patterns.”

Earlier this year, a Nature paper published by Google reported that a specialized LLM could help a sample group of 20 clinicians create more comprehensive differential lists, increasing their accuracy from 36% to nearly 52%. As the technology keeps evolving, research suggests that combinations of existing LLMs could significantly improve their diagnostic accuracy when creating differential lists.

This exceptional ability to find patterns in vast amounts of data can make AI a valuable tool for the education of medical professionals by helping them hone their diagnostic skills. While a student will normally only see a few cases of, say, appendicitis, during a typical clinical rotation, AI can give them a chance to practice diagnosing it over and over with varied patient presentations.

“Medicine is all about repetition and pattern recognition, the more cases you see the better you become at diagnosing them,” stressed Succi. Both Brigham and Harvard’s medical schools have started testing and implementing this technology with their students.

Enhancing diagnostic tools

One of the most promising contributions of AI to diagnostics is its ability to enhance the performance of existing diagnostic tools across medical disciplines. An area where the technology is already spearheading change is radiology, where the demand for diagnostic imaging and turnaround times are growing faster than hospitals can handle, driven by an aging population and the expansion of preventive care.

“In the U.S. right now there is a significant shortage of radiologists, which is exacerbated by the amount of studies we have to read,” said Samir Shah, MD, a radiologist with over 20 years of experience and chief medical officer of Qure.ai. “If we can make a radiologist’s job easier, that can help us improve [diagnostic] quality without adding time to our work.”

Qure.ai has developed several AI solutions to automate the interpretation of X-rays and CT scans, speeding up diagnosis and access to treatments. A major focus of the company is lung cancer, which is a very aggressive condition often diagnosed only at an advanced stage and the leading cause of cancer death in the U.S.

Some AI tools can assist with tasks like measuring and identifying lung nodules, stratifying their risk, and monitoring their progression over time, all of which combined typically take a significant portion of a radiologists’ valuable time. Others rely on LLMs to mine electronic health records (EHR) for data, finding and flagging patients who may have an actionable nodule in the lungs.

Another area where AI could have a big impact is cardiology, where it can aid established techniques like electrocardiograms (ECGs/EKGs) that have been around for over 100 years. “People can now look at an ECG signal from a wearable device, but what has not really changed much is how you reach a diagnosis from that signal,” said Maulik Nanavaty, PhD, CEO of Anumana.

Using EKG and echocardiogram data from millions of patients, the company is creating AI models that can find subtle patterns otherwise invisible to the human eye. Developed in collaboration with the Mayo Clinic, Anumana’s AI model has been cleared by the FDA for the detection of low ejection fraction, an early indicator of heart failure that commonly goes undiagnosed.

“AI can take the expertise of the best cardiologists in the world and put it in the hands of any physician,” said Nanavaty. Deep learning can assist with difficult diagnoses, such as pulmonary hypertension, which is often confirmed years after onset due to overlapping symptoms with other conditions. For these patients, unlocking early diagnosis and treatment can significantly increase their life expectancy and quality of life.

Addressing safety concerns

The growing use of LLMs has brought up concerns about “hallucinations,” where the models make information up when responding to a prompt. While newer models have lower error rates, ensuring that these tools provide reliable information is crucial in healthcare. Across the board, experts seem to agree that the reigning challenge to developing highly accurate algorithms is obtaining large amounts of quality data to train AI models.

“At the end of the day, your AI is only as good as your data,” said David Spetzler, PhD, president of Caris Life Sciences, a company leveraging molecular profiling techniques and AI to develop next-generation precision medicine tools. He noted that, currently, most patient data is stored in EHR systems, which are often built to collect data for billing and reimbursement rather than to comprehensively track a patient’s full history, symptoms, and outcomes over time.

To overcome this limitation, Caris Life Sciences has undertaken the task of generating that data in house, using whole exome sequencing (WES) and whole transcriptome sequencing (WTS) to analyze tumor tissue from over half a million cancer patients, essentially tracking their progress and treatment response over the years. Although building such large datasets takes a significant amount of time and resources, the company plans to keep scaling up its data collection. “Because the heterogeneity of cancer is just so big, we will need millions of patients’ worth of data in order to find all the nuances possible,” said Spetzler.

The company’s AI tools are helping identify the tissue of origin in cancers where the primary tumor is unknown, and allowing hundreds of patients who had been initially misdiagnosed access to the right treatment for their condition. From multicancer detection to therapy selection and disease monitoring, this type of technology is unifying and redefining how cancer is diagnosed.

Mindbowser, a healthcare consulting company, is seeking to address the current limitations of clinical data collection by adapting existing systems to meet the needs of AI development. “We are building complete AI-first EHR systems, foundationally correcting how data comes in and what kind of data is extracted,” said Ayush Jain, Mindbowser’s CEO.

The company is developing multiple features that cover the entire patient’s journey, like an assistant that can help physicians check for drug interactions when prescribing a medicine and a chatbot that provides relevant information to patients in between visits. Any LLMs that directly interact with patients can be trained to keep all answers within a knowledge repository created by the organization, eliminating the risk of hallucinations.

Introducing this kind of change can be challenging in an industry that still relies on fax machines to send and receive information, Jain noted. However, having a single comprehensive solution in place can significantly help hospitals and other healthcare institutions, whose IT teams often struggle to manage software from multiple providers, each covering tasks beyond EHRs such as storing medical images, setting appointments, billing, and filing insurance claims.

Mastering implementation

Regardless of how well an AI tool performs, improper implementation can turn it into a time sink for clinicians. Understanding and adapting to existing clinical workflows can have a much larger impact on success than the selection of which specific algorithm to use.

While some doctors may be excited to work with AI from the get go, others may be wary of new technology or reluctant to change the way in which they have been working. This can be especially true if they have had unsavory experiences with other tools in the past, like adding unnecessary tasks to their already full schedules and distracting from their job of caring for the patient.

“Great AIs fail all the time because they are not cleanly integrated into existing systems,” said Succi. “If you have to leave your medical records screen and open a browser or another program, it is very unlikely to be successful. Anything that adds even two or three extra clicks is really difficult to get adopted.”

He added that establishing well-defined evaluation and feedback stages during the implementation process is key to getting doctors on board with any new tools. Accountability is also a concern for patients and doctors alike, highlighting the importance of setting clear guidelines on responsibility for reviewing and approving any decisions made by an AI, and the actions to be taken when adverse outcomes are reported.

Furthermore, local validation of any new AI tools is essential to ensure the model works as intended for the target patient population. “I think the best approach is to treat AI like a high-risk medical device,” said Succi. “You have to prove it works in every single scenario.”

“Implementation requires going through a very rigorous process,” said Shah. “It touches so many clinicians and patients that we have to make sure that we are doing it ethically and within the rules of the healthcare system.”

Any successful implementation needs to account for the needs of the target organization. Shah recalls working with an emergency room in Toronto where the priority was to make sure no patients with bleeds in the head were discharged. This required adjusting the sensitivity and specificity of the AI algorithm to favor false positives over false negatives, and educating doctors working with the AI on where the algorithm was more likely to make mistakes. This helped the doctors make better choices for their patients.

“Different areas have different needs, and the main problem for clinicians is that they are overwhelmed most of the time with features they really do not need,” corroborated Jain. “A lot of it goes back to figuring out how to reduce the cognitive burden for the doctors.”

For him, the key to success is to never use AI as a temporary bandage to a healthcare organization’s problems, but to commit fully to its adoption once a decision has been made. As the technology keeps evolving, he believes that the most successful solutions will be those that can balance the interests of all stakeholders, from upper management to the end users.

Stepping into the future

One thing seems clear: AI is here to stay, and its role in healthcare will only continue to grow and evolve. Epic, one of the largest providers of EHR software in the U.S., recently announced the launch of hundreds of AI features like virtual assistants for healthcare providers and patients alike. Others are sure to soon follow in these steps.

Going forward, Nanavaty envisions medicine becoming more precise at the point of care, enabling patients to get tests done much earlier in their health journeys: “Allowing these tools to come in early will change how medicine is practiced.” Succi and Shah see AI increasingly moving towards working with multimodal data, integrating information across doctors’ notes, lab results, imaging, and more to provide better diagnostic answers. With the rise of wearables and home sensors, continuous monitoring of patients will allow vast amounts of data to be fed into AI models to significantly improve performance.

“We are on the cusp of a revolution where the technology available today can have dramatic effects on patients,” said Spetzler. As more drugs become available to patients, the standard of care will shift towards more precise diagnoses earlier in the patient’s journey. Ultimately, he envisions a future where diseases can be treated before they even fully manifest in the patient: “Precision medicine is going to turn into precision prevention.”