The Role of Physician-AI Interaction in the Treatment of Multiple Myeloma

The treatment paradigm for multiple myeloma has shifted in recent years. Notably, the gap between clinical trials and real-world practice continues to expand.

A drawback of randomized controlled trials (RCTs) is their inability to address every clinical scenario — patient care is often far more nuanced and complex. With the advent of artificial intelligence (AI) technologies, the integration of machine learning models in clinical decision support systems (CDSS) could be a potential solution to this problem.

In a report published in Blood, Barbara D. Lam, MD, of the department of medicine at Beth Israel Deaconess Medical Center in Boston, Massachusetts, and colleagues developed a CDSS that displays simulated survival and adverse event data from a clinical trial and machine learning model.

In a pilot study, Dr Lam and her colleagues evaluated how physicians utilize the available data to make treatment decisions for patients with multiple myeloma.

Testing the System

To test the system, physicians were recruited from the internal medicine and hematology-oncology departments at an academic medical center. They were presented with varying combinations of RCT and machine learning data in increasing “tiers” of information for 12 patients with multiple myeloma.

In tier 1, only RCT data was presented. In tier 2, participants were shown outcomes of a machine learning model, and in tier 3, they were provided with information about how the machine learning model was trained and validated. 

At each tier, clinicians were asked to choose a treatment (“red pill” or “blue pill”), rate their confidence in treatment on a scale from 1-10, and when machine learning data was available, rate their perceived reliability of the model.

Out of 284 physicians who were invited to participate, 32 (11.3%) took part in the study. Among the participants, 50% were internal medicine residents and 50% were hematology-oncology fellows and attendings. Most were White (69.0%), male (72.0%),  and all were less than 40 years of age.

Across various clinical scenarios, a few trends were observed. “Confidence in treatment was highest when RCT and [machine learning] findings were concordant,” the study authors wrote in their report. “Participants preferred the treatment that demonstrated a survival benefit, regardless of whether it was supported by RCT data or [a machine learning] model.” This was the case even before participants learned how the model was trained or validated.

Finally, participants chose the treatment that showed a survival benefit, regardless of whether it was supported by RCT data or a machine learning model.

Results in Context 

Overall, there has been limited investigation into how clinicians reconcile RCT and machine learning data, especially when the results are conflicting. Larger prospective randomized trials are necessary to bring more clarity to this question.

Undoubtedly, the integration of machine learning models into modern CDSS is intriguing and may offer a new path towards precision oncology. Dr Lam and her colleagues have showcased a prime example in this single-center pilot experience involving patients with multiple myeloma.

A broader question is how to best implement CDSS into clinical workflows. The sample size of the existing research is quite small, and some have questioned the clinical efficiency of such systems in their present state. Despite these difficulties, there is still ample opportunity to further develop and improve the utility of these systems.²

Notably, CDSS are currently unable to replace oncologists. The value of these systems lies in their ability to support clinical decision making and train young physicians.^2-3

Even if AI technology provides treatment suggestions, the most appropriate treatment needs to consider the patient’s physical and mental wellbeing, financial status, complications, and willingness to receive such treatment.^2,4

This article originally appeared on Hematology Advisor