Digital-health

Babylon Health has published in the journal Nature an interesting study on improving the accuracy of medical diagnoses through casual machine learning.

Machine learning promises to revolutionize decision making and clinical diagnosis. In the diagnosis process a doctor aims to explain the patient’s symptoms by determining the diseases that cause them.

Despite significant research efforts and strong commercial interest, IA-based diagnostic algorithms fail to achieve physicians’ accuracy in differential diagnosis, where there are multiple possible causes of a patient’s symptoms.

But why are the current algorithms not effective in differential diagnosis? All existing diagnostic algorithms, including those set on the Bayesian model and Deep Learning, are based on associative inference: they identify diseases based on their correlation with the patient’s symptoms and history.

Doctors, on the contrary, formulate the diagnosis by selecting the diseases that offer the best causal explanations for the patients’ symptoms. As noted by Pearl, associative inference is the simplest in a hierarchy of possible inference patterns. Counterfactual inference is at the top of this hierarchy and allows for the attribution of causal explanations to the data.

The researchers have shown in their study that the diagnosis is fundamentally a task of counterfactual inference and that the failure to dissociate the correlation from causality places strong constraints on the accuracy of the associative diagnostic algorithms, which sometimes result in sub-optimal or dangerous diagnosis.

The researchers then developed a counterfactual algorithm and compared it with a state-of-the-art associative diagnostic algorithm and a cohort of 44 physicians, using a test set of 1671 clinical cards.

The physicians achieve an average diagnostic accuracy of 71.40%, while the associative algorithm achieves a similar accuracy of 72.52%, ranking in 48% of the cohort physicians. The counterfactual algorithm reaches an average accuracy of 77.26%, ranking in the first 25% of the cohort and achieving expert clinical accuracy. These improvements are particularly pronounced for rare diseases, where diagnostic errors are more common and often more serious, with the counterfactual algorithm providing a better diagnosis for 29.2% of rare diseases and 32.9% of very rare diseases compared to the associative algorithm.

This new algorithm is not yet present in Babylon’s publicly available application and will only be released after all necessary regulatory approvals have been obtained.

If you want to read the entire study, you can find it here.