What Clinical Decision Support Systems (CDSS) look like, how they work, what constraints they have and what contribution they can make to improving doctors’ decision-making: all you need to know.
Article previewed on AgendaDigitale.eu
The first Clinical Decision Support Systems (CDSS) were developed in the 1980s. In recent years, thanks in part to advances in Artificial Intelligence (AI), their field of application has expanded considerably. This phenomenon is accompanied by a discussion on their real usefulness and reliability.
In order to answer these questions, it is necessary, first of all, to understand how CDSS are made, how they work, what constraints they have and what kind of contribution they can provide in improving the decision-making process of doctors.
Types of CDSS
CDSSs can be classified according to different criteria: active or passive, scope or purpose of use, type of architecture. Starting with these, we can divide CDSSs into two large families:
- Knowledge-based CDSSs
- CDSSs that are not knowledge-based.
The former type is based on rules (IF-THEN statements that can also be very articulated), which the system evaluates according to patient data to produce an action or output. Rules can be defined using evidence based on literature, practice or patient.
CDSSs that are not knowledge-based still require a data source, but the decision leverages artificial intelligence (AI), machine learning (ML), or statistical pattern recognition, rather than being programmed to follow expert medical knowledge.
Knowledge representation (EBM)
Medical knowledge gained through research and clinical practice – Evidence Based Medicine (EBM) – is conveyed through scientific articles that are published in medical journals. More than 800,000 scientific medical articles are produced each year in over 5,600 journals.
The scientific articles are structured according to precise rules and consist of text, tables and figures. Most of them are in English. Articles can refer to single studies, case-controlled studies, patient cohorts or randomised clinical trials (RCTs), be third-party reviews, summaries of multiple articles (evidence) or systematic reviews of multiple studies.
The value of knowledge is therefore given by its clinical relevance, i.e. the importance of the phenomenon observed or demonstrated, and its documentary relevance, i.e. the reliability of the sample and the analysis methodology that was followed.
Distillation of knowledge
In order to use medical knowledge in a CDSS it is necessary to carry out a process of “distillation” of the knowledge to allow its transformation into a series of logical concepts that can be applied to clinical cases. These concepts can be organised as “rules” that operate through an inferential engine or processed with artificial intelligence algorithms (neural networks, Bayesian networks, etc.).
The distillation process can be manual or semi-automatic. In the first case, a scientific board decides on the development of the rules and, starting from the evidence contained in guidelines, clinical protocols or scientific articles, operates in the following way:
- Identifies the clinical criteria that determine the application of the rule, e.g. patient’s age, therapy, bio-chemical values, etc. (inclusion criteria);
- Defines the contents that, when certain conditions occur, must be shown to the doctor. These contents are normally a synthesis of a more complex scientific rationale;
- Inserts references to the evidence underlying the rule;
- Classifies the importance of the contents according to international criteria, e.g. following the Grade methodology.
The semi-automated distillation process is carried out using AI technologies such as Natural Language Processing (NLP) which is able to extract information from digital texts by identifying medical concepts in documents, automatically extracting knowledge about medical procedures, drugs, body vital signs or medical conditions.
By combining NLP with Deep Learning, it is possible to identify clinically relevant attributes based on the surrounding context, e.g. discerning drugs prescribed in the past from those prescribed for the future or detecting the likelihood of a specific symptom or diagnosis, as expressed in the nuances of language.
These technologies automate a process that normally requires time-consuming manual review. Their effectiveness and reliability vary depending on the context, for example they achieve good accuracy when applied to extract information from drug product data sheets (SPCs) which have a fixed structure and a restricted lexicon, less so when applied to documents which may have different structures and lexicons.
As it is easy to understand, the reliability of the knowledge base is fundamental for the accuracy and effectiveness of the CDSS; for this reason, after the automatic distillation, a manual revision process always follows, conducted by qualified people (doctors, pharmacists) who validate the contents and refine the work produced by the AI.
Non knowledge-based CDSSs
CDSSs that do not use a knowledge base use a form of artificial intelligence called machine learning, which allows computers to learn from past experience and/or find patterns in clinical data. This eliminates the need to write rules and enlist the help of experts.
These systems are not able to explain the reasons for their conclusions and, for this reason, their use is rather controversial. They are often used for research purposes or as post-diagnosis systems to suggest models for doctors to investigate.
The most popular CDSSs of this type are based on support vector machines, artificial neural networks and genetic algorithms.
Artificial neural networks use nodes and weighted connections between them to analyse patterns found in patient data to derive associations between the symptoms and a diagnosis.
Genetic algorithms are based on simplified evolutionary processes that use direct selection to obtain optimal CDSS results. Selection algorithms evaluate components of random sets of solutions to a problem. The solutions that emerge at the top are then recombined and mutated and are again processed. This happens over and over again until the correct solution is discovered. They are functionally similar to neural networks in that they are also ‘black boxes’ that attempt to derive knowledge from patient data.
Non-knowledge-based networks often focus on a narrow list of symptoms, such as the symptoms of a single disease, as opposed to the knowledge-based approach that covers the diagnosis of many different diseases.
Information and actions
CDSS return, as output, messages of various kinds (alarms, warnings and suggestions), guidelines (in full or summary form), orders and prescriptions (order sets), clinical workflows, calculators and patient views organised by clinical concepts.
Areas of use
Decision Support Systems (DSS) find application in different areas of clinical practice.
Diagnosis
In the diagnostic field, CDSS can help clinicians in the formulation of a diagnosis through a differential diagnosis process that is usually carried out using neural networks. The process starts with the detection of the patient’s signs and symptoms and continues through successive questions to exclude similar diseases or conditions that do not include the set of symptoms and signs found during the examinations, until it is clear which disease or condition really belongs to the patient.
These types of SHDs may be directed at doctors, to make a diagnosis or perform triage, or at patients, to help them understand what their condition might be. These types of DSS are also called Symptoms Checkers.
In the field of radiology, specialised DSS are able, thanks to artificial intelligence (AI) algorithms, in particular those based on deep learning, to recognise images (pattern recognition) for the detection, characterisation and monitoring of diseases. Methods including convolutional neural networks and variational autoencoders are used in this area.
A further diagnostic field is that of models based on deep learning algorithms on bio-markers that are able to make diagnoses using data that cannot be directly correlated to the disease.
Choice of treatment
CDSS support clinicians in choosing the optimal treatment for efficacy, safety and cost. This process may be based on reference treatment protocols, such as in oncology, and/or on medical evidence that measures outcomes and quantifies the risks associated with treatment. Possible treatments are filtered and “weighed” according to the patient’s clinical picture by means of inferential models that can be implemented with different technologies.
Treatment management
CDSSs in this category provide suggestions and warnings about the steps that need to be taken to manage treatment or therapy. These may include dosing of drugs, monitoring of vital or biochemical parameters, taking special precautions or actions, and how to detect possible side effects or adverse reactions. This information, as far as drugs are concerned, comes from product data sheets (SPCs) and scientific evidence from observational studies of the use of drugs in clinical practice.
A further area in this category is specialised DSS based on predictive models that determine the likelihood of complications or patient prognosis.
Population stratification
This category includes DSSs that work on patient populations to determine health risks, e.g. the probability of events such as strokes or heart attacks, the demand for health services and the cost-effectiveness of interventions related to prevention and care of patients and the health benefits they may bring.
Possible benefits
The benefits that CDSS can provide are potentially very significant. Several scientific studies indicate that inpatients receive only 50% of the recommended care[1]. Many decisions in medicine may be considered unnecessary or even harmful and be placed among the wastes of resources.
A report on a study carried out by the Parliamentary Commission of Inquiry into Health Errors indicates that in Italy 53% of doctors admit to prescribing drugs for defensive purposes, a figure that rises to 73% among specialists. 71% of doctors prescribe laboratory tests for defensive purposes, and 76.5% prescribe instrumental tests. The estimated cost of defensive medicine is 10.5 per cent of the health budget, or 0.75 per cent of the Gross Domestic Product.
Another critical aspect concerns drug therapies. Medication errors occur in 5% of hospital admissions and are responsible for more than 98,000 deaths per year[2]. 39% occur at the time of treatment. 39% occur at the time of prescription[3]. The most common errors concern drug-drug interactions, incompleteness, wrong choice of drug, dosage and posology.
Adverse drug reactions (ADRs) are responsible for 3.1 – 6.2% of hospital admissions. Among inpatients, severe ADRs range from 2.2 to 4.6 per 100 admissions[4] [5] [6].
For every 1,000 admissions to the emergency department, it is estimated that 2.4 to 3.4 are due to severe ADRs. The cost of ADRs alone varies between €8,000 and €12,000 per 100 admissions. For a 700 bed hospital the cost is between 2.2 and 3.3 million euros per year.
CDSS can offer benefits in four areas:
- Reducing costs, e.g. by reducing the number of days spent in hospital due to ADRs or medical complications, fewer diagnostic tests, the use of cheaper drugs, and lower legal costs.
- Increased safety through increased awareness and better management of clinical risks.
- Increased effectiveness in terms of improved outcomes, reduced average hospital stay, reduced hospital readmissions and mortality.
- Improved clinical appropriateness through Evidence Based Practice, carried out following the guidelines and indications contained in medical evidence.
Theory or reality?
This question is often at the centre of discussions concerning CDSS and is also the subject of several studies, either on specific implementations of such tools in hospitals, or as systematic reviews that try to examine several studies and formulate a conclusion.
A first aspect that makes it very difficult to compare different studies is the great heterogeneity of CDSSs on the market, to which one must then add the different ways of integrating these tools with electronic medical records, which represents an extremely important aspect for the overall effectiveness of the CDSS.
Further complicating the picture are the differences between the various areas of application of these tools, the attitude of the doctors who use them, the possible “contamination” of results when the study involves the division of patients into two groups – intervention and control – in which doctors only have the CDSS in the former but “learn” and “apply” the suggestions made to the latter.
There are studies that find a positive impact of the CDSS and others that find no statistically significant improvement on outcome or cost reduction.
Among the former, I would like to mention the first study conducted in Italy, at the hospital of Vimercate, by means of a randomised clinical trial involving about 6,500 patients, which demonstrated the impact of CDSS on doctors’ decisions and a slight reduction in the average hospital stay (0.4 days). You can find the study here.
Positive results are more frequent in restricted or specialised areas, such as the control of drug prescriptions, the diagnosis of certain forms of cancer or the optimisation of therapies for chronic patients, e.g. diabetes mellitus. At the diagnostic level, there are several studies showing that non-knowledge-based CDSSs perform equally or in some cases better than human ones.
An interesting review of the results obtained by CDSSs can be found in “An overview of clinical decision support systems: benefits, risks, and strategies for success” published in npj Digital Medicine 3, which you can read here.
Limitations and problems of CDSSs
Like all systems, CDSSs have their critical aspects. A first problem is the impact that consulting the CDSS can have on clinical practice in terms of time and physician concentration. For this reason, the most common approach is to discreetly notify the user of the presence of information, often using icons or colours to express the level of importance, then leaving it up to the physician to deepen and read what is suggested (“discreet push” model). Less useful are those integrations where the CDSS has to be activated without having any prior information (“pull model”, such as HL7 Infobutton), since the physician may not know that he/she needs help from the CDSS, or those where the CDSS interrupts the physician’s work through windows that overlap the electronic medical record (“invasive push model”).
Another very common problem is ‘alert fatigue’ caused by a large number of messages that the doctor does not share or does not consider important. This risk can be reduced by improving the precision of the messages, for example by better specifying the context to which they refer or by making an a priori selection of the most important ones.
The user interface of the CDSS is also very important. Systems that merely provide lists of suggestions, in text form, are ineffective and time consuming to operate. It is not only essential that the content is produced to be read at the ‘point of care’, i.e. that it is very concise and clear, without ambiguity, but also that the interface uses infographics, icons and advanced knowledge representation models.
A not secondary aspect concerns then the application of the suggestions shown by the CDSS. The need to report the actions suggested by the CDSS in the CPOE or in the electronic medical record can be time-consuming and, in any case, error-prone. For this reason, the challenge in new generation CDSSs is to make knowledge “actionable”, avoiding data entry by doctors. This requires further ‘processing’ of knowledge, its transposition into codified actions and a much ‘tighter’ level of integration with clinical systems than is usually the case for classical integrations.
Conclusions
Since it is not possible to lump everything together, the evaluation of the advisability of introducing a CDSS into one’s own information system cannot be separated from a careful analysis of the product chosen and the way in which it integrates with the electronic medical record.
A priori critical judgements or unconditional trust in these tools are two attitudes that should be avoided. CDSSs move up the agenda and the value that an information system can have and it is no coincidence that they are, for example, a requirement for obtaining level 6 of the HIMSS EMRAM model.
Bibliography
- For example: McGlynn EA, Asch SM, Adams J, Keesey J, Hicks J, De Cristofaro A et al. The quality of health care delivered to adults in the United States. N Engl J Med. 2003; 348(26): 2635-45
- James JT. A new, evidence-based estimate of patient harms associated with hospital care. J Patient Saf. 2013 Sep;9(3):122-8
- Leape LL, Bates DW, Cullen DJ et al, System analysis of adverse drug events, JAMA 1995, 274:35-43
- Bates DW, Spell N, Cullen DJ, Burdick E, Laird N, Petersen LA, Small SD, Sweitzer BJ, Leape LL. The costs of adverse drug events in hospitalized patients. Adverse Drug Events Prevention Study Group. JAMA. 1997 Jan 22-29;277(4):307-11
- R. Bordet, S. Gautier, H. Le Louet, B. Dupuis, J. Caron. Analysis of the direct cost of adverse drug reactions in hospitalised patients. European Journal of Clinical Pharmacology – March 2001, Volume 56, Issue 12, pp 935–941
- Claudia Giardina, Paola M. Cutroneo, Eleonora Mocciaro, Giuseppina T. Russo, Giuseppe Mandraffino, Giorgio Basile, Franco Rapisarda, Rosarita Ferrara, Edoardo Spina, Vincenzo Arcoraci. Adverse Drug Reactions in Hospitalized Patients: Results of the FORWARD (Facilitation of Reporting in Hospital Ward) Study. Front. Pharmacol., 11 April 2018
Digital-health 