Supervised machine learning for the prediction of infection on

admission to hospital: a prospective observational cohort study

TM RAWSON1,2 *, B HERNANDEZ3, LSP MOORE1,2, O BLANDY1, P HERRERO3, M GILCHRIST2, A

GORDON4, C TOUMAZOU3, S SRISKANDAN1,2, P GEORGIOU3, AH HOLMES1,2

Affiliations:

1. National Institute for Health Research Health Protection Research Unit in Healthcare Associated Infections

and Antimicrobial Resistance, Imperial College London, Hammersmith Campus, Du Cane Road, London.

W12 0NN. United Kingdom.

2. Imperial College Healthcare NHS Trust, Hammersmith Hospital, Du Cane Road, London.W12 0HS. United

Kingdom

3. Department of Electrical and Electronic Engineering, Imperial College London, South Kensington Campus,

London, SW7 2AZ, United Kingdom

4. Section of Anaesthetics, Pain Medicine & Intensive Care, Imperial College London, South Kensington

Campus, SW7 2AZ, United Kingdom

Corresponding author *:

Dr Timothy M Rawson, Health Protection Research Unit in Healthcare Associated Infections & Antimicrobial

Resistance, Hammersmith Hospital, Du Cane Road, London.W12 0NN. United Kingdom.

Email: timothy.rawson07@imperial.ac.uk

Telephone: 02033132732.

Running Title: Machine learning for infection diagnosis

1

Abstract

Background: Infection diagnosis can be challenging, relying on clinical judgement and non-specific

markers of infection. We evaluated a supervised machine learning (SML) algorithm for diagnosing

bacterial infection using routinely available blood parameters on presentation to hospital.

Method: A SML algorithm was developed to classify cases into infection versus no infection using

microbiology records and six available blood parameters (C-reactive protein, white cell count, bilirubin,

creatinine, alanine transaminase, and alkaline phosphatase) from 160,203 individuals. A cohort of

patients admitted to hospital over a six month period had their admission blood parameters

prospectively inputted into the SML algorithm. They were prospectively followed from admission to

classify those who fulfilled clinical case criteria for a community bacterial infection within 72-hours of

admission using a pre-determined definition. Predictive ability was assessed using receiver-operator-

characteristics (ROC) with cut-off values for optimal sensitivities and specificity explored.

Results: 104 individuals were included prospectively. The median (range) cohort age was 65 (21-98)

years. The majority of individuals were female (56/104;54%). 36/104 (36%) were diagnosed with

infection in the first 72-hours of admission. Overall, 44/104 (42%) individuals had microbiological

investigations performed. Treatment was prescribed for 33/36 (92%) of infected individuals and 4/68

(6%) of those with no identifiable bacterial infection. Mean (SD) likelihood estimates for those with and

without infection were significantly different. The infection group had a likelihood of 0.80 (0.09) and the

non-infection group 0.50 (0.29) (p<0.01; 95%CI:0.20–0.40). ROC AUC was 0.84 (95%CI:0.76–0.91).

Conclusion: A SML algorithm was able to diagnose infection in individuals presenting to hospital

using routinely available blood parameters.

Abstract: 249

Word count: 3654

2

Introduction

The diagnosis and empirical management of infection in hospital can be challenging,

particularly on admission. Diagnosis of infection generally relies on clinical acumen,

physiological parameters, and non-specific markers of inflammation, such as C-Reactive

Protein (CRP).1,2

Recently, there has been a focus on the early recognition and management of sepsis, which

forms the severe end of the infection spectrum that presents in hospital.3,4 For the

recognition of sepsis there are now a wide range of clinical decision tools, including the

National Early Warning Score 2 (NEWS 2) in the UK 5–13 and electronic decision support,

such as sepsis alert systems.8,14,15 These tools aim to support the early detection and

management of sepsis, which has been shown to reduce mortality in this condition.16,17

These decision support tools focus on the recognition of deterioration in physiological

parameters, such as blood pressure and respiratory rate.18 The implementation of the NEWS

2 score in the emergency department has demonstrated good sensitivity for recognition of

critically ill patients with sepsis.5,18,19

Outside of ‘sepsis’, there is little support for clinicians to make decisions about infection

diagnosis and management. For example, the diagnosis of urinary tract infections (UTI) and

respiratory tract infections can be more nuanced.20–22 Often the clinician does not have

specific evidence of infection, such as temperature or radiological evidence, and are

therefore forced to make a decision based on reported symptoms, non-specific clinical signs,

and altered inflammatory markers.1,2 Antimicrobial therapy is often commenced in advance

of microbiological confirmation, exposing the patient to potentially inappropriate or

unnecessary treatment.

With advancements in computer processing and artificial intelligence, attempts have been

made to develop intelligent tools that can learn and automate decision support without the

need for human input, including the diagnosis of infection.23,24 However, concerns have been

3

raised about the unsupervised nature of such systems, which may allow error to go

unnoticed or even propagate.25 This is because an unsupervised system by definition learns

in an unguided fashion, being trained using data that is not associated with any predefined

outcome label (e.g. infection versus no infection). Given this argument, there is a need to

explore methods of developing supervised systems that can learn in a controlled way and be

utilised by the physician to support them making well informed, patient-centred decisions.25

Given the diagnostic challenges associated with these types of infective presentation and

the need to optimise antimicrobial prescribing; we investigated the development of a

supervised machine learning algorithm, embedded within a clinical decision support system

for the diagnosis of infection on presentation to hospital.26,27 This study aimed to use routine

blood test data to support clinicians in diagnosing infection occurring within 72-hours of

presentation to hospital.

4

Method

Study setting

This study took place at Imperial College Healthcare NHS Trust (ICHNT), comprising three

university teaching hospitals. The hospital network shares an electronic health record

system, infection management policies, and staff whom work across sites. The study took

place between October 2017 and March 2018. Two researchers (TMR and BH) were

responsible for collection and primary analysis of data. The study was reported following the

strengthening the reporting of observational studies in epidemiology (STROBE) criteria for

observational cohort studies.28

Clinical decision support system

A clinical decision support system (CDSS), called Enhanced Personalised Integrated Care

for Infection Management at the Point-of-Care (EPIC IMPOC), was incorporated into ICHNT

electronic health record system. This CDSS contained machine learning modules designed

to support antimicrobial selection, dose optimisation tools, and a patient engagement

module. All of the designed modules utilised routinely available data to provide individualised

infection management recommendations to healthcare professionals.

Supervised machine learning algorithm

The development and cross validation of supervised machine learning algorithms for the

inference of infection using routinely available microbiology and blood test data has been

previously described in detail.26 Briefly, a Support Vector Machine (SVM) binary classifier

algorithm was developed and incorporated into the EPIC IMPOC CDSS for investigation

within this study following validation and pilot assessment. SVM classifiers are one of the

newest techniques within the field of supervised machine learning.29–31 SVM work by taking

data with categories already assigned to it (e.g. infection versus no infection) and then

maximise the distance, or “margin”, that sits either side of a linear divide that separates the

two classes of data (Figure 1).29–31 By maximising the distance between the two data

5

classes, SVM aims to reduce the expected error in estimations made by the algorithm. Once

the optimal way for separating the data classes is found (Figure 1), the data points which lie

on the edge of the divide, known as the support vector points, are selected and all other data

is discarded. These support vector points are then used for classification purposes when

new data are introduced to the algorithm.29–31 Whilst the improved accuracy of SVM is

desirable, there are also limitations. Firstly, it is not always possible to identify a linear divide

between two classifications. This often requires use of a transformed feature space, where a

Kernal function is added to the algorithm to facilitate classification.29–31

To develop the SVM algorithm, a range of relevant, electronically collected data was

identified and extracted to create a training data set. This included clinical bacterial

microbiology data were extracted from the central microbiology records for all clinical

samples received by North West London Pathology laboratory from 2009 to 2015. Blood test

parameters were also extracted for all patients within the Trust during this time period. To

select variables for linkage to microbiology records a three-step approach was taken. Firstly,

variables reported by physicians as being important during infection management were

identified.1 Secondly, two infection specialists were asked to review these variables and

corroborate the findings. Finally, relevant literature was reviewed to provide further evidence

in support of these selected variables.

Six variables were eventually selected based on their availability electronically and their use

in infection management. These variables were C-reactive protein (CRP), white cell count

(WCC), creatinine (Cr), alanine aminotransferase (ALT), bilirubin (BIL), alkaline phosphatase

(ALP).32–38 Lactate was also felt to be an important blood marker for inclusion, however, at

the time of development this was not routinely available for the majority of patients within the

electronic database.39–41 Furthermore, physiological parameters (heart rate, respiratory rate,

temperature, blood pressure, oxygen saturation) were not available electronically.4,42–44

Following selection of the variables, individual patient profiles were linked to bacterial

6

microbiology data. Linkage was performed in the following way. Initially all individual patient

blood test profiles (n = 1,251,830) were labelled as “no culture”. For individuals who had a

positive and confirmed bacterial microbiology results (n = >350,000) within 48-hours of a

blood test result, these were then labelled as “positive culture”. If further blood test results for

“positive culture” individuals were available before or after the 48-hour window, but in the

same admission, these were excluded. This yielded two groups of patient profiles that were

labelled either “no bacterial culture” or “positive bacterial culture”.

Curation of the dataset was then performed prior to further processing to remove corrupt

data. Corrupt data can be defined as erroneous, imprecise, or missing data.45–47 It has been

demonstrated to be important to address these issues within machine learning to ensure that

predictions by the system remain robust.48 This stage focused on:

1. Removal of outliers: This has been demonstrated to significantly increase the

robustness of machine learning tools.48 In this study, outliers are likely to be due to

human error in data input or erroneous results secondary to technical factors such as

diagnostic accuracy or contamination. We defined outliers using the IQR rule, which

takes any variables outside of 1.5 x IQR to be an outlier removing them from the

dataset.48

2. Missing data: Within this study we evaluated the impact of missing blood

parameters on the accuracy of the tool during cross validation as outlined below. Of

the six variables, the system provided stable results providing that ≥4 variables were

present.26 For training however, patient profiles were required to contain all 6

variables.

3. Class imbalance: Invariably, there was likely to be a class imbalance in numbers of

individuals between those with “no culture” and “positive culture”. To address this, an

approach called Synthetic Minority Over-Sampling Technique was used.48 This

approach combines under sampling of the majority classifier and oversampling the

7

minority classifier group and has been demonstrated to enhance the performance of

data that uses this.48

Following data curation, 160,203 individual patient profiles containing all six blood variables

were available to train and test the diagnostic ability of the SVM algorithm using a standard

10-fold cross validation approach.26 For the purpose of this study, the inference of likely

positive bacterial microbiology was taken as a proxy for inferring the likelihood of a bacterial

infection diagnosis in an individual patient.

Participant inclusion and follow up

To ensure sample size was adequate to power our analysis (see statistical analysis below)

four periods of data collection were undertaken between October 2017 and March 2018.

One researcher (TMR) identified the first 20-30 adult patients admitted to ICHNT on the

selected day using the hospital electronic health record system for inclusion in the study.

Paediatric, psychiatric admissions, and maternity patients were not included. Data collection

was planned to continue in cycles of recruitment until 100 or more patients had been

identified. Individuals’ blood parameters on admission to hospital were automatically

extracted by the CDSS and input into the supervised machine learning algorithm. This

provided an automated likelihood estimate for the diagnosis of bacterial infection. Individuals

were then prospectively followed by two researchers (TMR and BH) until discharge to

determine those who went on to be diagnosed with a community acquired bacterial infection

within this first 72-hours of admission. The timeframe of 72-hours was selected as this

represents our local definition for community acquired infection given that the SVM was

developed to predict infection based on blood results from admission to hospital. Routine

electronic data for individuals were anonymously extracted for analysis including

demographic, laboratory, clinical, treatment, and outcome parameters.

8

The presence of infection was assessed using a pre-determined clinical case definition.

Assessment was preformed independently by two researchers (TMR & BH). The criteria

defining infection were the presence of a clinically relevant microbiological culture with one

or more categories in keeping with a bacterial infection (i.e. the presence of temperature,

change in physiological parameters, raised inflammatory markers, radiological evidence, or

infective signs or symptoms). If microbiological culture was not available then two or more

categories were required for confirmation. Where antimicrobial therapy was prescribed, but

these criteria were not present or disagreement occurred; the researcher’s opinion was

triangulated by an independent infection specialist and the local antimicrobial policy

reviewed to compare the treating clinician’s action to current guidelines within the hospital.

This study was an observational evaluation of the ability of the system to predict infection in

patients presenting to hospital only. The treating physicians were blinded to the use of the

system to avoid influencing the diagnosis of infection, the request of blood tests, and

management decisions made.

Statistical analysis

Analysis was performed using SPSS 24.0 (IBM, Chicago). Figures were plotted using R (R

core team 2013) and Igor Pro7.0 (Wavemetrics, OR, USA). Performance of the SVM

classifier algorithm was assessed using the receiver-operator-characteristics (ROC) AUC

analysis. A ROC AUC was used to assess the sensitivity and specificity of the supervised

machine learning tool, with an optimal cut-off value explored to maximise the systems

sensitivity and specificity.49,50

To estimate the required sample size, a power calculation was performed. This calculation

used the methodology proposed by Bujang and Adnan for estimating a minimum sample

size when performing sensitivity and specificity analysis.51 For this study, we set the power at

80% and significance value at < 0.05. Based on pilot work, we set our null hypothesis value

as 0.70 for both sensitivity and specificity with an alternative hypothesis of 0.90 for both,

9

respectively. The prevalence of patients within our cohort likely to be admitted with infections

/ started on antibiotics during the admission process was estimated at approximately 50 -

60% based on point-prevalence work within ICHNT. Using the estimates made by Bujang

and Adnan, we aimed to include at least 62 individuals, 31 of which went on to develop

infections to ensure that the study was appropriately powered.51

For statistical analysis of descriptive data t-tests, Mann - Whitney U, and Chi-squared were

used on parametric and non-parametric data, where appropriate.

Ethics

Ethical approval for this study was granted by London-Chelsea Regional Ethics Committee

(REC: 17/LO/0047) and local service evaluation procedures were followed (registered

SE113). All CDSS features were developed in line with the current directives of the Data

Protection Act 1998, The Privacy and Electronic Communications Directive 2003, and the

EU Directive 2006/24/EC for data retention.52,53 Internally, approval for the system to transfer

and anonymise patient identifiable data (Caldicott approval) was granted by ICHNT

Information Governance (Ref. 23307 / 726505).

10

Results

Summary of the cohort

Between October 2017 and March 2018, 104 individuals were included. Median age (range)

was 65 (21-98) years. The majority were female (56/104; 54%) and admitted with primarily

medical diagnoses (79/104; 76%). Blood tests were ordered by the treating clinician without

interference from researchers. All individuals had at least 4/6 blood parameters available on

admission. All individuals had WCC and Cr available on the day of admission. CRP was not

requested by the admitting clinician in 8/104 (8%), ALT in 7/104 (7%), BIL in 4/104 (4%), and

ALP in 2/104 (2%). No individuals were excluded from the analysis.

Of the 104 included in this study, 36 (36%) were diagnosed with true infection within 72-

hours of admission. During this time period 44/104 (44%) individual patients had

microbiological sampling performed. Of the 104 individuals, 25 (24%) had blood culture

performed with 8/25 (32%) growing isolates. Sputum cultures were sent in 5/104 (5%) with

2/5 (40%) growing an isolate. Urine culture was sent in 14/104 (14%) with 8/14 (57%)

growing isolates. Other types of microbiological culture were sent in 12/104 (12%) of cases

with 8/12 (67%) returning positive isolates. Antibiotics were prescribed to 37/104 (37%)

individuals, with 31/37 (84%) being intravenous therapy. Two individuals died during the

hospital stay and after 72-hours in hospital. One of these was related to infection, the other

an underlying malignancy.

Table 1 compares management of those meeting the clinical case definition for bacterial

infection (n = 36) to those without (n = 68). Those with infection were older (71 vs. 61 years;

p < 0.01, 95%CI: 3.41 – 16.94) and had a greater median (IQR) CRP at 96 (47 – 245) mg / L

versus 5 (2 – 13) mg / L (p < 0.01); WCC at 12 (6 – 16) x109 / L versus 8 (7 – 10) x109 / L (p

= 0.04); and ALP 101 (76 – 140) versus 86 (65 – 107) IU / L (p = 0.01). Those with infection

had positive microbiological cultures in 19/36 (52%) of cases. Those without infection had

positive microbiological culture in 4/68 (6%) cases. Two of these were urine samples in

11

females with no symptoms, one was an MRSA screening swab in a patient who was

undergoing suppressive treatment, and a skin swab that grew a Gram-positive cocci and

was not further cultured as it was felt to be clinically insignificant. Those with bacterial

infection received antimicrobial therapy in 33/36 (92%) of cases, 28/33 (78%) being

intravenous therapy. Of those without evidence of bacterial infection, 4/68 (6%) were

prescribed antimicrobials with 3/4 (75%) receiving intravenous antimicrobials.

Supervised machine learning for the prediction of infection

On input of patient blood parameters during admission to hospital the supervised machine

learning algorithm provided probabilities of infection for each individual included within the

study. Of those that went on to develop an infection, the mean (SD) likelihood estimate was

0.80 (0.09) compared to 0.50 (0.29) in those without infection.

Figure 2 demonstrates the ROC of the supervised machine learning algorithm for the

diagnosis of infection with 72-hours of admission to hospital. The ROC AUC was 0.84

(95%CI: 0.76 – 0.91). Analysis of cut off-values for the ROC analysis demonstrated that

setting a cut-off estimate at 0.81 would demonstrate a sensitivity of 89% and specificity of

63%. A likelihood estimate cut-off at 0.82 demonstrated a sensitivity of 44% and specificity of

93%. Using both cut-off values, the three patients with bacterial infection but not prescribed

antibiotics were classified as infected. For patients prescribed antibiotics but not meeting the

clinical case definition for infection; a cut off of 0.81 meant that one of the four patients was

classified not infected. Using a cut off of 0.82 three out of the four of these individuals were

classified non-infected.

12

Discussion

Within our study, a supervised machine learning algorithm, embedded within a clinical

decision support system was able to identify bacterial infection using a small number of

blood parameters that are routinely available on admission to hospital. The type of decision

support provided by this system may offer a number of potential benefits to traditional

decision support mechanisms surrounds the investigation, diagnosis, and management of

infection.

This study highlights the challenge of diagnostic uncertainty in determining infection. In this

study, physicians performed well at diagnosing bacterial infection (92% sensitivity, 94%

specificity) in a cohort of general patients admitted to hospital. However, there were a small

proportion of individuals who were not identified by physicians as having a bacterial infection

upon admission to hospital (3/36; 8%). These individuals did therefore not receive

appropriate investigation for or treatment of infection. Similarly, there were a small number

who did not have evidence of bacterial infection but received antimicrobial therapy

regardless (4/68; 6%). Whilst only a small proportion of the total, when contextualised for all

admissions to our hospital Trust over a 1-year period (~50,000 patients / year), this is likely

to equate to a large number of patients either being under treated or exposed to

unnecessary antimicrobial therapy.

Furthermore, when considering those with bacterial infection, the process of microbiological

sampling was often neglected in advance of commencement of antimicrobial therapy. Of the

36 individuals with bacterial infection, three (8%) had no microbiological sampling performed

of any kind prior to or after commencement of antibiotic treatment. Only 19 (53%) patients

with bacterial infection yielded a positive culture. This means that targeted antimicrobial

therapy can often not be prescribed, given to the paucity of microbiological confirmation.

This has the potential to impact on the practice of prescribing meaning that empirical therapy

is often continued using broad spectrum antimicrobial agents, due to diagnostic

uncertainty.54 With the implementation of systems such as supervised machine learning

13

algorithms, decision support may be provided to healthcare providers indicating those

individuals with high likelihood of infection, potentially driving enhanced microbiological

sampling prior to clinical review / initiation of antimicrobial therapy. This would include

supporting decision making in more challenging situations, such as the individuals not

treated for bacterial infection on admission in this study.

Finally, with a growing focus on the development of decision support mechanisms for

antimicrobial management in limited resource settings, the role of an algorithm that can work

with limited numbers of variables may be attractive across a range of healthcare settings.

Within this study, only a single sample of four to six blood parameters proved appropriate for

the system to predict the likelihood of infection.

With the development of more advanced supervised machine learning tools, these classifier

systems’ may also demonstrate the ability to provide a broader scope of decision support for

antimicrobial prescribing. Similar classifier systems have already been explored for the

prediction of causative organism and serotype of viral infections. For example, TREAT, a

decision support system containing causal-probabilistic-networks has previously explored

the ability to predict the likelihood of blood stream infection and causative organism.23,24

Other examples include the use of Decision Tree Classifiers using binary classification

applied to blood test parameters to predict the diagnosis of Chlamydia pneumoniae 32 and

hepatitis B/C virus.33 These approaches have yielded mixed results, demonstrating an

overall potential for supporting decision making using these techniques. However, with the

development of more advanced techniques, such as SVM, and availability of population level

datasets may provide the ability to be able to expand classifiers to other aspects of decision

making, including selection of the optimal length of treatment and intravenous versus oral

delivery. Moreover, with concerns over the role of unsupervised machine learning tools and

the need for artificial intelligence to be developed to provide better evidence in support of

clinician led, patient-centred decision making; this approach could potentially provide better

14

evidence in support of infection management than is currently available for clinicians

assessing the patient on presentation to hospital.

This study has several limitations that must be considered. Firstly, although powered to

demonstrate significance during sensitivity and specificity analysis, a relatively small cohort

of patients were included in this study. Furthermore, the external validity is unclear given that

our algorithm was developed and tested from on data from three geographically similar

hospitals in West London. Secondly, this study was observational only, meaning that the

impact of the algorithm on clinician decision making was not assessed. Thirdly, although

several other variables were identified as desirable during the development of the algorithm

(such as lactate and physiological parameters) these variables were not electronically

available for all individuals during development of the algorithm. This is subsequently being

addressed as part of the future direction of this project. Fourth, the identification and

diagnosis of true bacterial infection in the absence of microbiological evidence can be

challenging. Whilst a systematic approach was taken to determine infection in this study, the

clinical case definitions used may have influenced our findings. Finally, direct comparison

with current clinical decision support tools is not practical, given that they focus primarily on

the detection of sepsis only.

Future work will now explore the integration of this system with other forms of decision

support housed within EPIC IMPOC and evaluate the direct impact of the algorithm on

infection management in secondary care.

15

Conclusion

A supervised machine learning algorithm embedded within a clinical decision support tool

can support the diagnosis of infection in patients presenting to hospital. Future work must

explore the potential impact of classifier systems like this on the decision making of clinicians

who manage infections as part of multi-modal decision support packages.

16

Declarations

Transparency declarationsAll authors have no conflicts of interest to declare.

Ethics approval and consent to participateThe study protocol for Enhanced, Personalized and Integrated Care for Infection Management at Point of Care

(EPIC IMPOC) was reviewed and approved by the Chelsea Regional Ethics Committee (REC) REF: 17/LO/0047.

FundingThis report is independent research funded by the National Institute for Health Research Invention for Innovation

(i4i) grant, Enhanced, Personalized and Integrated Care for Infection Management at Point of Care (EPIC

IMPOC), II-LA-0214-20008.

Availability of data and materialThe datasets used and/or analysed during the current study are available from the corresponding author on

reasonable request where not presented in the manuscript or figure.

Author contributions

TMR, LSPM, and AH developed the initial concept for the study. BH developed and validated the machine

learning algorithm used within the study. TMR, BH, LSPM, and OB collected and performed the initial analysis of

data presented in this study. TMR drafted the initial manuscript. All authors contributed significantly to the

revision and finalisation of the manuscript for submission to this journal.

AcknowledgementsThe authors would like to thank members of Imperial College NHS Healthcare Trust who participated in the study.

The authors would also like to acknowledge the National Institute of Health Research Imperial Biomedical

Research Centre and the National Institute for Health Research Health Protection Research Unit (NIHR HPRU) in

Healthcare Associated Infection and Antimicrobial Resistance at Imperial College London in partnership with

Public Health England and the NIHR Imperial Patient Safety Translational Research Centre. The views expressed

in this publication are those of the authors and not necessarily those of the NHS, the National Institute for Health

Research or the UK Department of Health.

17

References

1. Rawson TM, Charani E, Moore LSP, et al. Mapping the decision pathways of acute

infection management in secondary care among UK medical physicians: a qualitative study.

BMC Med 2016; 14: 208.

2. Charani E, Tarrant C, Moorthy K, Sevdalis N, Brennan L, Holmes AH. Understanding

antibiotic decision making in surgery—a qualitative analysis. Clin Microbiol Infect 2017; 23:

752–60.

3. Reinhart K, Daniels R, Kissoon N, et al. Recognizing Sepsis as a Global Health Priority —

A WHO Resolution. N Engl J Med 2017; 377: 414–7.

4. Raith EP, Udy AA, Bailey M, et al. Prognostic Accuracy of the SOFA Score, SIRS Criteria,

and qSOFA Score for In-Hospital Mortality Among Adults With Suspected Infection Admitted

to the Intensive Care Unit. JAMA 2017; 317: 290.

5. Kmietowicz Z. Identify sepsis in patients by using early warning scores, doctors are urged.

BMJ 2015; 351: h6237.

6. Saeed M, Villarroel M, Reisner AT, et al. Multiparameter Intelligent Monitoring in Intensive

Care II: A public-access intensive care unit database*. Crit Care Med 2011; 39: 952–60.

7. Henry KE, Hager DN, Pronovost PJ, et al. A targeted real-time early warning score

(TREWScore) for septic shock. Sci Transl Med 2015; 7: 299ra122-299ra122.

8. Sawyer AM, Deal EN, Labelle AJ, et al. Implementation of a real-time computerized

sepsis alert in nonintensive care unit patients*. Crit Care Med 2011; 39: 469–73.

9. Umscheid CA, Betesh J, VanZandbergen C, et al. Development, implementation, and

impact of an automated early warning and response system for sepsis. J Hosp Med 2015;

10: 26–31.

10. Westphal GA, Lino AS. Systematic screening is essential for early diagnosis of severe

sepsis and septic shock. Rev Bras Ter Intensiva 2015; 27.  96–101.

11. Singer M, Deutschman CS, Seymour CW, et al. The Third International Consensus

Definitions for Sepsis and Septic Shock (Sepsis-3). JAMA 2016; 315: 801.

12. Fleischmann C, Scherag A, Adhikari NKJ, et al. Assessment of Global Incidence and

18

Mortality of Hospital-treated Sepsis. Current Estimates and Limitations. Am J Respir Crit

Care Med 2016; 193: 259–72.

13. Amorim FF, Santana ANC. Automated early warning system for septic shock: the new

way to achieve intensive care unit quality improvement? Ann Transl Med 2017; 5: 17.

14. Mani S, Ozdas A, Aliferis C, et al. Medical decision support using machine learning for

early detection of late-onset neonatal sepsis. J Am Med Inform Assoc 2014; 21: 326–36.

15. Nelson JL, Smith BL, Jared JD, Younger JG. Prospective Trial of Real-Time Electronic

Surveillance to Expedite Early Care of Severe Sepsis. Ann Emerg Med 2011; 57: 500–4.

16. Ferrer R, Martin-Loeches I, Phillips G, et al. Empiric Antibiotic Treatment Reduces

Mortality in Severe Sepsis and Septic Shock From the First Hour: Results From a Guideline-

Based Performance Improvement Program. Crit Care Med 2014: 42 (8); 1749–55.

17. Seymour CW, Gesten F, Prescott HC, et al. Time to Treatment and Mortality during

Mandated Emergency Care for Sepsis. N Engl J Med 2017; 376: 2235–44.

18. Corfield AR, Lees F, Zealley I, et al. Utility of a single early warning score in patients with

sepsis in the emergency department. Emerg Med J 2014; 31: 482–7.

19. Royal College of Physicians. National Early Warning Score (NEWS) 2: Standardising the

assessment of acute-illness severity in the NHS. Updated report of a working party. London:

RCP; 2017.

20. Ottosen J, Evans H. Pneumonia Challenges in the Definition, Diagnosis, and

Management of Disease. Surg Clin N Am 2014; 94: 1305–17.

21. Rowe TA, Juthani-Mehta M. Urinary tract infection in older adults. Aging health 2013; 9:

519–28.

22. van Duin D. Diagnostic Challenges and Opportunities in Older Adults With Infectious

Diseases. Clin Infect Dis 2012; 54: 973–8.

23. Paul M, Andreassen S, Nielsen AD, et al. Prediction of bacteremia using TREAT, a

computerized decision-support system. Clin Infect Dis 2006; 42: 1274–82.

24. Paul M, Nielsen AD, Goldberg E, et al. Prediction of specific pathogens in patients with

sepsis: evaluation of TREAT, a computerized decision support system. J Antimicrob

19

Chemother 2007; 59: 1204–7.

25. Verghese A, Shah NH, Harrington RA. What This Computer Needs Is a Physician.

JAMA 2018; 319: 19.

26. Hernandez B, Herrero P, Rawson TM, et al. Supervised learning for infection risk

inference using pathology data. BMC Med Inform Decis Mak 2017; 17: 168.

27. Hernandez B, Herrero P, Rawson TM, et al. Data-driven Web-based Intelligent Decision

Support System for Infection Management at Point-Of-Care: Case-Based Reasoning

Benefits and Limitations. Biostec 2017 2017: 119.

28. von Elm E, Altman DG, Egger M, et al. The Strengthening the Reporting of

Observational Studies in Epidemiology (STROBE) statement: guidelines for reporting

observational studies. Lancet 2007; 370: 1453–7.

29. Kausar N, Belhaouari Samir B, Abdullah A, et al. A Review of Classification Approaches

Using Support Vector Machine in Intrusion Detection. In: Springer, Berlin, Heidelberg, 2011;

24–34.

30. Shawe-Taylor J, Sun S. A review of optimization methodologies in support vector

machines. Neurocomputing 2011; 74: 3609–18.

31. Platt JC. Probabilistic Outputs for Support Vector Machines and Comparisons to

Regularized Likelihood Methods. In: Advances in large margin classifiers., 1999; 61--74.

32. Richardson A, Hawkins S, Shadabi F, et al. Enhanced laboratory diagnosis of human

Chlamydia pneumoniae infection through pattern recognition derived from pathology

database analysis. In: Third IAPR International Conference on Pattern Recognition in

Bioinformatics (PRIB 2008). Monash University, 2008; 227–34.

33. Richardson AM, Lidbury BA. Infection status outcome, machine learning method and

virus type interact to affect the optimised prediction of hepatitis virus immunoassay results

from routine pathology laboratory assays in unbalanced data. BMC Bioinformatics 2013; 14:

206.

34. Seigel TA, Cocchi MN, Salciccioli J, et al. Inadequacy of Temperature and White Blood

Cell Count in Predicting Bacteremia in Patients with Suspected Infection. J Emerg Med

20

2012; 42: 254–9.

35. Minemura M, Tajiri K, Shimizu Y. Liver involvement in systemic infection. World J

Hepatol 2014; 6: 632–42.

36. Shmuely H, Pitlik S, Drucker M, et al. Prediction of mortality in patients with bacteremia:

the importance of pre-existing renal insufficiency. Ren Fail 2000; 22: 99–108.

37. Sierra R, Rello J, Bailen M, et al. C-reactive protein used as an early indicator of

infection in patients with systemic inflammatory response syndrome. Intensive Care Med

2004; 30: 2038–45.

38. Kim WR, Flamm SL, Di Bisceglie AM, et al. Serum activity of alanine aminotransferase

(ALT) as an indicator of health and disease. Hepatology 2008; 47: 1363–70.

39. Filho RR, Rocha LL, Corrêa TD, et al. Blood Lactate Levels Cutoff and Mortality

Prediction in Sepsis—Time for a Reappraisal? a Retrospective Cohort Study. SHOCK 2016;

46: 480–5.

40. Blomkalns AL. Lactate - a marker for sepsis and trauma. 2006.

http://emcreg.org/publications/monographs/acep/2006/alb_acep2006.pdf

41. Trzeciak S, Dellinger RP, Chansky ME, et al. Serum lactate as a predictor of mortality in

patients with infection. Intensive Care Med 2007; 33: 970–7.

42. Williams JM, Greenslade JH, McKenzie J V., et al. Systemic Inflammatory Response

Syndrome, Quick Sequential Organ Function Assessment, and Organ Dysfunction. Chest

2017; 151: 586–96.

43. Finkelsztein EJ, Jones DS, Ma KC, et al. Comparison of qSOFA and SIRS for predicting

adverse outcomes of patients with suspicion of sepsis outside the intensive care unit. Crit

Care 2017; 21: 73.

44. Marik PE, Taeb AM. SIRS, qSOFA and new sepsis definition. J Thorac Dis 2017; 9:

943–5.

45. Johnson AEW, Ghassemi MM, Nemati S, et al. Machine Learning and Decision Support

in Critical Care. Proc IEEE 2016; 104: 444–66.

46. Borovicka T, Jirina M, Kordik P, et al. Selecting Representative Data Sets. In: Karahoca

21

A, ed. Advances in Data Mining Knowledge Discovery and Applications. InTech, 2012.

47. Pantic M. Introduction to Machine Learning. London; 2016.

48. Osborne JW, Overbay A. The power of outliers (and why researchers should ALWAYS

check for them) - Practical Assessment, Research &amp; Evaluation. Pract Assessment,

Res Eval 2004; 9. http://pareonline.net/getvn.asp?v=9&n=6

49. Hajian-Tilaki K. Receiver Operating Characteristic (ROC) Curve Analysis for Medical

Diagnostic Test Evaluation. Casp J Intern Med 2013; 4: 627–35.

50. Florkowski CM. Sensitivity, specificity, receiver-operating characteristic (ROC) curves

and likelihood ratios: communicating the performance of diagnostic tests. Clin Biochem Rev

2008; 29 Suppl 1: S83-7.

51. Bujang MA, Adnan TH. Requirements for Minimum Sample Size for Sensitivity and

Specificity Analysis. J Clin Diagnostic Res 2016; 10. YE01–YE06

52. DIRECTIVE 2006/24/EC OF THE EUROPEAN PARLIAMENT AND OF THE COUNCIL.

Off J Eur Union 2006.

53. The Privacy and Electronic Communications (EC Directive) Regulations 2003.

54. Morency-Potvin P, Schwartz DN, Weinstein RA. Antimicrobial Stewardship: How the

Microbiology Laboratory Can Right the Ship. Clin Microbiol Rev 2017; 30: 381–407.

22

Figure 1. Diagrammatic representation of the concept of Support Vector Machines.

23

Legend: Support Vector Machine (SVM) classifiers are trained using datasets where the class (or outcome) are already known. In this case, individual patients will have infection or no infection. The SVM aims to find a dividing line between the two data classes (called the hyperplane) in the training data set. Data points forming the margins of the two data classes between the hyperplane are then retained. These retained datapoints are known as support vectors and allow prediction of the classification of a new data set when it is introduced to the SVM classifier.

Adapted from Nugroho et al. Proceeding of Indonesian Scientific Meeting in Central Japan, 2003.

Table 1. Comparison of those with and without infection

Demographics   Infection n = 36 No infection n = 68Age Mean (SD) 71 (14) 61 (18)Gender Female (%) 19 (53) 37 (54)

Blood parameter resultsC-Reactive Protein (mg / L) Mean (SD) 142 (124) 12 (18)

Median (IQR) 96 (47-245) 5 (2-13)

White Cell Count (x109 / L) Mean (SD) 12 (7) 9 (4)Median (IQR) 12 (6-16) 8 (7-10)

Alanine Transaminase (IU /L) Mean (SD) 39 (38) 72 (228)Median (IQR) 26 (15-54) 23 (12-37)

Alkaline Phosphatase (IU /L) Mean (SD) 127 (81) 121 (176)Median (IQR) 101 (76-140) 86 (65-107)

Bilirubin (µmol / L) Mean (SD) 21 (21) 18 (22)Median (IQR) 11 (8-31) 10 (7-20)

Creatinine (µmol / L) Mean (SD) 180 (267) 123 (133)Median (IQR) 89 (68-171) 77 (66-115)

Clinical investigations performedBlood cultures taken n = (%) 23 (64) 2 (3)

Positive blood culture n = (%) 8 (35) 0 (0)Sputum cultures taken n = (%) 5 (14) 0 (0)

Positive sputum culture n = (%) 2 (40) 0 (0)Urine culture taken n = (%) 11 (31) 3 (4)

Positive urine culture n = (%) 7 (66) 2 (67)*Other culture taken n = (%) 8 (22) 4 (6)

Positive other culture$ n = (%) 6 (75) 2 (50)*

Management and outcomeGiven antibiotics n = (%) 33 (92) 4 (6)

IV antibiotics given n = (% of 37) 28 (78) 3 (75)

Died during admission n = (%) 2 (6) 0 (0)

24

Legend: SD = standard deviation; n = number; % = percentage; IV = intravenous; IQR = interquartile range; mg / L = milligrams per litre; L = litre; IU / L = International Units per litre; µmol / L = micromoles per litre

$ = Other positive cultures: Wound swab x 3 (Serratia marcescens, MRSA & beta-haemolytic streptococcus, Staphylococcus aureus), Joint fluid (Staphylococcus aureus), Biliary drain fluid (E.coli), Pleural fluid (Staphylococcus aureus)

* = Clinically insignificant; Urine cultures = asymptomatic bactiuria (E.coli) x2; MRSA screening swab; Unspecified wound swab (likely from a chronic venous ulcer that was not felt to be infected)

Figure 2. Receiver-operator-characteristics for the prediction of infection using a supervised

machine learning algorithm and routine blood parameters on admission to hospital.

25

Legend: ROC AUC = receiver operator characteristic area under the curveCut off values:Likelihood 0.812; Sensitivity = 89%, specificity = 63%Likelihood 0.820; Sensitivity = 44%, specificity = 93%