Review

Diagnostic accuracy and effectiveness of automated electronic sepsis alert systems: A systematic review

Corresponding Author

Anil N. Makam MD, MAS

Division of General Internal Medicine, University Texas Southwestern Medical Center, Dallas, Texas

Address for correspondence and reprint requests: Anil N. Makam, MD, 5323 Harry Hines Blvd., Dallas, TX 75390-9169; Telephone: 214-648-3272; Fax: 314-648-3232; E-mail: [email protected]Search for more papers by this author

Oanh K. Nguyen MD, MAS,

Oanh K. Nguyen MD, MAS

Division of General Internal Medicine, University Texas Southwestern Medical Center, Dallas, Texas

Search for more papers by this author

Andrew D. Auerbach MD, MPH,

Andrew D. Auerbach MD, MPH

Divsion of Hospital Medicine, University of California San Francisco, San Francisco, California

Search for more papers by this author

Anil N. Makam MD, MAS,

Corresponding Author

Anil N. Makam MD, MAS

Division of General Internal Medicine, University Texas Southwestern Medical Center, Dallas, Texas

Oanh K. Nguyen MD, MAS,

Oanh K. Nguyen MD, MAS

Division of General Internal Medicine, University Texas Southwestern Medical Center, Dallas, Texas

Search for more papers by this author

Andrew D. Auerbach MD, MPH,

Andrew D. Auerbach MD, MPH

Divsion of Hospital Medicine, University of California San Francisco, San Francisco, California

Search for more papers by this author

First published: 11 March 2015

https://doi.org/10.1002/jhm.2347

Citations: 55

Share a link

Email
Facebook
Twitter
LinkedIn
Reddit
Wechat

Abstract

BACKGROUND

Although timely treatment of sepsis improves outcomes, delays in administering evidence-based therapies are common.

PURPOSE

To determine whether automated real-time electronic sepsis alerts can: (1) accurately identify sepsis and (2) improve process measures and outcomes.

DATA SOURCES

We systematically searched MEDLINE, Embase, The Cochrane Library, and Cumulative Index to Nursing and Allied Health Literature from database inception through June 27, 2014.

STUDY SELECTION

Included studies that empirically evaluated 1 or both of the prespecified objectives.

DATA EXTRACTION

Two independent reviewers extracted data and assessed the risk of bias. Diagnostic accuracy of sepsis identification was measured by sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and likelihood ratio (LR). Effectiveness was assessed by changes in sepsis care process measures and outcomes.

DATA SYNTHESIS

Of 1293 citations, 8 studies met inclusion criteria, 5 for the identification of sepsis (n = 35,423) and 5 for the effectiveness of sepsis alerts (n = 6894). Though definition of sepsis alert thresholds varied, most included systemic inflammatory response syndrome criteria ± evidence of shock. Diagnostic accuracy varied greatly, with PPV ranging from 20.5% to 53.8%, NPV 76.5% to 99.7%, LR+ 1.2 to 145.8, and LR− 0.06 to 0.86. There was modest evidence for improvement in process measures (ie, antibiotic escalation), but only among patients in non–critical care settings; there were no corresponding improvements in mortality or length of stay. Minimal data were reported on potential harms due to false positive alerts.

CONCLUSIONS

Automated sepsis alerts derived from electronic health data may improve care processes but tend to have poor PPV and do not improve mortality or length of stay. Journal of Hospital Medicine 2015;10:396–402. © 2015 Society of Hospital Medicine

Supporting Information

References

1Torio CM, Andrews RM. National inpatient hospital costs: the most expensive conditions by payer, 2011: statistical brief #160. Healthcare Cost and Utilization Project (HCUP) Statistical Briefs. Rockville, MD: Agency for Healthcare Research and Quality; 2013.
Google Scholar
2Hall MJ, Williams SN, DeFrances CJ, Golosinskiy A. Inpatient care for septicemia or sepsis: a challenge for patients and hospitals. NCHS Data Brief. 2011;(62): 1–8.
PubMedWeb of Science®Google Scholar
3Martin GS, Mannino DM, Eaton S, Moss M. The epidemiology of sepsis in the United States from 1979 through 2000. N Engl J Med. 2003; 348(16): 1546–1554.
10.1056/NEJMoa022139
PubMedWeb of Science®Google Scholar
4Dellinger RP, Levy MM, Rhodes A, et al. Surviving sepsis campaign: international guidelines for management of severe sepsis and septic shock: 2012. Crit Care Med. 2013; 41(2): 580–637.
10.1097/CCM.0b013e31827e83af
PubMedWeb of Science®Google Scholar
5Rivers E, Nguyen B, Havstad S, et al. Early goal-directed therapy in the treatment of severe sepsis and septic shock. N Engl J Med. 2001; 345(19): 1368–1377.
10.1056/NEJMoa010307
CASPubMedWeb of Science®Google Scholar
6Pro CI, Yealy DM, Kellum JA, et al. A randomized trial of protocol-based care for early septic shock. N Engl J Med. 2014; 370(18): 1683–1693.
10.1056/NEJMoa1401602
CASPubMedWeb of Science®Google Scholar
7Turi SK, Von Ah D. Implementation of early goal-directed therapy for septic patients in the emergency department: a review of the literature. J Emerg Nurs. 2013; 39(1): 13–19.
10.1016/j.jen.2011.06.006
PubMedWeb of Science®Google Scholar
8Fong JJ, Cecere K, Unterborn J, Garpestad E, Klee M, Devlin JW. Factors influencing variability in compliance rates and clinical outcomes among three different severe sepsis bundles. Ann Pharmacother. 2007; 41(6): 929–936.
10.1345/aph.1K041
PubMedWeb of Science®Google Scholar
9Ferrer R, Artigas A, Levy MM, et al. Improvement in process of care and outcome after a multicenter severe sepsis educational program in Spain. JAMA. 2008; 299(19): 2294–2303.
10.1001/jama.299.19.2294
CASPubMedWeb of Science®Google Scholar
10Hooper MH, Weavind L, Wheeler AP, et al. Randomized trial of automated, electronic monitoring to facilitate early detection of sepsis in the intensive care unit*. Crit Care Med. 2012; 40(7): 2096–2101.
10.1097/CCM.0b013e318250a887
PubMedWeb of Science®Google Scholar
11Whiting PF, Rutjes AW, Westwood ME, et al. QUADAS-2: a revised tool for the quality assessment of diagnostic accuracy studies. Ann Intern Med. 2011; 155(8): 529–536.
10.7326/0003-4819-155-8-201110180-00009
PubMedWeb of Science®Google Scholar
12Owens DK, Lohr KN, Atkins D, et al. AHRQ series paper 5: grading the strength of a body of evidence when comparing medical interventions—agency for healthcare research and quality and the effective health-care program. J Clin Epidemiol. 2010; 63(5): 513–523.
10.1016/j.jclinepi.2009.03.009
PubMedWeb of Science®Google Scholar
13Meurer WJ, Smith BL, Losman ED, et al. Real-time identification of serious infection in geriatric patients using clinical information system surveillance. J Am Geriatr Soc. 2009; 57(1): 40–45.
10.1111/j.1532-5415.2008.02094.x
PubMedWeb of Science®Google Scholar
14Nelson JL, Smith BL, Jared JD, Younger JG. Prospective trial of real-time electronic surveillance to expedite early care of severe sepsis. Ann Emerg Med. 2011; 57(5): 500–504.
10.1016/j.annemergmed.2010.12.008
PubMedWeb of Science®Google Scholar
15Nguyen SQ, Mwakalindile E, Booth JS, et al. Automated electronic medical record sepsis detection in the emergency department. PeerJ. 2014; 2: e343.
10.7717/peerj.343
PubMedWeb of Science®Google Scholar
16Thiel SW, Rosini JM, Shannon W, Doherty JA, Micek ST, Kollef MH. Early prediction of septic shock in hospitalized patients. J Hosp Med. 2010; 5(1): 19–25.
10.1002/jhm.530
PubMedWeb of Science®Google Scholar
17Berger T, Birnbaum A, Bijur P, Kuperman G, Gennis P. A Computerized alert screening for severe sepsis in emergency department patients increases lactate testing but does not improve inpatient mortality. Appl Clin Inform. 2010; 1(4): 394–407.
10.4338/ACI-2010-09-RA-0054
CASPubMedWeb of Science®Google Scholar
18McRee L, Thanavaro JL, Moore K, Goldsmith M, Pasvogel A. The impact of an electronic medical record surveillance program on outcomes for patients with sepsis. Heart Lung. 2014; 43(6): 546–549.
10.1016/j.hrtlng.2014.05.009
PubMedWeb of Science®Google Scholar
19Sawyer AM, Deal EN, Labelle AJ, et al. Implementation of a real-time computerized sepsis alert in nonintensive care unit patients. Crit Care Med. 2011; 39(3): 469–473.
10.1097/CCM.0b013e318205df85
PubMedWeb of Science®Google Scholar
20Brun-Buisson C. The epidemiology of the systemic inflammatory response. Intensive Care Med. 2000; 26(suppl 1): S64–S74.
10.1007/s001340051121
PubMedWeb of Science®Google Scholar
21Nanji KC, Slight SP, Seger DL, et al. Overrides of medication-related clinical decision support alerts in outpatients. J Am Med Inform Assoc. 2014; 21(3): 487–491.
10.1136/amiajnl-2013-001813
PubMedWeb of Science®Google Scholar
22Siebig S, Kuhls S, Imhoff M, Gather U, Scholmerich J, Wrede CE. Intensive care unit alarms–how many do we need? Crit Care Med. 2010; 38(2): 451–456.
10.1097/CCM.0b013e3181cb0888
PubMedWeb of Science®Google Scholar
23Singal G, Currier P. How can we best use electronic data to find and treat the critically ill?*. Crit Care Med. 2012; 40(7): 2242–2243.
10.1097/CCM.0b013e318256b99b
PubMedWeb of Science®Google Scholar
24Ash JS, Sittig DF, Dykstra R, et al. Identifying best practices for clinical decision support and knowledge management in the field. Stud Health Technol Inform. 2010; 160(pt 2): 806–810.
PubMedGoogle Scholar
25Wright A, Phansalkar S, Bloomrosen M, et al. Best practices in clinical decision support: the case of preventive care reminders. Appl Clin Inform. 2010; 1(3): 331–345.
10.4338/ACI-2010-05-RA-0031
PubMedWeb of Science®Google Scholar

Citing Literature

Volume10, Issue6

June 2015

Pages 396-402

Filename	Description
jhm2347-sup-0001-suppinfo1.doc44 KB	Supplementary Information
jhm2347-sup-0002-suppinfo2.docx48.6 KB	Supplementary Information

Diagnostic accuracy and effectiveness of automated electronic sepsis alert systems: A systematic review

Abstract

BACKGROUND

PURPOSE

DATA SOURCES

STUDY SELECTION

DATA EXTRACTION

DATA SYNTHESIS

CONCLUSIONS

Supporting Information

References

Citing Literature

References

Information

About Wiley Online Library

Help & Support

Opportunities

Connect with Wiley

Diagnostic accuracy and effectiveness of automated electronic sepsis alert systems: A systematic review

Abstract

BACKGROUND

PURPOSE

DATA SOURCES

STUDY SELECTION

DATA EXTRACTION

DATA SYNTHESIS

CONCLUSIONS

Supporting Information

References

Citing Literature

References

Related

Information