Download PDF
Appl Clin Inform 2023; 14(04): 705-713
DOI: 10.1055/s-0043-1771395
Research Article

Behavioral Health Decision Support Systems and User Interface Design in the Emergency Department

Nicholas W. Jones
1   Department of Health Services, Policy, and Practice, Brown University School of Public Health, Providence, Rhode Island, United States
,
Sophia L. Song
2   Warren Alpert Medical School of Brown University, Providence, Rhode Island, United States
,
Nicole Thomasian
3   Department of Anesthesiology, New York Presbyterian-Weill Cornell Medical Center, New York, New York, United States
,
Elizabeth A. Samuels
4   Department of Emergency Medicine, Warren Alpert Medical School of Brown University, Providence, Rhode Island, United States
,
Megan L. Ranney
4   Department of Emergency Medicine, Warren Alpert Medical School of Brown University, Providence, Rhode Island, United States
› Author Affiliations Funding This work was supported by the Advance-CTR x, Big Data Pilot Project Award, U.S. Department of Health and Human Services, National Institutes of Health, National Institute of General Medical Sciences (grant no.: U54GM115677).
› Further Information
    Permissions and Reprints

Abstract

Objective The objective of this qualitative study is to gauge physician sentiment about an emergency department (ED) clinical decision support (CDS) system implemented in multiple adult EDs within a university hospital system. This CDS system focuses on predicting patients' likelihood of ED recidivism and/or adverse opioid-related events.

Methods The study was conducted among adult emergency physicians working in three EDs of a single academic health system in Rhode Island. Qualitative, semistructured interviews were conducted with ED physicians. Interviews assessed physicians' prior experience with predictive analytics, thoughts on the alert's placement, design, and content, the alert's overall impact, and potential areas for improvement. Responses were aggregated and common themes identified.

Results Twenty-three interviews were conducted (11 preimplementation and 12 postimplementation). Themes were identified regarding each physician familiarity with predictive analytics, alert rollout, alert appearance and content, and on alert sentiments. Most physicians viewed these alerts as a neutral or positive EHR addition, with responses ranging from neutral to positive. The alert placement was noted to be largely intuitive and nonintrusive. The design of the alert was generally viewed positively. The alert's content was believed to be accurate, although the decision to respond to the alert's call-to-action was physician dependent. Those who tended to ignore the alert did so for a few reasons, including already knowing the information the alert contains, the alert offering information that is not relevant to this particular patient, and the alert not containing enough information to be useful.

Conclusion Ultimately, this alert appears to have a marginally positive effect on ED physician workflow. At its most beneficial, the alert reminded physicians to deeply consider the care provided to high-risk populations and to potentially adjust their care and referrals. At its least beneficial, the alert did not affect physician decision-making but was not intrusive to the point of negatively impacting workflow.

Keywords

decision support systems - clinical - emergency department - hospital - informatics - attitude - interview

Protection of Human and Animal Subjects

Study participants were deidentified prior to analysis, with identifying information contained in a password-protected file within a secure file-sharing environment. This study was reviewed and approved by the relevant Institutional Review Board. Study participants were compensated for their participation via gift card.


Supplementary Material



Publication History

Received: 27 February 2023

Accepted: 06 June 2023

Article published online:
06 September 2023

© 2023. Thieme. All rights reserved.

Georg Thieme Verlag KG
Rüdigerstraße 14, 70469 Stuttgart, Germany

 

  • References

  • 1 Sutton RT, Pincock D, Baumgart DC, Sadowski DC, Fedorak RN, Kroeker KI. An overview of clinical decision support systems: benefits, risks, and strategies for success. NPJ Digit Med 2020; 3: 17
    CrossrefPubMedGoogle Scholar
  • 2 Shah SN, Amato MG, Garlo KG, Seger DL, Bates DW. Renal medication-related clinical decision support (CDS) alerts and overrides in the inpatient setting following implementation of a commercial electronic health record: implications for designing more effective alerts. J Am Med Inform Assoc 2021; 28 (06) 1081-1087
    CrossrefPubMedGoogle Scholar
  • 3 Rosenthal B, Skrbin J, Fromkin J. et al. Integration of physical abuse clinical decision support at 2 general emergency departments. J Am Med Inform Assoc 2019; 26 (10) 1020-1029
    CrossrefPubMedGoogle Scholar
  • 4 Fletcher GS, Aaronson BA, White AA, Julka R. Effect of a real-time electronic dashboard on a rapid response system. J Med Syst 2017; 42 (01) 5
    CrossrefPubMedGoogle Scholar
  • 5 Whitcomb WF, Lucas JE, Tornheim R, Chiu JL, Hayward P. Association of decision support for hospital discharge disposition with outcomes. Am J Manag Care 2019; 25 (06) 288-294
    PubMedGoogle Scholar
  • 6 Amland RC, Hahn-Cover KE. Clinical decision support for early recognition of sepsis. Am J Med Qual 2019; 34 (05) 494-501
    CrossrefPubMedGoogle Scholar
  • 7 Henry KE, Hager DN, Pronovost PJ, Saria S. A targeted real-time early warning score (TREWScore) for septic shock. Sci Transl Med 2015; 7 (299) 299ra122
    CrossrefPubMedGoogle Scholar
  • 8 Jenssen BP, Kelleher S, Karavite DJ. et al. A clinical decision support system for motivational messaging and tobacco cessation treatment for parents: pilot evaluation of use and acceptance. Appl Clin Inform 2023; 14 (03) 439-447
    Article in Thieme ConnectPubMedGoogle Scholar
  • 9 Shear K, Rice H, Garabedian PM. et al. Usability testing of an interoperable computerized clinical decision support tool for fall risk management in primary care. Appl Clin Inform 2023; 14 (02) 212-226
    Article in Thieme ConnectPubMedGoogle Scholar
  • 10 Levin S, Toerper M, Hamrock E. et al. Machine-learning-based electronic triage more accurately differentiates patients with respect to clinical outcomes compared with the emergency severity index. Ann Emerg Med 2018; 71 (05) 565-574.e2
    CrossrefPubMedGoogle Scholar
  • 11 Amarasingham R, Patzer RE, Huesch M, Nguyen NQ, Xie B. Implementing electronic health care predictive analytics: considerations and challenges. Health Aff (Millwood) 2014; 33 (07) 1148-1154
    CrossrefPubMedGoogle Scholar
  • 12 Bates DW, Saria S, Ohno-Machado L, Shah A, Escobar G. Big data in health care: using analytics to identify and manage high-risk and high-cost patients. Health Aff (Millwood) 2014; 33 (07) 1123-1131
    CrossrefPubMedGoogle Scholar
  • 13 Blackburn J, Ousey K, Goodwin E. Information and communication in the emergency department. Int Emerg Nurs 2019; 42: 30-35
    CrossrefPubMedGoogle Scholar
  • 14 Geissert P, Hallvik S, Van Otterloo J. et al. High-risk prescribing and opioid overdose: prospects for prescription drug monitoring program-based proactive alerts. Pain 2018; 159 (01) 150-156
    CrossrefPubMedGoogle Scholar
  • 15 Weiner SG, Baker O, Bernson D, Schuur JD. One-year mortality of patients after emergency department treatment for nonfatal opioid overdose. Ann Emerg Med 2020; 75 (01) 13-17
    CrossrefPubMedGoogle Scholar
  • 16 Obermeyer Z, Powers B, Vogeli C, Mullainathan S. Dissecting racial bias in an algorithm used to manage the health of populations. Science 2019; 366 (6464) 447-453
    CrossrefPubMedGoogle Scholar
  • 17 Manrai AK, Funke BH, Rehm HL. et al. Genetic misdiagnoses and the potential for health disparities. N Engl J Med 2016; 375 (07) 655-665
    CrossrefPubMedGoogle Scholar
  • 18 Gijsberts CM, Groenewegen KA, Hoefer IE. et al. Race/ethnic differences in the associations of the Framingham risk factors with carotid IMT and cardiovascular events. PLoS One 2015; 10 (07) e0132321
    CrossrefPubMedGoogle Scholar
  • 19 Hussain MI, Reynolds TL, Zheng K. Medication safety alert fatigue may be reduced via interaction design and clinical role tailoring: a systematic review. J Am Med Inform Assoc 2019; 26 (10) 1141-1149
    CrossrefPubMedGoogle Scholar
  • 20 Safi S, Thiessen T, Schmailzl KJ. Acceptance and resistance of new digital technologies in medicine: qualitative study. JMIR Res Protoc 2018; 7 (12) e11072
    CrossrefPubMedGoogle Scholar
  • 21 Benda NC, Das LT, Abramson EL. et al. “How did you get to this number?” Stakeholder needs for implementing predictive analytics: a pre-implementation qualitative study. J Am Med Inform Assoc 2020; 27 (05) 709-716
    CrossrefPubMedGoogle Scholar
  • 22 Jeffery AD, Novak LL, Kennedy B, Dietrich MS, Mion LC. Participatory design of probability-based decision support tools for in-hospital nurses. J Am Med Inform Assoc 2017; 24 (06) 1102-1110
    CrossrefPubMedGoogle Scholar
  • 23 Fernandes M, Vieira SM, Leite F, Palos C, Finkelstein S, Sousa JMC. Clinical decision support systems for triage in the emergency department using intelligent systems: a review. Artif Intell Med 2020; 102: 101762
    CrossrefPubMedGoogle Scholar
  • 24 Weiss A, Jiang HJ. Overview of Clinical Conditions With Frequent and Costly Hospital Readmissions by Payer, 2018. Healthcare Cost & Utilization Project: Agency for Healthcare Research and Quality (AHRQ); 2021
    PubMedGoogle Scholar
  • 25 Soares III WE, Melnick ER, Nath B. et al. Emergency department visits for nonfatal opioid overdose during the COVID-19 pandemic across six US Health Care Systems. Ann Emerg Med 2022; 79 (02) 158-167
    CrossrefPubMedGoogle Scholar
  • 26 Shortliffe EH, Sepúlveda MJ. Clinical decision support in the era of artificial intelligence. JAMA 2018; 320 (21) 2199-2200
    CrossrefPubMedGoogle Scholar
  • 27 Nowell LS, Norris JM, White DE, Moules NJ. Thematic analysis: striving to meet the trustworthiness criteria. Int J Qual Methods 2017; 16 (01) 1-13
    CrossrefPubMedGoogle Scholar
  • 28 Braun V, Clarke V. Using thematic analysis in psychology. Qual Res Psychol 2006; 3 (02) 77-101
    CrossrefPubMedGoogle Scholar
  • 29 Pourmand A, Quan T, Amini SB, Sikka N. Can emoji's assess patients' mood and emotion in the emergency department? An emoji based study. Am J Emerg Med 2020; 38 (04) 842-843
    CrossrefPubMedGoogle Scholar
  • 30 Lai D, Lee J, He S. Emoji for the medical community-challenges and opportunities. JAMA 2021; 326 (09) 795-796
    CrossrefPubMedGoogle Scholar
  • 31 Skiba DJ. Face with tears of joy is word of the year: are emoji a sign of things to come in health care?. Nurs Educ Perspect 2016; 37 (01) 56-57
    CrossrefPubMedGoogle Scholar
  • 32 Bakken S. Need for innovation in electronic health record-based medication alerts. J Am Med Inform Assoc 2019; 26 (10) 901-902
    CrossrefPubMedGoogle Scholar
  • 33 Miller SD, Murphy Z, Gray JH. et al. Human-centered design of clinical decision support for anemia screening in children with inflammatory bowel disease. Appl Clin Inform 2023; 14 (02) 345-353
    Article in Thieme ConnectPubMedGoogle Scholar
  • 34 Levy-Fix G, Kuperman G, Elhadad N. Machine Learning and Visualization in Clinical Decision Support: Current State and Future Directions. arXiv preprint; 2019
    PubMedGoogle Scholar
  • 35 Chan CT, Carlson J, Lee T, Vo M, Nasr A, Hart-Cooper G. Usability and utility of human immunodeficiency virus pre-exposure prophylaxis clinical decision support to increase knowledge and pre-exposure prophylaxis initiations among pediatric providers. Appl Clin Inform 2022; 13 (05) 1141-1150
    Article in Thieme ConnectPubMedGoogle Scholar