Usability Evaluation of a Community Pharmacy Health Information Exchange Interface Prototype

 

 Permissions and Reprints

Abstract

Objective Few community pharmacies have access to health information exchange (HIE) data. We conducted a first-of-its-kind usability evaluation of an HIE interface prototype (referred to throughout as the “HIE-Pioneer mock-up”) developed with pharmacists and pharmacy technicians to aid future implementation in community pharmacies.

Methods Community pharmacists and pharmacy technicians were recruited to complete usability evaluations with the HIE-Pioneer mock-up. Each usability evaluation lasted up to 60 minutes. System usability scale (SUS) scores were collected from each participant following each usability evaluation session and summarized with descriptive statistics. Usability evaluation videos were reviewed for common usability attributes, such as the impact of identified usability problems, learnability, and efficiency. Time on task, task success rates, and prototype utilization were also recorded.

Results Sixteen total participants completed usability testing across three community pharmacies. The average SUS score was 69.7 (scale 0–100, where 100 is the best), with pharmacists on average reporting higher satisfaction than technicians (74.1 vs. 65.3, respectively). Altogether, we identified 23 distinct usability problems. Key problems identified included needed clarification in tool label names and accessibility of HIE links within the existing workflow. Overall, the usability of the HIE-Pioneer mock-up generally fostered pharmacy professionals' ease of learning and efficiency.

Conclusion Our study identified key areas, and potential solutions, to improve the usability of the HIE-Pioneer mock-up. Overall, pharmacy professionals viewed the HIE-Pioneer mock-up positively, with good satisfaction ratings. The HIE-Pioneer mock-up provides a blueprint for future HIE implementation in community pharmacy settings, which would increase community pharmacy teams' access to HIE data nationwide. Community pharmacy access to bi-directional HIE is expected to improve communication among more health care professionals involved in patient care and equip pharmacy professionals with needed information for improved clinical decision-making.

Protection of Human and Animal Subjects

This study was reviewed and approved by the Indiana University Institutional Review Board.

Publication History

Received: 20 February 2023

Accepted: 22 July 2023

Accepted Manuscript online:
01 August 2023

Article published online:
11 October 2023

© 2023. The Author(s). This is an open access article published by Thieme under the terms of the Creative Commons Attribution-NonDerivative-NonCommercial License, permitting copying and reproduction so long as the original work is given appropriate credit. Contents may not be used for commercial purposes, or adapted, remixed, transformed or built upon. (https://creativecommons.org/licenses/by-nc-nd/4.0/)

Georg Thieme Verlag KG
Rüdigerstraße 14, 70469 Stuttgart, Germany

• References

• 1 Health Information Exchange. The Office of the National Coordinator for Health Information Technology; . Accessed January 17, 2023 at: https://www.healthit.gov/topic/health-it-and-health-information-exchange-basics/health-information-exchange
  Google Scholar
• 2 Berenbrok LA, Tang S, Gabriel N. et al. Access to community pharmacies: A nationwide geographic information systems cross-sectional analysis. J Am Pharm Assoc (Wash DC) 2022; 62 (06) 1816-1822.e2
  CrossrefPubMedGoogle Scholar
• 3 Hettinger KN, Snyder ME, Russ-Jara AL. A Formative Usability Evaluation of a Health Information Exchange Prototype for use in Community Pharmacies [poster abstract]. Poster presented at: American Medical Informatics Association Symposium Meeting; November 7, 2022; Washington, DC
  PubMedGoogle Scholar
• 4 Vest MH, Weber T, Savage S. et al. Development of an innovative partnership between a health-system department of pharmacy and external community pharmacies. Am J Health Syst Pharm 2022; 79 (22) 2040-2046
  CrossrefPubMedGoogle Scholar
• 5 Craddock DS, Hall RG. Pharmacists without access to the EHR: practicing with one hand tied behind our backs. Innov Pharm 2021; (e-pub ahead of print) DOI: 10.24926/iip.v12i3.4141.
  CrossrefPubMedGoogle Scholar
• 6 Hohmeier KC, Spivey CA, Boldin S, Moore TB, Chisholm-Burns M. Implementation of a health information exchange into community pharmacy workflow. J Am Pharm Assoc (Wash DC) 2017; 57 (05) 608-615
  CrossrefPubMedGoogle Scholar
• 7 Fanizza FA, Ruisinger JF, Prohaska ES, Melton BL. Integrating a health information exchange into a community pharmacy transitions of care service. J Am Pharm Assoc (Wash DC) 2018; 58 (04) 442-449
  CrossrefPubMedGoogle Scholar
• 8 Milosavljevic A, Aspden T, Harrison J. Community pharmacist-led interventions and their impact on patients' medication adherence and other health outcomes: a systematic review. Int J Pharm Pract 2018; 26 (05) 387-397
  CrossrefPubMedGoogle Scholar
• 9 Hughes CA, Guirguis LM, Wong T, Ng K, Ing L, Fisher K. Influence of pharmacy practice on community pharmacists' integration of medication and lab value information from electronic health records. J Am Pharm Assoc (Wash DC) 2011; 51 (05) 591-598
  CrossrefPubMedGoogle Scholar
• 10 Roberts MF, Reeves K, Divine H. Community pharmacists' lack of access to health records and its impact on targeted MTM interventions. J Am Pharm Assoc (Wash DC) 2019; 59 (4S): S81-S84
  CrossrefPubMedGoogle Scholar
• 11 Sethman NC, Hettinger KN, Snyder ME. Planning for health information exchange: Perspectives of community pharmacists in the Community Pharmacy Enhanced Services Network of Indiana. J Am Pharm Assoc (Wash DC) 2022; 62 (05) 1615-1622
  CrossrefPubMedGoogle Scholar
• 12 Baker C, Loresto Jr F, Pickett K, Samay SS, Gance-Cleveland B. Facilitating health information exchange to improve health outcomes for school-aged children: school nurse electronic health record access. Appl Clin Inform 2022; 13 (04) 803-810
  Article in Thieme ConnectPubMedGoogle Scholar
• 13 Schleyer TKL, Rahurkar S, Baublet AM. et al. Preliminary evaluation of the Chest Pain Dashboard, a FHIR-based approach for integrating health information exchange information directly into the clinical workflow. AMIA Joint Summits on Translational Science Proceedings. AMIA Joint Summits on Translational Science 2019:656–664
  PubMedGoogle Scholar
• 14 Cresswell KM, Bates DW, Sheikh A. Ten key considerations for the successful implementation and adoption of large-scale health information technology. J Am Med Inform Assoc 2013; 20 (e1): e9-e13
  CrossrefPubMedGoogle Scholar
• 15 Bacci JL, Berenbrok LA. Innovative advances in connectivity and community pharmacist patient care services: implications for patient safety. Pharmacotherapy 2018; 38 (08) 867-874
  CrossrefPubMedGoogle Scholar
• 16 Webber EC, Peterson RJ, Lory K, Kanis J, Saysana M, Schneider KS. A unique use of regional information exchange by a statewide health system serving refugees: “Operation Allies Welcome”. Appl Clin Inform 2023; 14 (03) 487-493
  Article in Thieme ConnectPubMedGoogle Scholar
• 17 HIE Access for Pharmacists-Benefits and Barriers. Accessed January 17, 2023 at: https://www.ethin.org/benefits-and-barriers-hie-access-forpharmacists/#:∼:text=As%20such%2C%20the%20HIEs%20exclude%20pharmacists%20as%20well.,to%20access%20are%20substantial%2C%20the%20benefits%20are%2C%20too
  PubMedGoogle Scholar
• 18 Vogel LH, Reed WC. Management of Information in Health Care Organizations. In: Shortliffe EH, Cimino JJ, eds., Chiang MF. co-ed. Biomedical Informatics: Computer Applications in Healthcare and Biomedicine. New York: Springer; 2021
  CrossrefGoogle Scholar
• 19 Lowry SZ, Quinn MT, Ramaiah M. et al. A Human Factors Guide to Enhance EHR Usability of Critical User Interactions when Supporting Pediatric Patient Care. NIST Interagency/Internal Report (NISTIR) 2012: 7865
  PubMedGoogle Scholar
• 20 Bakken S, Dykes PC, Collins Rosetti S. et al. Patient-Centered Care Systems. In: Shortliffe EH, Cimino JJ, eds., Chiang MF. co-ed. Biomedical Informatics: Computer Applications in Healthcare and Biomedicine. New York: Springer; 2021
  CrossrefGoogle Scholar
• 21 Russ AL, Saleem JJ. Ten factors to consider when developing usability scenarios and tasks for health information technology. J Biomed Inform 2018; 78: 123-133
  CrossrefPubMedGoogle Scholar
• 22 Brown B, Balatsoukas P, Williams R, Sperrin M, Buchan I. Multi-method laboratory user evaluation of an actionable clinical performance information system: Implications for usability and patient safety. J Biomed Inform 2018; 77: 62-80
  CrossrefPubMedGoogle Scholar
• 23 Indiana Health Information Exchange. Accessed January 19, 2023 at: https://ihie.org
  PubMedGoogle Scholar
• 24 PioneerRx Expands Pharmacy Network to 5,000 Independent Pharmacies. Accessed January 17, 2023 at: https://www.pioneerrx.com/blog/pioneerrx-expands-pharmacy-network-to-5-000-independent-pharmacies#:∼:text=Finding%20its%20start%20in%202008%2C%20PioneerRx%20has%20grown,active%20in%205%2C000%20community%20pharmacies%20across%20the%20country
  PubMedGoogle Scholar
• 25 Clinical Data Search. Indiana Health Information Exchange. Accessed January 17, 2023 at: https://www.ihie.org/wp-content/uploads/2022/04/CDS-Flyer-2021.pdf
  PubMedGoogle Scholar
• 26 IHIE HealthDart. Accessed January 17, 2023 at: https://www.ihie.org/products-overview/
  PubMedGoogle Scholar
• 27 Krauss ZJ, Abraham M, Coby J. Clinical Pharmacy services enhanced by electronic health record (EHR) access: an innovation narrative. Pharmacy (Basel) 2022; 10 (06) 170
  CrossrefPubMedGoogle Scholar
• 28 Talmon J, Ammenwerth E, Brender J, de Keizer N, Nykänen P, Rigby M. STARE-HI—statement on reporting of evaluation studies in health informatics. Int J Med Inform 2009; 78 (01) 1-9
  CrossrefPubMedGoogle Scholar
• 29 Indiana CPESN. Accessed January 17, 2023 at: https://cpesn.com/networks/cpesn-indiana/
  PubMed
• 30 Faulkner L. Beyond the five-user assumption: benefits of increased sample sizes in usability testing. Behav Res Methods Instrum Comput 2003; 35 (03) 379-383
  CrossrefPubMedGoogle Scholar
• 31 Nielson J. Demonstrate Think Aloud by Showing Users a Video. Accessed January 17, 2023 at: https://www.nngroup.com/articles/thinking-aloud-demo-video/
  PubMedGoogle Scholar
• 32 Fan M, Shi S, Truong KN. Practices and challenges of using think-aloud protocols in industry: an international study. J User Exper 2020; 15 (02) 85-102
  PubMedGoogle Scholar
• 33 Nielsen J. Estimating the number of subjects needed for a thinking aloud test. Int J Hum Comput Stud 1994; 41: 385-397
  CrossrefPubMedGoogle Scholar
• 34 System Usability Scale. Accessed January 17, 2023 at: https://www.usability.gov/how-to-and-tools/methods/system-usability-scale.html
  PubMedGoogle Scholar
• 35 Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG. Research electronic data capture (REDCap)—a metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform 2009; 42 (02) 377-381
  CrossrefPubMedGoogle Scholar
• 36 Harris PA, Taylor R, Minor BL. et al. The REDCap consortium. Building an international community of software partners. J Biomed Inform 2019; 95: 103208
  CrossrefPubMedGoogle Scholar
• 37 Russ AL, Zillich AJ, Melton BL. et al. Applying human factors principles to alert design increases efficiency and reduces prescribing errors in a scenario-based simulation. J Am Med Inform Assoc 2014; 21 (e2): e287-e296
  CrossrefPubMedGoogle Scholar
• 38 Nielsen J. Ch 2: What is usability?. In: Usability engineering. San Francisco, CA: Morgan Kaufmann; 1993: 23-48
  Google Scholar
• 39 Russ AL, Baker DA, Fahner WJ. et al. A Rapid Usability Evaluation (RUE) Method for Health Information Technology. AMIA Annual Symposium Proceedings. AMIA Annual Symposium 2010:702–706
  PubMedGoogle Scholar
• 40 Bangor A, Kortum P, Miller J. Determining what individual SUS scores mean: adding an adjective rating scale. J Usability Stud 2009; 4 (03) 114-123
  PubMedGoogle Scholar
• 41 Curran GM, Alle ME, Mukherjee S. et al. QUERI Series: a process for developing an implementation intervention. Implement Sci 2008; 3: 17
  CrossrefPubMedGoogle Scholar
• 42 Lefchak B, Bostwick S, Rossetti SC. et al. Assessing usability and ambulatory clinical staff satisfaction with two electronic health records. Appl Clin Inform 2023; 14 (03) 494-502
  Article in Thieme ConnectPubMedGoogle Scholar
• 43 Sutton RT, Pincock D, Baumgart DC, Sadowski DC, Fedorak RN, Kroeker KI. An overview of clinical decision support systems: benefits, risks, and strategies for success. NPJ Digit Med 2020; 3 (17) 17
  CrossrefPubMedGoogle Scholar
• 44 Fragidis LL, Chatzoglou PD. Implementation of a nationwide electronic health record (EHR). Int J Health Care Qual Assur 2018; 31 (02) 116-130
  CrossrefPubMedGoogle Scholar
• 45 Thinking Aloud. Accessed January 17, 2023 at: https://www.sciencedirect.com/topics/computer-science/thinking-aloud#:∼:text=An%20obvious%20disadvantage%20of%20thinking%20aloud%20is%20that,used%20to%20get%20an%20impression%20of%20problem%20states
  PubMedGoogle Scholar
• 46 Jaspers MWM. A comparison of usability methods for testing interactive health technologies: methodological aspects and empirical evidence. Int J Med Inform 2009; 78 (05) 340-353
  CrossrefPubMedGoogle Scholar
• 47 Russ AL, Jahn MA, Patel H. et al. Usability evaluation of a medication reconciliation tool: embedding safety probes to assess users' detection of medication discrepancies. J Biomed Inform 2018; 82: 178-186
  CrossrefPubMedGoogle Scholar
• 48 Nielsen J, Landauer TK. A mathematical model of the finding of usability problems. Proceedings presented at: ACM INTERCHI'93 Conference; April 24–29, 1993; Amsterdam, The Netherlands; 206-213
  PubMedGoogle Scholar

• Supplementary Material