Download PDF
Appl Clin Inform 2017; 08(03): 981-993
DOI: 10.4338/ACI2017050080
Research Article
Schattauer GmbH

The 9-Item Physician Documentation Quality Instrument (PDQI-9) score is not useful in evaluating EMR (scribe) note quality in Emergency Medicine

Katherine J. Walker
1   Emergency Department, Cabrini Hospital, Melbourne, Australia
4   Monash University, Department of Epidemiology and Preventative Medicine, Melbourne, Australia
,
Andrew Wang
2   Cabrini Institute, Melbourne, Australia
3   Monash University Faculty of Medicine, Nursing and Health Sciences, Melbourne, Australia
,
William Dunlop
1   Emergency Department, Cabrini Hospital, Melbourne, Australia
,
Hamish Rodda
1   Emergency Department, Cabrini Hospital, Melbourne, Australia
5   Box Hill Hospital, Melbourne, Australia
,
Michael Ben-Meir
1   Emergency Department, Cabrini Hospital, Melbourne, Australia
,
Margaret Staples
2   Cabrini Institute, Melbourne, Australia
4   Monash University, Department of Epidemiology and Preventative Medicine, Melbourne, Australia
› Author Affiliations Funding The Cabrini Medical Staff Association funded AW. The Phyllis Connor Memorial Fund, Equity Trustees, the Cabrini Institute, Foundation and hospital also funded this work. No input was provided into study design or write-up by funders.
Further Information

Publication History

received: 17 May 2017

accepted in revised form: 02 August 2017

Publication Date:
20 December 2017 (online)

   Permissions and Reprints

Summary

Background: Scribes are assisting Emergency Physicians by writing their electronic clinical notes at the bedside during consultations. They increase physician productivity and improve their working conditions. The quality of Emergency scribe notes is unevaluated and important to determine.

Objective: The primary objective of the study was to determine if the quality of Emergency Department scribe notes was equivalent to physician only notes, using the Physician Documentation Quality Instrument, Nine-item tool (PDQI-9).

Methods: This was a retrospective, observational study comparing 110 scribed to 110 non-scribed Emergency Physician notes written at Cabrini Emergency Department, Australia. Consultations during a randomised controlled trial of scribe/doctor productivity in 2016 were used. Emergency physicians and nurses rated randomly selected, blinded and de-identified notes, 2 raters per note. Comparisons were made between paired scribed and unscribed notes and between raters of each note. Characteristics of individual raters were examined. The ability of the tool to discriminate between good and poor notes was tested.

Results: The PDQI-9 tool has significant issues. Individual items had good internal consistency (Cronbach’s alpha=0.93), but there was very poor agreement between raters (Pearson’s r=0.07, p=0.270). There were substantial differences in PDQI-9 scores allocated by each rater, with some giving typically lower scores than others, F(25,206)=1.93, p=0.007. The tool was unable to distinguish good from poor notes, F(3,34)=1.15, p=0.342. There was no difference in PDQI-9 score between scribed and non-scribed notes.

Conclusions: The PDQI-9 documentation quality tool did not demonstrate reliability or validity in evaluating Emergency Medicine consultation notes. We found no evidence that scribed notes were of poorer quality than non-scribed notes, however Emergency scribe note quality has not yet been determined.

Citation: Walker KJ, Wang A, Dunlop W, Rodda H, Ben-Meir M, Staples M. The 9-Item Physician Documentation Quality Instrument (PDQI-9) score is not useful in evaluating EMR (scribe) note quality in Emergency Medicine. Appl Clin Inform 2017; 8: 981–993 https://doi.org/10.4338/ACI2017052017050080

Keywords

Clinical Documentation and Communications - Scribe - Emergency Medicine - Quality - Safety Culture

Human Subject Protections

The study was performed in compliance with the World Medical Association Declaration of Helsinki on Ethical Principles for Medical Research Involving Human Subjects, and was approved by Cabrini Human Research Ethics Committee (06–27–07–15).


Contributions

KW, MS, WD, HR and AW conceived the study and designed the trial. KW, AW: ethics. KW, MBM: funding. AW recruitment, randomization, database design. KW: data collection. MS, WD: statistics, data analysis. AW, WD, KW drafted manuscript. All authors contributed to its revision. KW takes responsibility for the manuscript.


 

  • References

  • 1 Tischendorf JJ, Crede S, Herrmann P, Bach N, Bomeke C, Manns MP, Schaefer O, Trautwein C. Encoding of diagnosis by medical documentation assistant or ward physician. Dtsch med Wochenschr 2004; 129 (33) 1731-5.
    Article in Thieme ConnectPubMedGoogle Scholar
  • 2 Walker K, Johnson M, Dunlop W, Staples M, Rodda H, Turner I, Ben-Meir M. Feasibility evaluation of a pilot scribe-training program in an Australian emergency department. Aust Health Rev. 2017 doi: 10.1071/AH16188. PubMed PMID: 28355527.
    PubMedGoogle Scholar
  • 3 Arya R, Salovich DM, Ohman-Strickland P, Merlin MA. Impact of scribes on performance indicators in the emergency department. Academic emergency medicine 2010; 17 (05) 490-4.
    CrossrefPubMedGoogle Scholar
  • 4 Bastani A, Shaqiri B, Palomba K, Bananno D, Anderson W. An ED scribe program is able to improve throughput time and patient satisfaction. The American journal of emergency medicine 2014; 32 (05) 399-402.
    CrossrefPubMedGoogle Scholar
  • 5 Heaton HA, Nestler DM, Jones DD, Lohse CM, Goyal DG, Kallis JS, Sadosty AT. Impact of scribes on patient throughput in adult and pediatric academic EDs. The American journal of emergency medicine 2016; 34 (10) 1982-5.
    CrossrefPubMedGoogle Scholar
  • 6 Heaton HA, Nestler DM, Jones DD, Varghese RS, Lohse CM, Williamson ES, Sadosty AT. Impact of Scribes on Billed Relative Value Units in an Academic Emergency Department. J Emerg Med. 2016 doi: 10.1016/j.jemermed.2016.11.017. PubMed PMID: 27988262.
    PubMedGoogle Scholar
  • 7 Heaton HA, Nestler DM, Lohse CM, Sadosty AT. Impact of scribes on emergency department patient throughput one year after implementation. The American journal of emergency medicine. 2016 doi: 10.1016/j.ajem.2016.11.017. PubMed PMID: 27856140.
    PubMedGoogle Scholar
  • 8 Hess JJ, Wallenstein J, Ackerman JD, Akhter M, Ander D, Keadey MT, Capes JP. Scribe Impacts on Provider Experience, Operations, and Teaching in an Academic Emergency Medicine Practice. West J Emerg Med 2015; 16 (05) 602-10.
    CrossrefPubMedGoogle Scholar
  • 9 Walker K, Ben-Meir M, O’Mullane P, Phillips D, Staples M. Scribes in an Australian private emergency department: A description of physician productivity. Emergency medicine Australasia 2014; 26 (06) 543-8.
    CrossrefPubMedGoogle Scholar
  • 10 Walker KJ, Ben-Meir M, Phillips D, Staples M. Medical scribes in emergency medicine produce financially significant productivity gains for some, but not all emergency physicians. Emergency medicine Australasia 2016; 28 (03) 262-7.
    CrossrefPubMedGoogle Scholar
  • 11 Shuaib W, Hilmi J, Caballero J, Rashid I, Stanazai H, Ajanovic A, Moshtaghi A, Amari A, Tawfeek K, Khurana A, Hasabo H, Baqais A, Mattar AA, Gaeta TJ. Impact of a scribe program on patient throughput, physician productivity, and patient satisfaction in a community-based emergency department. Health Informatics Journal. 2017. Epub March 1,. 2017 doi: 10.1177/1460458217692930.
    PubMedGoogle Scholar
  • 12 Bank AJ, Obetz C, Konrardy A, Khan A, Pillai KM, McKinley BJ, Gage RM, Turnbull MA, Kenney WO. Impact of scribes on patient interaction, productivity, and revenue in a cardiology clinic: a prospective study. ClinicoEconomics and outcomes research 2013; 5 (01) 399-406.
    CrossrefPubMedGoogle Scholar
  • 13 Bank AJ, Gage RM. Annual impact of scribes on physician productivity and revenue in a cardiology clinic. ClinicoEconomics and outcomes research 2015; 7: 489-95.
    CrossrefPubMedGoogle Scholar
  • 14 Koshy S, Feustel PJ, Hong M, Kogan BA. Scribes in an ambulatory urology practice: patient and physician satisfaction. The Journal of urology 2010; 184 (01) 258-62.
    CrossrefPubMedGoogle Scholar
  • 15 Carnes KM, de Riese CS, Werner TW. A Cost-Benefit Analysis of Medical Scribes and Electronic Medical Record System in an Academic Urology Clinic. Urology Practice. 2015 2(3).
    PubMedGoogle Scholar
  • 16 Misra-Hebert AD, Amah L, Rabovsky A, Morrison S, Cantave M, Hu B, Sinsky CA, Rothberg MB. Medical scribes: How do their notes stack up?. J Fam Pract 2016; 65 (03) 155-9.
    PubMedGoogle Scholar
  • 17 Yan C, Rose S, Rothberg MB, Mercer MB, Goodman K, Misra-Hebert AD. Physician, Scribe, and Patient Perspectives on Clinical Scribes in Primary Care. J Gen Intern Med 2016; 31 (09) 990-5.
    CrossrefPubMedGoogle Scholar
  • 18 Dunlop W, Hegarty L, Staples M, Levinson M, Ben-Meir M, Walker K. Medical scribes have no impact on the patient experience of an emergency department. Emergency Medicine Australasia. 2017 doi: 10.1111/1742–6723.12818.
    PubMedGoogle Scholar
  • 19 Hess JJ, Akhter M, Ackerman J, Wallenstein J, Ander D, Keadey M, Capes JP. Prospective evaluation of a scribe program’s impacts on provider experience, patient flow, productivity, and teaching in an academic emergency medicine practice. Annals of emergency medicine 2013; 1: S1.
    CrossrefPubMedGoogle Scholar
  • 20 Heaton HA, Samuel R, Farrell KJ, Colletti JE. Emergency department scribes: A two-step training program. Annals of emergency medicine 2015; 1: S159.
    CrossrefPubMedGoogle Scholar
  • 21 Walker KJ, Dunlop W, Liew D, Staples MP, Johnson M, Ben-Meir M, Rodda HG, Turner I, Phillips D. An economic evaluation of the costs of training a medical scribe to work in Emergency Medicine. Emerg Med J 2016; 33 (12) 865-9.
    CrossrefPubMedGoogle Scholar
  • 22 Stetson PD, Morrison FP, Bakken S, Johnson SB, eNote Research T. Preliminary development of the physician documentation quality instrument. J Am Med Inform Assoc 2008; 15 (04) 534-41.
    CrossrefPubMedGoogle Scholar
  • 23 Stetson PDB. et al. Assessing Electronic Note Quality using the Physician Documentation Quality Instrument (PDQI-9). Applied Clinical Informatics 2012; 3: 164-74.
    Article in Thieme ConnectPubMedGoogle Scholar
  • 24 Combs T. Recognizing the Characteristics of Quality Documentation. J AHIMA 2016; 87 (05) 32-3.
    PubMedGoogle Scholar
  • 25 Kargul GJ, Wright SM, Knight AM, McNichol MT, Riggio JM. The hybrid progress note: semiautomating daily progress notes to achieve high-quality documentation and improve provider efficiency. Am J Med Qual 2013; 28 (01) 25-32.
    CrossrefPubMedGoogle Scholar
  • 26 Burke HB, Hoang A, Becher D, Fontelo P, Liu F, Stephens M, Pangaro LN, Sessums LL, O’Malley P, Baxi NS, Bunt CW, Capaldi VF, Chen JM, Cooper BA, Djuric DA, Hodge JA, Kane S, Magee C, Makary ZR, Mallory RM, Miller T, Saperstein A, Servey J, Gimbel RW. QNOTE: an instrument for measuring the quality of EHR clinical notes. J Am Med Inform Assoc 2014; 21 (05) 910-6.
    CrossrefPubMedGoogle Scholar
  • 27 Jamieson T, Ailon J, Chien V, Mourad O. An electronic documentation system improves the quality of admission notes: a randomized trial. J Am Med Inform Assoc 2017; 24 (01) 123-9.
    CrossrefPubMedGoogle Scholar
  • 28 Bierman JA, Hufmeyer KK, Liss DT, Weaver AC, Heiman HL. Promoting Responsible Electronic Documentation: Validity Evidence for a Checklist to Assess Progress Notes in the Electronic Health Record. Teach Learn Med 2017; 12: 1-13.
    PubMedGoogle Scholar