Improving Clinical Trial Participant Tracking Tools Using Knowledge-anchored Design Methodologies

 

 Permissions and Reprints

Summary

Objective: Rigorous human-computer interaction (HCI) design methodologies have not traditionally been applied to the development of clinical trial participant tracking (CTPT) tools. Given the frequent use of iconic HCI models in CTPTs, and prior evidence of usability problems associated with the use of ambiguous icons in complex interfaces, such approaches may be problematic. Presentation Discovery (PD), a knowledge-anchored HCI design method, has been previously demonstrated to improve the design of iconic HCI models. In this study, we compare the usability of a CTPT HCI model designed using PD and an intuitively designed CTPT HCI model.

Methods: An iconic CPTP HCI model was created using PD. The PD-generated and an existing iconic CTPT HCI model were subjected to usability testing, with an emphasis on task accuracy and completion times. Study participants also completed a qualitative survey instrument to evaluate subjective satisfaction with the two models.

Results: CTPT end-users reliably and reproducibly agreed on the visual manifestation and semantics of prototype graphics generated using PD. The performance of the PD-generated iconic HCI model was equivalent to an existing HCI model for tasks at multiple levels of complexity, and in some cases superior. This difference was particularly notable when tasks required an understanding of the semantic meanings of multiple icons.

Conclusion: The use of PD to design an iconic CTPT HCI model generated beneficial results and improved end-user subjective satisfaction, while reducing task completion time. Such results are desirable in information and time intensive domains, such as clinical trials management.

Citation: Payne PRO, Embi PJ, Johnson SB, Mendonca E, Starren J. Improving clinical trial participant tracking tools using knowledge-anchored design methodologies. Appl Clin Inf 2010; 1: 177–196 http://dx.doi.org/10.4338/ACI-2010-02-RA-0012

• References

• 1 Chung TK, Kukafka R, Johnson SB. Reengineering clinical research with informatics. J Investig Med 2006 Sep 1 54 (Suppl. 06) 327-333.
  PubMed
• 2 Payne PR, Johnson SB, Starren JB, Tilson HH, Dowdy D. Breaking the translational barriers: the value of integrating biomedical informatics and translational research. J Investig Med 2005; May 53 (Suppl. 04) 192-200.
  CrossrefPubMedSearch in Google Scholar
• 3 Sung NS, Crowley Jr WF, Genel M, Salber P, Sandy L, Sherwood LM. et al. Central challenges facing the national clinical research enterprise. JAMA 2003 Mar 12 289 (Suppl. 10) 1278-1287.
  PubMed
• 4 Khan SA, Kukafka R, Payne PR, Bigger JT, Johnson SB. A day in the life of a clinical research coordinator: observations from community practice settings. Medinfo 2007; 12 Pt 1 247-251.
  PubMedSearch in Google Scholar
• 5 Khan SA, Payne PR, Johnson SB, Bigger JT, Kukafka R. Modeling clinical trials workflow in community practice settings. Proc AMIA Symp 2006; Jan 1 419-423.
  PubMedSearch in Google Scholar
• 6 Kukafka R, Johnson SB, Linfante A, Allegrante JP. Grounding a new information technology implementation framework in behavioral science: a systematic analysis of the literature on IT use. J Biomed Inform 2003; Jun 36 (Suppl. 03) 218-227.
  CrossrefPubMedSearch in Google Scholar
• 7 Chow SC, Liu JP. Design and Analysis of Clinical Trials: Concepts and Methodologies. 2 ed. Hoboken, NJ: John Wiley & Sons, Inc.; 2003
  Search in Google Scholar
• 8 Brandt CA, Argraves S, Money R, Ananth G, Trocky NM, Nadkarni PM. Informatics tools to improve clinical research study implementation. Contemporary clinical trials. 2006 Apr 1 27 (Suppl. 02) 112-122.
  PubMed
• 9 Payne PRO, Greaves AW, Kipps TJ. editors. CRC Clinical Trials Management System (CTMS): An Integrated Information Management Solution for Collaborative Clinical Research. AMIA 2003 Symposium; 2003; Washington, DC, USA.:
  PubMed
• 10 Payne PR, Greaves AW, Kipps TJ. CRC Clinical Trials Management System (CTMS): an integrated information management solution for collaborative clinical research. AMIA Annu Symp Proc 2003; 967.
  PubMedSearch in Google Scholar
• 11 Starren JB, Payne PR, Kaufman DR. Human computer interaction issues in Clinical Trials Management Systems. 2006
  PubMedSearch in Google Scholar
• 12 NIGMS.. National Institute of General Medical Sciences. Bethesda, MD: National Institutes of Health; 2010. Available from: http://www.nigms.nih.gov
  Search in Google Scholar
• 13 Shneiderman B. Designing the user interface : strategies for effective human-computer-interaction. 3rd ed. Reading, Mass: Addison Wesley Longman; 1998
  Search in Google Scholar
• 14 Payne PR, Starren JB. Quantifying visual similarity in clinical iconic graphics. J Am Med Inform Assoc 2005; May-Jun 12 (Suppl. 03) 338-345.
  CrossrefPubMedSearch in Google Scholar
• 15 Payne PR, Starren JB. Presentation discovery: building a better icon. CHI ‘06 extended abstracts on Human factors in computing systems; Motreal, Quebec, Canada: ACM; 2006
  Search in Google Scholar
• 16 Starren JB, Payne PR. Quantifying visual similarity in clinical iconic graphics. AMIA Annu Symp Proc 2003; 1016.
  PubMedSearch in Google Scholar
• 17 Gosbee JW. Applying CHI in Health Care: domain issues, resources and requirements. CHI 98 conference summary on Human factors in computing systems; Los Angeles, California, United States: ACM Press; 1998
  Search in Google Scholar
• 18 Cole WG. editor. Metaphor Graphics & Visual Analogy For Medical Data. Annu Symp Comput Appl Med Care. 1987
  PubMedSearch in Google Scholar
• 19 Cole WG, Stewart JG. Human performance evaluation of a metaphor graphic display for respiratory data. Methods Inf Med 1994; Oct 33 (Suppl. 04) 390-396.
  Thieme ConnectPubMedSearch in Google Scholar
• 20 Kelly GA. The psychology of personal constructs. [1st ] ed. New York: Norton; 1955
  Search in Google Scholar
• 21 Marcus A. Icons, symbols, and signs: visible languages to facilitate communication interactions. 2003; 10 (Suppl. 03) 37-43.
  PubMedSearch in Google Scholar
• 22 Patel VL, Arocha JF, Kaufman DR. A primer on aspects of cognition for medical informatics. J Am Med Inform Assoc 2001; Jul-Aug 8 (Suppl. 04) 324-343.
  CrossrefPubMedSearch in Google Scholar
• 23 Chandler D. Semiotics: The Basics. 1 ed. New York: Routledge; 2003
  Search in Google Scholar
• 24 Leite JC. editor. A Semiotic-based framework to user interface design. NordiCHI; 2002. Arhus, Denmark.:
  Search in Google Scholar
• 25 Marcus A. Human communications issues in advanced UIs. Commun ACM 1993; 36 (Suppl. 04) 100-109.
  PubMedSearch in Google Scholar
• 26 Elting LS, Bodey GP. Is a picture worth a thousand medical words? A randomized trial of reporting formats for medical research data. Meth Info Med 1991; 30 (Suppl. 02) 145-150.
  PubMedSearch in Google Scholar
• 27 Litt HI, Schmidt DF. editors. Application of the Visual Chart in an Ambulatory OB-GYN Clinic. Annu Symp Comput Appl Med Care. 1995
  PubMedSearch in Google Scholar
• 28 Abad-Mota S, Kulikowski C, Gong L, Stevenson S, Mezrich R, Tria A. et al., editors. Iconic reporting: a new way of communicating radiologic findings. Annu Symp Comput Appl Med Care. 1995
  PubMedSearch in Google Scholar
• 29 Barr P, Noble J, Biddle R. Icons R icons. Adelaide, Australia: Australian Computer Society, Inc.; 2003
  Search in Google Scholar
• 30 Horton WK. The icon book : visual symbols for computer systems and documentation. New York: J. Wiley; 1994
  Search in Google Scholar
• 31 Starren J. From Multimodal Sublanguages to Medical Data Presentations. New York: Columbia University; 1997
  Search in Google Scholar
• 32 Payne PR, Starren JB. Modeling categorical sorting behavior. Medinfo 2004; 2004 CD 1805.
  PubMedSearch in Google Scholar
• 33 Deitzer JR, Payne PR, Starren JB. Coverage of clinical trials tasks in existing ontologies. Proc AMIA Symp. 2006 Jan 1: 903.
  PubMedSearch in Google Scholar
• 34 Payne PR, Deitzer JR, Mendonca EA, Starren JB. Consensus-based construction of a taxonomy of clinical trial tasks. Proc AMIA Symp. 2006 Jan 1: 1059.
  PubMedSearch in Google Scholar
• 35 Rugg G, McGeorge P. The sorting techniques: a tutorial paper on card sorts, picture sorts and tiem sorts. Expert Systems 1997; May 14 (Suppl. 02) 80-93.
  CrossrefPubMedSearch in Google Scholar
• 36 JMP 7ed: SAS.
  PubMed
• 37 Romesburg HC. Cluster Analysis for Researchers. 2 ed. Raleigh, North Carolina: Lulu Press; 2004
  Search in Google Scholar
• 38 Morae.. TechSmith. 2010
  PubMedSearch in Google Scholar
• 39 Chin JP, Diehl VA, Normal KL. editors. Development of an instrument measuring user satisfaction of the human-computer interface. CHI Conference on Human Factors in Computing Systems 1988. Washington D C.: ACM Press.;
  Search in Google Scholar
• 40 Likert R. A Technique for the Measurement of Attitudes. Archives of Psychology 1932; 140 (June).
  PubMedSearch in Google Scholar
• 41 Perneger TV. What’s Wrong with Bonferroni Adjustments. BMJ 1998; 316 April 1236-1238.
  CrossrefPubMedSearch in Google Scholar
• 42 Sankoh AJ, Huque MF, Dusbey SD. Some comments on frequently used multipled endpoint adjustment methods in clinical trials. Stat Med 1997; 16 (Suppl. 22) 2529-2543.
  CrossrefPubMedSearch in Google Scholar
• 43 Telea AC. Data visualization : principles and practice. Wellesley, Mass.: A K Peters; 2008
  Search in Google Scholar
• 44 Harris Z, Mattick P. Science Sublanguages and the Prospects for a Global Language of Science. Annals of the American Academy of Political and Social Science. 1988 Jan 495: 73-83.
  PubMedSearch in Google Scholar
• 45 Harris Z. Theory of Language Structure. American Philosophical Quarterly 1976; 13 (Suppl. 04) 237-255.
  PubMedSearch in Google Scholar
• 46 Hripcsak G, Wilcox A. Reference standards, judges, and comparison subjects: roles for experts in evaluating system performance. J Am Med Inform Assoc 2002; Jan-Feb 9 (Suppl. 01) 1-15.
  CrossrefPubMedSearch in Google Scholar
• 47 Dicks RS. Mis-usability: on the uses and misuses of usability testing. Proceedings of the 20th annual international conference on Computer documentation Toronto, Ontario, Canada: 584960 ACM; 2002: 26-30.
  PubMedSearch in Google Scholar
• 48 Glaser B, Strauss A. The Discovery of Grounded Theory: Strategies for Qualitative Research. Piscataway, NJ: Aldine Transaction; 1967
  Search in Google Scholar