Health Information Technology Evaluation Handbook

Governments and clinical providers are investing billions of dollars in health information technologies (HIT). This is being done with the expectation that HIT adoption will translate into healthier patients experiencing better care at lower cost. In the initial push to roll out HIT, the reliability of these claims was often not substantiated by systematic evaluation and testing. As the first wave of widespread adoption of HIT comes to an end and the next wave begins, it is more important than ever that stakeholders evaluate the results of their investment, evaluate their success (or failure), and make decisions about future directions.

Structured evaluations of a project’s impact are an essential element of the justification for investment in HIT. A systematic approach to evaluation and testing should allow for comparison between different HIT interventions with the goal of identifying and promoting those which improve clinical care or other outcomes of interest. The question of the day is no longer "why perform evaluations," but "how to perform evaluations."

This updated book provides an easy-to-read reference outlining the basic concepts, theory, and methods required to perform a systematic evaluation of HIT. Chapters cover key domains of HIT evaluation: study structure and design, measurement fundamentals, results analysis, communicating results, guidelines development, and reference standards. Updated case studies and examples are included demonstrating the successes or failures of these investments. The authors also include new initiatives put in place by the government and discuss how they are being adopted and used by health systems.

Vitaly Herasevich is a Professor of Anesthesiology and Medicine in the Department of Anesthesiology and Perioperative Medicine, Division of Critical Care, Mayo Clinic, Rochester, Minnesota. He has been involved in medical informatics for over 20 years, with a specific concentration on applied clinical informatics in critical care and the science of healthcare delivery.

He was born in Belarus, where he earned his MD and PhD degrees, and joined the Mayo Clinic in 2006. He codirects the Clinical Informatics in Intensive Care program as part of a research group that works to decrease complications and improve outcomes for critically ill patients through systematic research and quality improvement. He is interested in studying and developing clinical syndromic surveillance alerting systems ("sniffers"), clinical data visualization (novel patient-centered electronic medical records), and hybrid predictive and prescriptive analytics. He is co-inventor of many technologies including AWARE platform, resulting in technologies commercialization. He has co-authored over 100 peer-reviewed articles and authored three books and number of book chapters. As part of an education effort, Dr. Herasevich creates and teaches medical informatics classes at the Mayo Medical School and Mayo Graduate School. He is a Fellow of the Society Critical Care Medicine, Fellow of American Medical Informatics Association and a Senior Member of IEEE serving on a number of committees.

Brian Pickering is an Associate Professor of Anesthesiology in the Department of Anesthesiology and Perioperative Medicine, Division of Critical Care, Mayo Clinic, Rochester, Minnesota. Dr. Pickering was born in Dublin, Ireland. He completed his medical education at Trinity College Dublin prior to his residency and fellowship training in anesthesiology and critical care at the College of Anesthetists, Royal College of Surgeons Ireland. He moved with his family to Rochester, Minnesota, in 2008 to complete further fellowship training in critical care at Mayo Clinic, where he currently practices. Dr. Pickering’s primary research area is focused on improving the processes of care in the acute care setting to improve patient health and outcomes while reducing medical costs. Leveraging the sophisticated electronic infrastructure available at Mayo Clinic, Dr. Pickering’s laboratory group have developed novel user interfaces, smart alerts, and reporting dashboards that target deviations from best practice and support healthcare providers to deliver a consistently high level of clinical performance to the bedside. With an established culture of innovation and patient-centered care models, the laboratory has unparalleled access to clinicians, clinical settings, homegrown technologies, and clinical information systems. This environment facilitates rapid bedside-to-bench and bench-to-bedside development cycles. The laboratories work has already led to practice improvement in Mayo Clinic ICUs through the implementation and adoption of a number of acute care clinical and administrative applications; several high profile funding awards; and intellectual property, which has been patented and licensed successfully by Mayo Clinic ventures. Together with institutional IT, the multidisciplinary team have developed a platform, AWARE, which addresses key safety concerns including, information overload, errors of omission, and workload associated with electronic medical record use.