Authors & Affiliations
Cognitive Performance Group
Portsmouth, VA
Portsmouth, VA
Georges A. Potworowski, PhD, Tarah N. Daly, MS, Michael A. Pritchett, MS
Sharpe Medical Consulting
Dallas, TX
Dallas, TX
Charles Motsinger, MD
Oracle
Austin, TX
Austin, TX
Brian Levy, MD, Eric Y. Hwang, MD, Danielle Benfanti, MS
Office of Mental Health and Suicide Prevention,
Veterans Health Administration
Washington, D.C.
Veterans Health Administration
Washington, D.C.
Quinn Bastian, PhD, Pearl McGee-Vincent, PsyD
Office of Health Informatics,
US Department of Veterans Affairs
Washington, D.C
US Department of Veterans Affairs
Washington, D.C
Apurva Desai, LL.M, M.S., PMP, Diane Montella, MD
____________
Has your policy, intervention, service, or product...
not supported desired workflow?
resulted in unexpected errors or adverse effects?
met resistance when you tried to implement it?
A human-centered design approach would help make it...
integrate with your workflow to make it more efficient and effective.
prevent, identify, mitigate, and recover from errors and adverse effects.
be embraced by stakeholders because they were partners from the start.
____________
Problem
The Joint Commission requires that “outcomes of care, treatment, or services be monitored over the course of service using a standardized instrument – a practice generally known as measurement-based care [MBC].”
The Veterans Health Administration (VHA) Business Owners (BOs) want a single standard and experience of mental health care for Veterans and clinicians across the VHA.Currently, a patchwork of electronic mental health testing tools (MH Tools) is used across the VHA. A single standard and experience of mental health care requires selecting and standardizing the MH Tool(s) to be used across the VHA.
To identify the best MH Tool, the VHA needs to build a set of selection criteria based on the requirements and needs of different stakeholders involved in MBC (e.g., clinicians, Veterans, administrators).
Approach
The goal of this project was to use a human-centered design (HCD) process to identify and organize a set of appropriate selection criteria into a MH Tool evaluation framework (Framework).
This poster describes the effort to understand how different clinicians, in different contexts, currently use MH Tools. This includes where and how different MH Tools do—and could better—support clinician workflow and contribute to the quality of Veteran care.
These insights aim to 1) inform what criteria to include in the framework, 2) give concrete examples of how and how much clinicians value a given criterion, and 3) depict the current state of how the BOs think the framework and these insights (can) support the evaluation of MH Tools.
Human-Centered Design (HCD)
“An approach to interactive systems that aims to make systems usable and useful by focusing on the users, their needs and requirements, and by applying human factors/ergonomics, and usability knowledge and techniques. This approach enhances effectiveness and efficiency, improves human well-being, user satisfaction, accessibility and sustainability; and counteracts possible adverse effects of use on human health, safety and performance.” (ISO 9241-210:2019)
· Design is based upon an explicit understanding of users, tasks, and environments.
· Users are involved throughout design and development.
· Design is driven and refined by user-centered evaluation.
· Process is iterative.
· Design addresses the whole user experience.
· Design team includes multi-disciplinary skills and perspectives.
References
International Organization for Standardization. (2019). ISO 9241–210 Ergonomics of Human-System interaction - Part 210: Human-Centred Design for Interactive Systems.
The Joint Commission. CTS.03.01.09. Care, Treatment, and Services. In 2020 Comprehensive Accreditation Manual for Behavioral Health Care (E-edition). Oak Brook, IL: Joint Commission Resources, 2020 .
This work was developed under Cognitive Medical Systems’ contract with the US Dept. of Veterans Affairs, Contract #36C77620D0022, Order# 36C77622N3030.
____________
Human-Centered Design Process
Deliverables
1. The Framework
The evaluation framework is a decision aid to help the VHA evaluate MH Tools based on functionality, technical criteria, integration, usability, and value to key stakeholders to recommend future development/acquisitions of MH Tools that meet VHA consistency and requirement standards.
2. Clinician End-User Storyboards
A storyboard depicts one clinician’s workflow, for a given purpose, in a given context, and where and how one or two exemplar MH Tool(s) do/es—and could potentially— support that workflow. The set of storyboards makes a storybook.
3. Storybook Table
This table collects data from across the current (and future) storyboards in one place to allow the BOs to summarize and facilitate analysis of data from across the storybook.
Framework Storyboard
The Framework Storyboard is a formative depiction of the BOs actual or intended MH Tool evaluation workflow, and where and how the Framework, Storyboards, and Storybook Table do and could support that evaluation. It helps identify gaps and potential next steps.
____________
Possible Next Steps
1. Additional Interviews
· More clinicians from other MBC settings using other MH Tools.
· Veterans who have experience with MH Tools in different MBC settings.
· Administrators or data analysts who (would) use MH Tool aggregate data for organizational performance improvement efforts.
2. Framework and supporting tool usability testing
· Develop use cases for the Framework (e.g., existing VHA MH Tool, adapting a MH Tool, new MH Tool, building a MH Tool).
· Evaluate BOs’ use of the Framework and supporting tools with use cases.
3. Validation of Storyboards
· Show storyboards to their respective interviewees to gain deeper insight, including quantitative value ratings of features.
4. Framework Development
· Develop a questionnaire to have stakeholders weigh the value of different MH Tools features as captured in Framework line items / criteria.
· Add line item weights by stakeholder group to Framework.
· Develop a UX testing protocol for MH Tools that have met minimum must-have criteria set in the Framework.