Usability of EHRs Meeting at NIST

I'm here in Gaithersburg Maryland today at NIST - where the clocks are always accurate.

The context is a "Community Building Workshop" on usability of Electronic Health Records.

Longtime Docnotes readers know that I've been thinking/writing/talking about this for a long time.  Most recently - I testified (pdf) to the HIT Policy Committee's Implementation Workgroup on this topic.  

It's impressive that this has come to be a compelling topic for discussion - but there remains quite a bit fo work to do.

Notes from today's meeting:

Jodi Daniel from ONC gave a nice little intro on ONC's rationale for being involved here.  Bottom line:  

1. As usability of EHRs improves - so does adoption and the effectiveness of EHRs in the delivery of value to the nation in the form of improved efficiency and quality. 
2. Let's work together to find the right solutions
   
   Jodi is super smart.  I'm alwasy impressed with her contributions to such conversations.  She is a listener and a learner - and I always know that she is thoughtfully incorporating what she hears into the policies that eventually make their way out of ONC.  Goot to hear her make the intro from ONC and it's good to see her so engaged.
   
   

Matt Quinn gave a good intro from NIST.  What NIST is - why NIST is engaged - and what is NISTs role in this work.  Like Jodi - I've grown to know and respect Matt of the the past few years.  His pitch:

David Brick, a cardiologist from NYC gave a well-intoentioned talk on some problems he's onserved in EHRs.  He provided examples of how EHR-derived growth charts can cause both displeasure and safety problems.  David's keystone example was a system in which a 5.5 POUND patient's data could be expressed as 5.5 KILOGRAMS when a user toggles between Metric and English systems.  While I agree there is a usability issue here - his  example is a bit of a straw man.  As one considers the continuum of user experience from functional - through usable - to meaningful - this example isn't even functional.  It's flawed deign - or falwed implementation.  Period.  It's a bug that needs to be fixed.

So while one might argue that this is ALSO a usability issue - we need to be careful not to lower the bar so much that such examples become part of the usability conversation.   Would user-centered design have prevented this flaw?  Sure .. but I would suggests that we need to assume or even demand functionality in this conversation - and sink our teeth into usability -  the next hurdle that the industry needs to jump.

Ben Shneiderman from University of Maryland up next.

    Ben is a dynamic and outspoken speaker on these topics.  

He describes the industry leading works of several vendors such as Apple.  I recall the early Apple usability guidelines well - and Ben makes a good point that design guidance is a good thing.    But should an industry have design requirements?  He doesn't go so far as to say that - but he actually comes rather close.

Ben makes a set of suggestions:

•     EHR developers should publish a set of EHR usability guidelines
•     EHR developers should permit examination & discussion of products their products by researchers
•     EHR developers should report on usability study results - so that others can understand strengths and weaknesses of these products and design teams.
•     EHR developers should promote consistency with industry-led common user inerface guidleines
•     EHR developers should report failures periodically with common metrics
•     EHR developers should work with government agencies to coordiate sspceficif industry practices ..

Ben expressed frustration that vendors have not shared with him any examples, access to demonstration systems without signing NDA, access to documentation, or even details screenshots of EHRs.

Ben is a passionate and articulate guy - and his heart is clearly in the right place - but it's simply not all so simple as he portrays - and I would argue that the vendor community may not be so cooperative with him as he would like - because his demeanor is combative rather than collaborative.  How can one trust that a collaboration with Ben won't turn into a marketing tragedy?

Next up ... 

Mohammed Walji - SHARP-C

SHARP-C is an ONC-sponsored grant program @ UT - Houston.  I was an advisor for the project for a while - and I find it to be incredibly interesting - yet somewhat academic and therefore not quite ready to inform the industry.  Yet.  Perhaps they will at some point and that will be valuable indeed.

Mohammed outlined a general approach that their team is taking to usability:

TURF framework for usability:

1. Task
2. User Representation
3. Function

 

 

Facets of usability:

1. Useful - Satisfying - Usable
2. Intrinsic complexity ==> Usefulness
3. Extrinisc complexity ==> Usableness

 --

Arien Malec from ONC - descibes the successful method that government and industry worked togethersuggests that a process here enables us to:

Raise objections and concerns early in the process

Ensure the resulting usability test approach supports multiple modalities (eg dicataion)

Learn from each other and creat UX and design best practice that create superior usability and UX

Concern:

Measurement may not capture the nuances of heatlhcare

Reply: Help us define the instrucments and measurements and methodilogies

ONC suggests that the community can help define the workflow and context - sensitive tests

 "vendor community says: don't let he government tell us what is good or bad - we want the market to tell us what is good or bad"

Users "we don't want the govt to incent us to use stinky software"

Community - not ONC - can define what this is - through the (proposed) Marketing Usability Workgroup

1. Share end-user and vendor concenrs
2. Define emchanisms to endusre an deffieicne and wwell dunctioning marked
3. Supply relevant and accurate information to end users
4. Reward excellence in ux
5. Spur innovation

Workgroups comprised of:

- academic researchers

- vendors

- Users / Implementers

- Government

- Human Factors professionals

Edna Boone - HIMSS

Speaking about advantages of professional collaboration community.  Good dovetail with Arien's talk.

Usability industry - Human Factors, Design and Usability people .. 

Proposal:  A community of profession led by HIMSS Usability Taskforce - responsible for providing domain experttiese, leadership and guidance to activities, inittiateas and collaboration within the speciality of HF, usability and design ..  

Ron Kaye - FDA devices human factors guy - here to describe how FDA manages human factors issues in the medical device arena.

 

Jorge Ferrer - VA

Jorge provided an interesting literature review on recent papers that have been published in the domain of usability - and focused on a usability framwork developed at the VA - with a (too long to for me to trascribe) list of recommendations for "next steps" in usability in HIT.

 Janet Campbell - Epic

"What are the needs of software developers?"  

Usability is a journey rather than a destination.

We are not (nor will we ever be) perfect.

"Our users are smart people.  Physicians have hig standards and low tolerance for dysfunctional design."

What do providers need?

1. Safety 
2. Empowerment - customization of a user experience FOR ME
3. Adapt systems to changes or external requirelents.  Systems need to be flexible and adaptable.
4. Disabilities should be understood and anticipated (accomodate poor vision or bad mouse skills, etc).

Dialogue between user and vendor is fluid and useful.  When a third party gets involved in the conversation - it make it more complex - and sometimes the messages get less clear.  

Concern about measurement, guidelines, standards .. measuring things that are un-measureable .. or comparing systems to some sort of idealized design.  There exists a public-private partnership that is working.  There is an unspoken message here that the industry has failed.  In fact - it has not.  Is the government barking up the wrong tree?  Aiming to solve a problem that isn't in its scope to solve?  A government assessment may send the message that a certain design standard will meet the needs of all users.

We think there is a role for ONC and NIST - let's look at the common requirements - as defined by Meaningful Use - to see how we can optimize THESE processes.

 Mary Kate Foley.  VP for User Experience at athenaHealth.

Educate, Motivate, Improve.

Mary Kate describes the challenges of implementing UCD principles in an organization that previously didn't use them.

Observations:

    EHRs lag in usability

        Inhibits adoption

        Inhibits productivity

        Contributes to clinical risk

AHRQ report on EHR vendor practices and perspectives

Market factors will exert appropriate pressure 

Can we anticipate and acellerate?

Apply our UCD principles to the problem - how can we do more UCD @ EHR vendors?

    Understand the problem.  Complex needs, complex user bases.

    Focus on the target audience

        Usability maturity:  

1. Low (don't know what they don't know - no champion but may be skeptcs .. and misguided enthusiasts)
2. Medium ( a few practitioners - org still learning - many skeptics .. still some misguided enthusiasm)
3. High (ux recognized as a llever for business performance.  experienced dedicated ux people. team .. integrated with RND) .. Employ reliable UCD patterns.

Design with tartget in mind

Low - Educate

Medium Educate & Motivate

High - Motivate.

Get each vendor started where traction is likely to be greatest within vendors:  usabilty TESTING .. 

  -- 

Lyle Berkowitz

 
Shows the usability periodic table from HIMSS (see page 3 of this document)

Practice guidance:

• Engage your users from the start (not just  the physician geek).
• Consider the practice goals
• include ux questions in your RFP
• Perform usability tests when evaluating new software
• Observe other practices using the products
• Discuss findings with the vendors

--
Work has units .. Clicks .. Time .. Eergy. Effort, frustration, failure .. 

 

Lana Lowry - NIST

Bob Schumacher, User Centric

Emily Patterson, Ohio State

Bob North, Human Centered Strategies

Chris Gibbons, Johns Hopkins

 (editorial:  I like Lana - I think she is a bright, passionate advocate for doing things right.  Her slide deck, however, was the worst of the day.  It violated nearly every principle of a "usable" presentation.  She needs a copy of Presentation Zen.  I think I'll buy her a copy.  No kdding.

Lana's presentation was the first of a few sessions that got to the meat of why we were all here.  All of the previous sessions were (I think) meant to set a level playing field and make sure that the audience was all on the same page (what is the definition of usability, etc) and had been exposed to various perspectives on the matter - HF experts, government folks, vendors, etc.  In general - this worked - but it could have been done much more effiiciently.  We could have had 2 hrs of intro rather than four - and I would have preferred to spend much more time on the "meat" of the matter rather than just a few quick presentations in the afternoon.

Proposed EHR Usability Evaluation Protocol 

These presentation (I'll post later today in more detail about them) focused on the EUP and what it is.  Key message:

a) EHRs should be tested by people with advanced training in usability, human factors, cognitive science.

b) The focus should be on testing for errors.  The key here is patient safety

c) There will be a collaborative community effort that is created to define the details