The paper is publicly available at this link in PDF.
The authors are Blackford Middleton (Harvard Medical School), Meryl Bloomrosen (AMIA), Mark A Dente (GE Healthare IT), Bill Hashmat (CureMD Corp.), Ross Koppel (Dept. of Socology, Univ, of Pennsylvania), J Marc Overhage (Siemens Health Services), Thomas H Payne (U. Washington IT Services), S Trent Rosenbloom (Vanderbilt Informatics), Charlotte Weaver (Gentiva Health Services) and Jiajie Zhang (University of Texas Health Science Center at Houston).
The paper states what has been obvious to this author - and many others - for many years:
ABSTRACT: In response to mounting evidence that use of electronic medical record systems may cause unintended consequences, and even patient harm, the AMIA Board of Directors convened a Task Force on Usability to examine evidence from the literature and make recommendations. This task force was composed of representatives from both academic settings and vendors of electronic health record (EHR) systems. After a careful review of the literature and of vendor experiences with EHR design and implementation, the task force developed 10 recommendations in four areas: (1) human factors health information technology (IT) research, (2) health IT policy, (3) industry recommendations, and (4) recommendations for the clinician end-user of EHR software. These AMIA recommendations are intended to stimulate informed debate, provide a plan to increase understanding of the impact of usability on the effective use of health IT, and lead to safer and higher quality care with the adoption of useful and usable EHR systems.
The paper is a respectable start at acknowledgement of the issues ... albeit years late.
That said:
I noted some typical language in the article characteristic of the reluctance of the health IT industry and its friends to directly confront these issues. I wrote a letter to the authors that, as I indicate below, not unexpectedly went unanswered except for one individual -- not even a physician -- who's gone out on a limb professionally in the interest of patient's rights, and as a health IT "iconoclast" (i.e., patient advocate) suffered for doing so (link). The lack of a response to the letter is itself representative, in my opinion, of a pathology that renders more rights to the healthcare computer and its makers than patients. More on this below.
First, I note I am rarely if ever cited by the academics. They are not prohibited from doing so. I've probably been writing on these issues -- poorly done health IT, improper leadership, the turmoil created, etc., publicly for longer than anyone else in the domain.
I also note that the paper is somewhat in the form of an analytical debate. Analytical debates are relatively ineffective in this domain. They are like popcorn thrust against a battleship. The paper, also, appearing as it does in a relatively obscure specialty journal (Journal of the American Medical Informatics Association), will probably get more exposure from this blog post than the entire readership of that journal. The authors need to be relating these issues in forums that are widely read by citizens and government, not in dusty academic journals - that is, assuming they want the messages to widely diffuse.
In my review of the article, I note the following:
... In an Agency for Healthcare Research and Quality (AHRQ) workshop on usability in health IT in July 2010, all eight participating vendors agreed that usability was important and many suggested it was a competitive differentiator, although some considered that usability was in the eye of the beholder and that the discipline of usability evaluation was an imperfect science, with results that were not useful.
A paper like this should have clearly repudiated antiquated viewpoints like that, not merely made note of them. Not taking a stand is a sign of weakness...or sympathy.
As a matter of fact, if leaders such as this had paid attention to the 'iconoclasts' and their 'anecdotes', my own mother might not have gone through horrible suffering and death, with me as sad witness as I related to them in my letter below.
... End-users of EHR systems are ultimately accountable for their safe and effective use, like any tool in clinical care.
I see a linguistic sleight of hand via use the word "tool" to describe HIT and trying to blend or homogenize this apparatus with other "tools" in clinical care. The HIT "tool" is unlike any other since no transaction of care can occur without it going through this device, and as such, all care is totally dependent on it. Further, unlike pharma and medical devices, this "tool" is unvetted and unregulated but its use forced upon many users.
... [AMIA] subcommittees reviewed the literature on usability in health IT, current related activities underway at various US Federal agencies, lessons learned regarding usability and human factors in other industries, and current federally funded research activities.
Did they speak with the source of the most candid information? The plaintiff's and defendant's Bars?
Need I even ask that question?
Need I even ask that question?
... Recent reports describe the safe and effective use of EHR as a property resulting from the careful integration of multiple factors in a broad sociotechnical framework
This is not merely 'recent' news. The field of Social Informatics (link), that has studied IT in its social contexts for decades now, has offered observations on the importance of considering multiple factors in a broad sociotechnical framework. The authors all know this - or should know this, or should have made it their business to know this. The statement sounds somewhat protective of the HIT and hospital industries for their longstanding negligence towards those issues.
... User error may result in untoward outcomes and unintended negative consequences. These may also occur as a result of poor usability, and may also be an emergent property only demonstrated after system implementation or widespread use.
I note the use of the term "user error" and lack of the term "use error" with significant disdain. As I wrote here regarding the views of a HIT industry exexcutive holding the mystical "American Medical Informatics Certification for Health Information Technology", NIST itself now defines "use error" (as opposed to "user error") as follows:
“Use error” is a term used very specifically by NIST to refer to user interface designs that will engender users to make errors of commission or omission. It is true that users do make errors, but many errors are due not to user error per se but due to designs that are flawed, e.g., poorly written messaging, misuse of color-coding conventions, omission of information, etc. From "NISTIR 7804: Technical Evaluation, Testing and Validation of the Usability of Electronic Health Records." It is available at http://www.nist.gov/healthcare/usability/upload/Draft_EUP_09_28_11.pdf (PDF).
In the article, indefinites were exchanged with what should have been stronger, declarative statements, and vice versa:
User error ... may also represent a potential health IT-related error yet to happen.
I most decidedly wish they'd stop this "may" verbiage in policy papers like this.
... Anecdotal reports suggest that these application differences [where clinicians use more than one commercial EHR system] result in an increased training burden for EHR users.
"Anecdotal"? How about "obvious to a third grader?"
"Anecdotal" in academic papers often is a term of derision for inconvenient truths such as reports of health IT problems. Its use often reflects a need for authors using the term (per a senior clinician from Victoria, Australia on the 'anecdotes' issue, link) "to attend revision courses in research methodology and risk management."
... Some suggest that the expected gains sought with the adoption of EHR are not yet realized.
"Some"? How about "credible experts?" "Suggest?" They merely hint at it? How about "opine?"
... The design of software applications requires both technical expertise and the ability to completely understand the user’s goal, the user’s workflow, and the socio-technical context of the intent
In the meantime, AMIA has been promoting national rollout of a technology where, most often, the latter does not apply.
To ... transform our healthcare delivery system ... clinicians need to use usable, efficient health IT that enhances patient safety and the quality of care.
This is the typical hyperenthusiast mantra. Where's the proof? And, transform into what, exactly? Vague rhetoric like this in allegedly scientific papers is most unwelcome.
Some experts suggest that improving the usability of EHR may be critical to the continued successful diffusion of the technology.
More weak talk. Why not come right out and say "Credible experts opine that ...."?
... While some EHR vendors have adopted user-centered design when developing health information technologies, the practice is not universal and may be difficult to apply to legacy systems.
From the patient advocacy perspective, that's their problem...it's a risk of being in this business. Patients should not be expected to be used as experimental subjects while IT sellers figure out what other industry sectors have long mastered. Further, they should be held accountable for failures that result in harm. Another risk of doing business in this sector that clinicians have long learned to live with...
... Some believe it is difficult or impossible to reliably compare one product with another on the basis of usability given the challenges in assessment of products as implemented.
Nothing is "impossible" and again, if it's "difficult", that's the industry's problem. There is risk of being in the business of medicine or medical facilitation; nobody promised a rose garden, and a rose garden should not be expected.
... Many effects of health IT can be considered to be ‘emergent’ or only discovered after monitoring a system in use
One might ask, where's the industry and AMIA been regading postmarket surveillance (common in other health sectors) for the past several decades?
... AMIA believes it is now critical to coordinate and accelerate the numerous efforts underway focusing on the issue of EHR usability.
Only "now?"
... Establish an adverse event reporting system for health IT and voluntary health IT event reporting
No, no, no ...voluntary reporting doesn't work. Even mandatory reporting is flawed, but it's better than voluntary.
I am invariably disappointed by recommendations like this. I've observed repeatedly, for example, that "volunatary reporting" of EHR problems already exists - in the form of the FDA MAUDE database - and most HIT sellers' reports are absent. See my posts on MAUDE here, here and here. (Also, the only one that seems to report may have ulterior motives, i.e., restraint of trade.)
... A voluntary reporting process could leverage the AHRQ patient safety organizations (PSO) ... This work should be sponsored by the AHRQ.
These folks clearly don't want any teeth in this. AHRQ is a research-oriented government branch, not a regulator, nor does it have regulatory expertise.
AMIA recommends:
In 2013 this is valuable information in the same sense that advice to use sterile technique during neurosurgery is valuable.Research and promote best practices for safe implementation of EHR
"Promoting best practices" has been done for decades. Not mentioned is avoiding worst practices. I've long written these are not the same thing, as toleration of the inappropriate leadership by health IT amateurs (a term I use in the same sense that I am a Radio Amateur, not a telecommunications professional), politics, empire-building and other dysfunction that goes on in health IT endeavors negates laundry lists of "best practices."
What is required is to research and abolish worst practices, including the culture and dynamics of the 'health IT-industrial complex.' I made this point in my very first website in 1998. It appears the authors don't get it and/or won't admit to the dysfunction that goes on in health IT projects.
... The adoption of useful and usable EHR will lead to safer and higher quality care, and a better return on investment for institutions that adopt them.
"Will?" With respect to my observation above about the paper's prominent misuse of indefinites vs. stronger declarative terms, the word "may" would have been the appropriate term here. As I wrote about similar statements from ONC in the NEJM in my 2010 post "Science or Politics? The New England Journal and The 'Meaningful Use' Regulation for Electronic Health Records", I'm quite disappointed seeing speculation and PR presented as fact from alleged scientists and scientific organizations.
Finally, I wrote the following email letter to the authors, to which (except for Ross Koppel) I received no reply. While Dr. Koppel (a PhD) graciously expressed sympathy for my me and mother, the others (many MD's) were silent.
Perhaps the silence is the best indicator of their concern for the rights of computers and HIT merchants relative to the rights of people:
Mon, Jan 28, 2013 at 1:12 PM
Dear authors,
I've reviewed the new paper "Enhancing patient safety and quality of care by improving the usability of electronic health record systems: recommendations from AMIA" and wanted to express thanks for it.
It's a good start. Late, but a good start at returning the health IT domain to credibility and evidence-based practice.
It's too bad it didn't come out years earlier. Perhaps my mother would not have gone through a year of horrible suffering and death, with me as sad witness, due to the toxic effects of bad health IT.
Perhaps you should hear how horrible it was to hear my mother in an extended agitated delirium; to hear her cry hysterically later on when the rehab people told her that her weight was 95 pounds; to have to make her a "no code" and put her on hospice-care protocols, and then to have watched her aspirate a sleeping pill when she was agitated, and die several days later of aspiration pneumonia and sepsis ... in the living room of my home ... and then watch the Hearse take her away from my driveway...as a result of bad health IT.
I will be writing more thoughts on your article at the Healthcare Renewal blog, of course, but wanted to raise three issues:
1. The use of "may" and "will" is reversed, and conflating the term "anecdote" with risk management-relevant case reports.
- They may also represent a potential health IT-related error yet to happen. ---> They likely represent a potential health IT-related error yet to happen
- Anecdotal reports suggest that these application differences result in an increased training burden for EHR users. ---> Credible reports indicate...
- Some suggest that the expected gains sought with the adoption of EHR are not yet realized. ---> Credible experts opine ....
- Some experts suggest that improving the usability of EHR may be critical to the continued successful diffusion of the technology. ---> "Credible experts opine that ..."
- The adoption of useful and usable EHR will lead to safer and higher quality care, and a better return on investment for institutions that adopt them. ---> The adoption of useful and usable EHR may lead to safer and higher quality care
You really need to show more clarity ... and guts ... in papers like this, and drop the faux academic weasel words.
2. You neglected to speak to the best source for information on EHR-related harms, evidence spoliation, etc... med mal attorneys.
3. You also neglected to speak to, or cite, the writings of a Medical Informaticist on bad health IT now going back 15 years - and whose mother was injured and died as a result of the issues you write about - me. In fact I am rarely cited or mentioned by anyone with industry interests.
An apparent contempt for 'whistleblowers' such as myself makes me wonder ... what kind of people are the leaders of health IT, exactly?
Do they value computer's rights over patients'?
It is not at all clear to me which has been the primary motivator of many of the health IT leaders.
I think the rights which I value are quite clear.
Sincerely,
Scot Silverstein
I neglected to mention the horror of seeing my mother put in a Body Bag before being taken to the Hearse in my driveway.
-- SS
