That study had the rather tame, reasonable conclusion:
... These findings raise the possibility that, as currently implemented, electronic access does not decrease test ordering in the office setting and may even increase it, possibly because of system features that are enticements to ordering. We conclude that use of these health information technologies, whatever their other benefits, remains unproven as an effective cost-control strategy with respect to reducing the ordering of unnecessary tests.
The ONC response (posted here at present) to such a mild conclusion about an experimental technology seemed very splenetic.
As I mentioned at the aforementioned post, ONC has thrown good science under the bus before, for political purposes in my view:
... On the other hand, coming from a political office that clearly does not understand how to conduct qualitative research and creates political promotion pieces masquerading as "research", such a statement is not surprising. See "ONC: "The Benefits Of Health Information Technology: A Review Of The Recent Literature Shows Predominantly Positive Results" at this link, where essential research methodologies were thrown under the bus for publication in Health Affairs.
At least the deviations from rigorous research methodologies were admitted:“... Our findings must be qualified by two important limitations: the question of publication bias [e.g., bias in evidence selection - ed.], and the fact that we implicitly gave equal weight to all studies regardless of study design or sample size.”
Unfortunately, the media, politicians, financial decisionmakers and others are likely not to really comprehend, in-depth, the full significance of that sentence.
I am in the unfortunate situation of again having to stuff ONC's - um, stuff - back into the bull.
First, hat tip to Histalk where I found the link below:
National Coordinator for HIT Farzad Mostashari, MD takes issue with the recently published report that found doctors with online access to patients’ charts ordered more tests. Mostashari disputes the study, which raised questions as to whether or not EHRs cut costs. Mostashari’s contends that the study was based on 2008 data and before the start of the Meaningful Use program and thus does not address certified EHRs’ capabilities for data exchange and clinical data support.
The fatal passage in the linked ONC piece at http://www.healthit.gov/buzz-blog/meaningful-use/study-facts/ is this:
"Also, the study data were from 2008, before the passage of the HITECH Act and the linking of payment incentives to the meaningful use of EHRs."
This seems a variation of the typical excuse-making in IT - "they were using v. 1.0; it's all fixed in the later version."
The outcomes of (Orwellian-named) "Meaningful Use" (MU) have not been studied, to my knowledge. Futher, the criteria chosen for "Meaningful Use" were primarily best guesses as to what could be beneficial. ("Meaningful Use" should have been more accurately termed "good faith use.")
Claiming that 2008 data on EHR-related test ordering is invalid because "Meaningful Use" was not in effect at the time is, in fact, jumping to an unsupported conclusion that "Meaningful Use" will counter whatever multiple medical/social factors caused the increased ordering in the first place -- because, of course, MU is "Meaningful" and deterministically guaranteed to work out, nationally, as planned, among all outpatient and inpatient settings.
This seems a form of "begging the question":
Begging the Question is a fallacy in which the premises include the claim that the conclusion is true or (directly or indirectly) assume that the conclusion is true. This sort of "reasoning" typically has the following form.
- Premises in which the truth of the conclusion is claimed or the truth of the conclusion is assumed (either directly or indirectly).
- Claim C (the conclusion) is true.
This sort of "reasoning" is fallacious because simply assuming that the conclusion is true (directly or indirectly) in the premises does not constitute evidence for that conclusion. Obviously, simply assuming a claim is true does not serve as evidence for that claim. This is especially clear in particularly blatant cases: "X is true. The evidence for this claim is that X is true."
I note that the MU criteria are themselves evolving and not finalized. Making predictions about the future is the domain of fortune tellers with crystal balls, not scientists:
Health IT WILL reduce costs and improve caregivers’ decisions and patients’ outcomes. It is written in the stars!ONC seems to think it is capable of such certainty, as I wrote in mid-2010 at "Science or Politics? The New England Journal and "The 'Meaningful Use' Regulation for Electronic Health Records":
... The widespread use of electronic health records (EHRs) in the United States is inevitable. EHRs will improve caregivers’ decisions and patients’ outcomes. Once patients experience the benefits of this technology, they will demand nothing less from their providers. Hundreds of thousands of physicians have already seen these benefits in their clinical practice.
In its current piece, ONC goes on to state:
Reducing Test Orders Is Not the Way that Health IT Is Meant to Reduce Costs
The ultimate impact of EHRs on reducing cost will be through improvements in the coordination and quality of care, and the prevention of unnecessary and costly complications and hospitalizations. [Note the mysterious disappearance of the word "tests" - ed.] Providers who are embracing new delivery and payment models such as Accountable Care Organizations and Patient-Centered Medical Homes know that meaningful use of EHRs is a critical foundation for being able to improve quality while reducing cost.
That is, simply, a lie. Reducing test utilization has long been claimed as a benefit of EHR's.
For example, from http://www.healthit.gov/patients-families/benefits-health-it:
EHRs reduce unnecessary tests and procedures. Have you ever had to repeat medical tests ordered by one doctor because the results weren’t readily available to another doctor? Those tests may have been uncomfortable and inconvenient or have posed some risk, and they also cost money. Repeating tests—whether a $20 blood test or a $2,000 MRI--results in higher costs to you in the form of bigger bills and increased insurance premiums. With EHRs, all of your care providers can have access to all your test results and records at once, reducing the potential for unnecessary repeat tests.
It is also another example of "moving the goalposts", a defense often used by those without a sound argument; by politicians; and sometimes by - scoundrels.
(I observed another "moving of the goalposts" at my aforementioned post "Another Health IT Mythbuster: Doctors order more X-rays, not fewer, with computer access." In that post I observed Michael Furukawa, a health economist in the ONC office, stating that the researchers’ focus was not "deep enough" to support the study’s conclusions. He wrote: “The data are sound, the methods are appropriate, but the focus is limited,” he said. “They only looked at one piece of health IT.”)
Well, yes, and the piece they looked at should set off red flags that the assumptions about health IT and savings might be erroneous - not generate excuses that the goalposts were too close, and need to be moved further away.
This is not to mention that it appears most healthcare errors have little to do with documentation, as I outlined in my Dec. 2010 post "Is Healthcare IT a Solution to the Wrong Problem?"
I particularly take issue with the ONC statement that:
... this was not a randomized trial, but an observational study (the National Ambulatory Medical Survey) that was not designed to answer the question of cost, or associations between EHRs and quality. As a result, many other variables that could affect physician behavior could not be examined in this study
Yet in the aforementioned ONC "predominantly positive results" literature survey described at this link, ONC "implicitly gave equal weight to all studies regardless of study design or sample size." Their review thus included qualitative studies that were probably not meant to be evaluative, and observational studies subject to severe methodological bias, yet all were weighted equally. Further, studies contradicting the 'narrative' of health IT efficacy and beneficence such as in this list appear to have been excluded.
This appears a prime example of, at best, "the pot calling the teakettle black", and raises doubts as to ONC's objectivity and perhaps even intellectual honesty. Of course, they are a political office with a mission, so perhaps this can be understood. (Not excused, but understood.)
This appears a prime example of, at best, "the pot calling the teakettle black", and raises doubts as to ONC's objectivity and perhaps even intellectual honesty. Of course, they are a political office with a mission, so perhaps this can be understood. (Not excused, but understood.)
ONC also views risk management-critical reports of health IT-caused harm as "anecdotal", a scientific and ethical faux pas (or is it willful blindness?) of major proportions.
As I wrote at my April 2001 post "Making a Stat Less Significant: Common Sense on "Side Effects" Lacking in Healthcare IT Sector":
... This view [of negative HIT reports being 'anecdotes'] extends all the way up to the Director of the Office of the National Coordinator for Health IT, who glibly stated per the Aug. 2010 Huffington Post Investigative Fund article FDA, Obama Digital Medical Records Team at Odds over Safety Oversight that FDA's own reports of health IT related injuries and deaths were “anecdotal":ONC director Blumenthal, the point man for the administration, has called the FDA’s injury findings “anecdotal and fragmentary.” He told the Investigative Fund that he believed nothing in the report indicated a need for regulation.
Those "injury findings" appear in an FDA Internal Memo made available by the aforementioned Huffington Post Investigative Fund and archived at the following link:
Internal FDA memorandum on HIT risks (PDF) to Jeffrey Shuren MD JD (Director, Center for Devices and Radiological Health). Health Information Technology (H-IT) Safety Issues. "This is an Internal Document Not Intended for Public Use." Feb. 23, 2010.
(My description/summary of the memorandum is at my Aug. 2010 post "Internal FDA memorandum of Feb. 23, 2010 to Jeffrey Shuren on HIT risks. Smoking gun?")
The definitive take-down of the "anecdote" canard is at this link.
One could wonder if a criteria for work at ONC is a Ddulite disposition ('Luddite' with first four characters reversed):
Ddulites: Hyper-enthusiastic technophiles who either deliberately ignore or are blinded to technology's downsides, ethical issues, and repeated local and mass failures.
Instead of logical fallacy and "spin", perhaps the ONC would be better served by sponsoring some (independent, objective) researchers to actually conduct research supporting their claims about the effects of the "Meaningful Use" program.
Finally, perhaps ONC should ask my mother what she thinks about EHRs. She has specially-acquired firsthand expertise. They can find her here.
-- SS
Addendum 3/9/12:
At my Feb. 2011 post "Does EHR-Incited Upcoding (Also Known as "Fraud") Need Investigation by CMS, And Could it Explain HIT Irrational Exuberance " is another side effect on costs of EHR's in inpatient settings, specifically, the ED.
-- SS
Addendum 3/11/12:
Another article in 2010 by Himmelstein, Woolhandler & Wright had reached related conclusions:
Hospital Computing and the Costs and Quality of Care: A National Study
David U. Himmelstein, MD, Adam Wright, PhD,Steffie Woolhandler, MD, MPHThe American Journal of Medicine Volume 123, Issue 1 , Pages 40-46, January 2010
Background
Many believe that computerization will improve health care quality, reduce costs, and increase administrative efficiency. However, no previous studies have examined computerization's cost and quality impacts at a diverse national sample of hospitals.
Methods
We linked data from an annual survey of computerization at approximately 4000 hospitals for the period from 2003 to 2007 with administrative cost data from Medicare Cost Reports and cost and quality data from the 2008 Dartmouth Health Atlas. We calculated an overall computerization score and 3 subscores based on 24 individual computer applications, including the use of computerized practitioner order entry and electronic medical records. We analyzed whether more computerized hospitals had lower costs of care or administration, or better quality. We also compared hospitals included on a list of the “100 Most Wired” with others.
Results
More computerized hospitals had higher total costs in bivariate analyses (r=0.06, P=.001) but not multivariate analyses (P=.69). Neither overall computerization scores nor subscores were consistently related to administrative costs, but hospitals that increased computerization faster had more rapid administrative cost increases (P=.0001). Higher overall computerization scores correlated weakly with better quality scores for acute myocardial infarction (r=0.07, P=.003), but not for heart failure, pneumonia, or the 3 conditions combined. In multivariate analyses, more computerized hospitals had slightly better quality. Hospitals on the “Most Wired” list performed no better than others on quality, costs, or administrative costs.
Conclusion
As currently implemented, hospital computing might modestly improve process measures of quality but does not reduce administrative or overall costs.
-- SS
