 |
| The Jury, by John Morgan |
The Agency for Healthcare Quality and
Research (AHRQ) has released its draft report on the Diagnosis and
Treatment of Myalgic Encephalomyelitis/Chronic Fatigue Syndrome
(ME/CFS).
For those who find it hard to keep
track of the plethora of reports, committees, panels, and reviews
currently underway, the AHRQ report provides the basis for the
Pathways to Prevention, or P2P Workshop. The P2P Workshop has been convened by the NIH for the purpose of making a report that will be
used to evaluate research grants for ME/CFS, and upon which
pharmaceutical companies will base their clinical trials.
The P2P panel is composed entirely of
non-experts. The preference for non-experts was intended, according
to Susan Maier, Executive Secretary for the NIH Advisory Committee on
Research on Women’s Health, to act like a “jury,” by which she
meant the composition of the panel was meant to be unbiased.
Leaving aside for a moment the
absurdity of consigning a research case definition for a disease to
people who aren't even physicians, the jury system itself does not
work. Trial by jury is one of the most inefficient systems of justice in
the world. Swayed by bombastic arguments, prejudice, and crocodile tears, juries of twelve unbiased peers routinely find innocent
people guilty, and allow the guilty to go free. As a method for
determining who gets grants for medical research, anything even remotely resembling the jury system is
entirely inappropriate.
I have touched upon some of the
short-comings of the draft report below. I've tried to keep it brief.
(There is so much wrong with this report that if I were to write a
thorough critique, it would be longer than the report itself.)
You can read the AHRQ draft report and appendices here.
You can make comments (until October 20) here.
Jennie Spotila gives a great overview of how the AHRQ report will be used, and explains why the draft report is bad science.
All the reasons why the P2P is
dangerous
Jennie Spotila explains why the AHRQ
report is a recipe for disaster, and how they pulled a “bait and
switch” by changing their Key Questions.
Mary Dimmock and Jennie Spotila explain
how refusing to consider how CFS and ME differ will affect the
results of the P2P Workshop.
Note: Anyone who wishes to reprint this blog post may do so. (Remember to link back to the original and provide attribution.)
_______________________
A BRIEF CRITIQUE OF THE AHRQ DRAFT REPORT
By Erica Verrillo
Point 1: The draft report shows limited
understanding of the illness
The report begins with the erroneous
statement that “The term ME was first used in the 1930s after an
outbreak of neuromyesthenia ...”
A quick google search would have
revealed that “myalgic encephalomyelitis” was first used in a
letter to the editor entitled, “A New Clinical Entity?” published
in the Lancet in 1956. The authors – Emile Nihoul, Lise
Quersin-Thiry, S.Chalmers Parry and Robert A. Good – were referring
to what became known as Royal Free Disease, an outbreak that occurred in London's Royal Free Hospital in the
1950s. Obviously, none of the members of the panel felt inclined to
check their facts.
The report goes on to say that
“Uncertainty persists regarding the etiology and whether the
condition reflects a single pathologically discrete syndrome, subsets
of the same illness, or a nonspecific condition shared by other
disease entities.” This statement reflects a thorough
misunderstanding of how the illness is perceived by people who
investigate and treat it. There are no experts in ME/CFS who would
claim that it is a “nonspecific condition shared by other disease
entities.” There are, however, numerous researchers who author
papers on “chronic fatigue” and “fatiguing illnesses” such as
cancer and MS. The authors of the AHRQ draft report, unlike experts
in the field, do not have enough background to be able to distinguish
between “chronic fatigue” and ME/CFS.
While the introduction to the report is
not crucial, it does indicate that the people writing it not only had
no knowledge of the illness, but that they did not want to spend any
time acquiring it. The willful ignorance that is demonstrated
in the introduction permeates the entire report.
Point 2: Problematic Search Methods
In order to find abstracts and
articles, the AHRQ searched three main databases using the terms:
fatigue; Fatigue Syndrome, Chronic; and Encephalomyelitis. With the
notable exception of PsycINFO, a database of abstracts of literature in the field of psychology produced by the American Psychological Association, these are the same databases used by
the Drug Class Review: Drugs for Fibromyalgia: Final Original Report
published by the Oregon Health & Science University in 2011. Ovid
and EBM/Cochrane are large medical databases, though they don't
necessarily include every study conducted on a given illness or
condition. Only controlled trials are included in the Cochrane databases.
The most glaring problem with the
search is that it included studies on “fatigue.” Indeed, a number
of studies included in the review were on “fatiguing illnesses”
rather than ME/CFS. Like the introduction, the search reflects a
state of confusion on the part of the authors. The confusion is not altogether surprising, given that researchers also appear to be confused about the difference between CFS and chronic fatigue. Nonetheless, experts in the field are not confused. They are aware that while ME has been used abroad since the 1950s, it has not been used as a diagnosis here in U.S. Specialists have been limited to CFS as a diagnosis, like it or not.
A second problem is that with the
perennial lack of NIH funding for ME/CFS controlled trials, much of
the information about treating the disease is based on clinical
observations. None of these were included. nor were
studies that were controlled, but which did not meet the set of
criteria for inclusion in the review – such as addressing the Key
Questions.
Point 3: Studies used for the report are
inadequate to address the Key Questions
The studies that
the reviewers included were not only too few, they were completely
inadequate to properly address the Key Questions.
The Key Questions
to be addressed by the report are as follows:
1. What methods are available to
clinicians to diagnose ME/CFS and how do the use of these methods
vary by patient subgroups?
a) What are widely accepted diagnostic
methods and what conditions are required to be ruled out or excluded
before assigning a diagnosis of ME/CFS?
b) What is the accuracy and concordance
of diagnostic methods?
c) What harms are associated with
diagnosing ME/CFS?
2. What are the (a) benefits and (b)
harms of therapeutic interventions for patients with ME/CFS and how
do they vary by patient subgroups?
a) What are the characteristics of
responders and non-responders to interventions?
There are problems with the wording of some of these questions. For example, in a country in which 80% of the physicians don't believe that CFS is a real disease, what could "widely accepted" be referring to? And, "What harms are associated with diagnosing ME/CFS?" seems to have an a priori assumption that diagnosing the disease may in itself cause harm. But aside from the oddness of the wording, the studies they chose do not adequately address the questions.
There are problems with the wording of some of these questions. For example, in a country in which 80% of the physicians don't believe that CFS is a real disease, what could "widely accepted" be referring to? And, "What harms are associated with diagnosing ME/CFS?" seems to have an a priori assumption that diagnosing the disease may in itself cause harm. But aside from the oddness of the wording, the studies they chose do not adequately address the questions.
The criteria for exclusion from the
review included, among others, that the study did not last
not long enough (therapeutic trial of less than 12 weeks), was published
before 1988, had wrong study design, or did not address a Key
Question. (There were 8 more exclusions.)
From among the thousands of studies
that have been conducted, the criteria limited the review to a
scant 64 studies. Some of the landmark studies that were excluded
were all of the studies demonstrating immune dysfunction (e.g. NK
cell deficiency studies by Brenu et al.), studies of viral
reactivation and antiviral treatments (e.g. all Lerner and Jessop
studies, Kerr parvovirus B19 study), studies documenting brain
abnormalities (e.g. Lange's MRI study), and all of the papers
published by Tom Kindlon on harms associated with GET and CBT. Not
even appearing on the excluded list were the ground-breaking 2-day
CPET studies conducted by Keller, Stevens and Snell, Peckerman's
cardiac insufficiency studies, and the recent Watanabe study on CNS
inflammation.
The fact that some of the most
significant studies in the ME/CFS literature did not even appear on
the excluded list was mind-boggling. Of the studies that
appeared on the exclusion list, the reasons given were various, but
among the most frequently cited were that the studies did not address
the Key Questions. Yet, several studies that directly addressed the Key
Questions were omitted (for example, 2-Day CPET studies were not
even considered), while studies that did not directly address the Key Questions were included. This arbitrariness permeated the entire study selection
process.
(Going though the studies that were
accepted I found three that did not meet the criteria for inclusion
without reading further than the first page. I also found studies in
the excluded section that met the criteria. Having no experience with
ME/CFS, the panel lacked the ability to distinguish relevant from
irrelevant studies.)
Point #4: Contradictory and unsupported conclusions
In terms of treatment, the report was heavily weighted toward psychological studies. Out of the 36 studies used to address Key Question 2, 14 concerned CBT. Considering that the PACE trial was included, but not any of its critiques, it is not surprising that the report favored CBT:
In terms of Key Question 1, the report suffered from similar inconsistencies, For example, the report concludes with the statement that “the negative effects of being given a diagnosis of ME/CFS appear to be ... universal.”
Point #4: Contradictory and unsupported conclusions
In terms of treatment, the report was heavily weighted toward psychological studies. Out of the 36 studies used to address Key Question 2, 14 concerned CBT. Considering that the PACE trial was included, but not any of its critiques, it is not surprising that the report favored CBT:
"Based on 13 trials, cognitive and behavioral therapy (CBT), either group or individual; self-instruction booklets; pragmatic rehabilitation: peer-to-peer counseling: and symptom consultation provide improvement in fatigue, function, quality of life, and employment in adult patients with ME/CFS."Yet, after a detailed examination of the actual results of the trials, the report went on to conclude that:
"In summary, head-to-head trials had mixed results with two trials finding improvement with GET, two trials finding improvement with CBT, and one trial finding no differences between CBT, GET, and usual care. In considering non-head-to-head trial data, there is low strength evidence that CBT and GET provide similar improvement in measures of fatigue and/or functioning."Those reading this report will base their recommendations on the first statement, as the second statement requires wading through a lot of statistics. They won't even realize that the conclusion that CBT provides "improvement in fatigue, function, quality, and employment" is ultimately derived from the results of a single, deeply manipulated, study. (The PACE trial.)
In terms of Key Question 1, the report suffered from similar inconsistencies, For example, the report concludes with the statement that “the negative effects of being given a diagnosis of ME/CFS appear to be ... universal.”
I could not find a single study from
among the list of included studies that would support the conclusion
that the diagnosis of ME/CFS had negative effects universally.
In the Asbring and Narvanen study entitled “Women's experiences of stigma in relation to chronic fatigue
syndrome and fibromyalgia,” the authors concluded that
“ The women experienced stigmatization primarily before receiving a
diagnosis.” [Emphasis mine] In addition, they stated that “Stigma
consisted of questioning the veracity, morality, and accuracy of
patient symptom descriptions and of psychologizing symptoms.”
The Dickson et al. study, “Stigma and
the delegitimation experience: An interpretative phenomenological
analysis of people living with chronic fatigue syndrome,” reported
that “participants reported delay, negotiation and debate over
diagnosis: further, they perceived their GPs to be sceptical,
disrespectful and to be lacking in knowledge and interpersonal
skills.” [Emphasis mine]
In the Green et al. study, “Stigma and
Chronic Fatigue Syndrome,” the authors state that “Most subjects
(77%) were labeled as ‘psychological cases’ by one or more of the
physicians (mean = 8) consulted, but of the 4 stigma measures, only
disclosure was related to physician labeling.” This means that
patients only felt stigmatized by their physicians after they
attempted to educate them about ME/CFS.
There was nothing in these studies to support the claim that the diagnosis itself had negative effects. Rather, it was the delay in diagnosis, and subsequent debate on the part of family and physicians – as well as the delegitimation resulting from a trivializing name – that led to negative effects.
In sum
The conclusions reached in this review
were the result of a poorly framed set of key questions, a literature
search that managed to exclude the most fundamental research studies,
and a misinterpretation of the studies that were eventually
deemed acceptable for inclusion. As a whole, this report is fundamentally, and irredeemably, flawed - even given its narrow search results.
These major shortcomings are the
inevitable result of appointing a
group of people who have no expertise in ME/CFS to evaluate 26 years
of research. ME/CFS is a disease that demands expertise. It cannot be
evaluated be a panel of non-experts.
To find out what you can do to oppose the P2P, click HERE.
To find out what you can do to oppose the P2P, click HERE.