All of these attempts have been quashed. Nonetheless, Mr. Matthees continues to make his case, and make it well. [See below]
There are many reasons to demand that the PACE trial submit its full results, not the least of which is that the published results of the PACE trial are misleading (if not outright tweaked). Bad science is one thing, but the wholesale acceptance of bad science is something else entirely.
With the spate of publicity surrounding the trial, doctors, clinics, and health agencies have uncritically embraced CBT and graded exercise as legitimate treatments for ME/CFS. Here are just a few examples in the US:
"The most effective treatment for chronic fatigue syndrome appears to be a two-pronged approach that combines psychological counseling with a gentle exercise program.
CDC Toolkit
- Graded exercise. A physical therapist can help determine what types of exercise are best for you. Inactive people often begin with range-of-motion and stretching exercises for just a few minutes a day. If you're exhausted the next day, you're doing too much. Your strength and endurance will improve as you gradually increase the intensity of your exercise over time.
- Psychological counseling. Talking with a counselor can help you figure out options to work around some of the limitations that chronic fatigue syndrome imposes on you. Feeling more in control of your life can improve your outlook dramatically."
"Cognitive behavioral therapy, or CBT, is an individualized, structured, goal-oriented form of therapy often prescribed to help chronically ill patients cope with illness and develop behaviors and strategies that help improve symptoms. CBT has been shown to be effective for some patients with CFS, but it must be paced, personalized, and tailored to the individual’s level.
Graded exercise therapy (GET) has shown to be very helpful to some CFS patients. Graded activity and exercise is defined as starting from a very low, basic level of exercise and/or activity and gradually increasing it to a level where people can go about their daily life."University of Maryland Medical Center
"Seeing a therapist who is trained in cognitive-behavioral therapy (CBT) can help CFS patients regain a sense of control over their lives. A number of studies [it added] have reported the benefits of a graded exercise program, in which patients gradually perform more intense exercises as their abilities improve. Research has found that most CFS patients who are able to engage in exercise, particularly aerobic exercise, report less fatigue and better daily functioning and fitness. Exercise works best for CFS when combined with CBT and education."
With our most respected agencies and clinics recommending CBT and GET it is no wonder that virtually all of the treatment information found online reflects the results of the PACE trial.
In short, the PACE trial is more than bad science. It is a piece of propaganda that is directly modeled after the Big Lie: “If you tell a lie big enough and keep repeating it, people will eventually come to believe it." (Joseph Goebbels)
The lie that cognitive behavioral therapy and GET are "the most effective therapies," "appear to help," "are very helpful" for ME/CFS is one that will continue to inform physicians and harm patients as long as the actual results of the PACE trial remain under wraps.
[Please see below for Mr. Matthees' argument for why full disclosure is in the public interest.]
From: Mr Matthees
Dr. Roach (syndicated health columnist)
"If, after a thorough evaluation, your diagnosis turns out to be chronic fatigue syndrome, the most effective therapies are cognitive behavioral therapy and a graded exercise program."
"Two types of therapy appear to benefit CFS patients. One is psychological counseling to help you cope with CFS and improve your mindset. The other is physical therapy. A physical therapist can evaluate you and create an exercise routine for you that gradually increases in intensity. This is known as graded exercise therapy, or GET."Medicinenet
"In general the therapy will be a combination of psychological counseling (to help with the day-to-day burden CFS imposes on the patient's life) and mild, guided exercise..."
In short, the PACE trial is more than bad science. It is a piece of propaganda that is directly modeled after the Big Lie: “If you tell a lie big enough and keep repeating it, people will eventually come to believe it." (Joseph Goebbels)
The lie that cognitive behavioral therapy and GET are "the most effective therapies," "appear to help," "are very helpful" for ME/CFS is one that will continue to inform physicians and harm patients as long as the actual results of the PACE trial remain under wraps.
[Please see below for Mr. Matthees' argument for why full disclosure is in the public interest.]
Dear Mr Smallcombe / QMUL,
Thankyou for the response to FOI 2014/F73, which was refused as
exempt under s.40(2) and s.41(1) of the Freedom of Information Act
2000 (FOIA), drawing upon the the Data Protection Act 1998 (DPA),
and with references to GMC and MRC guidance: "as the information
consists of sensitive medical data provided in confidence by
participants in the PACE trial" and cannot be fairly processed by
the data controller for purposes other than what it was explicitly
collected for. Unfortunately, I am dissatisfied with the response
and would like to request an internal review and further
clarification with reference to the following:
• It is entirely possible to disclose anonymised trial data without
breaching the DPA or fairness of the FOIA, the DPA's definition of
personal data cannot be extended to cover situations where the
disclosed data does not identify any individual, the disclosure of
anonymised data is not a disclosure of personal data even when the
data controller holds the key to re-identification, consent is not
necessary to release anonymised data when it is unlikely to lead to
re-identification, and the DPA does not require anonymisation to be
completely risk free but mitigated until the risk of
re-identification is remote. [1] This FOI request involves a
heavily redacted dataset with most variables removed, or a
selection of trial data which has undergone de-identification /
anonymisation, and ceases to be personal data at the point of
disclosure so the DPA does not apply. [2]
• QMUL have claimed that re-identification from the requested
(pseudo)anonymised data is a significant risk. However, there is no
obvious or plausible method for a member of the public to do this
without additional data that is held securely at QMUL and highly
unlikely ever to come into the possession of myself or other
members of the public. According to the ICO's Knowledge Base on FOI
Policy with reference to the Data Protection Act, the onus is on
public authorities to explain how re-identification would occur.
[3] Speculative assertions about alleged 'extremists' may reflect
the potential existence of a 'motivated intruder', but if such
people exist they are highly unlikely to succeed in
re-identification and therefore such speculation is irrelevant. In
a previous unrelated ICO ruling, the Commissioner attempted to play
the role of 'motivated intruder' and was unsuccessful. [4] Any fair
attempts to play the role of 'motivated intruder' for the purposes
of assessing this FOI request would also fail. The selected data
being requested was chosen so that the results could be calculated
from the bare minimum of data required (except without the optional
criteria for recovery from a clinical perspective, and with the
addition of 6MWD data).
• It is also the responsibility of the public authority to
establish how disclosure would lead to adverse consequences. QMUL
have stated that disclosure without explicit consent is unfair and
"very likely" to cause "great distress" to participants to their
obvious detriment, with adverse consequences such as revealing that
an individual suffers from a certain health condition, and
therefore constitutes a probable actionable legal offense. These
assertions appear to be based primarily on the assumption that the
requested data is strictly personal data that can identity trial
participants when disclosed. As the requested data would not
plausibly lead to identification of any individuals, the given
hypothetical examples of adverse consequences do not apply, and it
remains unclear how disclosure would constitute a serious invasion
of privacy causing considerable distress if no individuals can be
identified. The fairness test can be satisfied by removing
identifiable information about individuals and/or by anonymising
the data, "for example, by removing the name but leaving the rest
of the information". [5]
• Much of the requested data is already one step removed from the
information directly provided by trial participants e.g. fatigue
and physical function numerical scores are calculated from multiple
answers to different questions on a paper questionnaire, the
specific answers of which cannot be reliably extrapolated from the
summed numerical scores. The requested data consists of either
dichotomous or continuous values which are common and/or
fluctuating (it is not comparable to genetic information or a
National Insurance Number). The data is quantitative not
qualitative; it is not for example the written accounts of personal
opinions, identifiable contextual experiences about personal lives,
or the handwriting of living individuals.
• QMUL stated that explicit consent will not be requested from
trial participants for this particular FOI request. Assuming that
such consent is actually necessary for the release of
non-identifiable anonymised trial data (which according to multiple
references no longer qualifies to be classified as personal data),
the Ministry of Justice guidance on s.40 also points out that "a
public authority should not be able to engineer a situation in
which data cannot be disclosed by failing to notify the data
subjects". [5]
• QMUL stated that "Principle 2 of the Data Protection Act 1998
also does not allow data controllers to process personal data for
further incompatible purposes." However, other public authorities
tried to use this argument to refuse the disclosure of anonymised
data and failed, because anonymisation and/or redaction did not
count as a form of processing. [6]
• The GMC's guidance on confidentiality and MRC's guidance on data
sharing, cited by QMUL, did not appear to prohibit the disclosure
of non-identifiable anonymised data to members of the public.
Obviously this does not mean unrestricted access to all data
collected, but it certainly opens up the possibility of releasing
selected non-identifiable anonymised data.
• The implications of anonymisation on confidentiality is somewhat
less clear, but the s.41 exemption can only be applied if there is
a good chance of a successful actionable breach of confidence, [7]
which QMUL believes is the case. However, at least 2 expert
commentaries suggest that anonymisation of confidential data can
replace the need for consent, and can change the nature of the data
so that in most contexts it is no longer 'personal data' and thus
not subject to the legal duties of data protection. [8][9] It it
also appears that the public interest can override the requirement
for confidentiality. [10][11]
• The public interest strongly favours disclosure, but the argument
is complex and involves specific details about the trial which
cannot be easily summarised into a paragraph. Therefore I will add
an additional annotation below describing the public interest
argument in about 2800 words and providing appropriate
references. Patients and clinical commissioners have a right to
accurate information about treatments promoted to them as
rehabilitative or potentially curative, but it can be demonstrated
that the (apparently post hoc and possibly unapproved)
redefinitions of improvement and recovery in the PACE trial were
too lax (e.g. recovery thresholds overlapping with the trial
eligibility criteria for severe chronic disabling fatigue and not
guaranteeing no longer having CFS), that these outcomes were
inaccurately presented as based on strict or conservative
thresholds, and that the most controversial change to the physical
function criteria was erroneously based on a misinterpretation of
summary statistics from a population study. The FOIA appears to be
most plausible method for finding out the results as promised in
the original stricter PACE trial protocol. It is doubtful whether
disclosure would actually deter future research. Conversely, it
could be counter-argued that research candidates may feel
discouraged from participating in controversial research topics if
previous trials have involved major, questionable, and possibly
unapproved, deviations from the pre-approved original protocol.
Yours sincerely, Mr Matthees.
References (abbreviated to save space):
1.
http://ico.org.uk/for_organisations/data...
2.
http://ico.org.uk/news/blog/2012/anonymi...
3.
http://ico.org.uk/foikb/PolicyLines/FOIP...
4.
http://ico.org.uk/~/media/documents/deci...
5.
http://www.justice.gov.uk/downloads/info...
6. http://www.cloudlegal.ccls.qmul.ac.uk/Re...
7.
http://www.justice.gov.uk/downloads/info...
8. http://www.apira.co.uk/userfiles/files/C...
9.
http://www.patient.co.uk/doctor/records-...
10.
http://ico.org.uk/for_organisations/guid...
11.
http://ico.org.uk/for_organisations/guid...
Thankyou for the response to FOI 2014/F73, which was refused as
exempt under s.40(2) and s.41(1) of the Freedom of Information Act
2000 (FOIA), drawing upon the the Data Protection Act 1998 (DPA),
and with references to GMC and MRC guidance: "as the information
consists of sensitive medical data provided in confidence by
participants in the PACE trial" and cannot be fairly processed by
the data controller for purposes other than what it was explicitly
collected for. Unfortunately, I am dissatisfied with the response
and would like to request an internal review and further
clarification with reference to the following:
• It is entirely possible to disclose anonymised trial data without
breaching the DPA or fairness of the FOIA, the DPA's definition of
personal data cannot be extended to cover situations where the
disclosed data does not identify any individual, the disclosure of
anonymised data is not a disclosure of personal data even when the
data controller holds the key to re-identification, consent is not
necessary to release anonymised data when it is unlikely to lead to
re-identification, and the DPA does not require anonymisation to be
completely risk free but mitigated until the risk of
re-identification is remote. [1] This FOI request involves a
heavily redacted dataset with most variables removed, or a
selection of trial data which has undergone de-identification /
anonymisation, and ceases to be personal data at the point of
disclosure so the DPA does not apply. [2]
• QMUL have claimed that re-identification from the requested
(pseudo)anonymised data is a significant risk. However, there is no
obvious or plausible method for a member of the public to do this
without additional data that is held securely at QMUL and highly
unlikely ever to come into the possession of myself or other
members of the public. According to the ICO's Knowledge Base on FOI
Policy with reference to the Data Protection Act, the onus is on
public authorities to explain how re-identification would occur.
[3] Speculative assertions about alleged 'extremists' may reflect
the potential existence of a 'motivated intruder', but if such
people exist they are highly unlikely to succeed in
re-identification and therefore such speculation is irrelevant. In
a previous unrelated ICO ruling, the Commissioner attempted to play
the role of 'motivated intruder' and was unsuccessful. [4] Any fair
attempts to play the role of 'motivated intruder' for the purposes
of assessing this FOI request would also fail. The selected data
being requested was chosen so that the results could be calculated
from the bare minimum of data required (except without the optional
criteria for recovery from a clinical perspective, and with the
addition of 6MWD data).
• It is also the responsibility of the public authority to
establish how disclosure would lead to adverse consequences. QMUL
have stated that disclosure without explicit consent is unfair and
"very likely" to cause "great distress" to participants to their
obvious detriment, with adverse consequences such as revealing that
an individual suffers from a certain health condition, and
therefore constitutes a probable actionable legal offense. These
assertions appear to be based primarily on the assumption that the
requested data is strictly personal data that can identity trial
participants when disclosed. As the requested data would not
plausibly lead to identification of any individuals, the given
hypothetical examples of adverse consequences do not apply, and it
remains unclear how disclosure would constitute a serious invasion
of privacy causing considerable distress if no individuals can be
identified. The fairness test can be satisfied by removing
identifiable information about individuals and/or by anonymising
the data, "for example, by removing the name but leaving the rest
of the information". [5]
• Much of the requested data is already one step removed from the
information directly provided by trial participants e.g. fatigue
and physical function numerical scores are calculated from multiple
answers to different questions on a paper questionnaire, the
specific answers of which cannot be reliably extrapolated from the
summed numerical scores. The requested data consists of either
dichotomous or continuous values which are common and/or
fluctuating (it is not comparable to genetic information or a
National Insurance Number). The data is quantitative not
qualitative; it is not for example the written accounts of personal
opinions, identifiable contextual experiences about personal lives,
or the handwriting of living individuals.
• QMUL stated that explicit consent will not be requested from
trial participants for this particular FOI request. Assuming that
such consent is actually necessary for the release of
non-identifiable anonymised trial data (which according to multiple
references no longer qualifies to be classified as personal data),
the Ministry of Justice guidance on s.40 also points out that "a
public authority should not be able to engineer a situation in
which data cannot be disclosed by failing to notify the data
subjects". [5]
• QMUL stated that "Principle 2 of the Data Protection Act 1998
also does not allow data controllers to process personal data for
further incompatible purposes." However, other public authorities
tried to use this argument to refuse the disclosure of anonymised
data and failed, because anonymisation and/or redaction did not
count as a form of processing. [6]
• The GMC's guidance on confidentiality and MRC's guidance on data
sharing, cited by QMUL, did not appear to prohibit the disclosure
of non-identifiable anonymised data to members of the public.
Obviously this does not mean unrestricted access to all data
collected, but it certainly opens up the possibility of releasing
selected non-identifiable anonymised data.
• The implications of anonymisation on confidentiality is somewhat
less clear, but the s.41 exemption can only be applied if there is
a good chance of a successful actionable breach of confidence, [7]
which QMUL believes is the case. However, at least 2 expert
commentaries suggest that anonymisation of confidential data can
replace the need for consent, and can change the nature of the data
so that in most contexts it is no longer 'personal data' and thus
not subject to the legal duties of data protection. [8][9] It it
also appears that the public interest can override the requirement
for confidentiality. [10][11]
• The public interest strongly favours disclosure, but the argument
is complex and involves specific details about the trial which
cannot be easily summarised into a paragraph. Therefore I will add
an additional annotation below describing the public interest
argument in about 2800 words and providing appropriate
references. Patients and clinical commissioners have a right to
accurate information about treatments promoted to them as
rehabilitative or potentially curative, but it can be demonstrated
that the (apparently post hoc and possibly unapproved)
redefinitions of improvement and recovery in the PACE trial were
too lax (e.g. recovery thresholds overlapping with the trial
eligibility criteria for severe chronic disabling fatigue and not
guaranteeing no longer having CFS), that these outcomes were
inaccurately presented as based on strict or conservative
thresholds, and that the most controversial change to the physical
function criteria was erroneously based on a misinterpretation of
summary statistics from a population study. The FOIA appears to be
most plausible method for finding out the results as promised in
the original stricter PACE trial protocol. It is doubtful whether
disclosure would actually deter future research. Conversely, it
could be counter-argued that research candidates may feel
discouraged from participating in controversial research topics if
previous trials have involved major, questionable, and possibly
unapproved, deviations from the pre-approved original protocol.
Yours sincerely, Mr Matthees.
References (abbreviated to save space):
1.
http://ico.org.uk/for_organisations/data...
2.
http://ico.org.uk/news/blog/2012/anonymi...
3.
http://ico.org.uk/foikb/PolicyLines/FOIP...
4.
http://ico.org.uk/~/media/documents/deci...
5.
http://www.justice.gov.uk/downloads/info...
6. http://www.cloudlegal.ccls.qmul.ac.uk/Re...
7.
http://www.justice.gov.uk/downloads/info...
8. http://www.apira.co.uk/userfiles/files/C...
9.
http://www.patient.co.uk/doctor/records-...
10.
http://ico.org.uk/for_organisations/guid...
11.
http://ico.org.uk/for_organisations/guid...
From: Mr Matthees
Reasons why the public interest strongly favours disclosure of the
requested data:
(FOI 2014/F73) 18th June 2014
Assertions about complete recovery and/or functional remission from
any chronic debilitating illness with a poor prognosis that is
regarded as difficult to treat should be taken seriously and be
based on reasonably stringent definitions. However, a recent 2014
systematic review of studies on recovery from CFS (including the
PACE trial) concluded that in general what the literature defined
as 'recovery' is better described as modest clinical improvement
only. There was no guarantee of 'recovery' per se, as
classification was based on limited assessments, less than a full
restoration of health, and self-reports lacking objective measures
in function which when used in behavioural intervention studies
suggested no changes (prompting the authors to conclude that these
therapies for CFS were not rehabilitative as often claimed). [1] An
earlier 2012 systematic review concluded that a "comprehensive
rehabilitation programme only rarely results in full recovery". [2]
The PACE trial has been repeatedly presented as offering
'definitive' answers on the controversial issue of 'rehabilitative'
treatments for CFS (e.g. in the official website FAQ, [3] in the
trial statistical analysis plan, [4] by the Science Media Centre,
[5] and on ABC radio [6]). The published definition of
recovery/remission was presented by the principal investigators
White et al. as 'comprehensive and conservative' and purported to
use stricter thresholds than a previous study on recovery from CFS
by Knoop et al. published in 2007. [7] However, multiple
significant issues have been identified with the recovery criteria
which strongly challenge or contradict these presentations and have
not been adequately addressed by the authors. [1,8,9] Disclosure of
the requested data will greatly help the resolution of these
issues.
The thresholds used for the 'normal range' score of fatigue and
physical function inappropriately overlapped with the trial
eligibility criteria for 'severe fatigue' and 'significant
disability'. The recovery definition allowed participants to be
classified as 'recovered' without reporting clinically significant
improvements to fatigue and physical function, as such improvements
were not required and allowed a 5 point decline in physical
function. No longer meeting Oxford criteria for CFS in the trial
did not necessarily mean no longer meeting Oxford criteria or
suffering from CFS in the clinic, because additional criteria for
fatigue and physical function were required, and participants were
classified as 'no longer meeting Oxford criteria' if they failed to
meet a single one of these thresholds e.g. moving from a score of
65 to 70 points in physical function but remaining unwell. 11% of
excluded candidates failed to meet these additional criteria
despite otherwise meeting Oxford criteria, which itself also
requires fatigue to be the only principal symptom (which is not a
requirement of any other CFS case definition. [10] and 80% of
candidates who were definitely or provisionally diagnosed with CFS
before the trial were excluded from participation, with the most
common reason being not meeting Oxford criteria for CFS).
Improvement on the clinical global impression scale does not
guarantee a recovery from CFS or any improvement in the primary
outcome measures of fatigue and physical function. The optional
requirements of not meeting CDC criteria for CFS or London criteria
for ME were superfluous because these were not an entry
requirement, tend to be more difficult to meet than the Oxford
criteria in the first place, and were not applied properly in the
trial. [7,11]
The relevant trial oversight bodies approved the original 2007
protocol published in BioMed Central, which included a much more
stringent definition of clinically significant improvement
('positive outcome') and complete 'recovery'. [12] According to
BioMed Central, "publishing study protocols will help to improve
the standard of medical research by ... enabling readers to compare
what was originally intended with what was actually done, thus
preventing both 'data dredging' and post-hoc revisions of study
aims". [13] The purpose of pre-publishing a protocol is to avoid
accusations of cherry picking the results, but when the protocol is
ignored this clearly cannot be guaranteed. The thresholds for
clinical improvement on an individual patient level for the primary
measures of fatigue and physical function were abandoned and
replaced with weaker thresholds which have been criticized for
being minimal. [14,15] Similarly, all components of the recovery
definition were significantly modified in a manner which made them
substantially less stringent and easier to qualify. Of particular
note, the threshold for normal physical function was dropped from
85 to 60 out of 100 points, a score low enough that 13% of
participants were already within the 'normal range' at baseline
despite meeting trial eligibility criteria for 'significant
disability' (65 points or less). [16] In contrast, participants
originally had to improve a minimum of 20 to 25 points to physical
function to be classified as recovered. Other researchers of CBT
for CFS have even classified a score of 60 to 70 points as
indicative of 'severe' impairments in physical function. [17,18]
Professor White previously requested that the threshold for a
'positive outcome' in physical function (later abandoned) be raised
from 70 to 75 points, because the entry criteria had been raised
from 60 to 65 to increase recruitment, so a 10 point gap between
entry criteria and 'positive outcome' scores was needed to avoid a
'trivial' difference. [19,20] Now there is a 5 point gap in the
opposite direction, which cannot not be described as a strict or
'conservative' threshold. Although it has been argued that
protocols can change in light of new information, it is unclear how
any of these changes could "more accurately reflect recovery" as
asserted in the paper by White et al. [7] Furthermore, as the
changes to the definition of recovery published in 2013 appear to
be largely based on controversial post hoc analyses conducted for a
previous paper on the trial results published in 2011, [21] it is
unclear whether these major deviations from the protocol were
approved by the relevant trial oversight bodies, and this confusion
surrounding the timing of changes has reached the level of
parliamentary debate in the House of Lords. [22]
As a previous claim made in the Lancet paper about the normative
dataset used from a population study had turned out to incorrect,
[23] it seemed prudent to examine the justification behind the most
controversial change to the recovery criteria. White et al.
asserted that the change to the threshold of normal physical
function was justified because a score of ≥85 "would mean that
approximately half the general working age population would fall
outside the normal range". [7] However this is incorrect, as
independent analyses of the English normative dataset cited by
White et al. revealed that over half score the maximum of 100
points. The median(IQR) score for the general working age
population sample is 100(90-100) not about 85 as implied (which
suggested an erroneous assumption that the mean and median were
equivalent), and only about 18% of the general working age
population sample had a score under 85. [24] The original threshold
of >=85 points appears to be reasonable and appropriate, as it
"represents the ability to carry out moderate activities, such as
lifting a table, carrying purchases, or bowling, without
limitations". [25] 92% of the healthy working age population score
85 points or more, and 61% score the maximum of 100 points. The
mean(SD) and median(IQR) scores for this population are 95.0(10.2)
and 100(95-100) respectively, with scores under 80 appearing to be
extreme outliers when defined as more than 3 x IQR below the
median. [24] It is highly unlikely that the PACE trial participants
classified as 'recovered' have a similar distribution of scores
compared to a healthy working age population.
White et al. stated that "we derived a mean (S.D.) score of 84 (24)
for the whole sample, giving a normal range of 60 or above for
physical function" and asserted that this sample was
"demographically representative". [7] However, the 'whole sample'
was a general population which included the elderly and chronically
disabled, [26] with age and illness having a major impact on
physical function scores in a way which decreases the mean and
increases the standard deviation, therefore lowering the threshold
of 'normal'. The mean(SD) age was 48.3(19.0) years, 32% were aged
60 years or more, and 22% reported chronic debilitating illnesses
(many of which would have medically excluded candidates from
participating in the PACE trial). [24][26] Whereas PACE trial
participants had a mean(SD) age of about 38(12) years at baseline,
only 3% were aged 60 years or more, [27] and were previously
screened for common chronic debilitating illnesses in the
population which would have excluded them from a CFS diagnosis.
Although described as a 'conventional' method, [23] White et el.
have applied a simple parametric statistical method to a dataset
without any apparent consideration for what the authors of the
cited paper (Bowling et al.) described as a heavily skewed
distribution, [26] which was specifically warned against in a paper
previously co-authored by Professor White [28] and has been
described elsewhere as a "fundamental misuse of statistics". [29]
Furthermore, the use of normative data from a general population
sample with important demographic differences to PACE trial
participants (age distribution and presence of common debilitating
illnesses) has never been justified in any of the publicly
available PACE trial literature. It is unclear why the authors did
not stop and think twice before using a 'recovery' threshold that
was unusually low and overlapped with their own trial criteria for
'significant disability'. A score of 60 points means reporting
significant limitations in multiple domains (somewhere between
minor limitations for 8/10 questions or major limitations in 4/10
questions), [30] which is unusual for healthy people of working age
and an unsuitable threshold for a genuine recovery. White et al.
[7] incorrectly claimed that their threshold was more
"conservative" i.e. stricter than the previous work of Knoop et al.
[28] The latter paper actually used the same mean plus or minus 1
SD formula as PACE did (not mean plus or minus 2 SD as claimed by
White et al.), and relied on a healthy population instead of a
general population to calculate a higher threshold of 80 points in
physical function as the normal threshold for recovered. Similarly,
serious questions have also been raised about the suitability of
the threshold for normal fatigue and the population used to derive
it. [8,31-33]
In response to the paper on 'recovery', Dr Esther Crawley from the
University of Bristol said that "Every patient with CFS/ME wants to
know how likely they are to recover." [34] Yet, many patients were
rather unsatisfied with the major deviations from the previously
established protocol, questioned the 'normal range' in particular,
and wanted to know the 'positive outcomes' and more importantly the
recovery rates as previously defined more stringently. A collection
of patient charities made a FOI request for this information in
2011, which included the results according to the original recovery
criteria [35] but were refused on the grounds that this information
was exempt under s.22 of the FOIA i.e. due for future publication.
[36] A similar FOI request in 2012 was refused on the grounds that
the information was not held in final form because the definition
of recovery had changed with a pending paper and there was no
intention on publishing the requested information in the future
(the refusal notice also incorrectly claimed that some of the
changes made the definition more stringent). [37] Another FOI
request in 2013 was refused on the grounds that the raw data
required to calculate these outcomes does exist but would require
over 18 hours to do so. [38] Therefore, this FOI request is for
selected raw data so that these calculations can be done without
QMUL.
ME/CFS is regarded as a controversial subject, but this controversy
is only further fuelled by the lack of transparency over trial
results presented as 'definitive' and the failure to publish the
measures specified in the original approved protocol. Given that
the published recovery thresholds appear to be fundamentally based
on previous post hoc analyses, coincide with less than expected
clinical improvements, and are generally at or below the level of
the trial entry criteria, it is difficult to believe that the
accusations of cherry picking or intentionally misleading
vulnerable patients and clinical commissioners (irrespective of
whether it is true or not) will simply go away without the
publication of these stricter outcomes. It is critical that
sufficient data is placed in the public domain to allow patients
and clinical commissioners to accurately assess recovery and the
sensitivity to any particular threshold.
There have been recent calls for medical research to be more
transparent and accessible and accountable, as per the AllTrials
campaign (www.alltrials.net). Although this does not necessarily
mean unrestricted public access to all the data of a trial,
AllTrials calls for "All trials past and present should be
registered, and the full methods and the results reported." The
Wellcome Trust takes a step further and calls for the full release
of all trial data. [39] The public interest in transparency around
drug trials has been well established by the European Medicines
Agency and the same principles should apply to psychotherapy and/or
behavioural interventions. [40] The PACE trial was publicly funded
research and the (anonymised) data should be openly available to
the maximum practical extent. Answers to remaining questions in
science are generally gained from further replication, but the PACE
trial cost taxpayers £5m, and due to its high cost and large size,
it is highly unlikely that another similar trial will be conducted
anytime soon. Therefore, the collected data should be explored to
the maximum extent possible. Without voluntary transparency, the
task of finding out the results as promised in the original PACE
trial protocol depends on members of the public, and the FOIA
appears to be most plausible method for seeing this happen in the
foreseeable future.
The ongoing confusion and controversy is adding to the suffering of
patients, and getting to the bottom of this issue is important
whatever the outcome may be. The results and interpretations do not
just affect those who are curious about research, but have national
and perhaps even global ramifications. Patients and clinical
commissioners of this chronic debilitating illness have a right to
accurate information about treatments which are promoted to them as
rehabilitative and potentially curative. This is required for them
to assess and give informed consent for treatments, or make
informed decisions about health care. Lax definitions of
recovery/remission and clinical improvement lead to unreasonable
expectations from patients by those who provide their care. In a
similar study known as the FINE trial [41] (which released the
results according to its own published protocol and failed to show
significant improvements with therapies similar to and sharing
elements with CBT/GET tested in the PACE trial), some participants
had doubts about the (overly optimistic) treatment rationales, and
therapists reported becoming angry and blaming participants as "the
bastards don't want to get better". [42]
It is doubtful whether disclosure would actually deter future
research. Conversely, it could be counter-argued that research
candidates may feel discouraged from participating in controversial
research if previous trials have involved major, questionable, and
possibly unapproved, deviations from the pre-approved original
protocol which made it much easier for the tested therapies to
appear successful, coincided with less impressive than expected
results, and led to the results being exaggerated. For example, the
published rates of trial participants within the 'normal range' for
fatigue and physical function (which overlapped with trial
eligibility criteria for severe chronic fatigue and significant
disability) was presented in 2011 at a Lancet press conference with
the principal investigators as returning back to normal, [43] and
this was then widely misinterpreted as a complete recovery or cure
in the national news media e.g. [44,45] and medical journals e.g.
[46,47] The Lancet editorial which accompanied the 2011 paper on
the PACE trial results inaccurately claimed that the 'normal range'
was a strict criterion for recovery based on scores from healthy
people, [48] but the Press Complaints Commission later ruled that
this comment was misleading and breached Clause 1 (Accuracy) of the
Code. [49] Such repeated misstatements of fact have negative
implications for how patients are treated by doctors, how funding
decisions are made, and for scientific accuracy concerning recovery
from ME/CFS. A poll conducted on the ME Association website during
March 2011 revealed that 89% of 751 respondents were significantly
concerned that the PACE trial results would adversely affect
treatment within the NHS. [50]
Unless the PACE group themselves promptly publish the original
protocol-defined 'positive outcomes', the original protocol-defined
'recovery' rates, and summary statistics on those classified as
recovered (both versions) compared with appropriate summary
statistics of healthy populations with a similar age distribution
as trial participants, then the disclosure of the requested data
allowing others to do the necessary calculations is certainly in
the public interest. Given that the lax definition of 'recovery'
fundamentally depends on a threshold for 'normal' physical function
which appears to be seriously flawed and inaccurately presented as
strict or conservative, with the reason for abandoning the original
protocol-defined threshold found to be erroneously based on a
misinterpretation of summary statistics from a population study,
the requested data will be important to help the public (patients,
carers, research community, healthcare staff, et cetera) further
assess the degree and nature of improvements in the PACE trial.
Please help resolve this controversy once and for all by granting
this FOI request.
References (abbreviated to save space):
1. http://www.ncbi.nlm.nih.gov/pubmed/24791...
2. http://www.ncbi.nlm.nih.gov/pubmed/22725...
3. http://www.pacetrial.org/faq/faq2.html
4. http://www.trialsjournal.com/content/14/...
5.
http://www.sciencemediacentre.org/expert...
6.
http://www.abc.net.au/radionational/prog...
7. http://www.ncbi.nlm.nih.gov/pmc/articles...
8. http://www.ncbi.nlm.nih.gov/pubmed/23363...
9.
http://www.meassociation.org.uk/2013/07/...
10.
http://www.iacfsme.org/BULLETINWINTER201...
11. http://www.ncbi.nlm.nih.gov/pubmed/21334...
12. http://www.biomedcentral.com/1471-2377/7/6
13. http://www.biomedcentral.com/authors/pro...
14.
http://www.thelancet.com/journals/lancet...(11)60689-2/fulltext
15.
http://www.thelancet.com/journals/lancet...(11)60685-5/fulltext
16.
https://listserv.nodak.edu/cgi-bin/wa.ex...
17. http://www.ncbi.nlm.nih.gov/pubmed/22354...
18. http://eurpub.oxfordjournals.org/content...
19.
https://listserv.nodak.edu/cgi-bin/wa.ex...
20.
http://www.ico.org.uk/~/media/documents/...
21. http://www.ncbi.nlm.nih.gov/pmc/articles...
22.
https://www.whatdotheyknow.com/request/t...
23.
http://www.thelancet.com/journals/lancet...
24. http://dx.doi.org/10.5255/UKDA-SN-3660-1
25. http://www.ncbi.nlm.nih.gov/pubmed/9054791
26. http://www.ncbi.nlm.nih.gov/pubmed/10528...
27. http://www.bmj.com/content/347/bmj.f5963...
28. http://www.ncbi.nlm.nih.gov/pubmed/17426...
29. http://www.bmj.com/content/347/bmj.f5963...
30.
http://www.rand.org/health/surveys_tools...
31.
http://www.thelancet.com/journals/lancet...(11)60688-0/fulltext
32. http://evaluatingpace.phoenixrising.me/a...
33. http://www.meactionuk.org.uk/Normal-fati...
34.
http://www.sciencemediacentre.org/expert...
35. http://www.meassociation.org.uk/?p=6171
36.
http://www.meassociation.org.uk/wp-conte...
37.
https://www.whatdotheyknow.com/request/p...
38.
https://www.whatdotheyknow.com/request/p...
39.
http://blog.wellcome.ac.uk/2013/08/05/fu...
40. http://www.ema.europa.eu
41. http://www.ncbi.nlm.nih.gov/pmc/articles...
42. http://www.ncbi.nlm.nih.gov/pmc/articles...
43. http://www.meactionuk.org.uk/pacepressco...
44. http://www.ncbi.nlm.nih.gov/pmc/articles...
45. http://www.bmj.com/content/342/bmj.d1168...
46.
http://www.guardian.co.uk/society/2011/f...
47.
http://www.dailymail.co.uk/health/articl...
48.
http://www.thelancet.com/journals/lancet...
49. http://www.pcc.org.uk/news/index.html?ar...
50. http://www.meassociation.org.uk/archive-...
requested data:
(FOI 2014/F73) 18th June 2014
Assertions about complete recovery and/or functional remission from
any chronic debilitating illness with a poor prognosis that is
regarded as difficult to treat should be taken seriously and be
based on reasonably stringent definitions. However, a recent 2014
systematic review of studies on recovery from CFS (including the
PACE trial) concluded that in general what the literature defined
as 'recovery' is better described as modest clinical improvement
only. There was no guarantee of 'recovery' per se, as
classification was based on limited assessments, less than a full
restoration of health, and self-reports lacking objective measures
in function which when used in behavioural intervention studies
suggested no changes (prompting the authors to conclude that these
therapies for CFS were not rehabilitative as often claimed). [1] An
earlier 2012 systematic review concluded that a "comprehensive
rehabilitation programme only rarely results in full recovery". [2]
The PACE trial has been repeatedly presented as offering
'definitive' answers on the controversial issue of 'rehabilitative'
treatments for CFS (e.g. in the official website FAQ, [3] in the
trial statistical analysis plan, [4] by the Science Media Centre,
[5] and on ABC radio [6]). The published definition of
recovery/remission was presented by the principal investigators
White et al. as 'comprehensive and conservative' and purported to
use stricter thresholds than a previous study on recovery from CFS
by Knoop et al. published in 2007. [7] However, multiple
significant issues have been identified with the recovery criteria
which strongly challenge or contradict these presentations and have
not been adequately addressed by the authors. [1,8,9] Disclosure of
the requested data will greatly help the resolution of these
issues.
The thresholds used for the 'normal range' score of fatigue and
physical function inappropriately overlapped with the trial
eligibility criteria for 'severe fatigue' and 'significant
disability'. The recovery definition allowed participants to be
classified as 'recovered' without reporting clinically significant
improvements to fatigue and physical function, as such improvements
were not required and allowed a 5 point decline in physical
function. No longer meeting Oxford criteria for CFS in the trial
did not necessarily mean no longer meeting Oxford criteria or
suffering from CFS in the clinic, because additional criteria for
fatigue and physical function were required, and participants were
classified as 'no longer meeting Oxford criteria' if they failed to
meet a single one of these thresholds e.g. moving from a score of
65 to 70 points in physical function but remaining unwell. 11% of
excluded candidates failed to meet these additional criteria
despite otherwise meeting Oxford criteria, which itself also
requires fatigue to be the only principal symptom (which is not a
requirement of any other CFS case definition. [10] and 80% of
candidates who were definitely or provisionally diagnosed with CFS
before the trial were excluded from participation, with the most
common reason being not meeting Oxford criteria for CFS).
Improvement on the clinical global impression scale does not
guarantee a recovery from CFS or any improvement in the primary
outcome measures of fatigue and physical function. The optional
requirements of not meeting CDC criteria for CFS or London criteria
for ME were superfluous because these were not an entry
requirement, tend to be more difficult to meet than the Oxford
criteria in the first place, and were not applied properly in the
trial. [7,11]
The relevant trial oversight bodies approved the original 2007
protocol published in BioMed Central, which included a much more
stringent definition of clinically significant improvement
('positive outcome') and complete 'recovery'. [12] According to
BioMed Central, "publishing study protocols will help to improve
the standard of medical research by ... enabling readers to compare
what was originally intended with what was actually done, thus
preventing both 'data dredging' and post-hoc revisions of study
aims". [13] The purpose of pre-publishing a protocol is to avoid
accusations of cherry picking the results, but when the protocol is
ignored this clearly cannot be guaranteed. The thresholds for
clinical improvement on an individual patient level for the primary
measures of fatigue and physical function were abandoned and
replaced with weaker thresholds which have been criticized for
being minimal. [14,15] Similarly, all components of the recovery
definition were significantly modified in a manner which made them
substantially less stringent and easier to qualify. Of particular
note, the threshold for normal physical function was dropped from
85 to 60 out of 100 points, a score low enough that 13% of
participants were already within the 'normal range' at baseline
despite meeting trial eligibility criteria for 'significant
disability' (65 points or less). [16] In contrast, participants
originally had to improve a minimum of 20 to 25 points to physical
function to be classified as recovered. Other researchers of CBT
for CFS have even classified a score of 60 to 70 points as
indicative of 'severe' impairments in physical function. [17,18]
Professor White previously requested that the threshold for a
'positive outcome' in physical function (later abandoned) be raised
from 70 to 75 points, because the entry criteria had been raised
from 60 to 65 to increase recruitment, so a 10 point gap between
entry criteria and 'positive outcome' scores was needed to avoid a
'trivial' difference. [19,20] Now there is a 5 point gap in the
opposite direction, which cannot not be described as a strict or
'conservative' threshold. Although it has been argued that
protocols can change in light of new information, it is unclear how
any of these changes could "more accurately reflect recovery" as
asserted in the paper by White et al. [7] Furthermore, as the
changes to the definition of recovery published in 2013 appear to
be largely based on controversial post hoc analyses conducted for a
previous paper on the trial results published in 2011, [21] it is
unclear whether these major deviations from the protocol were
approved by the relevant trial oversight bodies, and this confusion
surrounding the timing of changes has reached the level of
parliamentary debate in the House of Lords. [22]
As a previous claim made in the Lancet paper about the normative
dataset used from a population study had turned out to incorrect,
[23] it seemed prudent to examine the justification behind the most
controversial change to the recovery criteria. White et al.
asserted that the change to the threshold of normal physical
function was justified because a score of ≥85 "would mean that
approximately half the general working age population would fall
outside the normal range". [7] However this is incorrect, as
independent analyses of the English normative dataset cited by
White et al. revealed that over half score the maximum of 100
points. The median(IQR) score for the general working age
population sample is 100(90-100) not about 85 as implied (which
suggested an erroneous assumption that the mean and median were
equivalent), and only about 18% of the general working age
population sample had a score under 85. [24] The original threshold
of >=85 points appears to be reasonable and appropriate, as it
"represents the ability to carry out moderate activities, such as
lifting a table, carrying purchases, or bowling, without
limitations". [25] 92% of the healthy working age population score
85 points or more, and 61% score the maximum of 100 points. The
mean(SD) and median(IQR) scores for this population are 95.0(10.2)
and 100(95-100) respectively, with scores under 80 appearing to be
extreme outliers when defined as more than 3 x IQR below the
median. [24] It is highly unlikely that the PACE trial participants
classified as 'recovered' have a similar distribution of scores
compared to a healthy working age population.
White et al. stated that "we derived a mean (S.D.) score of 84 (24)
for the whole sample, giving a normal range of 60 or above for
physical function" and asserted that this sample was
"demographically representative". [7] However, the 'whole sample'
was a general population which included the elderly and chronically
disabled, [26] with age and illness having a major impact on
physical function scores in a way which decreases the mean and
increases the standard deviation, therefore lowering the threshold
of 'normal'. The mean(SD) age was 48.3(19.0) years, 32% were aged
60 years or more, and 22% reported chronic debilitating illnesses
(many of which would have medically excluded candidates from
participating in the PACE trial). [24][26] Whereas PACE trial
participants had a mean(SD) age of about 38(12) years at baseline,
only 3% were aged 60 years or more, [27] and were previously
screened for common chronic debilitating illnesses in the
population which would have excluded them from a CFS diagnosis.
Although described as a 'conventional' method, [23] White et el.
have applied a simple parametric statistical method to a dataset
without any apparent consideration for what the authors of the
cited paper (Bowling et al.) described as a heavily skewed
distribution, [26] which was specifically warned against in a paper
previously co-authored by Professor White [28] and has been
described elsewhere as a "fundamental misuse of statistics". [29]
Furthermore, the use of normative data from a general population
sample with important demographic differences to PACE trial
participants (age distribution and presence of common debilitating
illnesses) has never been justified in any of the publicly
available PACE trial literature. It is unclear why the authors did
not stop and think twice before using a 'recovery' threshold that
was unusually low and overlapped with their own trial criteria for
'significant disability'. A score of 60 points means reporting
significant limitations in multiple domains (somewhere between
minor limitations for 8/10 questions or major limitations in 4/10
questions), [30] which is unusual for healthy people of working age
and an unsuitable threshold for a genuine recovery. White et al.
[7] incorrectly claimed that their threshold was more
"conservative" i.e. stricter than the previous work of Knoop et al.
[28] The latter paper actually used the same mean plus or minus 1
SD formula as PACE did (not mean plus or minus 2 SD as claimed by
White et al.), and relied on a healthy population instead of a
general population to calculate a higher threshold of 80 points in
physical function as the normal threshold for recovered. Similarly,
serious questions have also been raised about the suitability of
the threshold for normal fatigue and the population used to derive
it. [8,31-33]
In response to the paper on 'recovery', Dr Esther Crawley from the
University of Bristol said that "Every patient with CFS/ME wants to
know how likely they are to recover." [34] Yet, many patients were
rather unsatisfied with the major deviations from the previously
established protocol, questioned the 'normal range' in particular,
and wanted to know the 'positive outcomes' and more importantly the
recovery rates as previously defined more stringently. A collection
of patient charities made a FOI request for this information in
2011, which included the results according to the original recovery
criteria [35] but were refused on the grounds that this information
was exempt under s.22 of the FOIA i.e. due for future publication.
[36] A similar FOI request in 2012 was refused on the grounds that
the information was not held in final form because the definition
of recovery had changed with a pending paper and there was no
intention on publishing the requested information in the future
(the refusal notice also incorrectly claimed that some of the
changes made the definition more stringent). [37] Another FOI
request in 2013 was refused on the grounds that the raw data
required to calculate these outcomes does exist but would require
over 18 hours to do so. [38] Therefore, this FOI request is for
selected raw data so that these calculations can be done without
QMUL.
ME/CFS is regarded as a controversial subject, but this controversy
is only further fuelled by the lack of transparency over trial
results presented as 'definitive' and the failure to publish the
measures specified in the original approved protocol. Given that
the published recovery thresholds appear to be fundamentally based
on previous post hoc analyses, coincide with less than expected
clinical improvements, and are generally at or below the level of
the trial entry criteria, it is difficult to believe that the
accusations of cherry picking or intentionally misleading
vulnerable patients and clinical commissioners (irrespective of
whether it is true or not) will simply go away without the
publication of these stricter outcomes. It is critical that
sufficient data is placed in the public domain to allow patients
and clinical commissioners to accurately assess recovery and the
sensitivity to any particular threshold.
There have been recent calls for medical research to be more
transparent and accessible and accountable, as per the AllTrials
campaign (www.alltrials.net). Although this does not necessarily
mean unrestricted public access to all the data of a trial,
AllTrials calls for "All trials past and present should be
registered, and the full methods and the results reported." The
Wellcome Trust takes a step further and calls for the full release
of all trial data. [39] The public interest in transparency around
drug trials has been well established by the European Medicines
Agency and the same principles should apply to psychotherapy and/or
behavioural interventions. [40] The PACE trial was publicly funded
research and the (anonymised) data should be openly available to
the maximum practical extent. Answers to remaining questions in
science are generally gained from further replication, but the PACE
trial cost taxpayers £5m, and due to its high cost and large size,
it is highly unlikely that another similar trial will be conducted
anytime soon. Therefore, the collected data should be explored to
the maximum extent possible. Without voluntary transparency, the
task of finding out the results as promised in the original PACE
trial protocol depends on members of the public, and the FOIA
appears to be most plausible method for seeing this happen in the
foreseeable future.
The ongoing confusion and controversy is adding to the suffering of
patients, and getting to the bottom of this issue is important
whatever the outcome may be. The results and interpretations do not
just affect those who are curious about research, but have national
and perhaps even global ramifications. Patients and clinical
commissioners of this chronic debilitating illness have a right to
accurate information about treatments which are promoted to them as
rehabilitative and potentially curative. This is required for them
to assess and give informed consent for treatments, or make
informed decisions about health care. Lax definitions of
recovery/remission and clinical improvement lead to unreasonable
expectations from patients by those who provide their care. In a
similar study known as the FINE trial [41] (which released the
results according to its own published protocol and failed to show
significant improvements with therapies similar to and sharing
elements with CBT/GET tested in the PACE trial), some participants
had doubts about the (overly optimistic) treatment rationales, and
therapists reported becoming angry and blaming participants as "the
bastards don't want to get better". [42]
It is doubtful whether disclosure would actually deter future
research. Conversely, it could be counter-argued that research
candidates may feel discouraged from participating in controversial
research if previous trials have involved major, questionable, and
possibly unapproved, deviations from the pre-approved original
protocol which made it much easier for the tested therapies to
appear successful, coincided with less impressive than expected
results, and led to the results being exaggerated. For example, the
published rates of trial participants within the 'normal range' for
fatigue and physical function (which overlapped with trial
eligibility criteria for severe chronic fatigue and significant
disability) was presented in 2011 at a Lancet press conference with
the principal investigators as returning back to normal, [43] and
this was then widely misinterpreted as a complete recovery or cure
in the national news media e.g. [44,45] and medical journals e.g.
[46,47] The Lancet editorial which accompanied the 2011 paper on
the PACE trial results inaccurately claimed that the 'normal range'
was a strict criterion for recovery based on scores from healthy
people, [48] but the Press Complaints Commission later ruled that
this comment was misleading and breached Clause 1 (Accuracy) of the
Code. [49] Such repeated misstatements of fact have negative
implications for how patients are treated by doctors, how funding
decisions are made, and for scientific accuracy concerning recovery
from ME/CFS. A poll conducted on the ME Association website during
March 2011 revealed that 89% of 751 respondents were significantly
concerned that the PACE trial results would adversely affect
treatment within the NHS. [50]
Unless the PACE group themselves promptly publish the original
protocol-defined 'positive outcomes', the original protocol-defined
'recovery' rates, and summary statistics on those classified as
recovered (both versions) compared with appropriate summary
statistics of healthy populations with a similar age distribution
as trial participants, then the disclosure of the requested data
allowing others to do the necessary calculations is certainly in
the public interest. Given that the lax definition of 'recovery'
fundamentally depends on a threshold for 'normal' physical function
which appears to be seriously flawed and inaccurately presented as
strict or conservative, with the reason for abandoning the original
protocol-defined threshold found to be erroneously based on a
misinterpretation of summary statistics from a population study,
the requested data will be important to help the public (patients,
carers, research community, healthcare staff, et cetera) further
assess the degree and nature of improvements in the PACE trial.
Please help resolve this controversy once and for all by granting
this FOI request.
References (abbreviated to save space):
1. http://www.ncbi.nlm.nih.gov/pubmed/24791...
2. http://www.ncbi.nlm.nih.gov/pubmed/22725...
3. http://www.pacetrial.org/faq/faq2.html
4. http://www.trialsjournal.com/content/14/...
5.
http://www.sciencemediacentre.org/expert...
6.
http://www.abc.net.au/radionational/prog...
7. http://www.ncbi.nlm.nih.gov/pmc/articles...
8. http://www.ncbi.nlm.nih.gov/pubmed/23363...
9.
http://www.meassociation.org.uk/2013/07/...
10.
http://www.iacfsme.org/BULLETINWINTER201...
11. http://www.ncbi.nlm.nih.gov/pubmed/21334...
12. http://www.biomedcentral.com/1471-2377/7/6
13. http://www.biomedcentral.com/authors/pro...
14.
http://www.thelancet.com/journals/lancet...(11)60689-2/fulltext
15.
http://www.thelancet.com/journals/lancet...(11)60685-5/fulltext
16.
https://listserv.nodak.edu/cgi-bin/wa.ex...
17. http://www.ncbi.nlm.nih.gov/pubmed/22354...
18. http://eurpub.oxfordjournals.org/content...
19.
https://listserv.nodak.edu/cgi-bin/wa.ex...
20.
http://www.ico.org.uk/~/media/documents/...
21. http://www.ncbi.nlm.nih.gov/pmc/articles...
22.
https://www.whatdotheyknow.com/request/t...
23.
http://www.thelancet.com/journals/lancet...
24. http://dx.doi.org/10.5255/UKDA-SN-3660-1
25. http://www.ncbi.nlm.nih.gov/pubmed/9054791
26. http://www.ncbi.nlm.nih.gov/pubmed/10528...
27. http://www.bmj.com/content/347/bmj.f5963...
28. http://www.ncbi.nlm.nih.gov/pubmed/17426...
29. http://www.bmj.com/content/347/bmj.f5963...
30.
http://www.rand.org/health/surveys_tools...
31.
http://www.thelancet.com/journals/lancet...(11)60688-0/fulltext
32. http://evaluatingpace.phoenixrising.me/a...
33. http://www.meactionuk.org.uk/Normal-fati...
34.
http://www.sciencemediacentre.org/expert...
35. http://www.meassociation.org.uk/?p=6171
36.
http://www.meassociation.org.uk/wp-conte...
37.
https://www.whatdotheyknow.com/request/p...
38.
https://www.whatdotheyknow.com/request/p...
39.
http://blog.wellcome.ac.uk/2013/08/05/fu...
40. http://www.ema.europa.eu
41. http://www.ncbi.nlm.nih.gov/pmc/articles...
42. http://www.ncbi.nlm.nih.gov/pmc/articles...
43. http://www.meactionuk.org.uk/pacepressco...
44. http://www.ncbi.nlm.nih.gov/pmc/articles...
45. http://www.bmj.com/content/342/bmj.d1168...
46.
http://www.guardian.co.uk/society/2011/f...
47.
http://www.dailymail.co.uk/health/articl...
48.
http://www.thelancet.com/journals/lancet...
49. http://www.pcc.org.uk/news/index.html?ar...
50. http://www.meassociation.org.uk/archive-...
Mr Matthees left an annotation ()
• If there is any doubt about the ease of which some trial participants could be disqualified from meeting Oxford criteria at followup (for failing any additional ad hoc criteria bolted onto the Oxford criteria for the purposes of the trial), QMUL previously stated: "A score of 70 or more on the SF-36 sub-scale would mean that the participant did not meet Oxford criteria. Similarly having a score of 5 or less on the Chalder fatigue questionnaire would mean that the participant did not meet Oxford criteria." [1] This is not usually part of the Oxford criteria and means that no longer meeting Oxford criteria at followup in the trial could be achieved by borderline cases improving a single increment on either scale but who could still be significantly affected by CFS and still otherwise meet Oxford criteria outside the trial setting. In the original trial protocol, a fatigue score of 5/11 on the CFQ (bimodal scoring) and a score of 70/100 on the physical function sub-scale of the SF-36 were still both regarded as abnormal scores, and required a fatigue score of 3/11 and a physical function score of 85/100 to be classified as recovered. [2]
• Re: "11% of excluded candidates failed to meet these additional criteria despite otherwise meeting Oxford criteria". This was based on Figure 1 of the 2011 Lancet paper, [3] but it is not entirely clear how many excluded candidates failed to meet each stated reason for exclusion, as only one reason is given for each exclusion but it is plausible that excluded candidates failed more than one. Out of the 2517 candidates who were excluded (all of which had previously been definitely or provisionally diagnosed with CFS), 43% were excluded for not meeting Oxford criteria, 10% were excluded for scoring over 65/100 points in physical function, and 1% were excluded for scoring under 6/11 points in fatigue, suggesting that some excluded candidates still had CFS.
• A video has been made demonstrating how a trial participant could improve a single increment in fatigue, decline a single increment in physical function, report feeling significantly better e.g. due to pain medication, and then be classified as 'recovered' despite remaining significantly unwell with CFS. [4] Some participants would have to improve more to cross the thresholds, but it demonstrates a problem with the lax recovery criteria and that combining multiple recovery criteria does not necessary negate the weaknesses of each individual criteria. The definition of 'recovery' is simply too lax to justify the use of the word 'recovered', particularly without objective measures (which when used have generally shown a lack of clinically significant improvement).
• Re: "The thresholds for clinical improvements on an individual patient level for the primary measures of fatigue and physical function were abandoned and replaced with weaker thresholds...". To further clarify in context with the sentence which preceded it about BioMed Central's statement on post hoc revisions, the latter thresholds were the 'clinically useful difference' i.e. 2/33 points in fatigue (Likert scoring) and 8/100 points in physical function, which did not appear in the final version of the statistical analysis plan, [5] and were described as post hoc in the 2011 Lancet paper. [3] It is unclear when these were introduced and whether they were approved by the relevant trial oversight bodies (see below for why doubts have been raised).
• Re: "as the changes to the definition of recovery published in 2013 appear to be largely based on controversial post hoc analyses conducted for a previous paper on the trial results published in 2011". To further clarify, the methods and thresholds for the normal range for fatigue and physical function used in the revised definition of recovery did not appear in the final version of the statistical analysis plan [5] and are exactly the same as the post hoc analyses that were introduced during the peer review stage [6] of the fast tracked 2011 Lancet publication [3] (i.e. not introduced by the authors themselves). This is why doubts have been understandably raised over whether the changes to the recovery thresholds were approved by the relevant trial oversight bodies and whether the recovery definition was revised after rather than before these post hoc analyses were conducted, which would contradict the published impression that the changes were made to the recovery criteria before analysing any data. [7,8]
1. https://listserv.nodak.edu/cgi-bin/wa.ex...
2. http://www.biomedcentral.com/1471-2377/7/6
3. http://www.ncbi.nlm.nih.gov/pmc/articles...
4. http://www.youtube.com/watch?v=d_7J5ELjArU
5. http://www.trialsjournal.com/content/14/...
6. http://www.meactionuk.org.uk/whitereply....
7. http://www.ncbi.nlm.nih.gov/pmc/articles...
8. https://www.whatdotheyknow.com/request/t...