This would be a good time to contact The Lancet, and ask them to re-evaluate their decision to publish what is a clearly flawed study. Bad science should not be tolerated anywhere, and certainly should not be disseminated through one of the world's most prestigious medical journals.
To contact The Lancet, send an email to the editor, Richard Horton: richard.horton@lancet.com. Refer to: "Comparison of adaptive pacing therapy, cognitive behaviour therapy, graded exercise therapy, and specialist medical care for chronic fatigue syndrome (PACE): a randomised trial" published by The Lancet, Volume 377, No. 9768, p 823–836, 5 March 2011. (Read the study here.)
You can sign a petition asking the Lancet to retract their publication of the PACE trial HERE
By David Tuller, DrPH, 21 OCTOBER 2015
David Tuller is academic coordinator of the concurrent masters degree program in public health and journalism at the University of California, Berkeley.
A few years ago, Dr. Racaniello let me hijack this space for a long piece about the CDC’s persistent incompetence in its efforts to address the devastating illness the agency itself had misnamed “chronic fatigue syndrome.” Now I’m back with an even longer piece about the U.K’s controversial and highly influential PACE trial. The $8 million study, funded by British government agencies, purportedly proved that patients could “recover” from the illness through treatment with one of two rehabilitative, non-pharmacological interventions: graded exercise therapy, involving a gradual increase in activity, and a specialized form of cognitive behavior therapy. The main authors, a well-established group of British mental health professionals, published their first results in The Lancet in 2011, with additional results in subsequent papers.
Much of what I report here will not be news to the patient and advocacy communities, which have produced a voluminous online archive of critical commentary on the PACE trial. I could not have written this piece without the benefit of that research and the help of a few statistics-savvy sources who talked me through their complicated findings. I am also indebted to colleagues and friends in both public health and journalism, who provided valuable suggestions and advice on earlier drafts. Today’s Virology Blog installment is the first half; the second half will be posted in two parts, tomorrow and the next day. I was originally working on this piece with Retraction Watch, but we could not ultimately agree on the direction and approach.
SUMMARY
This examination of the PACE trial of chronic fatigue syndrome identified several major flaws:
*The study included a bizarre paradox: participants’ baseline scores for the two primary outcomes of physical function and fatigue could qualify them simultaneously as disabled enough to get into the trial but already “recovered” on those indicators–even before any treatment. In fact, 13 percent of the study sample was already “recovered” on one of these two measures at the start of the study.
*In the middle of the study, the PACE team published a newsletter for participants that included glowing testimonials from earlier trial subjects about how much the “therapy” and “treatment” helped them. The newsletter also included an article informing participants that the two interventions pioneered by the investigators and being tested for efficacy in the trial, graded exercise therapy and cognitive behavior therapy, had been recommended as treatments by a U.K. government committee “based on the best available evidence.” The newsletter article did not mention that a key PACE investigator was also serving on the U.K. government committee that endorsed the PACE therapies.
*The PACE team changed all the methods outlined in its protocol for assessing the primary outcomes of physical function and fatigue, but did not take necessary steps to demonstrate that the revised methods and findings were robust, such as including sensitivity analyses. The researchers also relaxed all four of the criteria outlined in the protocol for defining “recovery.” They have rejected requests from patients for the findings as originally promised in the protocol as “vexatious.”
*The PACE claims of successful treatment and “recovery” were based solely on subjective outcomes. All the objective measures from the trial—a walking test, a step test, and data on employment and the receipt of financial information—failed to provide any evidence to support such claims. Afterwards, the PACE authors dismissed their own main objective measures as non-objective, irrelevant, or unreliable.
*In seeking informed consent, the PACE authors violated their own protocol, which included an explicit commitment to tell prospective participants about any possible conflicts of interest. The main investigators have had longstanding financial and consulting ties with disability insurance companies, having advised them for years that cognitive behavior therapy and graded exercise therapy could get claimants off benefits and back to work. Yet prospective participants were not told about any insurance industry links and the information was not included on consent forms. The authors did include the information in the “conflicts of interest” sections of the published papers.
Top researchers who have reviewed the study say it is fraught with indefensible methodological problems. Here is a sampling of their comments:
Dr. Bruce Levin, Columbia University: “To let participants know that interventions have been selected by a government committee ‘based on the best available evidence’ strikes me as the height of clinical trial amateurism.”
Dr. Ronald Davis, Stanford University: “I’m shocked that the Lancet published it…The PACE study has so many flaws and there are so many questions you’d want to ask about it that I don’t understand how it got through any kind of peer review.”
Dr. Arthur Reingold, University of California, Berkeley: “Under the circumstances, an independent review of the trial conducted by experts not involved in the design or conduct of the study would seem to be very much in order.”
Dr. Jonathan Edwards, University College London: “It’s a mass of un-interpretability to me…All the issues with the trial are extremely worrying, making interpretation of the clinical significance of the findings more or less impossible.”
Dr. Leonard Jason, DePaul University: “The PACE authors should have reduced the kind of blatant methodological lapses that can impugn the credibility of the research, such as having overlapping recovery and entry/disability criteria.”
************************************************************************
PART ONE:
The PACE Trial, Deconstructed
On Feb 17, 2011, at a press conference in London, psychiatrist Michael Sharpe and behavioral psychologist Trudie Chalder, members of the British medical and academic establishments, unveiled the results of a controversial clinical trial of more than 600 people diagnosed with chronic fatigue syndrome. The findings were being published in The Lancet. As with many things about the illness, the news was expected to cause a stir.
The study, known as the PACE trial, was the largest ever of treatments for chronic fatigue syndrome. The authors were among a prominent group of British mental health professionals who had long argued that the devastating symptoms were caused by severe physical deconditioning. They recognized that many people experienced an acute viral infection or other illness as an initial trigger. However, they believed that the syndrome was perpetuated by patients’ “unhelpful” and “dysfunctional” notion that they continued to suffer from an organic disease—and that exertion would make them worse. According to the experts’ theory, patients’ decision to remain sedentary for prolonged periods led to muscle atrophy and other negative systemic physiological impacts, which then caused even more fatigue and other symptoms in a self-perpetuating cycle.
An estimated one to 2.5 million Americans, a quarter of a million British, and an unknown number of others around the world suffer from chronic fatigue syndrome. The illness often leaves patients too sick to work, attend school, or take care of their children, with a significant minority home-bound for months or years. It is a terrible public health burden, costing society billions of dollars a year in medical care and lost productivity. But what causes it and what to do about it have been fiercely debated for decades.
Patients and many leading scientists view the debilitating ailment as caused by pathological disease processes, not by physical deconditioning. Studies have shown that the illness is characterized by immunological and neurological dysfunctions, and many academic and government scientists say that the search for organic causes, diagnostic tests and drug interventions is paramount. Some recent research has generated excitement. In February, for example, a Columbia-led team reported distinct patterns of immune system response in early-stage patients—findings that could ultimately lead to a biomarker able to identify the presence of the illness.
In contrast, the British mental health experts have focused on non-pharmacological rehabilitative therapies, aimed at improving patients’ physical capacities and altering their perceptions of their condition through behavioral and psychological approaches. The PACE trial was designed to be a definitive test of two such treatments they had pioneered to help patients recover and get back to work. British government agencies, eager to stem health and disability costs related to the illness, had committed five million pounds—close to $8,000,000 at current exchange rates–to support the research.
At the press conference, Sharpe and Chalder touted the two treatments—an incremental increase in activity known as “graded exercise therapy,” and a specialized form of cognitive behavior therapy—as effective in reversing the illness. Citing participant responses on questionnaires about fatigue and physical function, Chalder declared that, compared to other study subjects, “twice as many people on graded exercise therapy and cognitive behaviour therapy got back to normal.”
A Lancet guest commentary, whose contents were discussed in advance with the PACE authors, amplified the positive news, stating that about 30 percent of patients in the two rehabilitative treatment arms had achieved “recovery.” Headlines and stories around the world trumpeted the results.
“Fatigued patients who go out and exercise have best hope of recovery, finds study,” declared The Daily Mail. “Psychotherapy Eases Chronic Fatigue Syndrome, Study Finds,” stated The New York Times headline (I wrote the accompanying story.) According to BMJ’s report about the trial, some PACE participants were “cured” of the illness.
***
Some 300 miles to the northwest in Castleknock, a middle-class suburb of Dublin, Tom Kindlon read and re-read the Lancet paper and reviewed the upbeat news coverage. The more he reviewed and re-reviewed everything, the more frustrated and angry he became. The investigators, he observed, were spinning the study as a success in one of the world’s preeminent scientific publications, and the press was lapping it up.
“The paper was pure hype for graded exercise therapy and cognitive behavior therapy,” said Kindlon in a recent interview. “But it was a huge trial, getting huge coverage, and they were getting a very influential base to push their views.”
Kindlon had struggled with the illness for more than two decades. In 1993, his health problems forced him to drop his math studies at Dublin’s prestigious Trinity College; he’d been largely homebound since. With his acumen for statistics, Kindlon was known in the advocacy community for his nuanced understanding of the research.
He shared this passion with a small group of other science-savvy patients he’d met through online networks. Kindlon and the others were particularly worried about the PACE trial, first announced in 2003. They knew the results would wield great influence on government health policies, public attitudes, and future research—not only in Great Britain, but in the U.S. and elsewhere as well.
Like others in the patient and advocacy communities, they believed the evidence clearly pointed to an ongoing biological disease, not physical debility caused by deconditioning. They bristled with offense at the suggestion they would get better if only they could change their perceptions about their condition. And pushing themselves to be more active not only wasn’t helpful, they insisted, but could trigger a serious and extended relapse.
In the four years since the Lancet publication, Kindlon and others have pressed for an independent review of the trial data. They have produced a sprawling online literature deconstructing the trial’s methodology, submitted dozens of freedom-of-information requests for PACE-related documents and data, and published their criticisms on the websites and letters columns of leading medical journals. Their concerns, if true, would raise serious questions about the study’s findings.
***
For their part, the PACE investigators have released additional results from the trial. These have included a 2012 paper on economic aspects in PLoS One, a 2013 paper on “recovery” in Psychological Medicine, and a “mediation analysis” paper last January in The Lancet Psychiatry suggesting that reducing patients’ purported fears of activity mediated improvement.
But this investigation–based on many dozens of interviews and a review of thousands of pages of documents–has confirmed that some of the major criticisms of the trial are accurate. (The documents reviewed included, among others, the trial protocol, the manuals for the trial’s four arms, participant information and consent forms, meeting minutes of oversight committees, critical reports written by patients and advocates, transcripts of parliamentary hearings, and many dozens of peer-reviewed studies. Some documents were obtained by patients under freedom-of-information requests and either posted online or provided to me.)
Among the findings:
*The trial included a bizarre paradox: Participants’ baseline scores for physical function and fatigue could qualify them simultaneously as sick enough to get into the trial but already “recovered” on those indicators–even before any treatment. In other words, the thresholds for being “recovered” demonstrated worse health than the scores required in the first place to demonstrate the severe disability needed to enter the trial. This anomaly meant that some participants could get worse on physical function and fatigue during the trial and still be included in the results as being “recovered.” Data obtained by a patient through a freedom-of-information request indicated that 13 percent of the participants were already “recovered” for physical function or fatigue, or both, when they joined the study—a fact not mentioned in any of the published papers. (In the 2011 Lancet paper, participants who met these unusual thresholds were referred to not as having “recovered” but as being “within normal range.” In the 2013 Psychological Medicine paper, the same thresholds were re-purposed as indicators of “recovery.”)
*During the study, the PACE team published a “participants newsletter” that included glowing testimonials from earlier trial subjects about how the “therapy” and “treatment” had improved their lives. An article in the same newsletter also reported that the U.K. government’s newly released clinical guidelines for the illness recommended the two rehabilitative treatments under investigation, cognitive behavior therapy and graded exercise therapy, “based on the best available evidence.” (The article didn’t mention that a key PACE investigator also served on the U.K. government committee that endorsed the two PACE therapies.) The testimonials and the statements promoting the two therapies could have biased the responses of the 200 or so remaining participants, about a third of the total study sample.
*The investigators abandoned all the criteria outlined in their protocol for assessing their two primary measures of fatigue and physical function, and adopted new ones (in the 2011 Lancet paper). They also significantly relaxed all four of their criteria for defining “recovery” (in the 2013 Psychological Medicine paper). They did not report having taken the necessary steps to assess the impacts of these changes, such as conducting sensitivity analyses. Such protocol changes contradicted the ethos of BMC Neurology, the journal that published the PACE protocol in 2007. An “editor’s comment” linked to the protocol urged readers to review the published results and to contact the authors “to ensure that no deviations from the protocol occurred during the study.” The PACE team has rejected freedom-of-information requests for the results as promised in the protocol as “vexatious.”
*The study’s two primary outcomes were subjective, but in the 2007 published protocol the investigators also included several “objective” secondary outcomes to assess physical capacity, fitness and function; these measures included a six-minute walking test, a self-paced step test, and data on employment, wages and financial benefits. These findings utterly failed to support the subjective reports that the authors had interpreted as demonstrating successful treatment and “recovery.” In subsequently published comments, the authors then disputed the relevance, reliability and “objectivity” of the main objective measures they themselves had selected.
*In seeking informed consent, the investigators violated a major international research ethics code that they promised, in their protocol, to observe. A key provision of the Declaration of Helsinki, developed after WW II to protect human research subjects, requires that study participants be “adequately informed” of researchers’ “possible conflicts of interest” and “institutional affiliations.” The key PACE authors have longstanding financial and consulting ties to the disability insurance industry; they have advised insurers for years that cognitive behavior therapy and graded exercise therapy can get patients off benefits and back to work. In the papers published in The Lancet and other journals, the PACE authors disclosed their industry ties, yet they did not reveal this information to prospective trial subjects. Of four participants interviewed, two said the knowledge would have impacted their decision to participate; one retroactively withdrew her consent and forbade the researchers from including her data.
***
I did not interview Chalder, Sharpe, and Peter White, also a psychiatrist and the lead PACE investigator, for this story. Chalder and Sharpe did not respond to e-mails last December seeking interviews. White did respond, e-mailing back that, after consulting with his colleagues and reviewing my past reporting on the illness, “I have concluded that it would not be worthwhile our having a conversation…We think our work speaks for itself.” A second request for interviews, sent last week to the three investigators, also proved unsuccessful.
(I did have a telephone conversation with Chalder in January of this year, organized as part of the media campaign for the Lancet Psychiatry paper published that month by the PACE team. In Chalder’s memory of the conversation, we talked at length about some of the major concerns examined here. In my memory, she mostly declined to talk about concerns related to the 2011 Lancet paper, pleading poor recall of the details.)
Richard Horton, the editor of The Lancet, was also not interviewed for this story. Last December, his office declined an e-mail request for an interview. A second e-mail seeking comment, sent to Horton last week, was not answered.
***
Experts who have examined the PACE study say it is fraught with problems.
“I’m shocked that the Lancet published it,” said Ronald Davis, a well-known geneticist at Stanford University and the director of the scientific advisory board of the Open Medicine Foundation. The foundation, whose board also includes three Nobel laureates, supports research on ME/CFS and is currently focused on identifying an accurate biomarker for the illness.
“The PACE study has so many flaws and there are so many questions you’d want to ask about it that I don’t understand how it got through any kind of peer review,” added Davis, who became involved in the field after his son became severely ill. “Maybe The Lancet picked reviewers who agreed with the authors and raved about the paper, and the journal went along without digging into the details.”
In an e-mail interview, DePaul University psychology professor Leonard Jason, an expert on the illness, said the study’s statistical anomalies were hard to overlook. “The PACE authors should have reduced the kind of blatant methodological lapses that can impugn the credibility of the research, such as having overlapping recovery and entry/disability criteria,” wrote Jason, a prolific researcher widely respected among scientists, health officials and patients.
Jason, who was himself diagnosed with the illness in the early 1990s, also noted that researchers cannot simply ignore their own assurances that they will follow specific ethical guidelines. “If you’ve promised to disclose conflicts of interest by promising to follow a protocol, you can’t just decide not to do it,” he said.
Jonathan Edwards, a professor emeritus of connective tissue medicine from University College London, pioneered a novel rheumatoid arthritis treatment in a large clinical trial published in the New England Journal of Medicine in 2004. For the last couple of years, he has been involved in organizing clinical trial research to test the same drug, rituximab, for chronic fatigue syndrome, which shares traits with rheumatoid arthritis and other autoimmune disorders.
When he first read the Lancet paper, Edwards was taken aback: Not only did the trial rely on subjective measures, but participants and therapists all knew which treatment was being administered, unlike in a double-blinded trial. This unblinded design made PACE particularly vulnerable to generating biased results, said Edwards in a phone interview, adding that the newsletter testimonials and other methodological flaws only made things worse.
“It’s a mass of un-interpretability to me,” said Edwards, who last year called the PACE results “valueless” in publicly posted comments. “Within the circle who are involved in this field, it seems there were a group who were prepared to all sing by the hymn sheet and agree that PACE was wonderful. But all the issues with the trial are extremely worrying, making interpretation of the clinical significance of the findings more or less impossible.”
Bruce Levin, a professor of biostatistics at Columbia University and an expert in clinical trial design, said that unplanned, post-protocol changes in primary outcomes should be made only when absolutely necessary, and that any such changes inevitably raised questions about interpretation of the results. In any event, he added, it would never be acceptable for such revisions to include “normal range” or “recovery” thresholds that overlapped with the study’s entry criteria.
“I have never seen a trial design where eligibility requirements for a disease alone would qualify some patients for having had a successful treatment,” said Levin, who has been involved in research on the illness and has reviewed the PACE study. “It calls into question the diagnosis of an illness whose patients already rate as ‘recovered’ or ‘within normal range.’ I find it nearly inconceivable that a trial’s data monitoring committee would have approved such a protocol problem if they were aware of it.”
Levin also said the mid-trial publication of the newsletter featuring participant testimonials and positive news about interventions under investigation created legitimate concerns that subsequent responses might have been biased, especially in an unblinded study with subjective outcomes like PACE.
“It is highly inappropriate to publish anything during an ongoing clinical trial,” said Levin. “To let participants know that interventions have been selected by a government committee ‘based on the best available evidence’ strikes me as the height of clinical trial amateurism.”
At the least, the PACE researchers should have evaluated the responses from before and afterwards to assess any resulting bias, he added.
***
Recent U.S. government reports have raised further challenges for the PACE approach. In June, a panel convened by the National Institutes of Health recommended that researchers abandon a core aspect of the PACE trial design—its method of identifying participants through the single symptom of prolonged fatigue, rather than a more detailed set of criteria. This method, the panel’s report noted, could “impair progress and cause harm” because it identifies people with many fatiguing conditions, making it hard to interpret the findings.
Last February, the Institute of Medicine released its own study, commissioned by several health agencies and based on an extensive literature review, which described the illness as a serious organic disease, not a cognitive or behavioral disorder characterized by “unhelpful beliefs” that lead to sedentary behavior. Two members of the IOM panel, in discussing their report with Medscape, cast sharp doubt on the central argument advanced for years by the British mental health professionals: that physical deconditioning alone perpetuates the devastating symptoms.
Ellen Wright Clayton, the panel chair and a professor of pediatrics and law at Vanderbilt University, said lack of activity could not possibly explain the scope and severity of patients’ symptoms. “The level of response is much more than would be seen with deconditioning,” she told Medscape.Peter Rowe, a pediatrician at Johns Hopkins and an expert on the disease, called the deconditioning hypothesis “flawed” and “untenable.”
The PACE investigators have strongly defended the integrity of their research and say that patients and advocacy groups have harassed and vilified them for years without justification. In 2011, The Guardian reported that Sharpe had been stalked by a woman who brought a knife to one of his lectures. A 2013 report in The Sunday Times noted that psychiatrist Simon Wessely, a senior colleague and adviser to the PACE authors, had received death threats, and that “one person rang him up and threatened to castrate him.”
No one is known to have been charged in these and other cases of reported threats or harassment.
PART TWO:
The Origins of the PACE Trial
Tom Kindlon, six feet tall and bulky, can only stand up for half a minute before dizziness and balance problems force him back down. He has a round face, wire-rimmed glasses, an engaging smile, and beard scruff. Direct light hurts his eyes. He wears a baseball cap to shield them.
Kindlon, 43, still lives with his parents in the two-story, four-bedroom house where he grew up. His mum, Vera, is his primary caretaker. He remains close with his three younger siblings— Ali, 40, and twins David and Deirdre, who are 35. All live nearby and help out when needed.
For the last 15 years, Kindlon has harnessed his limited energy for what he perceives as his primary mission: reviewing, and responding to, the literature on the illness. He has published more than a dozen peer-reviewed letters in scientific publications and regularly posts on the public forums and “rapid response” sections of journal websites, politely debating, dissecting and debunking questionable research claims.
“I haven’t read a fiction book in 20 years,” he noted, during a series of conversations ranging across Skype, Facebook, Twitter, and e-mail. “I need to be blinkered in what I do and don’t read, to concentrate and use my mental energy for this material.”
As a teenager, Kindlon loved playing rugby, cricket, tennis and soccer. When he was 16, he spent five days in western Ireland on a hiking and sailing trip with high school classmates. It was February, damp and chilly, and he was already suffering from a cold or some other bug; back in Dublin, he felt worse and stayed home for several days.
When he returned to school, he discovered something weird: After a round of sports, he now experienced muscle pains and a paralyzing exhaustion unlike anything he’d previously encountered. “I’d be totally whacked by the end of the day,” he recalled.
He saw a physiotherapist and then an orthopedic surgeon, who told him to exercise more. He tried swimming, but that also left him depleted. In 1991, despite his health struggles, he entered Trinity College. He slogged through two years of math studies but suffered more and more from problems with memory and concentration. “I was forgetting things, making silly errors,” he said.
Toward the end of the second year, he could no longer hold a pen in his hand. He developed tendonitis, first in one arm, then in the other. When he drove, pushing the pedals caused severe ankle pain. “Everything was magnified now,” he said. “I was just breaking down.” He took a leave from Trinity. His health continued to slide.
***
Then Kindlon read something about myalgic encephalomyelitis, or ME—an alternate name for chronic fatigue syndrome frequently used in the U.K., meaning “inflammation of the brain and spinal cord, with muscle pain.” A specialist confirmed the diagnosis.
Since there are no approved medical tests, diagnosis has generally been made based on symptoms, after other possibilities have been excluded. A major clue in Kindlon’s case was his experience of a prolonged collapse after sports. Almost all patients report this unusual symptom, called “post-exertional malaise”–a sustained relapse or worsening after a minimal amount of exertion or activity.
It was September, 1994. Tom Kindlon was 22 years old. He could just about drag himself to the toilet a few times a day. He could hold a brief conversation, though he often couldn’t remember what he or anyone else had said.
Soon after his diagnosis, he heard about a local support group called the Irish ME Association. Vera attended a meeting to learn more. She became a fixture at the monthly gatherings, and soon was voted chair of the group; her son was appointed assistant chair. Though his condition gradually stabilized and sometimes even seemed to improve a little, he never felt well enough to attend meetings and worked instead from home.
At the time, the organization only had a few dozen members. “I felt the group could get bigger than just people sitting in circles,” Kindlon said. “We needed to raise awareness. I wanted people’s stories to be told.”
On May 12, 1996, designated by U.K. advocates as International ME Day, the small Irish group held a public event. Vera spoke on national radio. The Kindlons, mother and son, publicized the group’s work, and by 2000 the membership list topped 400.
Through a leadership listserv, Kindlon maintained contact with dozens of patient support and advocacy groups around the UK and elsewhere; the network kept him abreast of the major scientific, public health, and political developments related to the illness. Then he learned about the PACE trial.
***
In the mid-1980s, several outbreaks of a disabling and prolonged flu-like illness popped up across the U.S. Although clinicians treating some of the patients believed it was associated with the Epstein-Barr virus, which causes mononucleosis, CDC investigators were unable to identify a link with that or other pathogens.
The CDC team called the mysterious condition “chronic fatigue syndrome” after rejecting the name “myalgic encephalomyelitis,” coined after a similar outbreak at a London hospital in the 1950s. The key symptom of myalgic encephalomyelitis had been identified as extreme muscle fatigue after minimal exertion, with delayed recovery—essentially, a description of post-exertional malaise, Tom Kindlon’s main symptom. The CDC also rejected “post-viral fatigue syndrome,” another common name. In contrast, the World Health Organization, which had years earlier classified “benign myalgic encephalomyelitis” as a neurological disorder, deemed both post-viral fatigue syndrome and chronic fatigue syndrome to be synonyms. (The word “benign” eventually fell out of common use.)
In the U.S., the disease is now often being called ME/CFS by government agencies; the recent report from the Institute of Medicine suggested renaming it “systemic exertion intolerance disease,” or SEID. In the U.K, it is often called CFS/ME.
Patients have always hated the name chronic fatigue syndrome. For one thing, the word “fatigue” does not come close to describing the profound depletion of energy that marks the illness. A few years ago, best-selling author and long-time patient Laura Hillenbrand (Unbroken; Seabiscuit) once told The New York Times: “This disease leaves people bedridden. I’ve gone through phases where I couldn’t roll over in bed. I couldn’t speak. To have it called ‘fatigue’ is a gross misnomer.”
Patients, clinicians and scientists say the name is also inaccurate because the hallmark is not fatigue itself but more specifically what Tom Kindlon experienced—the relapses known as post-exertional malaise. (Patients also criticize the word ‘malaise,’ like ‘fatigue,’ as inaccurate and inadequate, and many prefer to call the symptom ‘post-exertional relapse.’) Other core symptoms are cognitive and neurological problems, sleep disorders, and in many cases muscle pain.
Researchers have not been able to identify a specific cause—at least in part because investigators have used many different criteria to define the illness and identify study subjects, making it hard to compare results. In many cases, as in the 1980s outbreaks, ME/CFS appears to be triggered by a viral or other infection from which people never recover. Since patients often don’t seek treatment and are not diagnosed until they have been sick for a long time, research on triggering events has often been based on self-reports of an initial infection rather than laboratory confirmation. However, a prospective 2006 study from Australian researchers and the CDC found that 11 percent of more than 250 patients who were followed after acute cases of mononucleosis, Q fever, and Ross River virus met diagnostic criteria for chronic fatigue syndrome six months later.
Although in some cases patients report a gradual start to the illness, a 2011 definition of myalgic encephalomyelitis developed by an international expert committee noted that “most patients have an acute infectious onset with flu-like and/or respiratory symptoms.” In fact, many experts believe ME/CFS is likely a cluster of related illnesses, in which one or more infections, or exposures to toxins, mold, stress, trauma or other physiological insults, spark the immune system into a persistent state of hyper-activation, with the resulting inflammation and other systemic effects causing the symptoms. Like the varying methods for defining the illness, the heterogeneity of potential triggering events among chronic fatigue syndrome populations has also complicated research. Without accurate sub-grouping, the findings from such samples can undermine rather than promote the search for causes, biomarkers and treatments.
The illness can fluctuate over time. Long-term patients sometimes experience periods of moderate remission, but few appear to recover completely. Most treatment has involved symptomatic relief.
Although research has been hampered by limited government support, studies over the years have documented a wide range of biological abnormalities as well as associations with a host of pathogens. But some promising leads have not panned out, most spectacularly several years ago when an apparent association with mouse retroviruses turned out to be the result of lab contamination—a devastating blow to patients.
***
For their part, the PACE investigators have collectively published hundreds of studies and reports about the illness, which they prefer to call chronic fatigue syndrome. In their model, the syndrome starts when people become sick—often from a virus, sometimes from other causes. This short-term illness leaves them exhausted; when the infection or other cause passes and they try to resume normal activity, they feel weakened and symptomatic again. This response is expected given their deconditioned state, according to the model, yet patients become fearful that they are still sick and decide they need more rest.
Then, instead of undergoing a normal recovery, they develop what the PACE authors have called “unhelpful beliefs” or “dysfunctional cognitions”–more specifically, the unhelpful belief that they continue to suffer from an infection or some other medical disease that will get worse if they exert themselves. Patients guided by these faulty cognitions further reduce their activity and, per the theory, become even more deconditioned, ultimately leading to “a chronic fatigue state in which symptoms are perpetuated by a cycle of inactivity, deterioration in exercise tolerance and further symptoms,” noted a 1989 article whose authors included Chalder and Simon Wessely, the PACE investigators’ longtime colleague.
The two rehabilitative therapies were designed to interrupt this downward spiral and restore patients’ sense of control over their health, in part through positive reinforcement and encouragement that recovery was possible. The course of cognitive behavior therapy, known as CBT, was specifically designed and structured to help chronic fatigue syndrome patients alleviate themselves of the “unhelpful beliefs” that purportedly kept them sedentary, and to encourage them to re-engage with daily life. (Standard forms of cognitive behavior therapy are recommended for helping people deal with all kinds of adversity, including major illness, yet doctors do not suggest that it is an actual treatment for cancer, multiple sclerosis, or renal failure.) The increase in activity known as graded exercise therapy, or GET, sought to counteract the deconditioning by getting people moving again in planned, incremental steps.
Through their extensive writings and their consulting roles with government agencies, Sharpe, Chalder, White, and their colleagues have long exerted a major impact on treatment. In the U.K., the National Health Service has primarily favored cognitive behavior therapy and graded exercise therapy, or related approaches, even in specialized clinics.
In the U.S., the Centers for Disease Control and Prevention has collaborated with White, Sharpe and some of their colleagues for decades. The agency recommends the two treatments on its website and in its now-archived CFS Toolkit for health professionals about how to treat the illness. The toolkit recommends contacting St. Bartholomew’s—the venerable London hospital that is one of White’s professional homes—for more information about graded exercise therapy.
White, the lead author of the Lancet paper, is a professor of psychological medicine at Queen Mary University of London and co-leads the chronic fatigue syndrome service at St. Bartholomew’s. Sharpe is a professor of psychological medicine at Oxford University, and Chalder is a professor of cognitive behavioral psychotherapy at King’s College London. Their faculty webpages currently credit them with, respectively, 90, 366 and 205 publications.
The PACE authors have been referred to as members of the “Wessely school”—or, less politely, the “Wessely cabal”– because of Simon Wessely’s prominence as a pioneer of this treatment approach for chronic fatigue syndrome. Wessely, a professor of psychological medicine at King’s College London, has published more than 700 papers, was knighted in 2013, and is the current president of the Royal College of Psychiatrists.
Over the years, members of the PACE team developed close consulting and financial relationships with insurance companies; they have acknowledged these ties in “conflict of interest” statements in published papers. They have advised insurers that rehabilitative, non-pharmacological therapies can help claimants with chronic fatigue syndrome return to work—as Sharpe noted in a 2002 UNUMProvident report on disability insurance trends.
In his article for the UNUMProvident report, Sharpe also criticized the “ME lobby” for playing a negative role in influencing patients’ self-perceptions of their condition, noting that “the patient’s beliefs may become entrenched and be driven by anger and the need to explain continuing disability.” Sharpe noted that economic and social factors, like receiving financial benefits or accepting the physiological illness claims made by patient groups, also represented roadblocks to both clinical improvement and the resolution of disability insurance claims.
“A strong belief and preoccupation that one has a ‘medical disease’ and a helpless and passive attitude to coping is associated with persistent disability,” Sharpe warned readers of the disability insurance report. “The current system of state benefits, insurance payments and litigation remain potentially major obstacles to effective rehabilitation…If the claimant becomes hostile toward employer or insurer the position is likely to be difficult to retrieve.”
***
Given the medical and social costs of the illness, the government wanted solid evidence from a large trial about treatments that could help people get better. In 2003, the U.K. Medical Research Council announced that it would fund the PACE trial—more formally known as “Comparison of adaptive pacing therapy, cognitive behavior therapy, graded exercise therapy, and specialist medical care for chronic fatigue syndrome: a randomized trial.”
Three other government agencies–Scotland’s Chief Scientist Office, England’s Department of Health, and the U.K. Department for Work and Pensions—chipped in. The West Midlands Multicentre Research Ethics Committee approved the final study protocol.
The investigators selected two self-reported measures, for physical function and fatigue, as their primary outcomes. For physical function, they chose a section of a widely used questionnaire called the Medical Outcomes Study 36-Item Short Form Health Survey, or SF-36; with this physical function scale, they designated a score of 60 or less out of 100 as representing sufficient disability for trial entry.
For fatigue, they selected the Chalder Fatigue Scale, developed by one of the PACE investigators, on which higher scores represented greater fatigue. The response to each of the scale’s 11 questions would be scored as 0 or 1, and a score of 6 or more was deemed sufficient evidence of disability for trial entry.
In the proposed trial, participants would be randomized into four arms. All would be offered a few meetings with a specialist—the baseline condition ultimately called “specialist medical care.” Participants in three of the arms would receive additional interventions, of up to 14 sessions over six months, with a booster session three months later. Everyone would be assessed one year after entering the trial—that is, six months after the end of the main period of treatment. Home-bound patients were not eligible, since participation required attendance at multiple clinic sessions.
Besides the two rehabilitative treatments of cognitive behavior therapy and graded exercise therapy, the investigators planned to include an intervention based on a popular self-help strategy known as “pacing.” While the first two approaches challenged patients to adjust their thinking and push themselves beyond what they believed they could do, pacing involved accepting and adapting to the physical constraints of the illness, paying attention to symptoms, and not exceeding personal energy reserves to avoid triggering a relapse.
***
Previous studies conducted by the authors and other researchers, although smaller than PACE, had found that graded exercise therapy and cognitive behavior therapy led to modest improvements in self-reported outcomes, as a 2001 review in JAMA noted. But the same review also warned that the positive results on subjective measures in these studies did not mean that participants had actually improved their physical capacities.
“The person may feel better able to cope with daily activities because they have reduced their expectations of what they should achieve, rather than because they have made any recovery as a result of the intervention,” stated the review. “A more objective measure of the effect of any intervention would be whether participants have increased their working hours, returned to work or school, or increased their physical activities.”
Aware of such concerns, the PACE investigators planned to include some measures of physical function and fitness not dependent on subjective impressions.
Beyond the question of how to measure the effects of the intervention, the therapies themselves remained highly controversial among patients. Many understood that cognitive behavior therapy could be a useful tool for coping with a serious condition but resented and dismissed the PACE authors’ suggestion that it could treat the underlying illness. Encouraging an increase in exercise or exertion was even more controversial. Patients considered it dangerous because of the possibility of relapse from post-exertional malaise. In surveys, patients who had received graded exercise therapy were more likely to report that it had made them worse rather than better.
The psychiatrists and other mental health experts acknowledged that patients often felt worse after starting an activity program. To them, the resurgence of symptoms reflected the deconditioned body’s natural response to renewed exertion, not an underlying disease process—a point strongly conveyed to patients. According to the PACE manual for clinicians administering graded exercise therapy, “Participants are encouraged to see symptoms as temporary and reversible, as a result of their current physical weakness, and not as signs of progressive pathology.”
***
Patients and advocates, aware of the previous work of the PACE team, responded to the Medical Research Council’s announcement with alarm. Fearing the research would lead to calls for more funding for cognitive behavior therapy and exercise therapy and nothing else, patient groups demanded that the agency back research into biological causes and treatments of ME/CFS instead–something it was not doing.
“We believe that the money being allocated to the PACE trial is a scandalous way of prioritising the very limited research funding that the MRC [Medical Research Council] have decided to make available for ME/CFS,” declared the ME Association, a major advocacy organization, in a statement widely disseminated on social media. The statement demanded that the trial be halted and the money “held in reserve for research that is likely to be of real benefit to people with ME/CFS.”
Despite the anger in the patient community, the investigators were able to enlist Action For ME, another major advocacy group, to help design the pacing intervention. They called their operationalization of the strategy “adaptive pacing therapy,” or APT.
The trial protocol described the pacing therapy as “essentially an energy management approach, which involves assessment of the link between activity and subsequent symptoms and disability, establishing a stable baseline of activity using a daily diary, with advice to plan and pace activity in order to avoid exacerbations.” But many patients argued that pacing was an inherently personal, flexible approach. Packaging it as a structured “treatment” administered by a “therapist,” with a focus on daily diaries and advance planning, would inevitably alter its effect, they said.
***
Patients and other researchers also objected to the PACE study’s choice of “case definition”—a set of research or diagnostic criteria designed to include everyone with an illness and exclude those without it. Many challenged the decision to identify participants using the single-symptom case definition of chronic fatigue syndrome called the Oxford criteria—the same broad case definition that last June’s NIH report recommended for retirement because it could “impair progress and cause harm.”
Over the years, there have been many definitions proposed for both chronic fatigue syndrome and myalgic encephalomyelitis, for both clinical and research use. The most widely used has been the CDC’s 1994 definition for chronic fatigue syndrome, which required six months of fatigue, plus any four of eight other symptoms: cognitive problems, muscle pain, joint pain, headache, tender lymph nodes, sore throat, post-exertional malaise, and sleep disturbances.
Many patients, researchers and clinicians experienced in treating the illness prefer more recent and restrictive definitions that seek to reduce misdiagnoses by requiring the presence of the core symptom of post-exertional malaise as well as neurological and cognitive dysfunctions, unlike the more flexible CDC definition. In contrast, the Oxford criteria, published in 1991 by PACE investigator Michael Sharpe and colleagues, required only one symptom: six months of medically unexplained, disabling fatigue. Proponents argued that this broad scope ensured that research results could be applied to the largest number of people potentially suffering from the illness. If other symptoms were present, as often happened, the criteria required that fatigue be the primary complaint.
According to DePaul psychologist Leonard Jason, the Oxford criteria blurred the boundaries between “chronic fatigue,” a symptom of many conditions, and the distinct illness known as “chronic fatigue syndrome.” In particular, he said, an Oxford criteria sample would likely include many people with primary depression, which can cause prolonged fatigue and often responds to interventions like those being tested in PACE. (In contrast, many people with ME/CFS get depressed as a secondary result of their illness experience.)
“The Oxford criteria clearly select for a lot of patients with primary depression, and people who are depressed do react very well to CBT and exercise,” said Jason, who has published widely on the ME/CFS case definition problem. Positive outcomes in the sample among depressed patients without ME/CFS could therefore lead to the unwarranted conclusion that the therapies worked for people with the disease, he added.
***
The PACE investigators were aware of these concerns, and they promised to study as well two subgroups of participants from their Oxford criteria sample who met additional case definitions: an updated 2003 version of the CDC’s 1994 definition for chronic fatigue syndrome, and a separate definition for myalgic encephalomyelitis. That way, they hoped to be able to draw conclusions about whether the therapies worked, no matter how the illness was defined.
Yet this approach presented its own challenges. Neither of the two other definitions required fatigue to be the primary symptom, as did the Oxford criteria. The myalgic encephalomyelitis definition did not even include fatigue per se as a symptom at all; post-exertional malaise, not fatigue, was the core symptom. And under the CDC definition, patients could present with any of the other symptoms as their primary complaint, as long as they also experienced fatigue.
Given these major differences in the case definitions, an unknown number of patients might have been screened out of the sample by the Oxford criteria but still met one of the other sets of criteria, making it hard to interpret the subgroup findings, according to other researchers. (The PACE investigators and I debated this methodological issue in an exchange of letters in The New York Times in 2011, after an article I wrote about case definition and the PACE trial.)
Bruce Levin, the Columbia University biostatistician, said the PACE investigators should not have assumed that the experience of a subgroup within an already defined population would match the experience of a group that hadn’t been pre-screened. “I would not accept an extrapolation to people diagnosed with alternative criteria from a subgroup comprising people satisfying both sets of criteria rather than just the alternative set of criteria,” he said, adding that reviewers should catch such questionable assumptions before publication.
Tomorrow: Publication of the PACE trial
To contact The Lancet, send an email to the editor, Richard Horton: richard.horton@lancet.com. Refer to: "Comparison of adaptive pacing therapy, cognitive behaviour therapy, graded exercise therapy, and specialist medical care for chronic fatigue syndrome (PACE): a randomised trial" published by The Lancet, Volume 377, No. 9768, p 823–836, 5 March 2011. (Read the study here.)
You can sign a petition asking the Lancet to retract their publication of the PACE trial HERE
Below are the first two parts of TRIAL BY ERROR: The Troubling Case of the PACE Chronic Fatigue Syndrome Study as they appeared on Dr. Vincent Racaniello's Virology Blog.
____________________
By David Tuller, DrPH, 21 OCTOBER 2015
David Tuller is academic coordinator of the concurrent masters degree program in public health and journalism at the University of California, Berkeley.
A few years ago, Dr. Racaniello let me hijack this space for a long piece about the CDC’s persistent incompetence in its efforts to address the devastating illness the agency itself had misnamed “chronic fatigue syndrome.” Now I’m back with an even longer piece about the U.K’s controversial and highly influential PACE trial. The $8 million study, funded by British government agencies, purportedly proved that patients could “recover” from the illness through treatment with one of two rehabilitative, non-pharmacological interventions: graded exercise therapy, involving a gradual increase in activity, and a specialized form of cognitive behavior therapy. The main authors, a well-established group of British mental health professionals, published their first results in The Lancet in 2011, with additional results in subsequent papers.
Much of what I report here will not be news to the patient and advocacy communities, which have produced a voluminous online archive of critical commentary on the PACE trial. I could not have written this piece without the benefit of that research and the help of a few statistics-savvy sources who talked me through their complicated findings. I am also indebted to colleagues and friends in both public health and journalism, who provided valuable suggestions and advice on earlier drafts. Today’s Virology Blog installment is the first half; the second half will be posted in two parts, tomorrow and the next day. I was originally working on this piece with Retraction Watch, but we could not ultimately agree on the direction and approach.
SUMMARY
This examination of the PACE trial of chronic fatigue syndrome identified several major flaws:
*The study included a bizarre paradox: participants’ baseline scores for the two primary outcomes of physical function and fatigue could qualify them simultaneously as disabled enough to get into the trial but already “recovered” on those indicators–even before any treatment. In fact, 13 percent of the study sample was already “recovered” on one of these two measures at the start of the study.
*In the middle of the study, the PACE team published a newsletter for participants that included glowing testimonials from earlier trial subjects about how much the “therapy” and “treatment” helped them. The newsletter also included an article informing participants that the two interventions pioneered by the investigators and being tested for efficacy in the trial, graded exercise therapy and cognitive behavior therapy, had been recommended as treatments by a U.K. government committee “based on the best available evidence.” The newsletter article did not mention that a key PACE investigator was also serving on the U.K. government committee that endorsed the PACE therapies.
*The PACE team changed all the methods outlined in its protocol for assessing the primary outcomes of physical function and fatigue, but did not take necessary steps to demonstrate that the revised methods and findings were robust, such as including sensitivity analyses. The researchers also relaxed all four of the criteria outlined in the protocol for defining “recovery.” They have rejected requests from patients for the findings as originally promised in the protocol as “vexatious.”
*The PACE claims of successful treatment and “recovery” were based solely on subjective outcomes. All the objective measures from the trial—a walking test, a step test, and data on employment and the receipt of financial information—failed to provide any evidence to support such claims. Afterwards, the PACE authors dismissed their own main objective measures as non-objective, irrelevant, or unreliable.
*In seeking informed consent, the PACE authors violated their own protocol, which included an explicit commitment to tell prospective participants about any possible conflicts of interest. The main investigators have had longstanding financial and consulting ties with disability insurance companies, having advised them for years that cognitive behavior therapy and graded exercise therapy could get claimants off benefits and back to work. Yet prospective participants were not told about any insurance industry links and the information was not included on consent forms. The authors did include the information in the “conflicts of interest” sections of the published papers.
Top researchers who have reviewed the study say it is fraught with indefensible methodological problems. Here is a sampling of their comments:
Dr. Bruce Levin, Columbia University: “To let participants know that interventions have been selected by a government committee ‘based on the best available evidence’ strikes me as the height of clinical trial amateurism.”
Dr. Ronald Davis, Stanford University: “I’m shocked that the Lancet published it…The PACE study has so many flaws and there are so many questions you’d want to ask about it that I don’t understand how it got through any kind of peer review.”
Dr. Arthur Reingold, University of California, Berkeley: “Under the circumstances, an independent review of the trial conducted by experts not involved in the design or conduct of the study would seem to be very much in order.”
Dr. Jonathan Edwards, University College London: “It’s a mass of un-interpretability to me…All the issues with the trial are extremely worrying, making interpretation of the clinical significance of the findings more or less impossible.”
Dr. Leonard Jason, DePaul University: “The PACE authors should have reduced the kind of blatant methodological lapses that can impugn the credibility of the research, such as having overlapping recovery and entry/disability criteria.”
************************************************************************
PART ONE:
The PACE Trial, Deconstructed
On Feb 17, 2011, at a press conference in London, psychiatrist Michael Sharpe and behavioral psychologist Trudie Chalder, members of the British medical and academic establishments, unveiled the results of a controversial clinical trial of more than 600 people diagnosed with chronic fatigue syndrome. The findings were being published in The Lancet. As with many things about the illness, the news was expected to cause a stir.
The study, known as the PACE trial, was the largest ever of treatments for chronic fatigue syndrome. The authors were among a prominent group of British mental health professionals who had long argued that the devastating symptoms were caused by severe physical deconditioning. They recognized that many people experienced an acute viral infection or other illness as an initial trigger. However, they believed that the syndrome was perpetuated by patients’ “unhelpful” and “dysfunctional” notion that they continued to suffer from an organic disease—and that exertion would make them worse. According to the experts’ theory, patients’ decision to remain sedentary for prolonged periods led to muscle atrophy and other negative systemic physiological impacts, which then caused even more fatigue and other symptoms in a self-perpetuating cycle.
An estimated one to 2.5 million Americans, a quarter of a million British, and an unknown number of others around the world suffer from chronic fatigue syndrome. The illness often leaves patients too sick to work, attend school, or take care of their children, with a significant minority home-bound for months or years. It is a terrible public health burden, costing society billions of dollars a year in medical care and lost productivity. But what causes it and what to do about it have been fiercely debated for decades.
Patients and many leading scientists view the debilitating ailment as caused by pathological disease processes, not by physical deconditioning. Studies have shown that the illness is characterized by immunological and neurological dysfunctions, and many academic and government scientists say that the search for organic causes, diagnostic tests and drug interventions is paramount. Some recent research has generated excitement. In February, for example, a Columbia-led team reported distinct patterns of immune system response in early-stage patients—findings that could ultimately lead to a biomarker able to identify the presence of the illness.
In contrast, the British mental health experts have focused on non-pharmacological rehabilitative therapies, aimed at improving patients’ physical capacities and altering their perceptions of their condition through behavioral and psychological approaches. The PACE trial was designed to be a definitive test of two such treatments they had pioneered to help patients recover and get back to work. British government agencies, eager to stem health and disability costs related to the illness, had committed five million pounds—close to $8,000,000 at current exchange rates–to support the research.
At the press conference, Sharpe and Chalder touted the two treatments—an incremental increase in activity known as “graded exercise therapy,” and a specialized form of cognitive behavior therapy—as effective in reversing the illness. Citing participant responses on questionnaires about fatigue and physical function, Chalder declared that, compared to other study subjects, “twice as many people on graded exercise therapy and cognitive behaviour therapy got back to normal.”
A Lancet guest commentary, whose contents were discussed in advance with the PACE authors, amplified the positive news, stating that about 30 percent of patients in the two rehabilitative treatment arms had achieved “recovery.” Headlines and stories around the world trumpeted the results.
“Fatigued patients who go out and exercise have best hope of recovery, finds study,” declared The Daily Mail. “Psychotherapy Eases Chronic Fatigue Syndrome, Study Finds,” stated The New York Times headline (I wrote the accompanying story.) According to BMJ’s report about the trial, some PACE participants were “cured” of the illness.
***
Some 300 miles to the northwest in Castleknock, a middle-class suburb of Dublin, Tom Kindlon read and re-read the Lancet paper and reviewed the upbeat news coverage. The more he reviewed and re-reviewed everything, the more frustrated and angry he became. The investigators, he observed, were spinning the study as a success in one of the world’s preeminent scientific publications, and the press was lapping it up.
“The paper was pure hype for graded exercise therapy and cognitive behavior therapy,” said Kindlon in a recent interview. “But it was a huge trial, getting huge coverage, and they were getting a very influential base to push their views.”
Kindlon had struggled with the illness for more than two decades. In 1993, his health problems forced him to drop his math studies at Dublin’s prestigious Trinity College; he’d been largely homebound since. With his acumen for statistics, Kindlon was known in the advocacy community for his nuanced understanding of the research.
He shared this passion with a small group of other science-savvy patients he’d met through online networks. Kindlon and the others were particularly worried about the PACE trial, first announced in 2003. They knew the results would wield great influence on government health policies, public attitudes, and future research—not only in Great Britain, but in the U.S. and elsewhere as well.
Like others in the patient and advocacy communities, they believed the evidence clearly pointed to an ongoing biological disease, not physical debility caused by deconditioning. They bristled with offense at the suggestion they would get better if only they could change their perceptions about their condition. And pushing themselves to be more active not only wasn’t helpful, they insisted, but could trigger a serious and extended relapse.
In the four years since the Lancet publication, Kindlon and others have pressed for an independent review of the trial data. They have produced a sprawling online literature deconstructing the trial’s methodology, submitted dozens of freedom-of-information requests for PACE-related documents and data, and published their criticisms on the websites and letters columns of leading medical journals. Their concerns, if true, would raise serious questions about the study’s findings.
***
For their part, the PACE investigators have released additional results from the trial. These have included a 2012 paper on economic aspects in PLoS One, a 2013 paper on “recovery” in Psychological Medicine, and a “mediation analysis” paper last January in The Lancet Psychiatry suggesting that reducing patients’ purported fears of activity mediated improvement.
But this investigation–based on many dozens of interviews and a review of thousands of pages of documents–has confirmed that some of the major criticisms of the trial are accurate. (The documents reviewed included, among others, the trial protocol, the manuals for the trial’s four arms, participant information and consent forms, meeting minutes of oversight committees, critical reports written by patients and advocates, transcripts of parliamentary hearings, and many dozens of peer-reviewed studies. Some documents were obtained by patients under freedom-of-information requests and either posted online or provided to me.)
Among the findings:
*The trial included a bizarre paradox: Participants’ baseline scores for physical function and fatigue could qualify them simultaneously as sick enough to get into the trial but already “recovered” on those indicators–even before any treatment. In other words, the thresholds for being “recovered” demonstrated worse health than the scores required in the first place to demonstrate the severe disability needed to enter the trial. This anomaly meant that some participants could get worse on physical function and fatigue during the trial and still be included in the results as being “recovered.” Data obtained by a patient through a freedom-of-information request indicated that 13 percent of the participants were already “recovered” for physical function or fatigue, or both, when they joined the study—a fact not mentioned in any of the published papers. (In the 2011 Lancet paper, participants who met these unusual thresholds were referred to not as having “recovered” but as being “within normal range.” In the 2013 Psychological Medicine paper, the same thresholds were re-purposed as indicators of “recovery.”)
*During the study, the PACE team published a “participants newsletter” that included glowing testimonials from earlier trial subjects about how the “therapy” and “treatment” had improved their lives. An article in the same newsletter also reported that the U.K. government’s newly released clinical guidelines for the illness recommended the two rehabilitative treatments under investigation, cognitive behavior therapy and graded exercise therapy, “based on the best available evidence.” (The article didn’t mention that a key PACE investigator also served on the U.K. government committee that endorsed the two PACE therapies.) The testimonials and the statements promoting the two therapies could have biased the responses of the 200 or so remaining participants, about a third of the total study sample.
*The investigators abandoned all the criteria outlined in their protocol for assessing their two primary measures of fatigue and physical function, and adopted new ones (in the 2011 Lancet paper). They also significantly relaxed all four of their criteria for defining “recovery” (in the 2013 Psychological Medicine paper). They did not report having taken the necessary steps to assess the impacts of these changes, such as conducting sensitivity analyses. Such protocol changes contradicted the ethos of BMC Neurology, the journal that published the PACE protocol in 2007. An “editor’s comment” linked to the protocol urged readers to review the published results and to contact the authors “to ensure that no deviations from the protocol occurred during the study.” The PACE team has rejected freedom-of-information requests for the results as promised in the protocol as “vexatious.”
*The study’s two primary outcomes were subjective, but in the 2007 published protocol the investigators also included several “objective” secondary outcomes to assess physical capacity, fitness and function; these measures included a six-minute walking test, a self-paced step test, and data on employment, wages and financial benefits. These findings utterly failed to support the subjective reports that the authors had interpreted as demonstrating successful treatment and “recovery.” In subsequently published comments, the authors then disputed the relevance, reliability and “objectivity” of the main objective measures they themselves had selected.
*In seeking informed consent, the investigators violated a major international research ethics code that they promised, in their protocol, to observe. A key provision of the Declaration of Helsinki, developed after WW II to protect human research subjects, requires that study participants be “adequately informed” of researchers’ “possible conflicts of interest” and “institutional affiliations.” The key PACE authors have longstanding financial and consulting ties to the disability insurance industry; they have advised insurers for years that cognitive behavior therapy and graded exercise therapy can get patients off benefits and back to work. In the papers published in The Lancet and other journals, the PACE authors disclosed their industry ties, yet they did not reveal this information to prospective trial subjects. Of four participants interviewed, two said the knowledge would have impacted their decision to participate; one retroactively withdrew her consent and forbade the researchers from including her data.
***
I did not interview Chalder, Sharpe, and Peter White, also a psychiatrist and the lead PACE investigator, for this story. Chalder and Sharpe did not respond to e-mails last December seeking interviews. White did respond, e-mailing back that, after consulting with his colleagues and reviewing my past reporting on the illness, “I have concluded that it would not be worthwhile our having a conversation…We think our work speaks for itself.” A second request for interviews, sent last week to the three investigators, also proved unsuccessful.
(I did have a telephone conversation with Chalder in January of this year, organized as part of the media campaign for the Lancet Psychiatry paper published that month by the PACE team. In Chalder’s memory of the conversation, we talked at length about some of the major concerns examined here. In my memory, she mostly declined to talk about concerns related to the 2011 Lancet paper, pleading poor recall of the details.)
Richard Horton, the editor of The Lancet, was also not interviewed for this story. Last December, his office declined an e-mail request for an interview. A second e-mail seeking comment, sent to Horton last week, was not answered.
***
Experts who have examined the PACE study say it is fraught with problems.
“I’m shocked that the Lancet published it,” said Ronald Davis, a well-known geneticist at Stanford University and the director of the scientific advisory board of the Open Medicine Foundation. The foundation, whose board also includes three Nobel laureates, supports research on ME/CFS and is currently focused on identifying an accurate biomarker for the illness.
“The PACE study has so many flaws and there are so many questions you’d want to ask about it that I don’t understand how it got through any kind of peer review,” added Davis, who became involved in the field after his son became severely ill. “Maybe The Lancet picked reviewers who agreed with the authors and raved about the paper, and the journal went along without digging into the details.”
In an e-mail interview, DePaul University psychology professor Leonard Jason, an expert on the illness, said the study’s statistical anomalies were hard to overlook. “The PACE authors should have reduced the kind of blatant methodological lapses that can impugn the credibility of the research, such as having overlapping recovery and entry/disability criteria,” wrote Jason, a prolific researcher widely respected among scientists, health officials and patients.
Jason, who was himself diagnosed with the illness in the early 1990s, also noted that researchers cannot simply ignore their own assurances that they will follow specific ethical guidelines. “If you’ve promised to disclose conflicts of interest by promising to follow a protocol, you can’t just decide not to do it,” he said.
Jonathan Edwards, a professor emeritus of connective tissue medicine from University College London, pioneered a novel rheumatoid arthritis treatment in a large clinical trial published in the New England Journal of Medicine in 2004. For the last couple of years, he has been involved in organizing clinical trial research to test the same drug, rituximab, for chronic fatigue syndrome, which shares traits with rheumatoid arthritis and other autoimmune disorders.
When he first read the Lancet paper, Edwards was taken aback: Not only did the trial rely on subjective measures, but participants and therapists all knew which treatment was being administered, unlike in a double-blinded trial. This unblinded design made PACE particularly vulnerable to generating biased results, said Edwards in a phone interview, adding that the newsletter testimonials and other methodological flaws only made things worse.
“It’s a mass of un-interpretability to me,” said Edwards, who last year called the PACE results “valueless” in publicly posted comments. “Within the circle who are involved in this field, it seems there were a group who were prepared to all sing by the hymn sheet and agree that PACE was wonderful. But all the issues with the trial are extremely worrying, making interpretation of the clinical significance of the findings more or less impossible.”
Bruce Levin, a professor of biostatistics at Columbia University and an expert in clinical trial design, said that unplanned, post-protocol changes in primary outcomes should be made only when absolutely necessary, and that any such changes inevitably raised questions about interpretation of the results. In any event, he added, it would never be acceptable for such revisions to include “normal range” or “recovery” thresholds that overlapped with the study’s entry criteria.
“I have never seen a trial design where eligibility requirements for a disease alone would qualify some patients for having had a successful treatment,” said Levin, who has been involved in research on the illness and has reviewed the PACE study. “It calls into question the diagnosis of an illness whose patients already rate as ‘recovered’ or ‘within normal range.’ I find it nearly inconceivable that a trial’s data monitoring committee would have approved such a protocol problem if they were aware of it.”
Levin also said the mid-trial publication of the newsletter featuring participant testimonials and positive news about interventions under investigation created legitimate concerns that subsequent responses might have been biased, especially in an unblinded study with subjective outcomes like PACE.
“It is highly inappropriate to publish anything during an ongoing clinical trial,” said Levin. “To let participants know that interventions have been selected by a government committee ‘based on the best available evidence’ strikes me as the height of clinical trial amateurism.”
At the least, the PACE researchers should have evaluated the responses from before and afterwards to assess any resulting bias, he added.
***
Recent U.S. government reports have raised further challenges for the PACE approach. In June, a panel convened by the National Institutes of Health recommended that researchers abandon a core aspect of the PACE trial design—its method of identifying participants through the single symptom of prolonged fatigue, rather than a more detailed set of criteria. This method, the panel’s report noted, could “impair progress and cause harm” because it identifies people with many fatiguing conditions, making it hard to interpret the findings.
Last February, the Institute of Medicine released its own study, commissioned by several health agencies and based on an extensive literature review, which described the illness as a serious organic disease, not a cognitive or behavioral disorder characterized by “unhelpful beliefs” that lead to sedentary behavior. Two members of the IOM panel, in discussing their report with Medscape, cast sharp doubt on the central argument advanced for years by the British mental health professionals: that physical deconditioning alone perpetuates the devastating symptoms.
Ellen Wright Clayton, the panel chair and a professor of pediatrics and law at Vanderbilt University, said lack of activity could not possibly explain the scope and severity of patients’ symptoms. “The level of response is much more than would be seen with deconditioning,” she told Medscape.Peter Rowe, a pediatrician at Johns Hopkins and an expert on the disease, called the deconditioning hypothesis “flawed” and “untenable.”
The PACE investigators have strongly defended the integrity of their research and say that patients and advocacy groups have harassed and vilified them for years without justification. In 2011, The Guardian reported that Sharpe had been stalked by a woman who brought a knife to one of his lectures. A 2013 report in The Sunday Times noted that psychiatrist Simon Wessely, a senior colleague and adviser to the PACE authors, had received death threats, and that “one person rang him up and threatened to castrate him.”
No one is known to have been charged in these and other cases of reported threats or harassment.
PART TWO:
The Origins of the PACE Trial
Tom Kindlon, six feet tall and bulky, can only stand up for half a minute before dizziness and balance problems force him back down. He has a round face, wire-rimmed glasses, an engaging smile, and beard scruff. Direct light hurts his eyes. He wears a baseball cap to shield them.
Kindlon, 43, still lives with his parents in the two-story, four-bedroom house where he grew up. His mum, Vera, is his primary caretaker. He remains close with his three younger siblings— Ali, 40, and twins David and Deirdre, who are 35. All live nearby and help out when needed.
For the last 15 years, Kindlon has harnessed his limited energy for what he perceives as his primary mission: reviewing, and responding to, the literature on the illness. He has published more than a dozen peer-reviewed letters in scientific publications and regularly posts on the public forums and “rapid response” sections of journal websites, politely debating, dissecting and debunking questionable research claims.
“I haven’t read a fiction book in 20 years,” he noted, during a series of conversations ranging across Skype, Facebook, Twitter, and e-mail. “I need to be blinkered in what I do and don’t read, to concentrate and use my mental energy for this material.”
As a teenager, Kindlon loved playing rugby, cricket, tennis and soccer. When he was 16, he spent five days in western Ireland on a hiking and sailing trip with high school classmates. It was February, damp and chilly, and he was already suffering from a cold or some other bug; back in Dublin, he felt worse and stayed home for several days.
When he returned to school, he discovered something weird: After a round of sports, he now experienced muscle pains and a paralyzing exhaustion unlike anything he’d previously encountered. “I’d be totally whacked by the end of the day,” he recalled.
He saw a physiotherapist and then an orthopedic surgeon, who told him to exercise more. He tried swimming, but that also left him depleted. In 1991, despite his health struggles, he entered Trinity College. He slogged through two years of math studies but suffered more and more from problems with memory and concentration. “I was forgetting things, making silly errors,” he said.
Toward the end of the second year, he could no longer hold a pen in his hand. He developed tendonitis, first in one arm, then in the other. When he drove, pushing the pedals caused severe ankle pain. “Everything was magnified now,” he said. “I was just breaking down.” He took a leave from Trinity. His health continued to slide.
***
Then Kindlon read something about myalgic encephalomyelitis, or ME—an alternate name for chronic fatigue syndrome frequently used in the U.K., meaning “inflammation of the brain and spinal cord, with muscle pain.” A specialist confirmed the diagnosis.
Since there are no approved medical tests, diagnosis has generally been made based on symptoms, after other possibilities have been excluded. A major clue in Kindlon’s case was his experience of a prolonged collapse after sports. Almost all patients report this unusual symptom, called “post-exertional malaise”–a sustained relapse or worsening after a minimal amount of exertion or activity.
It was September, 1994. Tom Kindlon was 22 years old. He could just about drag himself to the toilet a few times a day. He could hold a brief conversation, though he often couldn’t remember what he or anyone else had said.
Soon after his diagnosis, he heard about a local support group called the Irish ME Association. Vera attended a meeting to learn more. She became a fixture at the monthly gatherings, and soon was voted chair of the group; her son was appointed assistant chair. Though his condition gradually stabilized and sometimes even seemed to improve a little, he never felt well enough to attend meetings and worked instead from home.
At the time, the organization only had a few dozen members. “I felt the group could get bigger than just people sitting in circles,” Kindlon said. “We needed to raise awareness. I wanted people’s stories to be told.”
On May 12, 1996, designated by U.K. advocates as International ME Day, the small Irish group held a public event. Vera spoke on national radio. The Kindlons, mother and son, publicized the group’s work, and by 2000 the membership list topped 400.
Through a leadership listserv, Kindlon maintained contact with dozens of patient support and advocacy groups around the UK and elsewhere; the network kept him abreast of the major scientific, public health, and political developments related to the illness. Then he learned about the PACE trial.
***
In the mid-1980s, several outbreaks of a disabling and prolonged flu-like illness popped up across the U.S. Although clinicians treating some of the patients believed it was associated with the Epstein-Barr virus, which causes mononucleosis, CDC investigators were unable to identify a link with that or other pathogens.
The CDC team called the mysterious condition “chronic fatigue syndrome” after rejecting the name “myalgic encephalomyelitis,” coined after a similar outbreak at a London hospital in the 1950s. The key symptom of myalgic encephalomyelitis had been identified as extreme muscle fatigue after minimal exertion, with delayed recovery—essentially, a description of post-exertional malaise, Tom Kindlon’s main symptom. The CDC also rejected “post-viral fatigue syndrome,” another common name. In contrast, the World Health Organization, which had years earlier classified “benign myalgic encephalomyelitis” as a neurological disorder, deemed both post-viral fatigue syndrome and chronic fatigue syndrome to be synonyms. (The word “benign” eventually fell out of common use.)
In the U.S., the disease is now often being called ME/CFS by government agencies; the recent report from the Institute of Medicine suggested renaming it “systemic exertion intolerance disease,” or SEID. In the U.K, it is often called CFS/ME.
Patients have always hated the name chronic fatigue syndrome. For one thing, the word “fatigue” does not come close to describing the profound depletion of energy that marks the illness. A few years ago, best-selling author and long-time patient Laura Hillenbrand (Unbroken; Seabiscuit) once told The New York Times: “This disease leaves people bedridden. I’ve gone through phases where I couldn’t roll over in bed. I couldn’t speak. To have it called ‘fatigue’ is a gross misnomer.”
Patients, clinicians and scientists say the name is also inaccurate because the hallmark is not fatigue itself but more specifically what Tom Kindlon experienced—the relapses known as post-exertional malaise. (Patients also criticize the word ‘malaise,’ like ‘fatigue,’ as inaccurate and inadequate, and many prefer to call the symptom ‘post-exertional relapse.’) Other core symptoms are cognitive and neurological problems, sleep disorders, and in many cases muscle pain.
Researchers have not been able to identify a specific cause—at least in part because investigators have used many different criteria to define the illness and identify study subjects, making it hard to compare results. In many cases, as in the 1980s outbreaks, ME/CFS appears to be triggered by a viral or other infection from which people never recover. Since patients often don’t seek treatment and are not diagnosed until they have been sick for a long time, research on triggering events has often been based on self-reports of an initial infection rather than laboratory confirmation. However, a prospective 2006 study from Australian researchers and the CDC found that 11 percent of more than 250 patients who were followed after acute cases of mononucleosis, Q fever, and Ross River virus met diagnostic criteria for chronic fatigue syndrome six months later.
Although in some cases patients report a gradual start to the illness, a 2011 definition of myalgic encephalomyelitis developed by an international expert committee noted that “most patients have an acute infectious onset with flu-like and/or respiratory symptoms.” In fact, many experts believe ME/CFS is likely a cluster of related illnesses, in which one or more infections, or exposures to toxins, mold, stress, trauma or other physiological insults, spark the immune system into a persistent state of hyper-activation, with the resulting inflammation and other systemic effects causing the symptoms. Like the varying methods for defining the illness, the heterogeneity of potential triggering events among chronic fatigue syndrome populations has also complicated research. Without accurate sub-grouping, the findings from such samples can undermine rather than promote the search for causes, biomarkers and treatments.
The illness can fluctuate over time. Long-term patients sometimes experience periods of moderate remission, but few appear to recover completely. Most treatment has involved symptomatic relief.
Although research has been hampered by limited government support, studies over the years have documented a wide range of biological abnormalities as well as associations with a host of pathogens. But some promising leads have not panned out, most spectacularly several years ago when an apparent association with mouse retroviruses turned out to be the result of lab contamination—a devastating blow to patients.
***
For their part, the PACE investigators have collectively published hundreds of studies and reports about the illness, which they prefer to call chronic fatigue syndrome. In their model, the syndrome starts when people become sick—often from a virus, sometimes from other causes. This short-term illness leaves them exhausted; when the infection or other cause passes and they try to resume normal activity, they feel weakened and symptomatic again. This response is expected given their deconditioned state, according to the model, yet patients become fearful that they are still sick and decide they need more rest.
Then, instead of undergoing a normal recovery, they develop what the PACE authors have called “unhelpful beliefs” or “dysfunctional cognitions”–more specifically, the unhelpful belief that they continue to suffer from an infection or some other medical disease that will get worse if they exert themselves. Patients guided by these faulty cognitions further reduce their activity and, per the theory, become even more deconditioned, ultimately leading to “a chronic fatigue state in which symptoms are perpetuated by a cycle of inactivity, deterioration in exercise tolerance and further symptoms,” noted a 1989 article whose authors included Chalder and Simon Wessely, the PACE investigators’ longtime colleague.
The two rehabilitative therapies were designed to interrupt this downward spiral and restore patients’ sense of control over their health, in part through positive reinforcement and encouragement that recovery was possible. The course of cognitive behavior therapy, known as CBT, was specifically designed and structured to help chronic fatigue syndrome patients alleviate themselves of the “unhelpful beliefs” that purportedly kept them sedentary, and to encourage them to re-engage with daily life. (Standard forms of cognitive behavior therapy are recommended for helping people deal with all kinds of adversity, including major illness, yet doctors do not suggest that it is an actual treatment for cancer, multiple sclerosis, or renal failure.) The increase in activity known as graded exercise therapy, or GET, sought to counteract the deconditioning by getting people moving again in planned, incremental steps.
Through their extensive writings and their consulting roles with government agencies, Sharpe, Chalder, White, and their colleagues have long exerted a major impact on treatment. In the U.K., the National Health Service has primarily favored cognitive behavior therapy and graded exercise therapy, or related approaches, even in specialized clinics.
In the U.S., the Centers for Disease Control and Prevention has collaborated with White, Sharpe and some of their colleagues for decades. The agency recommends the two treatments on its website and in its now-archived CFS Toolkit for health professionals about how to treat the illness. The toolkit recommends contacting St. Bartholomew’s—the venerable London hospital that is one of White’s professional homes—for more information about graded exercise therapy.
White, the lead author of the Lancet paper, is a professor of psychological medicine at Queen Mary University of London and co-leads the chronic fatigue syndrome service at St. Bartholomew’s. Sharpe is a professor of psychological medicine at Oxford University, and Chalder is a professor of cognitive behavioral psychotherapy at King’s College London. Their faculty webpages currently credit them with, respectively, 90, 366 and 205 publications.
The PACE authors have been referred to as members of the “Wessely school”—or, less politely, the “Wessely cabal”– because of Simon Wessely’s prominence as a pioneer of this treatment approach for chronic fatigue syndrome. Wessely, a professor of psychological medicine at King’s College London, has published more than 700 papers, was knighted in 2013, and is the current president of the Royal College of Psychiatrists.
Over the years, members of the PACE team developed close consulting and financial relationships with insurance companies; they have acknowledged these ties in “conflict of interest” statements in published papers. They have advised insurers that rehabilitative, non-pharmacological therapies can help claimants with chronic fatigue syndrome return to work—as Sharpe noted in a 2002 UNUMProvident report on disability insurance trends.
In his article for the UNUMProvident report, Sharpe also criticized the “ME lobby” for playing a negative role in influencing patients’ self-perceptions of their condition, noting that “the patient’s beliefs may become entrenched and be driven by anger and the need to explain continuing disability.” Sharpe noted that economic and social factors, like receiving financial benefits or accepting the physiological illness claims made by patient groups, also represented roadblocks to both clinical improvement and the resolution of disability insurance claims.
“A strong belief and preoccupation that one has a ‘medical disease’ and a helpless and passive attitude to coping is associated with persistent disability,” Sharpe warned readers of the disability insurance report. “The current system of state benefits, insurance payments and litigation remain potentially major obstacles to effective rehabilitation…If the claimant becomes hostile toward employer or insurer the position is likely to be difficult to retrieve.”
***
Given the medical and social costs of the illness, the government wanted solid evidence from a large trial about treatments that could help people get better. In 2003, the U.K. Medical Research Council announced that it would fund the PACE trial—more formally known as “Comparison of adaptive pacing therapy, cognitive behavior therapy, graded exercise therapy, and specialist medical care for chronic fatigue syndrome: a randomized trial.”
Three other government agencies–Scotland’s Chief Scientist Office, England’s Department of Health, and the U.K. Department for Work and Pensions—chipped in. The West Midlands Multicentre Research Ethics Committee approved the final study protocol.
The investigators selected two self-reported measures, for physical function and fatigue, as their primary outcomes. For physical function, they chose a section of a widely used questionnaire called the Medical Outcomes Study 36-Item Short Form Health Survey, or SF-36; with this physical function scale, they designated a score of 60 or less out of 100 as representing sufficient disability for trial entry.
For fatigue, they selected the Chalder Fatigue Scale, developed by one of the PACE investigators, on which higher scores represented greater fatigue. The response to each of the scale’s 11 questions would be scored as 0 or 1, and a score of 6 or more was deemed sufficient evidence of disability for trial entry.
In the proposed trial, participants would be randomized into four arms. All would be offered a few meetings with a specialist—the baseline condition ultimately called “specialist medical care.” Participants in three of the arms would receive additional interventions, of up to 14 sessions over six months, with a booster session three months later. Everyone would be assessed one year after entering the trial—that is, six months after the end of the main period of treatment. Home-bound patients were not eligible, since participation required attendance at multiple clinic sessions.
Besides the two rehabilitative treatments of cognitive behavior therapy and graded exercise therapy, the investigators planned to include an intervention based on a popular self-help strategy known as “pacing.” While the first two approaches challenged patients to adjust their thinking and push themselves beyond what they believed they could do, pacing involved accepting and adapting to the physical constraints of the illness, paying attention to symptoms, and not exceeding personal energy reserves to avoid triggering a relapse.
***
Previous studies conducted by the authors and other researchers, although smaller than PACE, had found that graded exercise therapy and cognitive behavior therapy led to modest improvements in self-reported outcomes, as a 2001 review in JAMA noted. But the same review also warned that the positive results on subjective measures in these studies did not mean that participants had actually improved their physical capacities.
“The person may feel better able to cope with daily activities because they have reduced their expectations of what they should achieve, rather than because they have made any recovery as a result of the intervention,” stated the review. “A more objective measure of the effect of any intervention would be whether participants have increased their working hours, returned to work or school, or increased their physical activities.”
Aware of such concerns, the PACE investigators planned to include some measures of physical function and fitness not dependent on subjective impressions.
Beyond the question of how to measure the effects of the intervention, the therapies themselves remained highly controversial among patients. Many understood that cognitive behavior therapy could be a useful tool for coping with a serious condition but resented and dismissed the PACE authors’ suggestion that it could treat the underlying illness. Encouraging an increase in exercise or exertion was even more controversial. Patients considered it dangerous because of the possibility of relapse from post-exertional malaise. In surveys, patients who had received graded exercise therapy were more likely to report that it had made them worse rather than better.
The psychiatrists and other mental health experts acknowledged that patients often felt worse after starting an activity program. To them, the resurgence of symptoms reflected the deconditioned body’s natural response to renewed exertion, not an underlying disease process—a point strongly conveyed to patients. According to the PACE manual for clinicians administering graded exercise therapy, “Participants are encouraged to see symptoms as temporary and reversible, as a result of their current physical weakness, and not as signs of progressive pathology.”
***
Patients and advocates, aware of the previous work of the PACE team, responded to the Medical Research Council’s announcement with alarm. Fearing the research would lead to calls for more funding for cognitive behavior therapy and exercise therapy and nothing else, patient groups demanded that the agency back research into biological causes and treatments of ME/CFS instead–something it was not doing.
“We believe that the money being allocated to the PACE trial is a scandalous way of prioritising the very limited research funding that the MRC [Medical Research Council] have decided to make available for ME/CFS,” declared the ME Association, a major advocacy organization, in a statement widely disseminated on social media. The statement demanded that the trial be halted and the money “held in reserve for research that is likely to be of real benefit to people with ME/CFS.”
Despite the anger in the patient community, the investigators were able to enlist Action For ME, another major advocacy group, to help design the pacing intervention. They called their operationalization of the strategy “adaptive pacing therapy,” or APT.
The trial protocol described the pacing therapy as “essentially an energy management approach, which involves assessment of the link between activity and subsequent symptoms and disability, establishing a stable baseline of activity using a daily diary, with advice to plan and pace activity in order to avoid exacerbations.” But many patients argued that pacing was an inherently personal, flexible approach. Packaging it as a structured “treatment” administered by a “therapist,” with a focus on daily diaries and advance planning, would inevitably alter its effect, they said.
***
Patients and other researchers also objected to the PACE study’s choice of “case definition”—a set of research or diagnostic criteria designed to include everyone with an illness and exclude those without it. Many challenged the decision to identify participants using the single-symptom case definition of chronic fatigue syndrome called the Oxford criteria—the same broad case definition that last June’s NIH report recommended for retirement because it could “impair progress and cause harm.”
Over the years, there have been many definitions proposed for both chronic fatigue syndrome and myalgic encephalomyelitis, for both clinical and research use. The most widely used has been the CDC’s 1994 definition for chronic fatigue syndrome, which required six months of fatigue, plus any four of eight other symptoms: cognitive problems, muscle pain, joint pain, headache, tender lymph nodes, sore throat, post-exertional malaise, and sleep disturbances.
Many patients, researchers and clinicians experienced in treating the illness prefer more recent and restrictive definitions that seek to reduce misdiagnoses by requiring the presence of the core symptom of post-exertional malaise as well as neurological and cognitive dysfunctions, unlike the more flexible CDC definition. In contrast, the Oxford criteria, published in 1991 by PACE investigator Michael Sharpe and colleagues, required only one symptom: six months of medically unexplained, disabling fatigue. Proponents argued that this broad scope ensured that research results could be applied to the largest number of people potentially suffering from the illness. If other symptoms were present, as often happened, the criteria required that fatigue be the primary complaint.
According to DePaul psychologist Leonard Jason, the Oxford criteria blurred the boundaries between “chronic fatigue,” a symptom of many conditions, and the distinct illness known as “chronic fatigue syndrome.” In particular, he said, an Oxford criteria sample would likely include many people with primary depression, which can cause prolonged fatigue and often responds to interventions like those being tested in PACE. (In contrast, many people with ME/CFS get depressed as a secondary result of their illness experience.)
“The Oxford criteria clearly select for a lot of patients with primary depression, and people who are depressed do react very well to CBT and exercise,” said Jason, who has published widely on the ME/CFS case definition problem. Positive outcomes in the sample among depressed patients without ME/CFS could therefore lead to the unwarranted conclusion that the therapies worked for people with the disease, he added.
***
The PACE investigators were aware of these concerns, and they promised to study as well two subgroups of participants from their Oxford criteria sample who met additional case definitions: an updated 2003 version of the CDC’s 1994 definition for chronic fatigue syndrome, and a separate definition for myalgic encephalomyelitis. That way, they hoped to be able to draw conclusions about whether the therapies worked, no matter how the illness was defined.
Yet this approach presented its own challenges. Neither of the two other definitions required fatigue to be the primary symptom, as did the Oxford criteria. The myalgic encephalomyelitis definition did not even include fatigue per se as a symptom at all; post-exertional malaise, not fatigue, was the core symptom. And under the CDC definition, patients could present with any of the other symptoms as their primary complaint, as long as they also experienced fatigue.
Given these major differences in the case definitions, an unknown number of patients might have been screened out of the sample by the Oxford criteria but still met one of the other sets of criteria, making it hard to interpret the subgroup findings, according to other researchers. (The PACE investigators and I debated this methodological issue in an exchange of letters in The New York Times in 2011, after an article I wrote about case definition and the PACE trial.)
Bruce Levin, the Columbia University biostatistician, said the PACE investigators should not have assumed that the experience of a subgroup within an already defined population would match the experience of a group that hadn’t been pre-screened. “I would not accept an extrapolation to people diagnosed with alternative criteria from a subgroup comprising people satisfying both sets of criteria rather than just the alternative set of criteria,” he said, adding that reviewers should catch such questionable assumptions before publication.
Tomorrow: Publication of the PACE trial