Dr. Unger's presentation of the CDC study demonstrated the flaws that have been apparent in every federally-sponsored study of ME/CFS to date. The instruments used in the study are old stand-bys, and of dubious relevance. The reason they were chosen for this study is that, with one exception (the DePaul questionnaire), they are standard tools of the trade. The problem is that ME/CFS is not a standard illness. (To quote an old adage: You can't fit a round peg into a square hole.)
Several of the questionnaires covered anxiety and depression. The results proved - again - that ME/CFS is neither. Yet, Dr. Unger reported that mental health was fairly normal in ME/CFS patients with a distinct tone of surprise. Conspicuously absent from the CDC's roster was a single objective measurement (e.g. 2-day CPET).
The IOM panel, particularly those with ME/CFS expertise, asked pertinent questions, most of which Dr. Unger was unable to answer. Dr. Rowe posed the question of whether a sample of patients who had been ill, and treated, for a long time was appropriate for establishing diagnostic criteria. He pointed out that the illness changes over time. (Actually, it is the patients who change. As Dr. Cheney pointed out 20 years ago, our bodies continually adjust to an ongoing disease process.)
Dr. Unger was followed by Susan Maier (Deputy Director of the Office of Research on Women's Health (ORWH)), who gave a presentation that was so laden with jargon and bad syntax that it was nearly impossible to follow. (She compensated for her inability to form a coherent sentence by a great deal of hand waving.) Maier's subject was the Pathways to Prevention (P2P) panel that is being assembled to review research definitions for ME/CFS.
In all respects, the P2P process is even more far-fetched than the IOM's. The members of the panel not only can't be experts in the field of ME/CFS, they can't have anything whatsoever to do with the illness. What's more, they don't even have to be physicians. They can be "highly regarded lawyers, ethicists, methodologists" - and, one presumes, expert butchers, bakers and candlestick makers.
This hodge-podge panel will review material that has been culled (by yet more non-experts using "standard methodology") and listen to presenters, each of whom has exactly 20 minutes to state their case "pro or con." Maier explains that this process is similar to the jury system, with a prosecution and a defense. The jury knows nothing initially, but is educated - over the course of two days - in everything they need to know to make a judgment.
Not only is this the height of absurdity, it is very definitely not the jury model. In our justice system, the defense and prosecution are able to question the jury to discover if they have any prejudices that might influence their decision. With 85% of physicians in the U.S. believing that "CFS" is a mental illness, and the majority of the public probably believing the same, can we really expect this panel to be unbiased? In any event, there is no mechanism for finding out if they are.
And if this were a trial, as Maier proposed, wouldn't the defendants (ME/CFS patients) have the right to face their accuser? Doesn't a jury have to deliberate until they reach a decision - instead of being given a deadline of 24 hours? Wouldn't they have more than two days to hear arguments? Wouldn't the defendants have a say in what constituted evidence?
To employ a more apt judicial metaphor, the P2P panel isn't a jury - it's a kangaroo court.
______________________________
This article originally appeared on ProHealth.
Note: You can read summaries HERE of the introductory segment, including Dr. Rick Erdtmann, Director of the Board on the Health of Select Populations, Dr. Ellen Wright Clayton, IOM Committee Chair, and Dr. Nancy Lee, Designated Federal Officer to CFSAC, study sponsor [HHS] representative.
By Erica Verrillo
The second hour of the IOM public meeting held on January 27, 2014 consisted of reports from related federal projects: The CDC Multi-site Clinical Study and the Pathways to Prevention Program (P2P), formerly known as the NIH Evidence-Based Methodology Workshop.
Each of these studies has profound implications for the ME/CFS community. The CDC study provides data that will be used by both committees, as well as researchers and clinicians, and the P2P panel reviews definitions that will be used by researchers.
You can read the meeting agenda HERE.
You can watch the presentations HERE.
You can read information about the committee members HERE.
It is not too late to submit comments to the IOM Committee. Send your comments to: mecfs@nas.edu
Elizabeth Unger - Methodology for the CDC Multi-site Clinical Study (18:07) (Video)
Dr. Elizabeth Unger is Chief of the Chronic Viral Diseases Branch (CVDB), Centers for Disease Control (CDC). She began her talk by stating that physicians worldwide recognize an illness that is called CFS or ME, and that there are more similarities than differences in the case definitions for how this illness is described.
Dr. Unger pointed out that the illness is very complex and patients are heterogeneous. The heterogeneity of patients in clinical trials and research studies and in epidemiologic studies can confound results. This may be contributing to the lack of uniform advancement in studies of this illness.
Some of the confounding factors for research have been: severity of illness, duration of illness, co-morbid conditions, medications, and demographics (age, sex, etc.) In Dr. Unger’s opinion, medications have been the least investigated, and could be most problematic for understanding the illness.
Clinicians need a case definition that is as clear and simple as possible so that this illness is recognized and patients get the case they deserve.
When designing the multi-site study, the CDC asked themselves two questions:
- Is refining or adding criteria going to produce a homogeneous population?
- What do you do with patients who don’t fit any one definition?
Because the CDC realized case definition alone was not going to be sufficient, they decided to capitalize on the expertise of clinicians who had the greatest experience treating the illness.
The multi-site study examines standardized data from patients aged 18-70 from seven specialist practices in the U.S. The enrollment criteria relied on clinical expertise rather than on any one case definition. I am using the term “CFS,” but we made it clear to physicians that CFS, ME, post-infectious fatigue, or any of these synonyms or possibly related illnesses, were eligible to participate. The only exclusions were HIV-positive patients, and patients older than age 62 at diagnosis.
Participating Physicians:
- Benjamin Natelson
- Nancy Klimas
- Lucinda Bateman
- Charles Lapp
- Andreas Kogelnik
- Daniel Peterson
- Richard Podell
Stage 1: First Year of Study
Dr. Unger devoted the bulk of her talk to describing Stage I of the study, which was the first year of gathering data obtained by physical examinations, patient questionnaires, and data extracted from medical records (e.g. medical history, medications list, lab tests, family history, infection and immunization history).
Data was gathered from 471 participants who filled out questionnaires either at the time of the clinic visit or in the week before. These included: Patient Health Questionnaire Depression Scale, Generalized Anxiety Disorder 7-Item Scale, Self-Rating Depression Scale, CDC Symptom Inventory (which includes 19 symptoms experienced during the previous month), the Medical Outcomes Study 36-Item short form (which measures quality of life), the Multi-Dimensional Fatigue Inventory-20 (which measures general fatigue, physical fatigue, mental fatigue, reduced motivation and reduced activity), questions from the DePaul symptom inventory, NIH PROMIS forms (pain, fatigue and sleep), sleep questionnaires, and Brief Pain Inventory.
Demographics
The patient population consisted of slightly overweight, middle-aged (48.2) white females with college education. Half were married, and 75% were unemployed, though most did not receive unemployment benefits.
The mean age at diagnosis was 38.4%, with sudden onset reported in 65.4%. Mean duration of “fatigue” was 14.3 years, ranging from 2.5 to 52 years.
Dr. Unger commented that the patient population studied through these specialty clinics was not necessarily representative of the patient population as a whole. Most patients were highly educated and insured, and all had been seen and evaluated by other physicians.
Results
The highest PROMIS scores were in generalized fatigue, physical fatigue, and reduced activity. On the SF-36 mental health was relatively preserved, while vitality was reduced. Functional measures included the mean number of hours for vertical activity (7.5 hours), the mean number of hours for horizontal activity (12.8 hours). The mean days per week in which patients exercised was 3.3.
The CDC symptom inventory showed the most prominent symptoms to be Post-Exertion Malaise (PEM) and unrefreshing sleep, followed by muscle pain and memory and concentration problems.
In the context of other illnesses, PROMIS scores showed that CFS patients experienced greater fatigue, sleep disturbance, and pain impairment than patients with chronic pelvic pain, spinal cord injury, Muscular Dystrophy, Post-Polio Syndrome, and MS.
Q&A with Dr. Unger (11:01) (Video)
Question – Nancy Klimas: Have you been able to do an analysis of different case definitions on this data set?
Answer: We have not done that but it is certainly possible. The data is there. That requires a lot of interpretation that we feel will require a lot of dialogue with people, in other words, how to set the criteria for each of the points in the disease criteria.
Question – Nancy Klimas: You have the DePaul symptom inventory on the list. Lenny Jason has an algorithm on an excel spreadsheet that we can use to compare Fukuda to the 2003 Canadian.
Answer: It still requires some interpretation and discussion. The data is there.
Question – Lily Chu: What is the timing and how will the committee receive data? My second question is, I can understand why you are asking clinicians to come up with who fits the CFS diagnosis. Is there any thought given to asking two or three physicians looking at the data if they would agree that this person has CFS? My third question is, you have measures of activity, but other than upright activity, are there any subjective questionnaires of what people do day to day?
Answer: None of these patients were diagnosed by a single physician. They are referrals. I think they [the physicians] are competent in their diagnostic skills.
As for measures of activity, we do not have narrative of what patients do in a given day. But all the questionnaires have time frames. They vary from being vague, others are for two weeks, and the DePaul is for six months. Each has a narrow-minded way in which they are asking questions. This illness defies neat categories. We have to be careful interpreting the data we have.
About logistics – we have the baseline data ready. We can provide the committee with tables and answers.
Question – Dr. Clayton: Did the patients consent to data sharing?
Answer: Patients consented to data sharing within the study. We feel we can release the analyses, but not the raw data, to the IOM. We feel that’s within informed consent.
Question – Peter Rowe: I have a question about the relevance of prevalence data. You have got patients who have been treated and followed for some length of time. It seems to me it would be problematic to apply those to case definitions. My hope in clinic is that I will have some impact on some patients’ initial CFS symptoms within a short period of time. So the problem might be that if people have been treated by such an experienced group, some of their symptoms that might be defining symptoms for CFS/ME will be gone by the time you’ve captured the data. How are you approaching that dilemma?
Answer: It is a dilemma. We sort of try to do one step at a time. We are aware of that. I was glad we had 80 patients that were only 2 ½ years from the onset of their illness. In the next phases of the study we have asked physicians to enroll patients earlier on in their disease course. As a last point, we have the intake form which gives a picture of the patients when they first come in, but even there, because this is a referral clinic, this may not be what the illness looked like at first. It’s not only the patients’ response to therapy, but the illness itself changes with time. That is a limitation that we have.
Question – Cynthia Mulrow: I want to understand the representativeness of this sample and what likely spectrum of CFS they might represent. If a large spectrum has not been captured, then it may not be right for us to go overboard using these data to help inform diagnostic criteria.
Answer: We think that it is data that could be used to inform you but we don’t think that absolute decisions can be made solely on the basis of this data. It’s a beginning point. I think that the patients that are in this study should meet the case criteria, whatever the case definition is. That’s not to say others wouldn’t.
The other piece of data in the comparison we have is the data on our population-based surveillance. We use many of the same instruments, so we have a data from those two and we can see how they compare.
This is to me one of the most important steps the field can take is getting some consensus on what are some shared measurements that studies and researchers and clinicians can count on and use, so that we can start making comparisons. One thing we’ve been lacking is measurements of this illness we can use, that’s why I like the PROMIS instrument so much. It compares patients with other patient groups.
Question – Lily Chu: Regarding timing there are studies, and patient anecdotes, indicating that sore throat and flu-like symptoms go down with time and neurologic symptoms go up over time. To Cynthia’s point, there are some epidemiologic studies that suggest that minority and lower socio-economic groups are high risk for CFS and may have more severe cases. This group [CDC study population] is highly educated, has resources to see referrals, etc.
Susan Maier - NIH Evidence-Based Methodology Workshop (18:27) (Video)
Dr. Susan Maier is the Deputy Director of the Office of Research on Women's Health (ORWH). She began her presentation by clarifying that the Pathways to Prevention Program (P2P) is a collaboration between NIH Office of Disease Prevention and the Trans-NIH ME/CFS Research Working Group.
The goals of the program are to identify research gaps and methodological scientific weaknesses in a scientific area, to suggest research needs, and to move the field forward through an unbiased and evidence-based assessment of a public health problem.
The P2P Program is for any disease, illness or syndrome that needs a more in-depth understanding, keeping in mind that the goal is to serve the community of researchers.
Timeline
The P2P process was begun in June 2012 when the Trans-NIH ME/CFS Research Working Group and ORWH began formulating a proposal. The Trans-NIH ME/CFS Research Working Group proposal was accepted by the Office of Disease Prevention in December of 2012. The first working group meeting to plan the workshop was held in January 2014. The tentative date for the workshop is December 2014.
Making a Case for ME/CFS
- Necessity – lack of physician recognition, lack of biomarkers, lack of understanding about the scope and range of the illness
- Urgency – there are no FDA-approved treatments
- Public Health Problem – people are sick and there is an unmet need
The working group is composed of experts in the field, patients, advocates, caregivers and Federal partners (e.g. CFSAC, HHS )
They had three tasks.
- Refine the questions for the evidence review. The evidence review is performed by an evidence-based practice center – this is a contract done through AHRQ (Agency for Healthcare Research and Quality).
- Develop the workshop agenda: Topics, speakers and format.
- Nominate panel members.
Questions had to be developed and refined, because the questions form the basis for the agenda of the workshop. (Who will speak and on which topics.)
- How do ME and CFS differ?
- What tools will allow us to define subsets across the entire subset of CFS?
- What are the characteristics of patients who respond to specific treatments across the spectrum of CFS?
- What does research on ME/CFS tell us about clinical diagnosis of ME/CFS?
- Have previous research findings shaped current clinical practice or are research and clinical practice completely separate?
Nomination of Panel Members
Panel members are highly recognized experts in their areas. These are not experts in ME/CFS, and not necessarily medical doctors or researchers. They may be attorneys, methodologists, ethicists, public representatives – as long as they have no direct connection with CFS. None may be employees of the Federal government.
What happens next?
ODP will set a date for the speakers. Evidence-based practice center will cull the evidence using standard methodology. Every meeting must be approved, so this workshop must be approved by the Federal government. Once the approval has been granted all information will be made public, including speakers and evidence.
The end result is a set of recommendations from the panel based on what they have read and heard. A follow-up (1-9 months later) will then work on how to process the recommendations and put them into practice.
P2P Contacts:
Paris Watson, Senior Advisor, P2P, Office of Disease Prevention (ODP)
watsonpa@mail.nih.gov prevention@nih.gov
Susan E. Maier, Deputy Director of the Office of Research on Women's Health (ORWH) Susan.Maier@nih.gov
Mariela Shirley, Chair, Trans-NIH ME/CFS Working Group shirelym@od.nih.gov
Susan E. Maier - Q&A (14:59) (Video)
Question - Theodore G. Ganiats: At the end there are some recommendations. Are those recommendations for research agenda for the NIH, or are they for practice?
Answer: It has the potential to be both, but our focus is on research that is used for translation into clinical care means we have to focus on “besting the science.”
Question - Theodore G. Ganiats: In the group of panelists, there was nobody from primary care. Given that most patients with this disease, whatever we call it, will be seen I primary case, I think that adding primary care would be important.
Answer: Absolutely.
Question - Lily Chu: On the list of refining questions, it says, “What does research on ME/CFS tell us about clinical diagnosis of ME/CFS?” That seems very similar to the charge of the IOM. How does that work between the NIH and IOM?
My other question is that I’ve been reviewing the P2P website to understand how the process works – you can’t be on the panel if you are a clinician with any experience, or you cannot be a researcher that has published on it? That was concerning to me because this is a complex illness.
One of the requirements for the topics is that it be non-controversial. This is a field that is undergoing a lot of changes. My concern is that there is not going to be anyone with any clinical or research experience on the panel.
My third question is, according to my understanding, the panel writes the document in 24 hours after the workshop?
Answer: The panel receives the evidence report, and any other documentation that is submitted, six weeks before the workshop. At the workshop, the panel will listen to speakers, and speakers will provide additional information that may not be in the evidence report. For example the CDC study may not be published yet. But that information is very important for the panelists.
Yes, the report is written in 24 hours. They do have a day and a half total. That report will be posted on the website for public comment. After 14 days, it will be taken down and the comments will be reviewed. The report will incorporate that content.
There is a link between what the IOM is doing and what we are doing. So, we hope to be able to work with the IOM committee to share the synergy.
About the panel requirements – the panel members may not be clinicians who see patients with ME/CFS or researchers who study ME/CFS. They could be a clinician that treats pain disorders, or an oncologist that treats patients with cancer-related fatigue, they could be in a research area that has recently gone a case definition review, such as PCOS. They are highly regarded experts in their field, just not in ME/CFS. That content comes from the speakers.
Think about this: It’s a jury model. You have the defense, you have the prosecution. They know the case very well. The jury is sequestered; they don’t know, they don’t know anything. The jury hears the evidence. They make their decisions based on the evidence.
Question - Nancy Klimas: I’m challenged, because I am on both groups. My understanding is that IOM is not able to put anything out until its final report. It would be very difficult to “synergize.” But it can certainly hear, so it’s a one-way direction from P2P. There is so much that would be duplicated, literature reviews and so on, you have this group doing a comprehensive review of the entire literature, obviously that would be something this committee could really value.
The case definition charge itself is difficult. If this group is really focused on a clinical case definition, then the P2P is really focused on a research case definition.
Answer: The goal of the P2P is not to develop a research case definition, but to review it, review the evidence supporting all case definitions that have been used.
Question - Nancy Klimas: How do you see the one-way direction of this information flow? I am worried about the time lines. When will the Portland group have the evidence from ARHQ?
Answer: The evidence review will be completed roughly six weeks before the workshop. We are “planning” for December 2014.
Question - Nancy Klimas: Might it be possible in order to facilitate this group’s work, to ask that the case definition part be available to this committee sooner than that, because it is too late to be useful on our timeline.
Answer: The two processes are different. Our purpose is different. Our questions will be heavily focused on the research part. That doesn’t mean it won’t be useful for clinical use. Our evidence may not exactly dovetail what you are looking for.
Question - Nancy Klimas: Can those of us who are on both committees have permission to share?
Answer: The content of the meeting is not finalized. All the things we discussed are on sheets of paper and maybe a couple of computer discs. When it’s pulled together, at that point you can share it.
Question - Nancy Klimas: I’m totally confused.
Answer: Paris Watson can give you that specific guidance.
Question - Cynthia Mulrow: My understanding is that protocols for evidence reports are publicly available. Are there ways that we can get access to search strategies that are used in the evidence report tied to specific questions, so that we can peruse those before the final report comes out? My understanding is that these evidence centers develop evidence tables before they finalize the report. Is there a way that our committee can have access to those interim products?
Answer: I don’t know the answer to that. But I can certainly ask.
[Dr. Clayton announces a BREAK.]