There’s very little that science knows definitively about chronic fatigue syndrome, a disease characterized by often debilitating physical and mental exhaustion — researchers still haven’t been able to identify a cause, let alone a cure. But in 2011, a study published in The Lancet claimed that it had done exactly that: The data, the authors declared, showed that a combination of exercise and talk therapy could significantly alleviate the symptoms of the disease, and even cure it fully in up to 20 percent of cases. Immediately, the study was both hailed as a great leap forward and criticized as bad science.
And now, definitive proof has emerged that the latter camp was correct. In a column published in Stat today, writer Julie Rehmeyer — herself a CFS patient — explained how a supposed breakthrough blew up so spectacularly.
Soon after the study was published, Rehmeyer wrote, she and other CFS patients, skeptical of the study’s claims, began to examine it more closely. What they found looked a lot like manipulated data:
Before the trial of 641 patients began, the researchers had announced their standards for success — that is, what “improvement” and “recovery” meant in statistically measurable terms. To be considered recovered, participants had to meet established thresholds on self-assessments of fatigue and physical function, and they had to say they felt much better overall.
But after the unblinded trial started, the researchers weakened all these standards, by a lot. Their revised definition of “recovery” was so loose that patients could get worse over the course of the trial on both fatigue and physical function and still be considered “recovered.” The threshold for physical function was so low that an average 80-year-old would exceed it.
In addition, the only evidence the researchers had that patients felt better was that patients said so. They found no significant improvement on any of their objective measures, such as how many patients got back to work, how many got off welfare, or their level of fitness.
Here’s a rough timeline of what happened next. Patients began writing their own papers and columns arguing against the validity of the study, which was called the PACE trial; a few also filed unsuccessful requests to obtain the raw data from London’s Queen Mary University, where the lead researcher worked. But things remained fairly stagnant until last October, when health reporter David Tuller joined them, publishing a thorough takedown of PACE on the Virology blog in a two-part post titled “Trial by Error: The Troubling Case of the PACE Chronic Fatigue Syndrome Study.”
From there, things snowballed: The next month, the same blog published an open letter to The Lancet from a team of scientists from Columbia, Stanford, and other universities, arguing that the journal should “seek an independent re-analysis of the individual-level PACE trial data.” Then, a few weeks ago, Queen Mary — “under court order,” Rehmeyer noted — finally released the data for public scrutiny. And finally, today, a new, damning analysis went up on Virology, confirming that the numbers didn’t support the PACE authors’ dramatic claims. The post concluded: “It is now up to the PACE authors to explain why anyone should accept their published outcomes as accurate, reliable or legitimate.”
The end of this particular saga feels fitting, somehow, for a community of patients whose disease is so thoroughly misunderstood. It’s a victory bound up in failure: On the one hand, invalidating the PACE study validates their experience; on the other, proving the research was useless also means we’re still that much further away from knowing how to treat the disease.
But as all this was unfolding, the research community was also making smaller steps toward progress. Last February, the Institute of Medicine released a report arguing that CFS was indeed a real disorder and outlining its diagnostic criteria, and a study published around the same time found evidence for the biological roots of the disease. Earlier this month, as the Washington Post reported, a study in Proceedings of the National Academy of Sciences found similarities between the metabolisms of CFS patients and hibernating animals, suggesting that “curing people of CFS may be akin to waking the body up.” The paper noted that scientists are now trying to replicate the PNAS study with a larger sample size; in the meantime, the hypothesis — as with so many other things surrounding CFS — remains just that.
