Salt is bad for you. According to a 2010 article in the New England Journal of Medicine, lowering dietary salt intake by 3 grams per day could “reduce the annual number of deaths from any cause by 44,000 to 92,000.”
Or maybe not. A 2011 meta-analysis of seven clinical studies of salt reduction, published this week in the American Journal of Hypertension, found “no strong evidence that salt reduction reduced all-cause mortality.” One of the seven studies showed that a low-sodium diet was associated with an increase in the risk of death for certain patients.
Studies of diet–eggs, caffeine and sugar; of screening tests–annual mammograms, PSA testing and lung scans for smokers; even topics as diverse as circumcision or drinking eight glasses of water a day, have all produced conflicting results. How can this happen? How often does this happen?
“We call them reversals,” said Adam Cifu, associate professor of medicine and co-author of a recent research letter on the phenomenon, published in the Archives of Internal Medicine. “Some new therapies are replacements,” he said. “They are better than what came before, as demonstrated by large, well designed, controlled studies.” Other new approaches arrive expecting to become replacements, but biology, complexity and, over time, better studies transform them into reversals.
“They make sense in the laboratory,” Cifu said. “Doctors are eager to try things that should work, based on what we understand about the biology. But the human body is complicated; things that made perfect sense in theory may not work in quite the same way in the clinic. We wanted to find out how often this happened.”
So Cifu and colleague Vinay Prasad, a former Pritzker student and now internal medicine resident at Northwestern, turned to the leading American medical publication, the New England Journal of Medicine. They focused on the 124 articles that appeared in 2009 involving investigation of a new medical practice or a practice already in adoption. Of those 124 articles, 16 could be characterized as a reversal. So 13 percent, one out of eight, contradicted an emerging or accepted medical practice.
The reversals included medical therapies such as tight control of blood sugars for patients in an intensive care unit, invasive procedures such as efforts to reopen clogged arteries for patients with chronic total artery occlusion, and predictive tests such as randomized prostate cancer screening.
One resounding reversal involved back-to-back studies in the August 6, 2009, issue that took a close look at vertebroplasty, a treatment for pain caused by compression fractures of the spine, a common problem for older women. The procedure, which had been widely used for more than a decade, involved injecting bone cement through a small hole in the back into a fractured vertebra. “We spent billions of dollars on this,” Cifu said. Several small early studies implied good results, but there had never been a blinded, placebo-controlled, randomized trial.
However, once NEJM published results from the first such trials, the party was over. One study found “no beneficial effect of vertebroplasty over a sham procedure at 1 week or at 1, 3, or 6 months.” The other reported that results for patients treated with vertebroplasty were “similar to the improvements in a control group.” An editorial in the same issue predicted the procedure, “virtually always considered to be successful,” would now be “considered no better than placebo.”
Cifu and colleagues developed a Why-We-Got-It-Wrong-Initially chart summarizing the reasons behind each reversal.
By far the most common explanation was “confidence that the pathophysiological concepts underlying the practice were rational.” This was a factor – sometimes the only factor – in 13 of the 16 studies. Other causes included studying only short-term endpoints, no good data beforehand, improperly controlled previous studies, or excessive confidence in a single-center trial.
Another study in the same issue, however, showed that new and better data do not always put the brakes on, much less shift the runaway treatment truck into reverse. The Occluded Artery Trial (OAT), published in NEJM in 2008, showed that opening clogged coronary arteries days after an infarct produced no benefit. But a new study that evaluated nearly 30,000 catheterization patients found no significant decline in the procedure for such patients after publication of the trial results and consequent revised treatment guidelines.
Such disregard for better data is disappointing, the authors noted. “The implications for reversals are notable. Reversal implies error or harm to patients who underwent the procedure in question, during the years it was considered effective. Reversal also undermines trust in the medical system.”
Will studies like this one reverse the tendency to ride roughshod over reversals?
“It’s hard to say,” said Cifu. “I’m not that optimistic.”
Prasad V, Gall V, & Cifu A (2011). The Frequency of Medical Reversal. Archives of internal medicine PMID: 21747003