Search form

Sections

Reporting on Evidence Requires More Than Spin

Reporting on Evidence Requires More Than Spin

Reporting on evidence based medicine is tricky. Although randomized controlled trials are sometimes considered the final word on EBM, the reality is they may not be. A study published in PLoS in December 2012 found that randomized controlled trials are sometimes misrepresented in press releases and subsequent news coverage. 

The Science Media Centre UK recently promoted one of a continuing series of papers based on a large RCT, however the results remain controversial among researchers - as does the reporting.  

Although the SMC UK did offer a number of experts as sources, they apparently chose not to include any outside experts who questioned the results and failed to mention industry ties for one of the sources they listed - as well as the authors. Industry ties, whether they be Big Pharma or disability insurers, are not an automatic conflict of interest, but transparency is crucial as is balance in reporting.

Variables count

Whether the results of a trial can be extrapolated to everyone depends on how the many variables were handled. Very few trials cover all variables. For example if an RTC studies white, middle-aged, males does that mean the results can be extrapolated to women, children or different races? It is far too easy to oversell a treatment, particularly in the headline, when the journalist fails to ask hard questions or use independent sources.

In the case of this trial, how the cohort was defined could presumably impact how results were interpreted and who they applied to.  When the initial study first came out U.S. journalist David Tuller MPH, wrote the standard article for the New York Times. However, after an outside expert unaffiliated with the trial contacted him about some of the issues, Mr. Tuller wrote a second piece about a key problem with the trial conclusions. The Wall Street Journal coverage followed a similar path.

Why was this potentially important? A number of large patient surveys showed that the treatments touted may be harmful for some patients. Although the study researchers insist they are not, the potential failure to include the patients most likely to be harmed in the study means there is not yet an evidence base for the safety of the treatment in such populations. While journalists themselves don’t determine if a treatment is safe, safety concerns should always be taken seriously and once again outside experts with concerns should be included. And as another U.S. researcher said, the benefits were modest at best even in cohort studied in tertiary setting where the patient had to be well enough to repeatedly travel to recieve treatment. This also limits generalization as patients too ill to travel were not studied.

So was it efficacious or effective?

Efficacy is when treatments are compared in the same study. In this case, there were also some questions raised regarding whether appropriate comparisons were made and whether the application of treatments matched what other researchers use and study. Once again, interviewing outside researchers who study the treatment these authors compared theirs to would have clarified issues regarding suitability and efficacy. Whether these treatments are effective in a standard clinical setting is not known.

How were results measured?

Most clinical trials publish their protocols first followed by the trial at its conclusion. This allows journalists who read both to determine whether endpoints were changed and ask why, if they were. As well, not all endpoints are meaningful or may use a different standard than similar trials.

Regarding the original trial the authors wrote a letter to the editor protesting the second article Mr. Tuller wrote. Mr. Tuller's reply to the letter clarified why the trial as it was done did not meet the gold standard of evidence-based medicine. Mr. Tuller could not have done that had he not read the full paper and protocols as well as interviewing appropriate outside sources.

In this case, perhaps even more concerning than the dearth of appropriate coverage was that the SMC UK was involved in the spin.

Spin can come from many different sources and may be best countered by reading both the protocol and the full paper, using independent expert,s including researchers in the field whose research results are different than that of the authors and providing meaningful context, as Mr. Tuller did. 

Image by Jaqian via Flickr

Comments

PACE was not a randomized controlled trial. It was an open label study. CBT and GET helped no one in the trial and the trial contained no ME patients as Peter White the lead author admitted.

Thanks, Ms Benson, for taking the trouble to report on this important issue, and the way that the Science Media Centre UK promoted the findings but chose not to include any outside experts who might challenge the findings.

Thank you, Kate Benson, for focusing attention on Fiona Fox's spin machine.

Thank you Ms Benson for providing a balanced account concerning the events around the PACE trial. Too many journalists simply regurgitated the press release and missed the concerted PR campaign by a group of psychiatrists to enforce certain treatments on the largely unwilling patient community. Patient complaints of harm (including attempts to forcibly take children into psychiatric care) have been brushed aside for 20 years, as has medical evidence that such approaches harm (Nunez et al 2012 March edition of Clinical Rheumatology).

Leave A Comment

Announcements

New this year: a $1,000 reporting stipend! Apply now for the all-expenses-paid 2015 California Health Journalism Fellowship and spend five days in Los Angeles March 1-5, 2015 learning about how Obamacare is -- and isn't -- working for Californians and what other factors, besides health care access, contribute to good health.  

CONNECT WITH THE COMMUNITY

Member Activities

William Heisel has shared a blog post

Read it.

Lee Adcock has shared a story

Read it.

Minnur Yunusov's profile has been updated

Connect with Minnur Yunusov

Ruxandra Guidi has shared a microsite blog

Read it.

Anna Romano has shared a blog post

Read it.
More Member Activities

Follow Us

ReportingHealth

Reporting on Underserved Communities



How can journalists and foundations collaborate to deepen and improve reporting on underserved communities? Our USC Annenberg School of Journalism program in collaboration with The California Endowment, the Wyncote Foundation and Media Impact Funders, convened 75 leaders from both fields.

Read More »