| |
|
Research: beware of hype!
Wednesday, 5
 |
 |
| |
Editors of high profile journals need to show
more care in their presentation of papers to the media according to the
research.
|
The reporting of medical and other research is highly flawed
according to several papers in this week's edition of an American journal.
The main problems highlighted in the special peer review edition of the
Journal of the
American Medical Association (JAMA) are that research presentations
at scientific meetings often receive an unwarranted amount of media
coverage, media releases can exaggerate the importance of findings while
failing to include the actual study results, and the views presented in
research papers often do not represent the opinions of all of the scientists
who contributed to the work.
Many of the papers suggest that journal editors need to take a stronger role
in addressing the issues.
"The current press coverage of scientific meetings may be characterized as
'too much, too soon'," writes Dr Lisa Schwartz from the Department of
Veterans Affairs Medical Center, Vermont, of her work looking at scientific
meetings. "Results are frequently presented to the public as scientifically
sound evidence rather than as preliminary findings with still uncertain
validity."
Such meetings are intended to provide a forum for researchers to present new
work to colleagues, work which may have undergone only limited peer review.
Often the presentations represent a study in progress with only a brief
abstract to outline what is happening.
Dr Schwartz' study looked at the media coverage of five high profile
scientific medical meetings in 1998. With colleagues from Dartmouth Medical
School, New Hampshire, she found 252 news stories reporting on 147 research
presentations, an average of 50 news stories per meeting, despite the often
preliminary nature of the work. Nine or more stories about research
presentations appeared in each of the nation's five highest-circulation
newspapers.
The team followed the work presented to these five medical meetings over the
next three years to discover which research was subsequently published. Only
50 per cent of the 147 abstracts were published in high-impact journals, 25
per cent were published in low-impact journals, and 25 per cent remained
unpublished. The 39 publications that received front-page coverage at the
time also followed this pattern, indicating that coverage by the media does
not discriminate between quality and average research.
"Unfortunately, many projects fail to live up to their early promise; in
some cases fatal flaws emerge", the authors concluded. "Press coverage at
this early stage may leave the public with the false impression that the
data are in fact mature, the methods valid, and the findings widely
accepted. As a consequence, patients may experience undue hope or anxiety or
may seek unproven, useless or even dangerous tests and treatments".
Another paper by Lisa Schwartz (with others) looked at the press releases
from high profile medical journals. It reviewed recent releases to evaluate
how study findings are presented and whether limitations and potential
conflicts of interest are acknowledged.
They found that medical journals do not extend the effort of ensuring
accuracy in papers to their own press releases - the most direct way that
journals communicate with the media. Not all journals issue press releases,
but of those that do, the press office selects articles based on perceived
newsworthiness. The releases are written by press officers trained in
communications rather medicine.
Journals have general guidelines for parameters like length, but no
standards for acknowledging limitations of the research or for data
presentation. Dr Schwartz and colleagues found that less than a quarter of
the releases recognised study limitations. Of the approximately half that
reported differences between study groups only 55 per cent provided the
corresponding base rate, the format least prone to exaggeration. Industry
funding was noted in only 22 per cent of studies receiving funding.
"As a direct means of communication between medical journals and the media,
press releases provide an opportunity for journals to influence how the
research is translated into news," the authors write. "The most direct way
to improve the quality of journal press releases lies in enhanced editorial
oversight of the process."
A third article in the same issue of JAMA found that research papers
rarely represent the full range of opinions of all those scientists whose
work it claims to report. This study was the work of Mr Richard Horton, the
editor of The Lancet
who examined papers that had been published in the journal in the year 2000.
He found that important weaknesses in the science that had not been included
in the published article were often admitted on direct questioning. "I have
found evidence of censored criticism; obscured views about the meaning of
research findings; incomplete, confused, and sometimes biased assessment of
the implications of a study; and frequent failure to indicate directions for
future research," he wrote.
"Editors might also explore ways to recover the plurality of contributors'
opinions."
ABC Science Online
More Info...
Many
peer-reviewed asthma studies are flawed - News in Science 28/2/2000
Evidence
based medicine a mystery to GPs – News in Science 24/4/2002
Peer review
'biased, unjust and …wrong' – News in Science 23/2/2002
 Print-friendly
version
|
|
|
