5th World Conference of Science Journalists

 A peer review of peer review [85:35m]: Download

PRODUCER: Julie Egan

CHAIR: Jim Handman

SPEAKERS: Phil Campbell, Warwick Anderson, John Rennie

PANELLIST: Carol Nader

SESSION REPORT: Ask, then ask again

By Robert Frederick

“Who here, by show of hands, does not trust peer review?” John Rennie, editor in chief of Scientific American, asked that question of a hundred delegates in this session. No one raised a hand, but more than a few hands fluttered in uncertainty.

Peer review is the process though which editors screen papers and granting agencies select projects. Typically, this process involves getting several experts to evaluate the author’s work or proposal. But the process relies on trust and the author’s honesty, and many publications have been hoodwinked.

“Peer review definitely enhances the reliability of what appears, at least in the journals that I’m familiar with,” said Philip Campbell, Nature’s editor in chief. But “anything you see [in any journal]… is provisional.”

Warwick Anderson, former research scientists and current CEO of Australia’s National Health and Medical Research Council, echoed that sentiment. “I think [peer review] just shows all the normal failings of something that involves human beings.”

While extolled as the gold standard, speakers at the A peer review of peer review session also lambasted peer review as a gauntlet for researchers, a burden for scientists, and a crutch for journalists. But whether an article had been peer reviewed before publication, each speaker said true peer review begins with publication. That’s when other scientists repeat the experiment or attempt to build on the result.

Speaking after the session, delegate and AAAS’s Senior Communications Officer Earl Lane said journalists should view journal articles and his organization’s EurekAlert! service as “A starting point for a story. I don’t think we would portray it as anything more than that.”

In summarizing advice to journalists, panelist Jim Handman, CBC’s senior producer of Quirks and Quarks, said even after a work has been peer reviewed, journalists should keep asking that all-important question, ‘How do they know that?’