In Peer-Review we trust? Do peer-review journals perpetuate bad science

Should the general public trust in what peer-reviewed articles published in scientific journals say? I recommend a Photo: author

HELSINKI, Finland, February 13, 2012–A recent look at the Danish Cohort and the Greek proteomics study brought to light the eternal problem of the quality of peer-review in scientific journals. Should we trust in peer-reviewed articles published in scientific journals or should we not?

As a journal editor who asked scientist to perform peer-review of manuscripts submitted for publication, and a scientist continuously conducting peer-reviews at the request of the various journals, I suggest caution in reading these articles.

There are no perfect studies. It is always possible to improve on findings and get more information. However, it does not mean that existing studies, even without improvements, are bad studies. They generally are adequate, good or very good.

Other studies fall into the “bad” category.  These are studies where the study design, or methods of execution are scientifically incorrect. This makes the results and the conclusions misleading or simply wrong. There are different grades of bad studies, so some studies are worse than others

An example of a very bad study is the Danish Cohort. Because of serious flaws and errors in the design, it produced false results. Peer-review should have rejected this study because no improvements could make it better.

An example of a study that needs serious improvement is the Greek proteomics study. Part of the findings is correct but part is not, which leads to false, or at least highly overstated, conclusions. This study should have been returned to authors either for shortening and rewriting the conclusions or to conduct additional experiments.

Unfortunately, following the peer-review by journals’ editors and appointed reviewers, both studies were published. By publication in peer-review journals the results of both studies were unjustly “ennobled” to the category of credible scientific evidence.

This, in turn, led to false headlines and misleading information to the general public and to decision makers.

Readers may wonder how it is possible that poor quality manuscripts were published and become valid peer-reviewed scientific evidence.

The process of publication is, briefly and in simplified way, as follows:

  • Scientists submit a manuscript;
  • Journal editor reviews it and sends it out to at least 2 scientists for review and comments;
  • Upon receiving comments from reviewers, and using his own judgment, the editor decides whether the manuscript is rejected, returned for corrections or accepted for publication.

The general rule is that editor and reviewers should be impartial and base their comments and recommendations solely on the science. But editors and reviewers are only humans and can make mistakes.

That potential conflict of interest is an important issue.

However, the confidentiality of the peer-review hampers determination of its impact on the outcome of peer-review process.

The journal editor wields enormous power. He chooses reviewers and evaluates the reviews. The selection of reviewers may predetermine the fate of the article. If the editor is “sympathetic” and selects “sympathetic” reviewers, then the reviews might be better than when reviewers are “not sympathetic” to the topic or the authors of the study.

The whole process of review and publication of scientific manuscripts relies heavily on the integrity and commitment of the editors and reviewers to fairly evaluate the science and the science alone.

It is a great responsibility that lies on their shoulders. Unfortunately, too often reviewers are overworked and the reviews are done hastily, leading to publication of studies with poor science. Let’s not forget that job of reviewers is unpaid pro bono work for the scientific community.

Another important issue is that any study, no matter how poor scientifically, will be published because there will always be some journal that will accept and publish it. This is a sad reality.

Once published, studies automatically become valid references that will be used for years to come to support certain scientific views. Over time, the detailed content of the studies will become blurred but the conclusions will live on in the various reviews and reports.

In case of the studies concerning cell phone radiation and health, the so called “activists” and “anti-activists” alike, blinded by the target, entrenched in their positions, unable to admit that some of their opinions might be not correct, seem to accept any shred of evidence that supports their view as valid. No matter how bad the science behind it is.

They both are hiding behind the term “study published in peer-review journal” and use it as ultimate proof that the study is correct because it was peer-reviewed.

The general public wonders what is going on and whom to trust.

Such attitudes as this presented by “activists” and “anti-activists” is wrong. It misleads the general public and the decision makers alike. It is bad for science. Good scientific evidence is lost among the poor science or sacrificed for the short term gains.

Take home message: Limited trust is the best option. Ask an acknowledged scientist/expert for an opinion. Ask scientists who have pro and contra opinions. This is of paramount importance when writing a story for the news media. Do not take everything that authors say for granted.

Not everything that glitters is gold.

Read more from Dariusz Leszczynski in his science blog “BRHP - Between a Rock and a Hard Place” at  Dariusz is a Research Professor at the Radiation and Nuclear Safety Authority in Finland.

Follow Dariusz on twitter: @blogBRHP

Disclaimer: the opinions presented in this column are author’s own and should NOT be considered as the official opinions of the STUK - Radiation and Nuclear Safety Authority in Finland.

This article is the copyrighted property of the writer and Communities @ Written permission must be obtained before reprint in online or print media. REPRINTING TWTC CONTENT WITHOUT PERMISSION AND/OR PAYMENT IS THEFT AND PUNISHABLE BY LAW.

More from Between a Rock and a Hard Place
blog comments powered by Disqus
Dariusz Leszczynski

Dariusz Leszczynski is an expert in the biological and health effects of cell phone radiation.

Since 2009 he publishes a science blog dealing with the issue of cell phone radiation and health:  

Disclaimer: the opinions presented in this column are authors' own and SHOULD NOT be considered as opinions of any of his employers.

Contact Dariusz Leszczynski


Please enable pop-ups to use this feature, don't worry you can always turn them off later.

Question of the Day
Photo Galleries
Popular Threads
Powered by Disqus