Peer review is the time-honored way that researchers establish the credibility of their work. Originally that meant well-known researchers would endorse or at least "sign off on" the published works of their peers. Today there are so many credible researchers in such disparate fields of endeavor that it has become increasingly impossible for anyone to know them all or understand their work. In reaction, credibility is currently established by a work’s acceptance for publication or presentation at professional conferences or in reputable journals. The process is lengthy and cumbersome and can be delayed for months as authors wait for reviews or for conference dates to arrive. Additionally, works are often reviewed during a limited pre-publication period by a very small number of individuals in a secretive fashion that promotes hasty and sometimes political decisions. These are extraordinarily talented people who believe they are doing their best to be objective, but the process can often fail.
In a highly publicized, recent case, Hwang et al published fraudulent research on pluripotent stem cells in Science, which was later retracted. For other less publicized cases consult Richard Smith’s book “The Trouble with Medical Journals.” As the former editor of the British Medical Journal, he is in a unique position to report numerous cases of misbehavior by editors, reviewers and authors. A very interesting mathematical simulation of peer review by Neff and Olden, “Is Peer Review a Game of Chance”(pdf) shows that the review process can include a strong “lottery” component, independent of editor and referee integrity and recapitulates the fact that a stringent peer review process will keep out much of the bad science, but also a large amount of the good science. Obviously, a more transparent and open-ended peer review process is needed and that is where the “Social Web” can help.
As it becomes increasingly more practical via the web to evaluate researchers and their work based on the reviews of their actual peers, the need for endorsement by centralized big-name journals begins to diminish. Always reacting to web trends - in this case the rise of "Social Media" - Google has facilitated the launch of a new project called Google Peer Review. I say "facilitated" because Google's only involvement in the open-source endeavor thus far has been to provide hosting services for the project. Google Peer Review was started by a young programmer, Mike Gasher, and has been joined by several other interested individuals. The intention is to do for scientific publishing what the world wide web has done for media publishing by allowing “vetted” reviewers write a review of someone's work and digitally sign them together.
There is a lively discussion underway here and a wiki has also been established. You can learn how to join the project here.
No comments:
Post a Comment