Talk:Peer review failure

Questionable para from fraud section =

(William M. Connolley 08:37, 21 Jun 2004 (UTC)) I've moved this here for discussion:

While fraud obviously has severe negative consequences for the author of a paper, there are generally no adverse consequences to either the editor or the reviewers for recommending publication of a fraudulent paper, as detecting fraud is not a goal of the process, and the editors and reviewers almost never have enough information to detect outright fraud.

Who says that there are no consequences for editors (or reviewers - less sure). Quite likely, the editors get a black mark in their career record - what this text means probably is that nothing about that appears in the newspapers?

Hi William.

Fair point...when Nature withdrew the Schon papers, the editorial said

Nature this week finds itself in the unenviable (and unprecedented) position of formally retracting 
seven papers (see page 92).  All share the same first author, Jan Hendrik Schön (see Nature 419, 
417;  2002); in fact, this represents the entire body of work published by Schön in this journal.

which *does* look like severe negative consequences on the Journal, if not the Editor...nevertheless, the referees (being anonymous and voluntary, as much as anything) do escape censure....WMMV.

best

Robinh 10:23, 21 Jun 2004 (UTC)

Hi again. Another gem from Nature Editorial, 3 Oct 2002:

In some media reports, journalists and a few scientists who are unconnected with the
Schön investigations have taken the opportunity to make potentially damaging assertions
about journals, including Nature: that in order to compete or to publish exciting results,
journals will cut corners in peer review, overrule hostile reviewers or select sympathetic ones.

We at Nature unequivocally reject such charges. The publication history and files of these                                                                                                                                                                                         
particular papers and the editorial policies and interests of Nature are completely at odds with 
these assertions. Nature has nothing to gain by the pursuit of glamour at the expense 
of scientific quality, considering, not least, the criticisms, corrections and retractions we 
would then habitually be forced to publish.  There is more than enough rock-solid and splendid 
science to publish. Furthermore, it is a strict policy of Nature that our Letters and Articles are
selected for their outstanding scientific impact, sometimes also taking into account relevance to 
public policy issues, but never simply because the results will make headlines.

Robinh 13:16, 22 Jun 2004 (UTC)

Section on Peer review failures

Happy to see this new section started. Please do not unilaterally remove citation requests without providing citation or justifying and consensusing on the talk page. I've placed an Original Research tag on it and will follow up with requests for verification per WP:VER, since there will naturally be some disagreement about the section's content. Kenosis 15:21, 27 June 2006 (UTC)

The first paragraph of the section currently reads as follows: ... Kenosis 15:21, 27 June 2006 (UTC)

• A peer review failure occurs when a peer-reviewed article contains an obvious fundamental error(s) that undermines at least one of its main conclusions. Peer review is not considered a failure in cases of deliberate fraud by authors. Letters-to-the-editor that correct major errors in articles are a common indication of peer review failures. Few journals have an effective procedure to deal with peer review failures. Journal editors do not usually acknowledge peer review failures and rarely ask peer reviewers to reconsider an article in light of published criticisms. The author is allowed a published reply to a critical letter, but the reply is usually obfuscation rather than an admission of the article's flaws. Editors sometimes refuse to publish letters correcting badly flawed articles for apparently no reason other than to cover up peer review failures. Some refereed journals have a policy to never publish letters. Even when letters reveal peer review failures, they are often not effective in alerting readers because letters are not indexed along with the articles. Editors seldom publish retractions or corrections for flawed articles unless the authors request them. 15:21, 27 June 2006 (UTC)
• (1)The first part of this paragraph currently is a made-up definition of what constitutes a peer review failure. ... Kenosis 15:21, 27 June 2006 (UTC)
• (2)("Peer review is not considered a failure in cases of deliberate fraud by authors.") This sentence is speculative without supporting literature.
• (3)("Letters-to-the-editor that correct major errors in articles are a common indication of peer review failures.") This sentence is obvious and I support it.
• (4)("Few journals have an effective procedure to deal with peer review failures.") This sentence also is understood immediately and ought easily be agreed by consensus, but still could use examples and support.
• (5)(" Journal editors do not usually acknowledge peer review failures and rarely ask peer reviewers to reconsider an article in light of published criticisms.") This sentence is unsupported and accusatory. Please support it.
• (6)("The author is allowed a published reply to a critical letter, but the reply is usually obfuscation rather than an admission of the article's flaws. ") This sentence is a ridiculously accuasatory generalization without concrete support.
• (7)("Editors sometimes refuse to publish letters correcting badly flawed articles for apparently no reason other than to cover up peer review failures.") By this point in the paragraph it is obvious that the editor who wrote this has an obvious bias and an agenda here.
• (8)Several further generalizations and accusations follow, all of which require citation and support due to their accusatory and obviously controversial nature.

This section is important to the health of the article. Please let's get it right in a way that it will remain a stable contribution to the article's informativeness, usefulness and accuracy. ... Kenosis 15:21, 27 June 2006 (UTC)

• (1)The first part of this paragraph currently is a made-up definition of what constitutes a peer review failure. ... Kenosis 15:21, 27 June 2006 (UTC)

• This is based on the "Reasons for peer review" section which says the purpose of peer review is to spot mistakes or flaws. Doesn't it logically follow that if obvious, major flaws are not identified by peer review, then it has failed? Plantguy 17:46, 27 June 2006 (UTC)

• (2)("Peer review is not considered a failure in cases of deliberate fraud by authors.") This sentence is speculative without supporting literature.

• This sentence follows from the section on Peer review and fraud which states "the process is not designed to detect fraud." That is a widely held belief. Plantguy 17:46, 27 June 2006 (UTC)

• (3)("Letters-to-the-editor that correct major errors in articles are a common indication of peer review failures.") This sentence is obvious and I support it.

• (4)("Few journals have an effective procedure to deal with peer review failures.") This sentence also is understood immediately and ought easily be agreed by consensus, but still could use examples and support.

• What kind of evidence do you want here? I agree much of this has not been studied but mainly because the peer review system was not set up to deal with peer review failures, yet they happen. I have never seen a peer-reviewed journal with a policy on how to deal with peer review failures. I have a lot of personal experiences where editors refused to correct major factual errors in referreed articles without explanation or with the excuse that there is no room. Plantguy 17:46, 27 June 2006 (UTC)

• (5)(" Journal editors do not usually acknowledge peer review failures and rarely ask peer reviewers to reconsider an article in light of published criticisms.") This sentence is unsupported and accusatory. Please support it.

• I have never seen a journal editor apologize in print for an article that contained obvious, major errors (not fraud) or announce that an article had been reexamined by reviewers and found to be lacking so is being retracted. How can you cite a negative? Plantguy 17:46, 27 June 2006 (UTC)

• (6)("The author is allowed a published reply to a critical letter, but the reply is usually obfuscation rather than an admission of the article's flaws. ") This sentence is a ridiculously accusatory generalization without concrete support.

• What kind of support do you want here? I can prepare a list of published examples where an author replied to a letter with obfuscation. One excellent example is Temple, S.A. 1979. The dodo and the tambalacoque tree. Science 203: 1364. which is a reply to the letter pointing out flaws in the dodo article by Owadally, A.W. 1979. The dodo and the tambalacoque tree. Science 203: 1363-1364. Plantguy 17:46, 27 June 2006 (UTC)

• (7)("Editors sometimes refuse to publish letters correcting badly flawed articles for apparently no reason other than to cover up peer review failures.") By this point in the paragraph it is obvious that the editor who wrote this has an obvious bias and an agenda here.

• This is supported by later citations. The editor of Bioscene refused to publish a letter or consider an article pointing out factual errors in a 1993 article. Those corrections were published elsewhere. I can provide additional examples of this type. Plantguy 17:46, 27 June 2006 (UTC)

• (8)Several further generalizations and accusations follow, all of which require citation and support due to their accusatory and obviously controversial nature.

• I provided several footnotes for the second paragraph. What more do you think is needed? This section addresses an obviously controversial topic because journals do not want to admit that peer review sometimes fails for other than author fraud, often spectacularly so. In the book, The Double Helix, it was mentioned that a former Nobel Prize winner, Linus Pauling, published an article on a possible structure for DNA, but Watson immediately recognized that it violated the basic laws of chemistry. Another example of a spectacular peer review failure. Plantguy 17:46, 27 June 2006 (UTC)

Citations are duly noted for the examples in the second paragraph – good citations at that. Problem is, the entire first paragraph is arbitrary and judgmental as a summary paragraph for the fine examples that follow in the section. The most conspicuous instances of thrown-in judgments in the current first paragraph are:

• "... the reply is usually obfuscation rather than an admission of the article's flaws",
• "... and rarely ask peer reviewers to reconsider an article in light of published criticisms."
• "... for apparently no reason other than to cover up peer review failures."

These are only the most conspicuous. By my reading of it, the paragraph currently sounds like a diatribe. Surely this can written so as not to serve as concrete proof of certain WP editors' disdain for the pre-publication peer-review process. ... Kenosis 22:18, 27 June 2006 (UTC)

• I rewrote a lot of it and tried t

peer review failures

I think the examples in this section are unsupported and do not belong. Some of them are errors, not failures; some are disputed cases. Classifying them this way is OR. What is necessary to support them is references to sources that explicitly discuss them as failures of peer review. This is a matter of opinion, not of 2+2=4, and we cannot use our own unsupported views on these. Additionally, these are unrepresentative examples--there are some more drastic ones not discussed. The solution as far as this article is concerned is to branch these to a separate article so we can at least keep this article free of detailed controversy. I intend to do that split, subject to discussion here. How to handle the new article? Again, by making it a summary--the major cases deserve articles of their own. Some probably have them, and perhaps we can look for them and use a category. But thats for discussion on the new talk p. OK? DGG 06:13, 19 June 2007 (UTC)

Semantics

It appears the controversy over peer review may be a bit semantical. Peer review is a philosophy that when executed correctly, is an excellent way to conduct scientific publishing. All the controversy appears to be when the system for implementing it messes up (biased reviewers, bad reviewers, etc.).

In that case, a "peer review" is not really obtained.

I point this out as a matter of discussion about perhaps rephrasing the sections on discussing the controversies around the subject. That is, peer review itself, as a philosophy, seems not to be the controversy. It is the way in which it is implemented. I would propose editing the "Problems with Peer Review" to be "Problems with Implementation of Peer Review and in Peer Review Itself." And then editing the text to reflect the title.

Deletion of section - let's discuss this

Earlier today I added the following paragraph to the section on Criticisms of Peer Review that has been deleted twice:

Conspicious examples of censorship in the realm of peer review are articles with potential religious connotations. Deviations from mainstream scientific thought that can be related to religious viewpoints can be quickly censored by the "elites". Even if an article passes the peer review process, its publishing can have disastrous effects for the publisher. In August of 2005, Richard Sternberg, a research associate with the Smithsonian Institution, was fired after allowing an article to be published entitled "The Origin of Biological Information and the Higher Taxonomic Categories" that cast doubt on Darwinism. Senior scientists at the Smithsonian Institution called Sternberg a “shoddy scientist” and a “closet Bible thumper,” despite his protests that he is agnostic about Intelligent Design and does not hold any sort of creationist viewpoint.

In the edit comment box I asked for help finding the actual citations (the Washington Post, Washington Times, and the UK's Independent) so that we wouldn't have to have the secondary source of the AiG article. The AiG article is a secondary source that summarized the Post and Times and Independent articles with commentary - I asked for help finding the primary sources. Please explain why we should not include this paragraph with reliable sources. standonbible 18:32, 15 November 2006 (UTC)

Evaluating peer review through fundamentalist Christian lenses of AiG is inappropriate. AiG is only a reliable source for what they believe -- they do not belong on this page nor does any analysis that follows their points belong on this page. --ScienceApologist 18:36, 15 November 2006 (UTC)

Um ... this section is discussing criticisms of the peer review process and it specifically mentions elites suppressing dissent against mainstream views. An example of this would be good - and Sternberg is perfect for this. I was careful not to include any "fundamentalist Christian" spin on it but reported the facts - facts that should be directly sourced by everyone, not suppressed just because AiG happened to report on it. Would you rather me replace the AiG ref with a {{fact}} tag? Otherwise the AiG article should serve as a guide for editors to find sources until all the sources can be found. standonbible 18:42, 15 November 2006 (UTC)

You may think you were careful, but in fact your bias shines through. First of all, firing a person has nothing to do with peer review. Second of all, claiming religious bias is a specious claim favored by AiG and backed up by zero evidence. Scientists of many different religions have no problem publishing, the only people who have problems publishing are those that contravene the scientific method and inject their religion into their papers. That's an issue for the theistic realism and philosophical naturalism pages, it doesn't belong here. In short, the entire paragraph is a red herring and represents creationist POV-creep -- it frankly doesn't belong here. --ScienceApologist 18:45, 15 November 2006 (UTC)

You said, "firing a person has nothing to do with peer review". Ahem - he was fired for allowing the article to be peer-reviewed. And I never claimed bias against religion - that is something you read into it. Don't talk about "contravening the scientific method and injecting religion into their paper - Sternberg subscribes to neither ID nor Creationism. How can the article be a red herring if the very last paragraph talks about the "elites" squelching non-mainstream research - don't you think there should be an example of this?

How's this:

Conspicious examples of censorship in the realm of peer review are articles that deviate from mainstream thought and may have religious connotations. Publishers are hesitant to pass such articles due to opposition from the mainstream; in August of 2005, Richard Sternberg, a research associate with the Smithsonian Institution, was fired after allowing an article to be published entitled "The Origin of Biological Information and the Higher Taxonomic Categories" that cast doubt on Darwinism. Senior scientists at the Smithsonian Institution immediately called Sternberg a “shoddy scientist” and a “closet Bible thumper,” despite his protests that he is agnostic about Intelligent Design and does not hold any sort of creationist viewpoint.

I removed any hint of so-called "religious bias" and even re-referenced it. Now there is no mention of AiG. Does that satisfy you? How is that "creationist POV-creep" (to use your rather blunt language)? standonbible 20:31, 15 November 2006 (UTC)

Oh, and here's what that looks like with nowiki:

Conspicious examples of [[censorship]] in the realm of peer review are articles that deviate from mainstream thought and may have [[religion|religious]] connotations. Publishers are hesitant to pass such articles due to opposition from the mainstream; in [[August]] of [[2005]], [[Richard Sternberg]], a research associate with the [[Smithsonian Institution]], was fired after allowing an article to be published entitled "The Origin of Biological Information and the Higher Taxonomic Categories" that cast doubt on [[Evolution|Darwinism]]. Senior scientists at the Smithsonian Institution immediately called Sternberg a “shoddy scientist” and a “closet Bible thumper,”<ref>Michael Powell; The Washington Post; Aug 19, 2005; A.19 [[http://pqasb.pqarchiver.com/washingtonpost/access/884160621.html?dids=884160621:884160621&FMT=ABS&FMTS=ABS:FT&fmac=&date=Aug+19%2 :C+2005&author=Michael+Powell&desc=Editor+Explains+Reasons+for+%27Intelligent+Design%27+Article]]</ref> despite his protests that he is agnostic about [[Intelligent Design]] and does not hold any sort of [[creationism|creationist]] viewpoint.<ref>David Usborne; New York; The Independent (UK); Aug 20, 2005 [[http://news.independent.co.uk/world/americas/article307079.ece]]</ref>

Thanks! standonbible 20:49, 15 November 2006 (UTC)

I tried posting what I have above and was rv'd by JoshuaZ. He said it wasn't a notable enough example. Did you even read the sources? This involved the Smithsonian Institution and went as high as President Bush - what would be notable enough? We need an example of what is being talked about - explain how that is POV. standonbible^Talk! 21:18, 15 November 2006 (UTC)

Where does it say it went as high as Dubya? Nowhere. &#0149;Jim62sch&#0149; 21:32, 15 November 2006 (UTC)

Actually, yes-where. Take a look at [[1]]. standonbible^Talk! 22:07, 15 November 2006 (UTC)

In any case the NYT had this to say about it:

August 20, 2005

Intelligent Design and the Smithsonian

The Smithsonian Institution can't seem to disentangle itself from the clutches of the anti-evolution crowd. Earlier this year, the Smithsonian's natural history museum discovered to its dismay that it had agreed to be the host and co-sponsor of a movie intended to undercut the theory of evolution and make the case for intelligent design, the idea that an intelligent agent had a hand in designing the universe. Only after intelligent-design proponents started chortling on the Internet about their stunning coup in co-opting the Smithsonian did museum officials reverse course and withdraw their sponsorship, while allowing the film to be shown.

Now comes word that a little-known government office has accused the Smithsonian of retaliating against a scientist who slipped an article promoting intelligent design into an obscure journal that has only very loose connections to the Smithsonian. That judgment, by the United States Office of Special Counsel, a federal agency set up to protect whistle-blowers, is the latest twist in a case that started with the publication of the article last year in the Proceedings of the Biological Society of Washington.

The article contended that evolution theory could not account for the great proliferation of life forms during the so-called Cambrian explosion some 530 million years ago, and that an intelligent agent was the best explanation. It set off an uproar among evolutionary biologists and was later disowned by the professional society that published it.

The editor who authorized publication, Richard Sternberg, filed a complaint contending that he had suffered reprisals. In an 11-page letter not yet officially released, the Office of Special Counsel said it had found support for his complaint but was dropping the investigation because he was not an employee of the Smithsonian, just a research associate.

E-mail notes show that several scientists and managers at the Smithsonian were extremely embarrassed and eager to push Mr. Sternberg out of his research niche, and that some dug around for material to discredit him. That may lead critics of evolution to see Mr. Sternberg as a martyr.

But those who see no place for intelligent design in the realm of science -- and that includes us -- will ruefully give him credit for maneuvering a brief for intelligent design into a peer-reviewed scientific journal, although how rigorous that review was remains a point of contention.

[[2]]

Get the point dude? &#0149;Jim62sch&#0149; 21:32, 15 November 2006 (UTC)

If that isn't a POV article, I don't know what is. But fine - put the mainstream rebuttal in there if you'd like - maybe "Other scientists doubt Mr. Sternberg's claims that he is not a proponent of ID <ref>(nytimes link)</ref>". But the facts (verifiable, etc., etc.) still say that a research associate allowed a minority viewpoint through peer review, was fired and called a "shoddy scientist" and "closet Bible-thumper", and protested that he was not a proponent of ID. As such it is a perfect example of what this section is discussing; NYT's incredibly biased and pejorative article hardly changes that. Here's another revised version that might possibly fit your personal censorship standards:

A conspicious example of such censorship of iconoclastic research took place in August of 2005 at the Smithsonian Institution. Richard Sternberg, a research associate for the Smithsonian, was fired after allowing an article to be peer-reviewed and published entitled "The Origin of Biological Information and the Higher Taxonomic Categories". The article was controversial in that it cast doubt on the scientific theory of Evolution. Senior scientists at the Smithsonian Institution immediately called Sternberg a “shoddy scientist” and a “closet Bible thumper,” despite his protests that he is agnostic about Intelligent Design and does not hold any sort of unorthodox creationist viewpoint. Other scientists, however, see Sternberg's intentions as suspect.

Supporters of the peer-review process have pointed out ...

With nowiki:

A conspicious example of such censorship of iconoclastic research took place in [[August]] of [[2005]] at the [[Smithsonian Institution]]. [[Richard Sternberg]], a research associate for the Smithsonian, was fired after allowing an article to be peer-reviewed and published entitled "The Origin of Biological Information and the Higher Taxonomic Categories". The article was controversial in that it cast doubt on the scientific theory of [[Evolution]]. Senior scientists at the Smithsonian Institution immediately called Sternberg a “shoddy scientist” and a “closet Bible thumper,”<ref>Michael Powell; The Washington Post; Aug 19, 2005; A.19 [[http://pqasb.pqarchiver.com/washingtonpost/access/884160621.html?dids=884160621:884160621&FMT=ABS&FMTS=ABS:FT&fmac=&date=Aug+19%2 :::C+2005&author=Michael+Powell&desc=Editor+Explains+Reasons+for+%27Intelligent+Design%27+Article]]</ref> despite his protests that he is agnostic about [[Intelligent Design]] and does not hold any sort of unorthodox [[creationism|creationist]] viewpoint.<ref>David Usborne; New York; The Independent (UK); Aug 20, 2005 [[http://news.independent.co.uk/world/americas/article307079.ece]]</ref> Other scientists, however, see Sternberg's intentions as suspect.<ref>New York Times; Intelligent Design and the Smithsonian; August 20, 2005 [[http://select.nytimes.com/gst/abstract.html?res=FB0A13FA3B5A0C738EDDA10894DD404482&showabstract=1]]</ref> :::Supporters of the peer-review process have pointed out ...

There. I even called darwinism "the scientific theory of evolution" so there could be no doubt about the matter. Feel free to change the last sentence (other scientists, however, see Sternberg...) as much as you would like if you want it to be more accurate. What more could you guys ask for? standonbible^Talk! 22:07, 15 November 2006 (UTC)

Heck, I could even say "that questioned key assumptions regarding the Pre-Cambrian period" if you want to make it as unbiased as possible! standonbible^Talk! 22:38, 15 November 2006 (UTC)

This is an AiG criticism and as such does not belong in this article. Standonbible wouldn't have this line of reasoning if not for AiG, therefore it qualifies as AiG propaganda and shall be excluded from the article. --ScienceApologist 23:36, 15 November 2006 (UTC)

So, Mr. Apologist, if AiG reports that the moon-dust argument (if the moon really was billions of years old it would have 6 feet of dust on it) is based on faulty assumptions and should not be used, would that validate the moon-dust argument? Your overt bias against any inconoclastic viewpoints is fatiguing. If I had found the references first, somehow I don't think you would have objected so strongly. I would appreciate it if you would refrain from so quickly attacking anything that might possibly cast doubt on your personal bias. Your syllogism lacks validity and I would like to know what about the proposed addition violates what WP policy. standonbible^Talk! 00:13, 16 November 2006 (UTC)

To possibly clarify and expand on what SA was trying to say- there is an issue of good faith and POV pushing. The fact is that you are associated with AiG and prefer to put their material in many articles. The use of the new sources was therefore a step in the right direction. However, the fundamental problem of whether this is a notable example of a problem with peer review has not been addressed. Unlike some other problems like the Sokal incident or the incident with the French brothers (what was their name again?) which received widespread coverage the Sternberg matter received almost no coverage outside the usual venues where creationism/ID are discussed. JoshuaZ 00:17, 16 November 2006 (UTC)

Thanks, Joshua. I am associated with AiG in that I have several friends who work for AiG. However I don't exactly "prefer to put their material" in articles. I was reading the peer review article for personal reasons and thought an example of the "elites" censoring an unorthodox viewpoint would be in good taste - hence the use of the AiG article on Sternberg. I don't, as you seem to think, go poking around looking for a place to insert AiG POV. Now, since that is out of the way....

I agree that the fundamental problem is whether this is a notable example. According to WP:N, a subject must have been described by multiple independent reliable sources in order to be notable. The Sternberg incident has been documented and commented upon by the New York Times, the Washington Post, the Opinion-Journal, WorldNetDaily, the Discovery Institute, NPR, the Washington Times, the World Socialist Web Site, PhysOrg.com, and the National Review (just try Googling it). Somehow I think that qualifies as "multiple independent reliable sources". Why not? standonbible^Talk! 00:55, 16 November 2006 (UTC)

WP:N describes in general what is notable enough to have an article about a topic. Hence we have an article on the Sternberg controversy. That doesn't make it notable enough to be mentioned here. JoshuaZ 01:03, 16 November 2006 (UTC)

Could you tell me what the guidelines for notability in this case would be? standonbible^Talk! 01:11, 16 November 2006 (UTC)

If you insist, I suppose we could just put "for an example of this, see the Sternberg controversy" in the paragraph right before the proposed change. I think it is more specific this way, but.... standonbible^Talk! 01:11, 16 November 2006 (UTC)

I don't know if there is a specific guideline in regard to this matter but the undue weight section of WP:NPOV may be relevant. The other issue that was brought up above is whether this is even relevant to the discussion. The Sternberg case was problematic because it avoided the standard review process. Given that, it isn't clear to me why it should be included here. JoshuaZ 01:14, 16 November 2006 (UTC)

Undue weight really deals with a minority viewpoint of an issue - not really applicable here. As far as relevancy is concerned: given the circumstances it is understandable that there are alternate views on how the review process went here. But there isn't any reason to argue about that. The fact is that this is the clearest example I have seen of the powers-that-be censoring or squelching some portion of the review process, so unless a better example can be given and cited this example really ought to be included. Otherwise the preceding paragraph is a bunch of theoretical blah-blah without much meat. standonbible^Talk! 04:54, 16 November 2006 (UTC)

I mentioned above two far more prominent examples of failure of the peer review system. Including either or both would make sense prior to this one (although I wouldn't object to all three being mentioned). JoshuaZ 04:57, 16 November 2006 (UTC)

Mentioning all three sounds good to me! I don't know much about the other two examples, though, so can I put the Sternberg example in until the other two can be added by editors more knowledgeable on the subject? Whatever you think. standonbible^Talk! 05:07, 16 November 2006 (UTC)

The Sternberg case is not an example of censorship in peer review; that it is a particular viewpoint, Sternberg's and the pro-ID/creationist one, and a viewpoint you've been promoting across the science/evolution articles of the project. There's no recognition within the scientific community or even the popular press (barring a few op-ed puff pieces from ID fellow travelers) that the Sternberg case is anything more than what the publisher says and the evidence indicates -- that Sternberg stepped outside of normal process to give something his ID cronies have desperately sought. We won't be using the Sternberg case as an example of peer review censorship here. In fact, censorship in peer review is not a common complaint at all. FeloniousMonk 05:08, 16 November 2006 (UTC)

So are the New York Times, the Washington Post, the Opinion-Journal, WorldNetDaily, the Discovery Institute, NPR, the Washington Times, the World Socialist Web Site, PhysOrg.com, and the National Review all "ID fellow travelers"? There is no hint of an ID or creationist viewpoint in the proposed edit. Why do you disagree with JoshuaZ?

I've explained why: because as the Sternberg peer review controversy article explains, it is only Sternberg's and other creationist's opinion that his imbroglio is an example of censorship in peer review. In the scientific community, there's no acceptance of that viewpoint, which they rightly point out is in line with the ID crowd's (of which Sternberg is a notable member) agenda. I'm very, very familiar with the articles and sources, and the Washington Post article, the WSJ's Opinion-Journal article, WorldNetDaily, the Discovery Institute (!! as if...) and the National Review are all ID fellow travelers. If you are unaware that the Discovery Institute is the primary promoter of ID, and that Sternberg is affiliated with it, and that its fellows have written many of the very articles you cite, then you probably shouldn't be making objections or alterations to this article's content. FeloniousMonk 05:33, 16 November 2006 (UTC)

My mistake with mentioning the Discovery Institute; I Googled it and somehow mentally confused the Discovery Institute with a branch of PBS - don't know why.

Of course the Sternberg peer review controversy was controversial! But it doesn't matter if/that what happened actually qualifies as peer review censorship - it is still a good example of peer review controversy that should go into the "criticisms of peer review" section. You can't explain that away. A controversy for a section discussing controversy. standonbible^Talk! 05:43, 16 November 2006 (UTC)

Not really... One creationist abusing his position as editor to help out his buddies does not speak to any greater issues within peer review itself. It doesn't really belong here, it's more appropriate to the ID articles. FeloniousMonk 05:48, 16 November 2006 (UTC)

One creationist research associate with ID leanings being fired for allowing a controversial article through and for exercising his authority in a controversial manner is relevant to criticisms of peer review which say that the "elite" can easily squelch minority viewpoints. I suggest we wait until JoshuaZ gets back to see what he says. standonbible^Talk! 05:53, 16 November 2006 (UTC)

Clearly you're not up to speed on the Sternberg matter: 1) He was not a "research associate" at the journal, he was an editor, 2) he was not fired from the journal, was already leaving when published Meyers paper as his one of his last acts. 3) The Smithsonian, where Sternberg was an unpaid research associate did not fire him, he quit. If you can't get the basic facts in this case right, why should your proposal carry more weight than simply being your personal opinion. FeloniousMonk 15:59, 16 November 2006 (UTC)

On the subject the journal itself said: "The Council, which includes officers, elected councilors, and past presidents, and the associate editors would have deemed the paper inappropriate for the pages of the Proceedings because the subject matter represents such a significant departure from the nearly purely systematic content for which this journal has been known throughout its 122-year history." So they disagreed not only on the grounds that he was a poor choice to evaluate the article (because he is affiliated with the DI) but because it was simply inappropriate material. This really isn't a controversy over peer review, though people have tried to inflate it to that. Had he done the right thing and put it through normal procedures it would never have become an issue in the first place. --Davril2020 12:02, 16 November 2006 (UTC)

I don't understand something. You just said that the Council rejected the paper because it was "inappropriate" - it was departing from systematic Darwinistic thought. So therefore it had nothing to do with peer review? The point that is being made in this section is that sometimes the elites may, through their power over the system, squelch unorthodox viewpoints. When something like this was published, the "elites" did exactly that by firing the publisher - and you yourself admitted that it wasn't just his method of getting it through but the contents of the paper itself. Elites-use-power-to-squelch-inappropriate-content.

Until someone can write up a good summary of the two "more prominent examples" of elites abusing the peer review system to censor "inappropriate" content, this remains the best example of that. Perhaps an RfC would be good here so that we can get some outside viewers on this debate - or we should just wait until JoshuaZ gets back. standonbible^Talk! 13:18, 16 November 2006 (UTC)

I can only assume that you are unfamiliar with the journal's response. The comment on it being inappropriate had nothing to do with the article backing of ID, but the fact that the journal simply did not publish articles of that form or type within the journal. The journal was notable for its 'systematic content' - this was its specialism and virtually all it published. The article in question was a literature review. There were purely technical reasons as well as ethical reasons for being opposed to this. Frankly, had it been a literature review supporting a particular mechanism of evolution it would probably have failed peer review as well. --Davril2020 15:52, 16 November 2006 (UTC)

You've got a good point, Davril. That was information I was unaware of. Of course, the comments made by the other scientists calling Sternberg a "closet Bible thumper" would lead me to believe that their rejection of the article and firing of Sternberg wasn't purely for structural reasons concerning the paper ... but I think that in any case this argument will not be much more productive. It would probably be good if someone could do a write-up of the examples JoshuaZ cited. standonbible^Talk! 15:59, 16 November 2006 (UTC)

If we wanted to do an RfC, it could look like this:

Controversy: Whether the Sternberg incident is a noteworthy and relevant example of the scientific elite squelching viewpoints that challenge mainstream systematic thought by any means necessary.

Side one: This is a perfect example of such censorship because all parties involved admit that the content of the paper was a consideration in the action that was taken (firing Sternberg) and, despite argumentation over the peer review process that was taken, this type of incident serves to exemplify a flaw in the peer review process. Thus, it is good for inclusion in the Criticisms of Peer review section. standonbible^Talk! 13:18, 16 November 2006 (UTC)

Side two: As the Sternberg peer review controversy article explains, it is only Sternberg's and other creationist's opinion that his imbroglio is an example of censorship in peer review. In the scientific community, there's no acceptance of that viewpoint, which they rightly point out is in line with the ID crowd's (of which Sternberg is a notable member) agenda. The Sternberg matter received almost no coverage outside the usual venues where creationism/ID are discussed. (sign name here)

Whether there are more relevant or notable examples does not need to come up. Of course you can edit side two as much as you want. I suggest that we wait until JoshuaZ returns before filing any kind of RfC but it's something that we might keep in mind as a way to get a bit more dialogue on this subject. standonbible^Talk! 13:18, 16 November 2006 (UTC)

Regarding the proposed insertion:

• "Conspicious examples of censorship in the realm of peer review are articles with potential religious connotations" . "Conspicuous examples"? More than one? Really? There's one example, and it really wasn't "conspicuous". Storm in a tea pot.
• "Deviations from mainstream scientific thought that can be related to religious viewpoints can be quickly censored by the "elites"" - again, any examples of this? The Sternberg issue is about dishonesty and abuse of his position, not about "censorship" (and by "elites"? How funny that major political players who are part of the political estabishment call underpayed academics "elites".
• "Even if an article passes the peer review process, its publishing can have disastrous effects for the publisher" - there is no evidence that the Meyers article passed peer review - neither Sternberg nor Meyers is able to provide copies of the reviews. The article lacked scientific merit and was outside of the scope of the journal.
• "In August of 2005, Richard Sternberg, a research associate with the Smithsonian Institution, was fired ..." - nope, he stuck the paper in the journal on his way out the door.
• "Senior scientists at the Smithsonian Institution called Sternberg a “shoddy scientist” and a “closet Bible thumper,” despite his protests that he is agnostic about Intelligent Design and does not hold any sort of creationist viewpoint.[1]" - what does this have to do with peer review?

So, apart from the fact that this is irrelevant inasmuch as it is a triviality that has been used as a crutch to explain the total lack of science generated by the creationist movement (and, in fact, the failure of people to even apply for grants offered to support scientific endeavours by creationists), the proposed text is factually inaccurate. Get your facts straight first, then discuss the value of including this trivia. Guettarda 13:36, 16 November 2006 (UTC)

Another Section in Question

Another section which deserves some attention is the section on "Peer Review Failures", in which there are about three paragraphs talking about the Center for Regulatory Effectiveness (CRE)'s dissatisfaction with an editorial published in the Journal of the American Medical Association by Linda Rosenstock. While this may be of interest to some readers, I do not believe it warrants as long an explanation as is currently being provided. Its list of specific complaints, in particular, does not seem relevant to a general discussion of peer review failures. Could we not just add an external link to the CRE's website so interested parties can find out more about the controversy regarding the JAMA editorial? It seems to me that that would be more appropriate.

(Yes, I previously removed this section for the same reason, and now see that it is back. While the poster clearly made an effort to be more objective in his/her discussion, I don't believe such a detailed discussion of this particular controversy is warranted or appropriate for a general article on peer review. Does anyone else have thoughts on this? Thanks.Chrissy385 23:15, 5 December 2006 (UTC)

chuine + keenan

with respect to the last revert. There is a claim here that the publication of Chuine paper is a demonstration of a peer review failure on the say so of one frankly trivial counter point (Keenan) and the backing of a non peer reviewed hatchet job (Carter et al) on the stern report (see here for more rebuttals: http://www.world-economics-journal.com/ ). If every paper that was ever challenged was a peer review failure, then half the literature would be here. This claim appears to have done the rounds of a various segment of the blogosphere and yet has never been picked up outside of those echo chambers. Failures of peer review would seem to me to require widespread recognition that such a failure had indeed occurred. As a counter example, I would offer Soon and Baliunas (2003) which lead to the resignation of six editors at the journal concerned (Climate research) due to the poor peer review and unjustified statements therein. Curiously enough the editor responsible for that is an author on the Carter et al paper (De Frietas). Chuine et al is neither an important paper, nor an important enough 'failure' (even assuming the Keenan analysis is justified, which is not at all clear) for it to be mentioned here. As it stands it is verging on libelous. Thus this section should be removed as i) inappropriate - the 'failure' not having been demonstrated beyond a single (and trivial) critique, ii) irrelevance, this was not an important paper (a brief communication of some anecdotal interest but of limited implication) and iii) partiality, much worse (and clearly documented) failures are not listed (I mention one above, but there are many more, Friis-Christensen and Lassen (1991, Science), or Douglass et al, 2004 GRL for instance). Leaving this in is a travesty of good scholarship.

You are quick to accuse the text as being libelous, when the text gives no indication of signaling out Chuine for criticism. The argument you give justifies it being replaced rather than removed. You are free to replace it with more important failures, but simply removing it is unproductive to creating complete discussion of the topic.--Jorfer 21:49, 20 June 2007 (UTC)

(Yes, I previously removed this section for the same reason, and now see that it is back. While the poster clearly made an effort to be more objective in his/her discussion, I don't believe such a detailed discussion of this particular controversy is warranted or appropriate for a general article on peer review. Does anyone else have thoughts on this? Thanks.Chrissy385 23:15, 5 December 2006 (UTC)

peer review failures

There are quite a few additional ones to add. Before I add them, I want to consider how:

To avoid excessive controversy like the above, perhaps it might be better to make an article for each of the truly important cases, and give a link. It will focus the discussion where it belongs. The major ones are notable enough, and usually get coverage from various science news sources, not to mention newspapers. DGG 22:45, 15 January 2007 (UTC)

If an example generated controversy, then it generated notability. --Iantresman 23:14, 15 January 2007 (UTC)

Couldnt agree more--the question is simply where to best put them. I think the best way might be to give each their own article, and then have a category Peer review controversies, or a List of peer review controversies. There is a lot more to be said about peer review in general, and the individual examples and the comment will overbalance the rest. Especially when all the other go in: stem cells, Lucent, etc. etc. I'll see it if you reply here. DGG 00:49, 16 January 2007 (UTC)

I continue to be troubled by the paragraphs under the peer review failures section added by the CRE. I think this section is inappropriate for this article. Can we remove it until we decide where to add it? Chrissy385 19:40, 4 June 2007 (UTC)

accusations that a paper is a clear example of 'peer review failure' is something that is seen within the academic community as extremely serious, and could do serious harm to an authors reputation. Thus for a paper to be listed there, there must be good cause and serious backup. Neither applies in this case. I offered three examples of what I thought were more grevious cases - especially the Soon and Baliunas paper, and I perfectly happy to have that replace the current example. My experience is with the climate related literature and so I do not want to comment on the other examples. I note that the S+B paper is already described as a peer review failure on Baliunas's page and so it makes sense to cross link it here: [3]

Since there was no apparent objection to this suggestion, I have gone ahead and replaced the disputed reference, to one that I think everyone can agree on. If there are objections, or better referencing required , please let me know.

This is not the place to discuss the global warming controversy. Any examples based on it are--very obviously--divisive, and will be removed to maintain the integrity & NPOV of this article. (regardless of which side they are on). There are enough articles already to discuss this topic, or start a new one: Allegations of publication fraud in the global warming controversy, or some such thing. and put a link to it in this one. DGG 05:49, 26 June 2007 (UTC)

practical thing is to split

as discussed earlier, and since there has been no objection over the last 3 weeks, I have split the peer review failures and peer review fraud sections out into a new article Peer review failure, I have also copied over the relevant part of the talk page discussions.

Deletion of most of the article's text

I have removed almost all the text in this article as it was uncited and appeared to contain original research about what constitutes peer review failure. More seriously, the article also contained possibly cherry-picked examples of peer reivew failure which involved people who are still living and active organisations. As these were uncited they were particularly unsuitable for inclusion. --Nick Dowling 23:45, 30 September 2007 (UTC)

Certainly agree, but most of them are probably notable and sourced. certainly needs rewriting, by people without POV in any particular controversy. DGG (talk) 01:55, 1 October 2007 (UTC)

That might be the case, but the examples in the article weren't appropriately sourced - most of the sources were simply the paper in question, and not a source which stated that the paper was flawed and had been published as a result of peer review failure occuring. This article also needs sources to support its definitition of a peer review failure (including the common causes of failures). --Nick Dowling 02:09, 1 October 2007 (UTC)

two kinds of failure??

In the peer review article, it is explained that some see the peer review process as a form of censorship. Is this also called a p-r failure? Michelle Kuiper (talk) 23:57, 9 June 2008 (UTC)

Need one or more examples

This seems a very weak article with no mention of any actual examples of a peer review failure.

An article that has often been considered a peer review failure by other authors and was challenged by letters in print shortly after publication is

Temple, Stanley A. (1977): Plant-animal mutualism: coevolution with Dodo leads to near extinction of plant. Science 197(4306): 885-886.

Other articles have criticized it for the kind of fundamental flaws that should have been noticed by reviewers, including the lack of a control treatment and failure to cite basic literature on seed germination of the plant that contradicted the article's conclusion.

The article on Tambalacoque discusses the case. Plantguy (talk) 05:54, 11 September 2009 (UTC)

Merger proposal

I think this article should be merged into Peer review (along with the other articles proposed to be merged in). I can't think of any other instance where the general failure of an entity is considered an independent article. There isn't much coverage of this in reliable sources. I assume the abundance of peer review-related articles on Wikipedia is due to the abundance of scientists on Wikipedia. –CWenger (talk) 15:31, 27 March 2011 (UTC)

It needs to stand separately, at least until it's fleshed out a bit more. Last time I tried to consolidate all the peer review articles, I identified at least a half dozen of different aspects. No one wanted to help with a merge then, either.

Peer review failures are not just of one type (letting bad stuff slip through) but of the opposite type (filtering out good stuff). Jenner's work on smallpox vaccination was rejected by peer review, and David Michaels calls this a failure, too. --Uncle Ed (talk) 18:18, 26 April 2011 (UTC)

Just came across this and fully support the merge proposal as this is simply a fork article with very little sourced content. The examples are rather amusing: a 1790 bit on vaccination - rather ancient; and a humorous bit re: math cluelessness in med circles. Need more current serious examples ... if these are all there be ... what's the point of the article? Vsmith (talk) 18:00, 13 May 2011 (UTC)

If by "fork" you mean a POV fork, then let's take to AfD right away. Violations of NPOV should be removed without undue delay.

But there are half a dozen or more articles dealing with "review" of scientific, especially medical, matters. While there is certainly a lot of overlap there, the merging task will be mammoth. Who's going to do all that heavy lifting? --Uncle Ed (talk) 18:08, 13 May 2011 (UTC)