Wikipedia talk:Identifying reliable sources (medicine)/Archive 23

Archive 20

Archive 21

Archive 22

Archive 23

Archive 24

Archive 25

→

Archive 30

RFC: Questions about RFC closure - Country of origin

The following discussion is closed. Please do not modify it. Subsequent comments should be made on the appropriate discussion page. No further edits should be made to this discussion.

---

We do reject sources because they are from a specific country because there are sources that are poor sources and there are bias sources. According to the close the country of origin is a legit consideration where RS have identified it as an issue. There have been hard data (as contrasted to stereotypes) that have identified a systematic problem that is normally identified with an affiliated country of origin. Therefore, this edit seems to contradict the close. QuackGuru (talk) 17:00, 20 October 2015 (UTC)

The close that I wrote said no such thing. Besides, the suggestion that the edit that the close clearly and directly indicates is appropriate nonetheless contradicts the close by some extreme, odd interpretation is utterly nonsensical. Get the point. Discussion closed. --Elvey^(t•c) 03:34, 24 October 2015 (UTC)

User:Elvey, perhaps you would come back and clarify a few things for us. For example, you wrote "We cannot override WP:V or WP:RS." In what way would rejecting some sources, on some subjects, from countries with a strong reputation for the low quality their sources, constitute "overriding" WP:V or WP:RS? WhatamIdoing (talk) 21:35, 20 October 2015 (UTC)

Reread Albino's comment. To what end do you seek clarification? --Elvey^(t•c) 03:34, 24 October 2015 (UTC)

I am flummoxed as to why that single comment is singled out as of more weight than those of multiple other editors. This seems to me a very questionable closure. CFCF 💌 📧 12:02, 24 October 2015 (UTC)

I think that would likely be because Albino's comment stated that multiple editors were ignoring the fact that we are talking about high quality sources and some editors opposed low quality sources, which wasn't the purpose of the RfC. LesVegas (talk) 12:53, 24 October 2015 (UTC)

This is not true. The statement in the guideline is about high-quality types (emphasis in the original) of sources, not about high-quality sources. A meta-analysis is a high-quality type of source, but it can be a low-quality source (e.g., if it's outdated, poorly done, or irrelevant). WhatamIdoing (talk) 21:32, 28 October 2015 (UTC)

Yes, there are meta-analyses that are poor quality because of age, journal integrity, etc, but they are not poor quality simply on the basis of where they are published or the country of origin of its authors. That was the purpose of the RfC, and no, just as we cannot exclude sources because they receive industry funding, we also cannot exclude them because of country of publication. Not one single editor addressed why we don't reject industry funded sources on the basis of known bias, but we should ban sources based on speculative and unconfirmed publication bias due to country of origin. Not one editor. Note that Elvey mentioned as much in his close as well. There were multiple factors here really. LesVegas (talk) 19:23, 29 October 2015 (UTC)

It still completely fails to address the actual need for the addition. No discussion has been shown about not including high quality sources based on country of origin, this is a red herring and a useless bloating addition. CFCF 💌 📧 10:02, 26 October 2015 (UTC)

This discussion centres around attempts by SPA editors to crowbar low quality sources supporting various degrees of efficacy not shown in high quality sources of ALT-Med articles, particularly Acupuncture. Other attempts have been made, to WP:MEDRS for example. Characterising mainstream editors as racially prejudiced by SPA's has been happening for a while now, and is a particularly nasty tactic. -Roxy the dog™ (Resonate) 10:20, 26 October 2015 (UTC)

While I am surprised my comment was mentioned. I am not surprised with the close. My comment did point the problem with some of the responses. They were off topic. A problem that looks like its repeating down here. The RFC question specifically was about High quality sources. Low quality sources will be rejected regardless of what country they are from because they will not even pass WP:RS. AlbinoFerret 13:54, 26 October 2015 (UTC)

Are there examples of editors rejecting high quality sources because of racial prejudice? -Roxy the dog™ (Resonate) 14:30, 26 October 2015 (UTC)

A couple were mentioned during the RFC. I did point out I was unsure if it was widespread, but even if it isnt very widespread its a bad thing that should be stopped. It hurts the project in rejecting even a few high quality sources and makes the project look bad focusing in the ethnic angle. I think those editors who are concerned this will allow low quality sources in shouldt worry, low quality will always be excluded. Just point out the problems with the source that make it low quality and dont point out where they came from. Doing that give those pushing them a reason to argue. AlbinoFerret 14:46, 26 October 2015 (UTC)

I think this might work: "While country of origin per se is not a suitable reason to reject a source, it is appropriate to consider in cases where reliable sources have identified systematic problems in the medical literature associated with specific regions or countries." QuackGuru (talk) 20:31, 26 October 2015 (UTC)

While that may be what you want in the section, it does not say what the closer said. Here is the section:

"This addition should NOT be read as a PC ban on any mention of country of origin (or founding source, etc.) when necessary to refer to studies with hard data (as contrasted to stereotypes) that have identified a systematic problem that is normally identified with an affiliated country of origin, as mentioned by Richard Keatinge. Likewise, this addition should NOT be read as a changing the longstanding policy that sources from publications known to routinely publish and fail to retract material proven unreliable may be excluded."

What it appears Elvey is saying , and Elvey can correct me if I am wrong, is this close is not a ban on discussing problems with sources in a discussion of a source that talks about them. It doesnt appear to be a loophole to insert arguments about a source, based on a country, just because other crappy sources have come from there. What might be better to say is "It is better to look at the quality of a source, if the source is of low quality it should be excluded." AlbinoFerret 20:58, 26 October 2015 (UTC)

CORRECT on both counts. (The comma in the last sentence should be a semicolon or period.) --Elvey^(t•c) 02:27, 29 October 2015 (UTC)

Your proposal "It is better to look at the quality of a source, if the source is of low quality it should be excluded." is not about country of origin. QuackGuru (talk) 21:01, 26 October 2015 (UTC)

See the specific part of the close. See "when necessary to refer to studies with hard data (as contrasted to stereotypes) that have identified a systematic problem that is normally identified with an affiliated country of origin, as mentioned by Richard Keatinge." QuackGuru (talk) 21:02, 26 October 2015 (UTC)

Thats because the RFC has already said that country of origin is not a valid exclusion. I assumed that country of origin is added to the list of other things that should not be considered. The whole purpose of that section appears to be to tell people to look for high quality sources, then some things that should not be considered. I propose adding a sentence at the bottom to direct editors to, instead of looking at the country or funding, to look at the quality. AlbinoFerret 21:07, 26 October 2015 (UTC)

Yes, it says not to stop discussions about a topic that discusses low quality sources, not a loophole to allow discussions that we have already said should not take place like excluding a source based on ethnic origin. AlbinoFerret 21:09, 26 October 2015 (UTC)

You said "Thats because the RFC has already said that country of origin is not a valid exclusion." That what was written and that was what was added.

"While country of origin per se is not a suitable reason to reject a source, it is appropriate to consider in cases where reliable sources have identified systematic problems in the medical literature associated with specific regions or countries."[1] QuackGuru (talk) 21:11, 26 October 2015 (UTC)

My close is clear. It's not appropriate to --Elvey^(t•c) 02:27, 29 October 2015 (UTC)

You are adding a loophole that I dont believe the closer added. Lets wait for them to chime in as I have pinged the closer in a few posts ago. AlbinoFerret 21:14, 26 October 2015 (UTC)

Albino Ferret is right, in fact, it would violate the spirit of every single RfC done across wikipedia if we always added a reference tag and then just put whatever summary those opposed wanted. Adding in a summary of whatever you want to be read as a caveat is gaming an RfC outcome. Besides, the RfC wasn't about "should we say 'country of origin' and then have these caveats?" it was about the wording, "country of origin" specifically. And, to that, the answer was "yes," it needs to be added in. But since the consensus reading did mention other specifics and since some editors are persistent in wanting something else, I figured a link where readers could see the full consensus read/closing comments would be the best compromise. LesVegas (talk) 21:25, 26 October 2015 (UTC)

The outcome of the RfC is not constrained to be binary. It's intended to gauge consensus, and the closer found consensus to include 'country of origin', but with the caveat that it might be a legitimate consideration where "hard data" demonstrate a concern about biased literature. MastCell ^Talk 22:52, 26 October 2015 (UTC)

The point is that the addition is completely useless, and not supported by the RfC. QuackGuru – while I agree with the intent of your clarifications the fact is we are just introducing bloat. If any policy is to be taken seriously it needs to be succinct, and can't include hypothetical clauses that have never been proven to be needed. CFCF 💌 📧 22:48, 26 October 2015 (UTC) 

When the RfC is a question asking "should we add 'country of origin' to this list?", and the consensus reading says "Yes" removing it entirely is going against consensus. LesVegas (talk) 23:09, 26 October 2015 (UTC)

We're trying to get some clarification of some of the major issues with the close and a possible new clause, it isn't something we will do on a whim–we need to have a agree on what (if anything) to include. We should wait until clarification of what Elvey meant in the close summary. Currently the meaning of the text is very murky, and as such it may be challenged for not adequately summarizing the consensus. CFCF 💌 📧 23:15, 26 October 2015 (UTC)

(Late reply) Elvey, I'm trying to figure out what the relevance of your statement about "overriding WP:V and WP:RS" is. Imagine that you are trying to decide whether a source is reliable. Do you believe that considering the country of origin, e.g., to avoid citing the notoriously bad Soviet science, would somehow a constitute "overriding" WP:V? Could you point to any sentence in WP:V that would be violated or "overridden" by doing that?

Also, I have read Albino's comment, and you seem to have overlooked a critical difference between what the guideline says and what Albino wrote. The sentence in the guideline talks about high-quality types of sources (e.g., a meta-analysis is a high-quality type of source; a case study is a low-quality type of source). Albino talks about high-quality sources—a quality that takes far more into consideration than the type of the source. "High-quality types" and "high-quality sources" are not the same thing. It's possible to have a low-quality meta-analysis, and just like it's possible to have a top-quality case study. WhatamIdoing (talk) 21:32, 28 October 2015 (UTC)

I don't dispute that it's possible to have a low-quality meta-analysis. What's clear is there was consensus that "country of origin", per se, is not a valid method to identify a low-quality meta-analysis. --Elvey^(t•c) 02:27, 29 October 2015 (UTC)

What does "overriding WP:V and WP:RS" is? It seems to mean that MEDRS is wrong and that MEDRS overrides WP:V and WP:RS to exclude sources that meet WP:V and WP:RS. Therefore, country of origin such as from China are good even if they are poor quality and bias. QuackGuru (talk) 21:51, 28 October 2015 (UTC)

You didn't respond to my comment about your extreme, odd interpretation of what I said. I feel grossly misrepresented, and an apology would certainly have been welcome. "We cannot override WP:V or WP:RS" means just that. Surely none of you dispute that "We cannot override WP:V or WP:RS." There was consensus that "country of origin", per se, is not a valid reason to reject a source. I claim neither WP:V nor WP:RS contain any sentence consistent with allowing "country of origin", per se, as a valid reason to reject a source.

Again: You need to drop the stick. Insisting my close not clear by misrepresenting what I said with an extreme, odd interpretation is not going to fly, and you've just done that for the second time in this section, this time by claiming I've said MEDRS is wrong. STOP. It's disruptive and uncivil. I said no such thing. Not liking the close is not a valid reason to reopen it. I don't see any clarification request that hasn't been adequately addressed by AlbinoFerret or myself. Re-closing. --Elvey^(t•c) 02:27, 29 October 2015 (UTC)

No, you haven't answered my question. There are two basic ways to approach this. Either your comment about overriding WP:V and WP:RS is:

• pointless blather, with just as much relevance as you saying "Don't kick puppies" or "Be nice to your neighbors" in the middle of this (in which case, you should just remove it), or
• you actually meant to communicate something relevant, that editors need to know and understand (in which case, you need to explain what you meant, because everybody's confused).

The obvious assumption is that you meant what you said. What you said seems to be that (when relevant/appropriate/etc.) editors should not take notice of published academic research that says (for example) Soviet psychiatric research is a bunch of garbage (and garbage because it came out of a country that had difficulties with the concept of apolitical science), because you believe that doing so would be based on "country of origin" and that discarding sources on the grounds of country of origin somehow override WP:V and WP:RS.

Is that what you meant? WhatamIdoing (talk) 03:24, 29 October 2015 (UTC)

I'm not sure what Elvey meant, but I know some editors here have already exhausted this editor (a volunteer, as we all are) with complaints, so I'm willing to take a stab at it from my reading of Elvey's close. Documented issues with poor science conducted by a particular piece of research are a valid reason to exclude that individual piece of research because it doesn't meet the barometer of a high-quality source, per WP:V and WP:RS anyway. Excluding such research, however, because it is Russian or Soviet and they had scientific issues and therefore it all must be garbage science, is not an appropriate reason for rejecting it. In other words, what are the reasons that individual piece of research is invalid? And if it is documented invalid, it isn't high quality anyway. Funding sources have a well documented history of much worse; Elvey mentioned in their close that country of origin wasn't worse than funding, per consensus (since nobody seemed to tackle that question I raised.) LesVegas (talk) 19:38, 29 October 2015 (UTC)

This was the previous proposal: "While country of origin per se is not a suitable reason to reject a source, it is appropriate to consider in cases where reliable sources have identified systematic problems in the medical literature associated with specific regions or countries."

It is known that there is publication bias from Chinese journals. We cannot claim "country of origin" is not a problem given the evidence. QuackGuru (talk) 20:24, 29 October 2015 (UTC)

That's a 2005 source, and it does illustrate publication bias in some fields. For those fields it notes, it may apply to Chinese sources published prior to 2005. LesVegas (talk) 23:00, 29 October 2015 (UTC)

LesVegas, your reply does not address my question at all. So far as I can tell, there is nothing in WP:V or WP:RS that requires us to accept sources that have characteristics which editors deem suspicious. However, it leads me to a question for you: Imagine that (it's 1985 and...) you are looking at a source produced by the well-documented mess that was Soviet psychiatry on the question of people with schizophrenia who claim to be political prisoners. Do you think that specific source could be excluded:

(a) because it was produced by the Soviet psychiatric system, which is an extraordinarily well-documented disaster (and therefore the odds are very high that it, too, is bad) or

(b) only if that individual source were called out, by name, in another reliable source, as an example of bad research?

Another way to put this: If a source was produced by a Soviet psychiatric institution, is it "tarred with the same brush" as the rest of the field, or "innocent until proven individually guilty" of politically manipulated science? WhatamIdoing (talk) 00:39, 30 October 2015 (UTC)

User:WhatamIdoing, you're right in that WP:V and WP:RS don't require us to use sources from countries with issues, but they also don't require us to exclude them on that basis. These policies simply tell us what a high quality source is and what it is not. The way I read the close is based on the content of the RfC expressed over and over again, which is we have our rubrics for determining what a reliable source is. That's the WP:V and RS part. So we use WP:V and WP:RS to determine if it is a high quality source. These rubrics do not allow us to exclude sources that meet WP:RS standards because of their country of origin, much as they don't allow us to reject sources because of industry funding issues. And the way I see it, the issues with the Soviet disaster likely produced journals that wouldn't meet MEDRS standards anyway, certainly being called out by reliable sources. If this occurred today, you would undoubtedly find a slew of journals being slammed by academia for problems. Having occurred so long ago, I'm sure there's some, but the point is moot because those journals are so old they wouldn't be reliable sources anyway per WP:MEDDATE. LesVegas (talk) 19:06, 30 October 2015 (UTC)

I think your going to have to clarify your thought process here - I can't tell how your different statements are related. And neither RS nor MEDRS are about finding high quality sources, they're about finding acceptable quality sources for Wikipedia, with MEDRS focusing on medicine. CFCF 💌 📧 19:23, 30 October 2015 (UTC)

MEDRS is WP:RS but specific to medical claims. We don't use low quality sources. What other statements do you not understand? LesVegas (talk) 19:38, 30 October 2015 (UTC)

Wikipedia most certainly does use low-quality sources—every hour of the day.

MEDRS in particular, but RS as well, has a structural problem with purpose. Is the purpose to show you the "minimum acceptable quality", so that you can find the line between barely good enough and not quite good enough, so you'll know when to tag or remove dubious contributions? Or is the purpose to show you how to find the best possible sources, so you'll be able to find great sources for writing new content?

MEDRS is largely written to show you how to find "best possible" sources. (As a result, we have a problem with people rejecting "good enough" contributions, because they aren't "best possible".) RS is split more evenly, but it still fails to tell editors when it's talking about "good enough" and when it's talking about "best possible", which causes unpleasant disputes, in which both sides believe themselves to be (and actually are) correct. We are not going to solve this problem this week. WhatamIdoing (talk) 03:38, 1 November 2015 (UTC)

Elvey the clarification that I asked you about is from an earlier discussion, that some want to add a loophole to the "country of origin" by stating it still can be considered with language like this as a note "it is appropriate to consider in cases where reliable sources have identified systematic problems in the medical literature associated with specific regions or countries" saying that the last paragraph of your close says that. Is this what you wrote in that last paragraph of the close? AlbinoFerret 04:46, 29 October 2015 (UTC)

Editors are waiting for clarification. Editors do not agree with this change. See "The outcome of the RfC is not constrained to be binary. It's intended to gauge consensus, and the closer found consensus to include 'country of origin', but with the caveat that it might be a legitimate consideration where "hard data" demonstrate a concern about biased literature."[2] QuackGuru (talk) 19:32, 30 October 2015 (UTC)

Reread Albino's comment. To what end do you seek clarification, and of what? Clarification answers, AGAIN: "We cannot override WP:V or WP:RS" means just that. Surely none of you dispute that "We cannot override WP:V or WP:RS." There was consensus that "country of origin", per se, is not a valid reason to reject a source. I claim neither WP:V nor WP:RS contain any sentence consistent with allowing "country of origin", per se, as a valid reason to reject a source. No, I did not say that. There is no caveat in the close; just a clarification. --Elvey^(t•c) 02:37, 31 October 2015 (UTC)

Elvey, I seek clarification about exactly which sentence in WP:V or WP:RS would allegedly be "overridden" by rejecting a source on the grounds that it came from a particular country, where said country's academic publications were very widely accused of bias and politically motivated writing. The exact sentence, please, in the form of a direct quotation. Handwaving about "There isn't any rule that says you may exclude Soviet psychiatry sources if it superficially appears to meet all the other standards, and therefore you mayn't exclude Soviet psychiatry sources merely because the whole field is known to be crap." I want to know exactly which sentence this overrides.

NB that you cannot actually override rules that do not exist. If, as you say now, no such rule actually exists, then you should strike that sentence from your closing statement, because it is (at best) irrelevant. And if the main reason for your conslution is that dumping sources from disreputable countries actually would contradict this non-existent rule, then you should reverse the entire closing statement, because you were reasoning from false premises. (And in that case, I'd suggest letting someone else re-close it.) WhatamIdoing (talk) 03:31, 1 November 2015 (UTC)

Let me clarify: Even if you can show that reliable sources A thru D all say the whole field of X in country Y is known to be crap, then you still must not reject the the whole field of X in country Y on the basis that it's from country Y; rather you may (and should) reject the the whole field of X in country Y on the basis that reliable sources A thru D all say the whole field of X in country Y is known to be crap. The fact is, you do not seek clarification; rather you simply reject the clarification given. I claim neither WP:V nor WP:RS contain any sentence consistent with allowing "country of origin", per se, as a valid reason to reject a source. It's on you to find a sentence that allows "country of origin", per se, as a valid reason to reject a source. One cannot prove a negative. The two policies provide detailed guidance on what makes content verifiable via a reliable source. None of that guidance says anything to suggest that "country of origin", per se, is a valid reason to reject a source. Your straw man attack doesn't fly. Even if you can show that reliable sources A thru D all say the whole field of X in country Y is known to be crap, then you still must not reject the the whole field of X in country Y on the basis that it's from country Y; rather you may (and should) reject the the whole field of X in country Y on the basis that reliable sources A thru D all say the whole field of X in country Y is known to be crap. QuackGuru says "We do reject sources because they are from a specific country " per se. We must not do that. That is the consensus. --Elvey^(t•c) 22:58, 1 November 2015 (UTC)

Elvey, what's the practical difference between "We reject all sources from China about acupuncture because Chinese publications on acupuncture are unreliable" and "We reject all sources from China about acupuncture because Sources A through D say that Chinese publications on acupuncture are unreliable"? I'm seeing zero practical difference myself.

User:LesVegas, was it clear to you from Elvey's closing statement that sources about acupuncture from China would still end up being rejected, only with a slightly longer excuse? WhatamIdoing (talk) 02:46, 3 November 2015 (UTC)

User: WhatamIdoing I don't understand for the life of me what Elvey meant above, but I'm afraid your deciphering of Elf speak is just not the case at all. After all, Elvey stated that, "I was, rather, simply clarifying that mention of country of origin is not barred, even though it would now be explicit that decision making on the basis of country of origin, per se is not OK." That just means we can't bar or reject sources because of country of origin, much as we cannot reject them if they're industry funded, but that there is no PC ban on mentioning country of origin or anything else about or in sources, for that matter. LesVegas (talk) 17:24, 3 November 2015 (UTC)

Elvey has repeated that country of origin per se is not a reason to reject a source. I have finally figured out that Elvey is signalling that country of origin per quod is a valid reason to reject a source—i.e., because Soviet psychiatric sources are widely discredited. So you can't reject a source "because it's from the USSR" (country of origin per se); instead, you reject that source "because it's from the USSR and Soviet psychiatry has been widely discredited" (country or origin per quod). The end result is the same, but the excuse is wordier (and more accurate).

The application to Chinese acupuncture and the multiple denunciations of the publication bias and methodology issues should be obvious. WhatamIdoing (talk) 01:54, 6 November 2015 (UTC)

User:WhatamIdoing, I do agree that if methodology is specifically and widely criticized by the scientific community, and a piece of research uses that methodology, that's reason enough to reject. What should never happen (and this, I believe, is also what the close says) is for editors to assume all research originating from one country uses that very methodology. What is and should be allowed is for editors to say on talk, "hey, according to X source, a lot of environmental research from Sweden is flawed and needs to be examined to see if it adopts the same methods which have been widely discredited" and then reject those sources on that basis. But what we cannot do is reject all environmental research from Sweden because research from Sweden has been criticized for adopting a method that an individual piece of research from Sweden never used. That's what is meant by it, but if you disagree with what in saying and feel your interpretation is correct, I can always ask Elvey on his talk page if you want. LesVegas (talk) 13:43, 6 November 2015 (UTC)

So, you're back to saying that for a source to be discredited we need express mention of flaws in that source?CFCF 💌 📧 14:04, 6 November 2015 (UTC)

Yes, honestly I believe that's fair. Yet as the wording stands, it currently also prohibits editors from conducting a second peer review because of methods. While editors shouldn't be doing that because of their personal opinions on methodology, I think if the scientific community at large has criticized specific methods and a particular piece of research adopts those methods, we shouldn't be using that research. So under those specific grounds, I agree with you that we need further clarity for the RfC. So, I would be willing to compromise with you in this way: let's restore "country of origin" for now into MEDRS, per the previous RfC close, for now. As it stands, that is the way things are supposed to be. You and I can work together to re formulate this second RfC to make it clearer and contain genuine questions we both agree to disagree on, and that we hope to seek the community's opinion on because, even though we may disagree on exactly how to phrase it, I honestly do agree with you that we need some further clarification even with "country of origin" added into the sentence and I would like to see an RfC that brings clarity here to gain traction. Does that sound like a deal? LesVegas (talk) 15:50, 6 November 2015 (UTC)

I'm glad that we have agreed that some further clarification is necessary.

I don't think that we can restrict this solely to using a discredited methodology. The problem with Soviet psychiatry wasn't the reported "method": it was the political system that refused to publish research that disagreed with the party line. That's a problem that isn't reported in the methodology section of the paper itself. That's why so many sources said that nothing at all from Soviet psychiatry could be trusted, because it was impossible to know which results were tampered with or suppressed. WhatamIdoing (talk) 18:57, 6 November 2015 (UTC)

Greetings ladies and gentlemen! Sorry for chiming in so late. The Talk Page has grown rapidly during the past few days, and it's taken some time to keep track with the discussion. Anyway, it appears to me that there's been some rather heavy edit warring with respect to the former RfC close[3][4][5][6][7][8][9][10]. As far as I am concerned, the RfC close was quite clear on the fact that "country of origin" is no reason to omit any sources. Should there be well documented publication bias in a given country and field, then that sure serves as a red flag but never as reason to omit a source. Instead, where we pay attention to is the quality of the source: low-quality sources will always be discarded, no matter what's the country of origin. Likewise, high-quality sources shall be used, no matter how many low-quality sources there are published in the country of origin. Actually, this was paid attention to in the close as well; a lot of editors were arguing low-quality sources, and therefore I think the closer was correct in discounting those comments as off-topic.

If the same editors who opposed the proposal at the RfC, still keep opposing it after the close, it doesn't mean that the RfC is disputed and therefore the changes cannot be implemented. Meanwhile, I agree that the consensus per the last RfC should be incorporated until we get more clarity on the issue and the possible wording. Cheers! Jayaguru-Shishya (talk) 16:22, 8 November 2015 (UTC)

User: WhatamIdoing, you raise a good point about the Soviets and methods and this makes me think about even further angles for how we can clarify this. What seems apparent to me is that the Soviet research we wouldn't be using anyway, not because of the country (and time period) it originated from, but because it shouldn't be considered "high quality"and the sentence we are talking about is on high quality sources anyway. So that's how we disqualify it. Now, I'm not exactly sure MEDRS currently has clauses on how to determine if sources like that aren't high quality or not, so I think that's how we can fix it to where everyone's happy. Bogus Coca Cola funded research showing cokes don't cause obesity should also clearly not be allowed on MEDRS (and, yeah it was a primary study, but clearly could become part of a secondary source) and, as it's written, I don't see any reason it would be rejected. So, in my opinion, the issue will easily be resolved if we better address what precisely precludes a source from being a high quality one. I now think that's how we go about clearing up the RfC, not necessarily with another sentence and certainly not with altering the closed one. LesVegas (talk) 18:43, 10 November 2015 (UTC)

• The paragraph isn't about "high-quality sources". I believe I've mentioned this three or four times already, and I'm hoping that you will pay attention to this detail this time, even if it seems unimportant to you. The paragraph is about "high-quality types of studies". As in: Do not reject a meta-analysis (a high-quality type of evidence) in favor of a randomized controlled study (a low-quality type of evidence) because of your personal objections. This paragraph is not "about high-quality sources". It's about people trying to dump good evidence types (some of which are actually low-quality sources) in favor poor evidence (some of which are actually high-quality sources).
• How do you know that Soviet psychiatry is a low-quality source? You know it's a low-quality source because of its country of origin. You would, in fact, be rejecting it due to its country of origin (combined with the fact that said research on that subject, emanating from that country, has been specifically, by country name, declared to be garbage by experts.
• Perhaps, though, we don't actually need a clarification, so much as we need to agree that (a) this section is about editors' personal objections to sources, and (b) once you have a source that says all Soviet psychiatry sources—or all Chinese acupuncture sources—are biased, then it's no longer a "personal objection", and therefore it's perfectly fine to chuck Soviet sources about psychiatry out the window. WhatamIdoing (talk) 01:48, 11 November 2015 (UTC)

User:WhatamIdoing, the way in which we know a Soviet psychiatry source would be low quality is that the journals, methods used, etc, would be discredited in reliable sources. To be specific, The Russian Federation admitted that Soviet Psychiatry was used for political purposes. Yes, that is a pretty large umbrella. Were they talking about the practice of declaring political dissidents to have mental illness in order to imprison them? Yes, they were. Were they talking about every single piece of psychiatric research published during the entire Soviet era? Of course not. G.E. Sukhareva was an influential Soviet psychiatrist who worked in the Soviet era. Are his works and findings discredited? Andrey Yevgenyevich Lichko is another. Is all his research discredited? See, the problem with stereotyping sources and politicizing them in order to reject them is that not everything fits so neatly in those confines. Btw, I understand what MEDRS states as high quality here, and it always appeared to me you are the one who thinks it needs to be more stringent, whether you realize it or not. I was politely extending an olive branch to you for something it appears you want, and that's exactly why I'm saying perhaps we need to redefine, in that paragraph and elsewhere, what a high-quality is, so that the sentence "do not reject a high quality type of study due to, w, x, y and z" might mean something we can all agree on, even if it would mean changing "study" to "source". That's what I mean by clarification. And yes, we do, perhaps, need clarification because not all Soviet psychiatry sources are biased, not all Chinese sources on acupuncture are biased (bias is only listed in reliable sources as one of several possibilities during a particular period of time, a fact I have also repeated numerous times). I seriously don't know how you can possibly want to reject all Chinese studies on acupuncture, yet are ok with wording that gives carte-blanche usage for industry funded Coca-Cola or Big-Tobacco research. WhatamIdoing, you have always struck me as a very reasonable and highly intelligent editor, unlike many here. I would love to work with you on a reasonable compromise and fix this so that MEDRS actually works in principle moving forward. I'm sorry I had to point out the flaw and force a fix, but it was something I felt needed doing, and now it seems that with so many editors so emotional about the close we probably need some additional wording redefining "high quality" which we can all agree on. LesVegas (talk) 15:10, 11 November 2015 (UTC)

You are reasoning from false premises. The reason that we know Soviet psychiatry sources are unreliable is not because some of the methods or non-Soviet journals are discredited (not all Soviet psychiatry was published in Soviet-controlled journals). The reason that we know Soviet psychiatry sources are unreliable is because they were published under a political system (the political system in their "country of origin") that suppressed and manipulated the sources. It is (presumably) true that only some of the Soviet sources were directly tampered with; it is definitely true that some of the problem is due to refusing to publish research that had the "wrong" answer (a problem that is neither with the methods nor the journals). But we don't know which sources were tampered with, or how much suppression was involved, therefore all Soviet sources are unreliable for statements of psychiatric fact.

Or, to put it another way: When a researcher is exposed for serious, intentional fraud, reputable scientists stop citing all his publications, not just the one for which fraud has been proven. Why? Because you no longer know how much you can rely upon anything that the liar wrote. Maybe the researcher only told lies once, but maybe he told lies through his whole career. And you don't know, so you can't rely upon any of it.

In an exactly analogous manner, when a whole political system is exposed for serious, intentional fraud, intelligent people stop citing all the publications from that political system, not just the individual ones for which proof of fraud has already been published.

(I agree that a simpler summary of "high quality" is needed, but it's irrelevant to this paragraph. This paragraph is about preferring better evidence over weaker evidence, even if there's something about the better evidence that an editor personally objects to. And of all people on Earth, I should know the actual intent, because I wrote this paragraph. NB that the edit was made in 2010, but based upon multiple conversations throughout most of 2009.) WhatamIdoing (talk) 16:46, 11 November 2015 (UTC)

So what is wrong with saying that country of origin, basically a possible racial motive and very disturbing if it happens, is not an exclusion criteria in that context for high quality (add whatever word you want here sources, type, etc)? AlbinoFerret 16:53, 11 November 2015 (UTC)

Yes, I second Albino Ferret's question. What is wrong with saying this? And I really hope you're not saying the works of Ivan Pavlov, Vladimir Bekhterev or Pyotr Gannushkin, who are highly regarded and highly influential psychiatrists who did happen to work in the Soviet era, are unreliable. You're not actually saying their research during this era is unreliable are you? Do you see why this can't fit so easily into a tiny, neat box where everything should unilaterally be banned as unreliable, but instead needs some nuance? LesVegas (talk) 17:17, 11 November 2015 (UTC)

Once again

Let's try this again, from the top:

• The statement is not about high-quality sources. It is about high-quality types of sources, e.g., meta-analyses, including low-quality, poorly conducted meta-analyses. So what's wrong with high-quality sources? Nothing, except that the sentence isn't talking about high-quality sources. It's talking about high-quality, mid-quality and low-quality sources that happen to rank higher on those pyramids of evidence quality.
  • You should exclude low-quality sources that use a high-quality type of study design.
  • You should include high-quality sources that use a high-quality type of study design.
  • Using a high-quality type of study is not the sole determinant of whether something is a high-quality source. Meta-analysis = high-quality type. Meta-analysis ≠ high-quality source.
• If we exclude "country of origin" from the list of "personal objections" that editors must ignore, then both Chinese acupuncture and Soviet psychiatry are excluded. Why? Because once you've got a list of academic sources saying that these entire output of these two countries on these particular subjects, is tainted by political bias, then the editors are no longer rejecting them because of "personal objections". They're rejecting them because of objections that are verifiably held by actual subject-matter experts.
• If we don't exclude "country of origin" from the list of "personal objections" that editors must ignore, then Chinese acupuncture and Soviet psychiatry are still excluded. Why? Because once you've got a list of academic sources saying that these entire output of these two countries on these particular subjects, is tainted by political bias, then editors should avoid those sources. They should find sources from an academic and political system that isn't widely condemned for political bias.

So basically I'm concluding that this is an irrelevant addition. You want this change so that you can say "Look, we get to use Chinese acupuncture sources!" And the response will be, "That sentence is only about personal objections to Chinese acupuncture sources. I have no personal objections. I only object because these three peer-reviewed academic sources say that all Chinese acupuncture sources are suspect. I have purely impersonal objections. Therefore, we still won't use them." WhatamIdoing (talk) 06:20, 17 November 2015 (UTC)

User:WhatamIdoing I understand your point and you don't have to keep repeating yourself, that's a lot of effort you're having to needlessly expend, so I just want to make it clear that I understand your point. I'm not sure, however you understand mine, so let me repeat two simple questions: if academic sources have identified Soviet psychiatric research as unreliable, should we remove or reject research from Ivan Pavlov, Vladimir Bekhterev or Pyotr Gannushkin, well-regarded and influential in the psychiatric field, each of whom did happen to work in the Soviet era as well? And why do you support allowing industry funded research in certain fields when much worse bias has been shown there than those studies on Chinese acupuncture research you complain about? You do know that this was mentioned in Elvey's close, right? Ok, that's three questions, but you get the point. LesVegas (talk) 15:23, 17 November 2015 (UTC)

We can use Chinese or Soviet sources if they are of high-quality, but low-quality sources we shall discard without doubt. It's all a question of source quality, and this must be assessed on a case-by-case basis. The country of origin has no role in this, and the close of the last RfC also conforms this.

Should there be a study suggesting the possibility of publication bias, it sure serves a red flag and calls for extra attention with respect to the sources, but the source quality is what mattes in the end; not the country of the origin. The purpose of the statistical tests carried out to study possible publication bias is not to discard all the studies of the country in question (i.e. do not extrapolate the results outside the sample), and this misunderstanding on the nature of statistics appears to be behind some users suggestion to discard all sources from the countries in question.

They say that there are "three kinds of lies: lies, damned lies, and statistics". This is hardly true, though. Usually the problem is misunderstanding of statistics. Jayaguru-Shishya (talk) 19:56, 17 November 2015 (UTC)

Yes, LesVegas, the fact that those prominent scientists worked under the Soviet system is definitely a red flag. If the case of Pavlov (most of whose career was pre-Soviet anyway), his work was famously not manipulated, so we can counter sources that condemn Soviet psychiatry as a whole with sources that accept his specifically. But "Pavlov's work seems to have been okay" doesn't mean that you can use any other Soviet psych sources. (Also, Pavlov technically did physiology, not psychiatry; in fact, Pavlov died the year after the first-ever board-certified psychiatrists were approved in the U.S.) I don't know anything about the other two, but I note that they both died even earlier than Pavlov.

Jayaguru, if the source indicates publication bias based on country of origin, then the country of origin is indeed a red flag against the source. Like every other consideration, it is not necessarily a sufficient consideration, but if the source is published in a system with demonstrated bias, then that is a verifiable indication that the source may not be high-quality after all. WhatamIdoing (talk) 02:26, 18 November 2015 (UTC)

User: WhatamIdoing Thank you for your response to one of my questions and I appreciate your consistency with your answer. Regarding Pavlov, you are correct, much of his famous published works for which he won the Nobel Prize, occurred before the Soviet Union was formed. However, he did continue to publish up until 1924, two years into the formation of the USSR, and then he worked in the clinical field after that. Would those publications be allowed? The other two get a bit more complicated, as some of their publications were actually seminal works and published after the Soviet Union was formed. Bekhterev was credited for founding objective psychology before Watson, for discovering the role of the hippocampus in memory, for discovering ankylosing spondylitis (which, yeah, is more physiology than psychology related, I was just illustrating he's a big deal) and he just so happened to publish a posthumous work in 1927, nearly 5 years into the formation of the USSR. Gannushkin published important research in the late 20's and also a posthumous work in the early 30's on personality disorders and that research is credited as a War and Peace of personality disorders, the most complete exhaustive description of them. These are all influential works and, despite being psychology works published in the Soviet Union, they are well received and I see no academic objections to them, not even close. Therefore, wouldn't the best way for us to deal with these is to say they're allowed, but if editors find reliable sources that have serious objections to them, thereby making them not "high quality sources", they're disallowed? (which would mean a rewrite of the paragraph from type to source). I also note that you didn't answer my question about why funding sources are not a reason for rejecting a source, despite much stronger evidence in certain fields that this is clearly the case than we have for Chinese studies on acupuncture, yet you seem fine with disallowing those. I'm seriously not trying to pummel you or make you look like a hypocrite and I hope you don't think that's the case. My intention is just to suggest that perhaps you have overlooked a blind spot with regard to our wording, and that perhaps it should be something that is consistent. And also to point out that this was noted by Elvey in his close and any proposed implementation of wording should take this into consideration, and I'm not sure your proposal did. I can ask you down there, though, if you'd rather. LesVegas (talk) 13:56, 18 November 2015 (UTC)

If, e.g., Gannushkin's work is respected, then we ought to be able to find sources that balance the overall reputation ("All Soviet psychiatry is untrustworthy garbage") with sources that specifically praise his ("Soviet psychiatry is generally untrustworthy, but Gannushkin's work on personality disorders is actually good").

I've seen no similar claims for the Chinese source you want to include.

I don't believe that our guidance about funding sources has any bearing on this question. WhatamIdoing (talk) 23:23, 21 November 2015 (UTC)

Phrasing

We're having significant issues implementing the result of the RfC, in part because it was ill-formed, and in part because it had a very unclear close. I think the situation demands more discussion in order to sort out the proper phrasing. Before staring a new RfC or anything similar I think we should tally our suggestions:

This is the last suggestion:

Editors should not perform detailed academic peer review. Do not reject a high-quality study-type because of personal objections to: inclusion criteria, references, funding sources, country of origin or conclusions.

References

I think the major issue with this is bloat – we're basically wasting time explaining the obvious here. Is it needed at all, and why not forgo it entirely? Since country of origin pew say has never been proposed to be a reason to exclude sources, why should we waste valuable space when it's going to need such a lengthy explanation that the clause really it doesn't mean anything at all? CFCF 💌 📧 01:38, 31 October 2015 (UTC)

The wording is ambiguous. It does not improve or clarify anything. QuackGuru (talk) 01:42, 31 October 2015 (UTC)

All this talk about country of origin is misguided because it does not matter what this guideline says—if there is a reason to consider country of origin when assessing the reliabity of a particular source for verification of particular text, the country will be considered. A guideline cannot prohibit common sense. Naturally anyone saying "that source has to be rejected because it came from China" can be ignored, but there may well be reason to be cautious about some types of research which a guideline cannot rule out. Johnuniq (talk) 02:36, 31 October 2015 (UTC)

I support the proposal above. For now, I have added in "country of origin" to the article since that much was clear from the consensus read. The remainder, in my opinion, should link to the full wording of the close since a summarized version will never be agreed upon. The above version, in linking to full wording, does just that. I also like the footnote for conclusions as well, and think it serves a valuable purpose here. LesVegas (talk) 16:15, 31 October 2015 (UTC)

Archival

This is insane. I asked what clarification question was given no answer, but got no reply. Archiving. --Elvey^(t•c) 02:37, 31 October 2015 (UTC)

Elvey - your archiving of this discussion was a major side-step and not in line with Wikipedia's consensus-building. If we are unable to discuss this properly as to elucidate what wording is most appropriate - the only alternative will be to throw out the consensus-reading and start anew. For starters we had a 4 Support v 11 Oppose close in favor of the minority position - this seems completely bizarre when you look at it. Either we start a new RfC or we send it to the appropriate venue for a close dispute. CFCF 💌 📧 11:07, 31 October 2015 (UTC)

Consensus reads are not simple vote tallies. The majority of those opposed were talking about low quality sources which are not allowed per WP:V or WP:RS anyway, hence the reading. LesVegas (talk) 16:08, 31 October 2015 (UTC)

This was a flawed close. There is no consensus - by any definition of the term - in favour of the "yes" position, yet Elyey finds consensus. Elvey says we can't overturn WP:V and WP:RS. Excluding acupuncture studies published in China would be following those policies. We're expected to use the best sources and there is a cloud over those studies. Relying on them would be doing a serious disservice to our readers. (As I said above, we need to reconsider the weight we give to industry-funded psychopharmacology studies, too, but that's probably for another discussion - but maybe not. Maybe now is the right time for that, since both controversies hinge on publication bias/cherry picking/salami slicing.)

I see Elvey has tried to close this discussion. That's inappropriate.

Is it appropriate for Elvey to be making non-admin closures while they are under editing sanctions? I'm not sure of the details, but Elvey mentioned them on Jimmy's talk page. [11] --Anthonyhcole (talk · contribs · email) 02:38, 1 November 2015 (UTC)

I don't think there's a rule about NACs while under sanctions, but I'd consider it a bad idea myself due to the necessary level of community trust. In this case there are actually connections with the RfC - Elvey's topic ban followed some highly acrimonious interactions with User:Jytdog (one of the editors !voting Oppose), and the t-ban was supported by several other editors who also !voted Oppose here. I read the close as likely being an attempted supervote, especially after their subsequent actions - joining the edit warring over the RfC result, telling editors questioning the close to "drop the stick" and other less complimentary things, and ultimately trying to archive this discussion. But either way, the close unfortunately perpetuated the dispute rather than resolving it. Sunrise (talk) 12:03, 1 November 2015 (UTC)

Yes. The close was incompetent and nonsensical. Given the closer's history, that may be the best gloss on it. Alexbrn (talk) 14:31, 1 November 2015 (UTC)

Elvey removed the above comments on the basis they were personal attacks. These are not personal attacks, but focus on the fault of the close. Elvey , your actions are very possibly a violation of your COI-topic ban. The discussion regarding Chinese sources covered conflict of interest on the part of the researchers, you should not be involving yourself in discussions surrounding COI at all. CFCF 💌 📧 23:28, 1 November 2015 (UTC)

RfC Appropriate version for the new clause

Possible phrasings required to support the inclusion of a clause surrounding country of origin are multiple and we have so far no consensus on what to use. For this reason I have listed the following versions as possible:

(Note: other wording is also possible, feel free to add o the end of this list)

1. Clarification
   Do not reject a high-quality study-type because of objections to: inclusion criteria, references, funding sources, country of origin or conclusions except when they explicitly impact the quality of the source.
2. Omission of the addition
   Do not reject a high-quality study-type because of personal objections to: inclusion criteria, references, funding sources, or conclusions.
3. Link to extended discussion
   Do not reject a high-quality study-type because of personal objections to: inclusion criteria, references, funding sources, country of origin or conclusions.
4. Alternate clarification
   Do not reject a high-quality study-type because of personal objections to: inclusion criteria, references, funding sources, country of origin, or conclusions. However, you should consider these factors if reliable sources have specifically linked them to systematic problems in the medical literature.
5. Another alternate clarification
   Where reliable sources have identified systematic problems in the medical literature associated with specific regions or countries it may impact the quality of the source. (Without including <ref></ref> tags.)

References

• Support 1 or 2 - Oppose 3 Adding the link will result in noone reading the content and the entire sentence losing its meaning with new time-consuming debates blossoming. My reading, and I think the only sensible one is that the RfC overwhelmingly supported not including the statement on the basis that is was a hypothetical situation that had never occurred. CFCF 💌 📧 11:21, 1 November 2015 (UTC)

Also considering the difficulty of understanding the close as it is we can count on that the addition of a link will only confuse future readers. CFCF 💌 📧 12:58, 1 November 2015 (UTC)

• Comment This seeks, in less than a month, to undo a RFC by removing the consensus of the last RFC. AlbinoFerret 13:42, 1 November 2015 (UTC)
  • No, it does not. There are multiple alternatives and including the majority supported position expressed in the previous RfC is only proportionate. CFCF 💌 📧 14:01, 1 November 2015 (UTC) 
  • If I'd been asked, I would have recommended waiting a few weeks, so that this page isn't quite so busy with other discussions. But there is no actual rule against starting a second RFC the day after the first expires, or even before then.
    Actually, if I'd remembered that the proximate cause of Elvey's topic ban was a dispute with multiple participants (Jytdog, Doc James, and Alexbrn were all mentioned in the original AN complaint, and all opposed this change) and involved articles about drugs and medical devices, then I might have suggested following the directions at Wikipedia:Closing discussions#Challenging other closures to have the original close formally overturned and the discussion re-closed by someone else (ideally, an admin. NACs are discouraged from taking on contentious closes like this one). It rarely looks good when you close an RFC about a guideline in favor of a position held by only ~30% of participants; when you are also closing it against multiple people who were involved in getting you topic-banned, then it's even worse. The "smell test" matters for closers. WhatamIdoing (talk) 15:45, 1 November 2015 (UTC)

Yes the smell test may matter. But part of a closers job is to discount off topic comments. It is reasonable to discount comments about low quality sorces in a discussion that from the start was about high quality sources. AlbinoFerret 17:02, 5 November 2015 (UTC)

Since the sentence in question is about sources of all quality levels that use high-quality types of study designs, I disagree that complaints about low-quality meta-analyses are "off topic". If the NAC didn't understand the difference between "top of the pyramid vs bottom of the pyramid, as determined solely by the pyramids and ignoring the entire rest of this page and all sourcing policies" and "high-quality source, as defined not only by those evidence-oriented pyramids but also after taking into account all the other factors on all the sourcing policies and guidelines", then the NAC certainly had no business rendering a decision here. WhatamIdoing (talk) 06:25, 17 November 2015 (UTC)

I am curious, this RfC is titled "appropriate version for the new clause" but I am at a loss as to how any option but number 3 is actually a version of the new clause. Options like number 2 actually appear to be merely upset editors gaming against the previous close. Can anyone tell me where I'm wrong? LesVegas (talk) 14:12, 2 December 2015 (UTC)

• Support 3 only, and comment This RfC isn't valid and goes against proper procedure. It was generated by an editor who disrespects both consensus and policy and has started an illegitimate RfC to justify edit warring and disruption. Many options here don't actually address how to implement the wording from the previous close. Option 3 is the only one that addresses what the previous close said. Option 1, while it does put "country of origin" on equal footing as "funding sources" (which the previous close addressed), the wording "except where it explicitly impacts the quality of the source" was never once addressed in the RfC close. I wouldn't be opposed to it in principle, but it is frankly necessary to do an entire RfC on that alone to address how it could be gamed, and that's not the scope of this RfC. Option 4 is a similar issue. All other options (2, and 5) aren't valid whatsoever because they don't address implementation of the wording "country of origin" from the previous RfC close, but rather get rid of it entirely. LesVegas (talk) 17:42, 1 November 2015 (UTC)

We've been through multiple iterations trying to apply the result of the close, none of which has been stable or consensus-driven. An RfC could settle this matter, and this one also includes the ability to add other potential wordings. The only reason you would dislike this RfC is if you feared that it would undue the previous close reading, which really shouldn't be a problem if you have real consensus. The other venue available is of course a close review, and seeing as there is little visibility for this RfC that may be more appropriate. CFCF 💌 📧 18:25, 1 November 2015 (UTC)

No, you've been through multiple iterations. The previous RfC settled it on wording suggested in that RfC. I advised you that if you had a problem with it you could formally challenge it. Edit warring and starting another RfC isn't the way to go about this, and looks especially bad for you since there's an open ANI case over this very behavior. LesVegas (talk) 23:53, 1 November 2015 (UTC)

In all the examples that you gave in support of the previous RfC, concern about the country of origin was clearly justified. Hence you have not established why the change of language was necessary in the first place. Furthermore the close of the previous RfC was questionable as it used a circular argument for its justification. In a nutshell, it stated that the RfC only applied to high quality sources, but country of origin can be a legitimate consideration in determining whether the source is high quality. Finally the closure did not specify the exact language that should be implemented. This RfC simply tries to establish what language should be used. Boghog (talk) 06:30, 2 November 2015 (UTC)

• Support 1 and Oppose 3 As pointed out during the previous RfC, there are well documented cases of culture specific bias in biomedical publications and taking into account the source's country of origin in these cases is clearly justified. The problem with the current language is that it can be misused to argue that these biases should not be taken into account. The clarification wording removes the potential for abuse. Boghog (talk) 19:58, 1 November 2015 (UTC)

• Comment - One thing that seems to be missing in all the options presented so far is the issue context. Reliability often depends on context... when we examine a source, we always need to examine the exact wording of the information being supported by that source. Is the source being used to verify a fact? Is it being used to verify an opinion? A source that is unreliable in one context might be quite reliable in another context. If we apply this to the issue of "country of origin", it will mean that sources from a particular country may well be unreliable in one context (for example, when stating something as a blunt medical fact)... and yet quite reliable in another context (such as explaining a cultural opinion on something medically related). Blueboar (talk) 12:53, 2 November 2015 (UTC)
  • I'm sure you won't be surprised to hear that the whole sentence is being taken out of context. ;-)  This is in a section on assessing evidence quality (e.g., a meta-analysis of multiple randomized controlled trials is better evidence than one randomized controlled trial; one randomized controlled trial is better evidence than a case study). A few years ago, we had a problem with some editors "assessing evidence quality" by personally scrutinizing the methods, and then rejecting any source that included things they disapproved of. So we would end up with editors refusing to use (or to permit others to use) apparently excellent meta-analyses, and demanding that primary sources be preferred to secondary sources, because they "assessed the evidence quality" and decided that the meta-analysis was "low-quality evidence" and that the primary source was "high-quality evidence", because the meta-analysis used studies with the wrong inclusion criteria, or because the primary source was funded by an activist group instead of an industry group, or whatever their hobby horse was. So it's not actually meant to discuss article content at all; it's meant to help you screen sources so that you can write a better (i.e., more representative of biomedical reality) article. It's really about helping you find what's WP:DUE instead of what's WP:V. WhatamIdoing (talk) 03:23, 3 November 2015 (UTC)
  • The context of the original RFC is quite clear: LesVegas wants to include questionable sources that misrepresent the effectiveness of acupuncture, and wishes to rewrite policies and guidelines to help him justify doing so.—Kww(talk) 17:16, 3 November 2015 (UTC)
• Support 5: Where reliable sources have identified systematic problems in the medical literature associated with specific regions or countries it may impact the quality of the source is positive guidance that addresses the issue at hand.—Kww(talk) 17:16, 3 November 2015 (UTC)
• Support 5: per Kww. This is the clearest version if we really need further clarification. However, I note that none of the possibilities exclude us from using well-referenced concerns about systematic pseudoscience in defined areas, so they all strike me as acceptable. Version 4 is fine too. As nobody is seriously proposing that personal objections are an acceptable basis for an encyclopedia (LesVegas's interlocutor notwithstanding) the mention of "personal objections" in 2 and 3 strikes me as not required. Richard Keatinge (talk) 13:10, 12 November 2015 (UTC)
• Support 1 or 5. Oppose 3. It is clear that this particular change to this guideline was started to win an underlying content dispute. I agree that in general these criteria as personal opinions are dubious as reasons to apply, but when independent sources identify them as problematic, we need to follow those source as well. Yobol (talk) 01:35, 13 November 2015 (UTC)

I noted that you, along with CFCF, opposed the previous RfC. Option 5 goes entirely against it, and this RfC is actually about how to word the inclusion of "country of origin" from the previous close, not undermine it, as many options, including option 5, do. LesVegas (talk) 16:44, 2 December 2015 (UTC)

Your point being? CFCF 💌 📧 23:26, 2 December 2015 (UTC)

• In my humble opinion 5 shouldn't even be an option here. This request for comment is about implementing certain wording, not about getting rid of it. 166.170.47.193 (talk) 15:07, 15 November 2015 (UTC)
• Start from scratch, using the information we gained in this dispute about how some editors misunderstand this. I think that we need to re-write the entire sentence, because it's clear that the current language is too subtle. A couple of editors keep confusing "high-quality type" and "high-quality source". So how about we scrap the (demonstrably) confusing "high-quality type" language and say something like this?
  "Assessing evidence quality" means editors should determine quality of the type of study, and where that type falls on these evidence pyramid charts. Editors should not perform detailed academic peer review. Do not reject a source that is high on the evidence pyramid in favor of one that is lower on the evidence pyramid because of personal objections to: inclusion criteria, references, funding sources, or conclusions. This factor must be considered alongside other criteria when evaluating a source. It is possible for a source to rank high on the evidence pyramid and still be a low-quality source over all, e.g., because it is outdated or published by a disreputable source.
  I've underlined the bits that are new. I grant that it's not going to win any awards for brilliant prose, but I'm not planning to submit it for FA. The paragraph is about two inches below the two large, rainbow-colored pyramid diagrams about evidence. Can anyone read what I've written here and see ways for it to be misunderstood or misinterpreted? WhatamIdoing (talk) 06:50, 17 November 2015 (UTC)

Good suggestion WhatamIdoing. Its main problem is that it won't help anyone win their content dispute. Richard Keatinge (talk) 22:06, 17 November 2015 (UTC)

I support changing high quality type to high quality source, especially because this gives a stronger barrier to entry of all of our sources. However, I do not support omitting "country of origin" for two reasons: 1) it goes against the previous close, 2) it allows "funding sources" (which in some fields have shown major issues with bias) but disallows country of origin for same, albeit lesser, reasons. Why would we allow funding sources into our statement but not country of origin? LesVegas (talk) 14:05, 18 November 2015 (UTC)

Since the whole section is about types of studies, then why would we change it to talk about individual studies? A source can be the very best type of study and still be unreliable due to factors unrelated to evidence quality. WhatamIdoing (talk) 23:27, 21 November 2015 (UTC)

• Support 5. Reject all other proposals. Bias sources may be rejected where they is a known problem. WhatamIdoing, your proposal is too confusing for me. QuackGuru (talk) 05:22, 19 November 2015 (UTC)
  • It would be very helpful to me to know which of the proposed changes is most confusing to you. WhatamIdoing (talk) 23:27, 21 November 2015 (UTC)
    • Number 4 is most confusing (ambiguous). Your proposal is too complicated. QuackGuru (talk) 00:48, 22 November 2015 (UTC)
• Support 3 and second 1. Country of origin should not be a factor in disqualifying high quality sources. AlbinoFerret 14:16, 4 December 2015 (UTC)
• Support 3 only The other options are really invalid options since the content of these was not addressed during the previous RfC close, plus this RfC is about how exactly to word the previous close's decision (something the previous RfC also tackled, but certain tendentious editors are ignoring that fact.) When the previous RfC is a simple question asking, should we change MEDRS from:

Do not reject a high-quality type of study due to personal objections to the study's inclusion criteria, references, funding sources, or conclusions.

to

Do not reject a high-quality type of study due to personal objections to the study's inclusion criteria, references, funding sources, country of origin, or conclusions.

The answer to that simple question was Yes, so this RfC about what version to implement wasn't needed. But CFCF, who objected to the original RfC, decided to game the system and create an RfC that seeks to overturn the previous close, while cloaking this RfC as "which version of the new clause" do we use? This RfC is nothing more than an elaborate WP:GAME by an editor who objected without basis, was refuted, attempted to edit war his version into place, and hopes to have a version that circumvents due process.

Options 2 and 5 are particularly awful since they don't even contain the words "country of origin" whatsoever, something the prior close was adamant was necessary. I will also note that the OP, CFCF, stated this was a hypothetical situation that had never occurred., yet he continues to ignore the fact that, yes, it had occurred multiple times1 [2 3, all of which were outlined in the previous RfC, and whose close he edit warred and disruptively disputed, or else we wouldn't even be talking about this right now. LesVegas (talk) 04:43, 9 December 2015 (UTC)

• Support 3 Primarily, I am still in support of the previous close. If the community considers the best, however, to choose an alternative way for implementing it, I am in support of the alternative number 3. I'd like to point out that whereas number 3 is the only one that is actually searching for a way to implement the new clause, it isn't supported even by the original poster himself. The other alternatives do not really fit how the very RfC is formatted.
  • Brevity is the source of wit, so three points in regards to the RfC at hand:
    • 1. Existing RfC closure[12], and the conclusion was yes to include "the country of origin" to the text.
    • 2. The new RfC started by the very same person who objected the first one, but who wasn't pleased with the outcome. In my opinion, this seems gaming against the previous closure. Meanwhile, we have the previous closure in force.
    • 3. The RfC is misleadingly named "Appropriate version for the new clause", but the options offered by the original poster don't seem to be versions of the new clause at all, i.e. they do omit the part "country of origin" completely. Therefore, this RfC has nothing to do with implementing the new clause, but getting rid of it totally. Not just seems this RfC to be poorly formatted, but it is opened by the very same editor that objected the previous RfC and who now - unsatisfied by the outcome - seems to game against it.
• Jayaguru-Shishya (talk) 23:30, 2 January 2016 (UTC)
  • 3 is supporting the use of sources that violates MEDRS. Poorly sourced text from a country that has been identified to have systematic problems in the medical literature is against MEDRS. Why are editors supporting the use of sources that violate MEDRS. Adding a source to an article from a country that is known to be bias in the medical literature can be inappropriate. Trying to rewrite MEDRS to allow lower quality sources is inappropriate. QuackGuru (talk) 19:32, 4 January 2016 (UTC)

The discussion above is closed. Please do not modify it. Subsequent comments should be made on the appropriate discussion page. No further edits should be made to this discussion.

Medical and health-related content

I think we can clarify the wording in the text. It is more than medical content. QuackGuru (talk) 21:40, 4 January 2016 (UTC)

We just had an RFC that came to consensus that it is only Biomedical content. AlbinoFerret 21:43, 4 January 2016 (UTC)

(edit conflict)With respect, there has just been a lengthy RfC on exactly this question[13], which has closed to the effect that "health" is too broad. "Health-related" would seem to be a strict superset of this. - Ryk72 ^'c.s.n.s.' 21:46, 4 January 2016 (UTC)

That was only for text in the box. QuackGuru (talk) 21:48, 4 January 2016 (UTC)

That is incorrect, the edit that started the RFC can be found here.[14] the box edit was later. AlbinoFerret 21:51, 4 January 2016 (UTC)

With respect, I cannot concur that the question In the lead should we use "biomedical and health information" or "biomedical information"? is limited to the nutshell box. Similarly, on review of the discussion in the RfC, I cannot concur that the understanding of the participants in that discussion is demonstrably that it was limited in such a way. - Ryk72 ^'c.s.n.s.' 21:54, 4 January 2016 (UTC)

QuackGuru, I would be careful if I were you. You have a history of disrupting and specifically going against consensus readings you disagree with. LesVegas (talk) 15:18, 5 January 2016 (UTC)