Jump to content

Wikipedia talk:Identifying reliable sources (medicine)/Archive 10

Page contents not supported in other languages.
From Wikipedia, the free encyclopedia
Archive 5←Archive 8Archive 9Archive 10Archive 11Archive 12→Archive 15

Mayo: how did this happen ?

The guideline currently says:[1]

Consequently, they are usually poor sources and should always be used with caution, never used to support surprising claims, and carefully identified in the text as preliminary work. Peer-reviewed medical information resources such as WebMD, UpToDate, Mayo Clinic, and eMedicine are usually acceptable sources in themselves, and can be useful guides about the relevant medical literature and how much weight to give different sources; however, as much as possible Wikipedia articles should cite the more established literature directly.

Mayo Clinic was inserted with this edit based on no discussion that I am able to find. I did find this discussion a few days before the editor inserted the text about WEBMD, Mayo, etc. And I found this discussion about that section of MEDRS. We have had multiple conversations over the years, at WT:MED, here and on article talk pages, about Mayo, particularly in the psych realm. Why are we elevating one hospital's website above any and all others?

I can find no consensus for the insertion of Mayo in MEDRS, do not recall how or why it happened (in fact, did not realize it had happened), understand there were times in the past when discussions here weren't attended by large numbers of editors, and I am wondering how this happened and why it has stood. There is not often need or reason to be using Mayo information, and particularly not in the psych realm. As far as I can tell (unless someone else can produce another discussion), this text was inserted without discussion or consensus.

Also, is our wording clear enough, since in at least one discussion, editors are interpreting our caution as if Mayo is a fine source, on par with better sources available (or at best, failing to read the entire passage).[2] Even if we were to merge a few sentences to make the caution more clear, new editors may be more likely to digest the whole thing. We seem to be leaving the impression that borderline sources are on par with the many and much better sources available. Mayo does a few things right in a few areas, but they sure aren't internationally or nationally recognized in the psych realm, so why are we elevating them above other hospital websites? SandyGeorgia (Talk) 16:05, 20 November 2014 (UTC)

I agree with removing Mayo. Jytdog (talk) 16:17, 20 November 2014 (UTC)
I also agree with removing Mayo. The content on their website is of decidedly varied quality. - - MrBill3 (talk) 17:36, 20 November 2014 (UTC)
Also agree, and will remove as their appears to be a consensus. Yobol (talk) 17:45, 20 November 2014 (UTC)
All of those also have the issue of not have a static copy. I typically use none of the 4. Doc James (talk · contribs · email) 20:27, 20 November 2014 (UTC)
I think that it would be interesting to consider changing this sentence to say that these websites are often acceptable for basic, non-controversial information. I support giving users a source or two that they can read and understand, even if that means not "cit[ing] the more established literature directly" as often as possible. I'm also dubious about this statement that WebMD and the Mayo Clinic website can tell you what weight to give different sources. WhatamIdoing (talk) 04:35, 26 November 2014 (UTC)
As Mayo is gone, regarding the other three, I am not so sure about "usually acceptable sources" perhaps "often" or "sometimes". I share WhatamIdoing's concern about using them for weighting sources and Doc James concern about the non static nature. - - MrBill3 (talk) 08:50, 26 November 2014 (UTC)
"... I support giving users a source or two that they can read and understand, ..." Huh ? Anyone who is reading Wikipedia is on the internet already, and Google will cough up sources like Mayo for them if that's what they need. Shouldn't OUR Wikipedia articles be something they "can read and understand"? This argument makes no sense to me; it amounts to saying we encourage readers to link out because our articles may be crap. SandyGeorgia (Talk) 09:55, 26 November 2014 (UTC)
"sometimes acceptable for non controversial material" maybe Doc James (talk · contribs · email) 10:38, 26 November 2014 (UTC)
Agree we should not be using Mayo and WebMD and agree with their removal. Cas Liber (talk · contribs) 06:09, 29 November 2014 (UTC)
Sandy, I want to provide good lay-accessible sources to people who want to read more than what's in the Wikipedia article (especially if you didn't write that Wikipedia article yourself  ;-), and for students who are under orders not to cite Wikipedia. To put it another way, if you ever find a good, lay-accessible source about Tourette's, then I want them reading the one that you think is best, and not whichever one appears at the top of the search engine results. WhatamIdoing (talk) 13:11, 1 December 2014 (UTC)
I agree with WhatamIdoing and we just discussed this topic in the Other sources section above. I don't see that we need to prohibit decent or good laysources. Being overly strict in that way is not hugely beneficial, or always beneficial. Like I stated there, "[Do] I sometimes use WebMD sources on Wikipedia? I indeed do, usually as an interim for scholarly sources and/or as an adjunct to scholarly sources. Using a laysource in combination with a scholarly source has repeatedly proven beneficial for readers at various medical articles, or specifically at anatomy articles, especially in a WP:PAYWALL case. That's also why Template:Citation Style documentation/lay exists. WP:Reliable sources that accurately explain medical matters in plain English are always beneficial." But if WP:MEDRS editors form a WP:Consensus to be overly strict on this matter, there's not much I can do about it but adhere to that WP:Consensus. Flyer22 (talk) 13:32, 1 December 2014 (UTC)
Also, regarding SandyGeorgia's "16:05, 20 November 2014 (UTC)" post that cites a short discussion I was a part of, I was never under the impression that "borderline sources are on par with the many and much better sources available"; furthermore, the WP:MEDRS guideline has always been clear that they are not. My comprehension skills have always worked in that regard. And I know that WhatamIdoing, who was also involved in that short discussion, has always had comprehension skills that work in that regard. As for Mayo Clinic, I saw its listing in the guideline as an example, not a matter of WP:MEDRS stating that it is a hospital better than all the other hospitals when it comes to a laysource. The guideline did, and still does, state "such as" (meaning "for example"), after all. Flyer22 (talk) 14:03, 1 December 2014 (UTC)

Consumer reports

There is a discussion here about the use of this report by Consumer Reports [3]

It states "Our evaluation is based on an independent scientific review of the evidence on the effectiveness, safety, and adverse effects of opioids. A team of physicians and researchers at the Oregon Health & Science University Evidence-based Practice Center conducted the analysis as part of the Drug Effectiveness Review Project, or DERP."

Peoples thoughts? Doc James (talk · contribs · email) 10:55, 30 November 2014 (UTC)

Commented. The authorship is unclear from the document, which I find unsettling. JFW | T@lk 14:43, 30 November 2014 (UTC)
i just gotta say, the Drug Effectiveness Review Project is a good thing but the acronym is infelicitous :) Jytdog (talk) 10:11, 1 December 2014 (UTC)
I work for Consumer Reports, I used that reference, and I have insight into this. Most of what I do at Consumer Reports is unrelated to these classes of papers; I have used papers like this in 10-15 articles only, and almost always it was to add safety information on the side effects of drugs. There are several issues related to MEDRS here, but the one that was raised is "does it meet MEDRS?" As Doc James, Jfdwolff, and Formerly 98 have noticed, the publication is not a peer reviewed report nor is it obviously connected to one. When I added this about a year ago I had doubts and I still do, but I think this paper meets RS. The situation is that Consumer Reports published a layman report based on a market research paper which I think meets RS. Here is a citation to that market research paper:
  • Carson, Susan; Chou, Roger; Thakurta, Sujata; Low, Allison; Smith, Beth (July 2011). McDonagh, Marian; Helfand, Mark; Drug Effectiveness Review Project; Oregon Evidence - based Practice Center (eds.). Drug Class Review Long - Acting Opioid Analgesics (Final Update 6 Report ed.). Oregon Health & Science University. Archived from the original on 1 December 2014. Retrieved 1 December 2014. {{cite book}}: Unknown parameter |editorlink3= ignored (|editor-link3= suggested) (help)
The introduction to this paper says "To identify published studies, we searched MEDLINE, Cochrane Central Register of Controlled Trials, Cochrane Database of Systematic Reviews, Database of Abstracts of Reviews of Effects, and reference lists of included studies. We also searched the US Food and Drug Administration Center for Drug Evaluation and Research website for additional unpublished data and solicited dossiers of information from pharmaceutical manufacturers." Reading that, and considering the apparent quality of the paper and its source, I felt that it met MEDRS because it is a review (maybe a systematic review), very conservative, from a reputable source, and is used by state governments in the United States to guide policy.
The unusual aspects of this paper include that it is not distributed through academic journals, costs perhaps tens of thousands of dollars to read, has almost an exclusive governmental and institutional audience, and has a research focus which is at odds with academic medicine - "How to make recommendations for medicine when people have to choose based on cost".
When I used these papers mostly it was to talk about the side effects of classes of drugs. In my opinion, layman-targeted papers like those presented by Consumer Reports are more sensitive to the readership desires of consumers with regard to side effects than are academic medical papers, but also I was paid to cite this particular paper. When I went to the opioid article, I changed a lot throughout the article and as I recall, this was to sort the side effects.
In my opinion, there is an issue to be addressed in this case then a broader issue. The local issue is whether this privately published DERP report meets MEDRS. In my opinion, it does, because it hardly does anything except report side effects from other MEDRS sources and match them to market prices. The broader issue is deciding the circumstances under which layman health publications should be cited on Wikipedia when there are other more technical papers behind them which are the ultimate source of information. For partnerships, organizations like Consumer Reports (and WHO, MSF, medical specialty societies) need clarity. In my opinion, having multiple or bundled citations is a workable solution, wherein both the MEDRS source and a complementary MEDRS-derived layman-targeted source are both cited.
Does anyone have comments on this particular source and the others like it? Blue Rasberry (talk) 15:47, 1 December 2014 (UTC)
Bluerasberry, I agree that it could serve a secondary source but the authorship (in terms of accountability and traceability) is important so thanks for clarifying. Generally, a source like this should be used side-by-side by professional guidelines. JFW | T@lk 21:29, 1 December 2014 (UTC)
Jfdwolff Yes I confirm. If sources like this are used then they should be complements to a professional guideline which establishes MEDRS. Blue Rasberry (talk) 22:54, 1 December 2014 (UTC)
I worry we're going to get more stuff like ProPublica's "expose" of Tylenol, which quoted real MEDRS sources but linked them together with enough original synthesis to conclude that everyone should take NSAIDs instead, which conclusion an FDA working group explicitly rejected due to increased risk of gastric bleeding and MI. Formerly 98 (talk) 21:42, 1 December 2014 (UTC)
Formerly 98 I do not think the ProPublica paper could be confused with a health education publication. Here is that source.
This is journalism about a safety fear which the organization felt was not sufficiently recognized in health practice. This kind of journalism has a place in Wikipedia articles, but in society and culture sections and not in health information sections. It is not a MEDRS source nor does it purport to be a layman summary of any MEDRS source. In the case of Consumer Reports and most other community health organizations, the goal for health education outreach is to translate technical documents into plain English without changing the meaning or doing WP:SYNTHESIS of them. These organizations try to do what Wikipedia does - make the information accessible to layman audiences and base their publications on MEDRS sources.
You are right to worry generally, especially until it becomes more clear how health organizations should present themselves and their resources to Wikipedia. There are a lot of health organizations with major international influence but which have lower standards than Wikipedia. Blue Rasberry (talk) 22:53, 1 December 2014 (UTC)
Bluerasberry, I argued for keeping that ConsumerReportsHealth Opiods report.
My main reason was that it was written clearly enough for a layman to understand it, but covered content with the sophistication of a clinical article in a peer-reviewed journal.
Frankly, it's easier to read and understand than most Wikipedia articles. And it's probably more thoroughly fact-checked. I assume that the final version was reviewed by one or more doctors in the specialty.
I think the concern of Doc James, JFW and others is that they want the medical profession to accept Wikipedia as reliable, and that won't happen if we support our articles with material that wouldn't be accepted in a major peer-reviewed journal. I agree.
My concern is that I want Wikipedia to be readable and understandable by non-professional readers too. That's the mandate of WP:MOS and WP:NOTJOURNAL. I want to satisfy both audiences, and there are many publications that do that. Professionals and laymen can both read a Science magazine News & Comments article and get professional-level information. I also think a non-professional reader should be able to follow up WP citations for further information. Sources like The Lancet are not freely available, and often non-professional readers can't understand them. Sources like Consumer Reports or the New York Times are often free on the Internet, and educated non-professional readers can understand them.
I think the way to do it is to provide both. For the Opioid article, for example, I would cite the Annals of Internal Medicine article, and I would also cite the Consumer Reports article.
I think it's a good practice to add a popular source to a journal source if we see that the popular source is accurate, and it's supported by the journal sources. Gary Schwitzer, who is cited in WP:MEDMOS or WP:MEDRS, has published peer reviews of newspapers and other mass media, and he says that some of them are accurate and some aren't. You could challenge a popular source because it's inaccurate, but I don't think you should challenge an otherwise accurate article simply because it's a newspaper or popular source per se. I think that's the usual practice in Wikipedia, but it's never been clarified or reduced to a recommendation. So some editors arbitrarily delete even accurate news stories from reliable sources like Larry Altman from the New York Times.
I think we have a consensus that it's acceptable to add an accurate popular source to a journal source as a citation. Do we all agree?
I think we have a consensus that it's acceptable to add something like the ConsumerReportsHealth Opiods report as an External Link. Do we all agree? --Nbauman (talk) 04:47, 2 December 2014 (UTC)
I am happy with this as an external link and maybe to support some no controversial material.
It is not ideal but okay as it has had some peer review. I am not happy with the popular press such as the NYTs, even though it is a great paper it is not suitable for medical content. Doc James (talk · contribs · email) 05:13, 2 December 2014 (UTC)
How do you feel about adding a NYT story as a lay summary in addition to the journal source? We would only use a NYT story if it was accurate, by following for example Gary Schwitzer's checklist. --Nbauman (talk) 06:44, 2 December 2014 (UTC)
How do we know a NYTs story is accurate unless we have a good source to back it up? The popular press typically reports on primary sources rather than reviews and guidelines. Doc James (talk · contribs · email) 07:03, 2 December 2014 (UTC)
What I'm recommending is that we cite the peer-reviewed journal source, and also include the NYT story about the journal article. The journal article backs up the NYT story.
We read the journal article, read the NYT story, and decide whether the NYT story is accurate. WP:RS says that we can use newspapers with "caution." We don't automatically accept or reject the NYT story. We have to use judgment. For that matter, we have to use judgment in selecting peer-reviewed journal articles. You can't reduce medical writing to a set of simple rules that anyone can follow without understanding the underlying principles (which journals assume their readers understand). For example, journals will report on a secondary outcome, and assume their readers understand the difference between a primary and secondary outcome. Many Wikipedia editors don't understand that. A good newspaper story, as we often find in the NYT, will explain the difference between a primary and secondary outcome. (The classic example of people confusing primary and secondary outcomes is Avastin.)
The NYT does report on primary sources and guidelines. Do a Google search for "source:nytimes.com guidelines". For example: http://www.nytimes.com/2013/12/19/health/blood-pressure-guidelines-can-be-loosened-panel-says.html (hypertension guidelines) http://www.nytimes.com/2013/11/14/opinion/dont-give-more-patients-statins.html (statin guidelines).
The benefit of a NYT story (about guidelines, for example) is that, when the story is well done, they will describe not only the current guideline, but also the other guidelines and how they differ. I'll search PubMed or guideline.gov, but that doesn't always work. What do I do with 500 citations? Try searching guideline.gov for "opioid", "statin" or "hypertension" and see what you get. PubMed and guideline.gov are useful sources for people who already know the answer. A NYT story is usually written by someone (like Kolata) who has been covering the subject for years, has interviewed the main researchers many times, has attended the meetings where the research was first presented before it was published in the journal, already knows the answer, and understands what the issues and disagreements are.
For "NYT", I could substitute the Wall Street Journal, Reuters, MedPage Today, and a lot of other news media. Schwitzer and others have used rigorous peer-reviewed methods to analyze the news media, and they've given guidelines and checklists to tell whether a news story is reliable or not. Those medical reporters and their editors know Schwitzer and Goldacre, they're on panels together, and they work together.
Does that answer your objections? --Nbauman (talk) 18:44, 2 December 2014 (UTC)
If the NYTs covers a high quality secondary source such as in the case you mention than sure we could use both with the NYTs being the layurl.Doc James (talk · contribs · email) 19:05, 2 December 2014 (UTC)

What are we up to, in Project Medicine?

I just wanted to open a discussion focused on this. What are aiming for? Are we aiming to present the best evidence, or to present the consensus? A specific question - when something like a Cochrane review contradicts recent treatment guidelines, how should we handle that? As I have written above, I fear that to a certain extent, we have gone off-mission (!) in advocating for evidence-based medicine. Jytdog (talk) 16:14, 30 November 2014 (UTC)

We are looking to stick to scientific consensus, which may change over time (and often do). A Cochrane review carries a lot of weight. If it challenges a prevailing consensus, we update our articles to take the new findings into account. -A1candidate (talk) 17:15, 30 November 2014 (UTC)
I think we are looking at the professional rather than the scientific consensus (not always fully aligned). I think we owe enough kudos to EBM to resolve as a WikiProject that if Cochrane contradicts established practice, this should still be included with the specific caveat that this is a matter of dispute. JFW | T@lk 19:49, 30 November 2014 (UTC)
We are aiming to be a high quality useful source of medical information based on high quality sources. What makes a source high quality depends on many factors including: when it was published, who published it, how independ it is from financial COI, etc.
We are not necessarily recording current practice which often differs from both EBM and guidelines (think antipsychotics in dementia or antibiotics in the common cold). Surveying of physicians asking them what they do represents a sort of consensus and the one often used to define standard of care in a legal sense. This may be appropriate in certain situations but is usually under society and culture.
"Consensus" is not a clearly defined term. Both review articles and practice guidelines represent a consensus of sorts. A good meta analysis represents the consensus of high quality evidence. Practice guidelines often do the same while looking at a broader scope of practice. Doc James (talk · contribs · email) 06:27, 1 December 2014 (UTC)
Doc James, can you point to a single treatment guideline that calls for giving antibiotics for the common cold? Ignorant, minority clinical practice is not what anybody here is talking about and that is just a red herring. In my view, per WP:NPOV, CC is prominent and if a CC review contradicts treatment guidelines we should mention it, but the WP:WEIGHT should be that of a significant minority and not the main thrust of the section. In articles where we flip that, we have gone very astray, in my view. Jytdog (talk) 10:06, 1 December 2014 (UTC)
"Very astray"? My position is not to give greater weight but equal weight. This whole discussion appears to be making a mountain out of a molehill. Rather than working together to deal with the much bigger issue that lots of our content is based on old or primary sources must we really continue to "attack" each other over this fairly minor difference in opinion over which of the high quality sources is really the highest of the high quality sources? Sure we disagree. Doc James (talk · contribs · email) 10:27, 1 December 2014 (UTC)
of course we agree on cleaning up bad articles. the question is about what is good, and it is important. Jytdog (talk) 12:19, 1 December 2014 (UTC)
and Doc James I really am talking to you. You are probably the most visible pubic presence for Project Medicine and one of our most prolific editors, and I really think your emphasis on EBM over (or equal to) professional consensus is astray. in my view it is important to all of us that you are flying true to WP's mission. Please think about that. Jytdog (talk) 14:20, 1 December 2014 (UTC)
Looking at the RfC above it appears our community is divided on this issue. Yes I am of the opinion that high quality meta analysis should be balanced with guidelines based on a number of factors that need to be judged on a case by case basis. I disagree with you that providing the best available evidence such as in the activated protein C example is "correcting a great wrong". I am not sure if you are proposing this but I would strong object to removing information about what Cochrane concluded wrt breast cancer screening.
I have said publicly that "if we use the best available sources we will end up with the best available content". I do not have as narrow a view regarding what counts as professional or scientific consensus. Doc James (talk · contribs · email) 05:27, 2 December 2014 (UTC)
I like JFW's comment about presenting professional consensus rather than scientific consensus. I think that we need to present:
  1. what the current professional consensus is (e.g., what NICE says you ought to do when faced with the common cold or stage 4 lung cancer),
  2. what actually happens to patients (e.g., antibiotics get prescribed for the common cold every hour of the day; cancer patients aren't offered palliative care), and
  3. what the evidence says about efficacy (if anything, because good evidence doesn't exist for thousands of diseases).
Unlike Doc James, I'd give them weight in that order: it is more important to lay people to know what the professional consensus is, than to know what the latest claim to be The Truth™ about evidence is, especially if your doctor is unaware of the latest word (which will probably be contradicted by the next study) or is deliberately ignoring it. WhatamIdoing (talk) 13:22, 1 December 2014 (UTC)
Hold on, let's keep perspective here.

I do not think that wanting to present a high-quality summary (where available) of the state of the evidence in appropriate weight alongside what authoritative practice guidelines say is in any way going off the track of the purpose of WP:MED or in any way at odds with the purpose of Wikipedia. Not at all.

Jytdog I don't even get from his comments that Doc James necessarily wants to give weight to discussion of evidence over and beyond what practice guidelines say, in all cases. What Doc James said was (typos and all), "A blanket statement for one over the other is not a good idea as these issues should be addressed on a case by case bases." Jytdog I still don't think you're connecting to the point I've been making that a high-quality review of the state of the evidence base isn't the same kind of animal as what practice guidelines say, and it's not an either/or proposition.

And WhatamIdoing I think that use of "The Truth™" is a bit unfair--that tongue-in-cheek idiom gets used here to call attention to those who want an article to say something because they think it's true and/or personally feel it's important, even though they do not have a reliable source to back it up. That's not the case here, nobody is saying that articles should say something about evidence even though we don't have a source--we're talking about cases where we do have a high-quality source covering the evidence base.

In the end I think we're probably going to leave the guideline as it is. I don't think the guideline should prescriptively state that one kind of content is always more important than the other, it depends on what sourcing is available and then we apply WP:NPOV and weight in the article accordingly. Zad68 15:03, 1 December 2014 (UTC)

I replied above, where I should have a while ago. sorry about that. Jytdog (talk) 15:25, 1 December 2014 (UTC)
Zad, We are also dealing with situations when we have good sources, plural, that cover the evidence base and coming to contradictory conclusions.
I agree that providing information about the current state of evidence "in appropriate weight alongside what authoritative practice guidelines say" is ideal. Unfortunately, I have too often seen well-sourced multi-paragraph sections that say "what this thing is" and "how it is used today, in the real world" replaced by a couple of sentences that say "there's no evidence for this" (or "the evidence says that X is better than Y", or whatever one review says). Zero space for actual practice and 100% space for evidence—usually, where "evidence" is defined as the conclusion of exactly one review paper—is not "in appropriate weight" IMO. To put it another way, I'm thinking a little less about the words on this page and a little more about what some editors are doing in the articles. I want to move out work slightly back away from the "evidence only" model and towards a holistic summary (actual practice + official practice guidelines + evidence). WhatamIdoing (talk) 10:46, 4 December 2014 (UTC)
Yes all three are important and they need to be dealt with with appropriate weight. I do not see the proposed wording as suggesting this. The previous wording dose more so. Doc James (talk · contribs · email) 15:18, 4 December 2014 (UTC)
Evidence is generally internationally applicable, but what actually happens is very variable internationally, and in our case will inevitably tend to give only a US perspective. For example I doubt that giving anti-biotics for the common cold is at all common in the UK. Wiki CRUK John (talk) 11:51, 5 December 2014 (UTC)
Yeah? What about giving antibiotics for two days of infectious acute bronchitis (agent unknown) in a previously healthy patient with a temp of 38 C, good oxygenation, and light green phlegm? We assume no rapid test modalities are positive. Suppose (statistically and absent other info) you have one meta-analysis that found it helps, but only A LITTLE? [4]. Now what? You might say of course it depends on further factors like age and smoking status and FEV1, and blah, blah. Well, do you even have the EMB to differentiate on this stuff based on THAT? Mostly, no. Okay, suppose your patient is a smoker with COPD. Do courses of antibiotics reduce bronchitic exacerbations in COPD? We think so. [5]. But I had to look it up. And there are lots of other cases where we have no idea. SBHarris 21:45, 5 December 2014 (UTC)

A MEDRS-quality and highly WP:FRINGE medical review...

I have absolutely no idea what to do about this review: PMID 24624096; does this journal typically publish fringe perspectives?

It ignores virtually all the evidence involving pharmacogenomics and behavioral epigenetics from these reviews even though its newer than both: PMID 24459410 (addiction in general) and PMID 21459101 (epigenetics of behavioral addictions and behavioral therapies).

In comparison, the newest review on addiction epigenetics, PMID 25083822, argues that the current evidence is so strong that it supports using the level of accumbal ΔFosB gene expression as an addiction biomarker as well as using the magnitude of accumbal ΔFosB induction by a drug as a measure of how "addictive" a drugs is. Seppi333 (Insert 2¢ | Maintained) 20:14, 5 December 2014 (UTC)

Its impact factor is zero [6]. Would advice against using it. Doc James (talk · contribs · email) 20:26, 5 December 2014 (UTC)
I believe we had a related conversation here awhile ago [7] just for reference. It seems like our only resort in these cases is to try to figure out if the journal was reputable, otherwise wait for other scholars to comment on the article basically being junk. Having a zero impact factor would exclude it as a MEDRS source for me, but I'm not sure how we'd deal with sources in more marginal journals that say have an impact factor around 1. It's still possible for junk to get through. Kingofaces43 (talk) 20:36, 5 December 2014 (UTC)
happily, that article itself makes it clear that the authors are not in the mainstream (see 4th paragraph of the intro). The book from which it has been taken has been widely discussed in the media - in some ways it has been a useful corrective (e.g. the hype of neurolaw, neuromarketing, etc) but it has been taken too far (see here for example). and they don't seem to completely fringe - they write "Moreover, although severe addictions are partly rooted in genetic predispositions that are themselves manifested in brain functioning, these conditions can be profitably understood at multiple levels of analysis (e.g., psychological, social, cultural) in addition to the neural level." They are not throwing out "brain" altogether.... seems easily handled as a "significant minority view" - but (and again by the authors' own statements) their views should not be used to compete with or debunk what is actually mainstream. Jytdog (talk) 22:26, 6 December 2014 (UTC)

What are we up to, indeed? MEDRS and SYNTH as cudgels in deletion debates

One thing I hope we’re not doing is making it harder to write an encyclopedia. The general criteria at WP:NOCITE is you remove uncited material stuff when it violates BLP, is clearly vandalism, or you don’t believe it. You put in a “citation needed” otherwise. I very much hope that MEDRS is not going to be used like BLP to make inclusion criteria stricter for biomedical article evidence.

But I think it is. As it happens, I was just in the middle of an unpleasant situation at Vachellia nilotica where one editor wanted to remove an entire paragraph about using this plant as chewsticks, and did. He was convinced chewsticks are always made of Neem (another species), whereas they are in fact made of half a dozen plants, including V. nilotica (see teeth cleaning twig, as well as the Vachellia nilotica article). In the struggle to get references for that fact into the article, I had MEDRS cited against me by two other editors, not only by the original editor (who really, really wanted to be right at any cost), but also by an administrator I asked to look at the problem. The final article is poorer for it. I didn’t really make any clinical claims in the first place—just said people use the stuff for a toothbrush or chewstick. A clinical claim would have been how well it worked, but would Wikipedia be poorer if we included primary source journals addressing that? Or is it poorer if we suppress them, as is happening now? The original article said that chewsticks are gathered and sold in villages. We don’t have a source for that, so out it goes. Do you all really believe it’s not true, since the sources we do have make it clear V. nilotica chewsticks are used throughout India, and a toothpaste flavored with the plant is used as well? You'd think I was directing people about how to do their own appendectomies.

And SYNTH as a weapon, also. While I’m here, let me say a few more words about WP:SYNTH. Anybody who looks hard at WP:NOR should be aware that NOR is not SYNTH (SYNTH is a part of the NOR policy). A companion to WP:SYNTH is WP:SYNTHNOT. This little essay points out that SYNTH is not a rigid rule and it does not cover summary and explanation. Further, it should shout out loud, since it was originally made up to keep WP from publishing original or crackpot theories whole, not to keep a one-line possibly true statement (with only primary research backing in a clinical question?) from going into WP. The WP:SYNTHNOT article asks if your material actually could be published as original research, or if a good journal that publishes OR would laugh at it, as a bit of physics or medicine that knowledgeable people have known since before you were born. If the last, it’s not OR, and if it’s not OR it doesn’t matter HOW MUCH SYNTH you do in explaining it in WP--- everybody in the field knows the view or theory or whatever already, and it’s not original. Your original explanation of a fact which itself is NOT original, is not “original research.” So I can replace Einstein’s SR trains with rocket-ships and light-sabers if I like. Synthesis of complicated material into simple explanations is what we do on Wikipedia. If we found a good simple explanation for a really wide topic already in print, it would probably be in another encyclopedia, and if we quoted enough of it, we’d be in severe danger of unfair use, even if we fairly cited it.

Take a look at [8]. SYNTH is not any type of synthesis. In 2004, Jimbo Wales said "In many cases, the distinction between original research and synthesis of published work will require thoughtful editorial judgment.” Indeed.

If you want to see a face-palm case of this, the same editor who wanted to delete the toothbrush section of V. (Acacia) nilotica, wanted to delete the entire article on Inert gas asphyxiation. Even now, he insists that it’s not a defined subject (under any name), and that the idea that "inert gas asphyxiation" might be assumed to be asphyxiation with a hypoxic asphyxiant gas, or chemically or physiologically inert gas, is just too... synthetic. Those of us in the medical field, dealing with respiratory physiology, know it’s a WP:N subject (whatever you choose to call it)--- but this guy is not in the field.

All of this makes WP not-fun to edit. Each time it happens, I go back to the chemistry articles, and you get less and less medicine and physiology editing out of me. That’s true of a lot of editors. If you make it a royal pain to edit here with more and more of all kind of rules, and if you tolerate debates like the ones above, you’re going to have a lot of (former) medical editors just say: “You don’t pay me enough for this.” SBHarris 22:56, 5 December 2014 (UTC)

Sbharris I wonder if discussing some folk or indigenous uses, a better way to approach some of these is to conisder they are not part of mainstream practice - so hence don't fall under the aegis of WP:MEDMOS unless the article discusses them as having an actual effect. Cas Liber (talk · contribs) 22:36, 6 December 2014 (UTC)
That's not a good approach if medical claims are made. I did look at this & while I sypathize with User:Sbharris (and one of the editors involved is quite the attack dog) there was a problem with tying the source to the right species etc, or seemed to be, and rather than just saying people used the things to clean their teeth, claims that this had oral hygiene advantages were made. If you wait long enough, the attack dog is likely to get locked in his kennel for a while again ... But yes it is very wearing. Wiki CRUK John (talk) 22:50, 6 December 2014 (UTC)

Ben Goldacre's editorial or the article he references may be worth adding as a reference to the popular press section.

RDBrown (talk) 02:35, 9 January 2015 (UTC)

As an example of a reliable source or an unreliable one? Formerly 98 (talk) 02:39, 9 January 2015 (UTC)

Standardizing the five-year rule

Newer health sources are presumed to be better than older ones when all other things are equal. I just changed a line from " Look for reviews published in the last five years or so, preferably in the last two or three years" to "Look for reviews published in the last five years or so, with newer being better". I wanted to remove the 2-3 notice because most medical information is not updated that frequently, and the idea to express is "newer is better" and not to convey worry about using a 5-year old source when a slightly newer one has not been identified.

The "5 year" rule comes from PubMed's own search function, which has built-in features to find papers which are published either within the last 5 or 10 years. These are arbitrary time limits, but I support the 5-year rule because PubMed supported it first. The 2-3 year rule probably originated in Wikipedia and I would like to make the 5-year rule more standard here. Blue Rasberry (talk) 16:28, 14 January 2015 (UTC)

I agree with taking out the emphasis on "two or three years." Flyer22 (talk) 16:33, 14 January 2015 (UTC)

I'm quite uncomfortable with this. There are many areas that are not currently very active areas of research, for which 1) there are very few recent meta analyses, and 2) there is very little recent data of importance. In these cases the one or two, possibly non-representative viewpoints that were published recently drawing from pretty much exactly the same evidence base as the older ones will become Truth and other viewpoints will be shut out.

On some topics this change will have the effect of specifically giving an unjustifiably outsized voice to Cochrane, as it is in the habit of republishing its reviews every several years if only to say "there is no new data to consider". Normally Cochrane is a great source, but we know on the flu vaccine and neurimamidase areas for example, they are well outside the mainstream of medical opinion.

I think rather than writing this overly broad restriction into stone, it is better if we continue to use common sense on a case by case basis. I don't see the use of obsolete reviews as a problem that we have not dealt with effectively in the past. Formerly 98 (talk) 16:56, 14 January 2015 (UTC)

Formerly 98 It sounds like you are not commenting on what I did, but rather on the implication that this further entrenches the idea that newer sources are better. I thought that the 5-year rule was standard already, and it was my intent to leave that part alone and just remove the distinction between 3 year old sources and 5 year old sources.
My motive in clearing this up was to give advice to new editors, most of whom are just being introduced to the idea that sources of information in journals can have varying quality. I wish that I could express whatever ought to be said about the age of journal articles for that beginner audience and avoid any pretense of taking a position with nuance.
I thought that it was accurate to say that when multiple sources exist, and they all seem to have been produced with comparable quality, then it is better to cite the newer one, especially if no one has any other basis for preference for one source over the others. I presume you agree with this much, right? If so, can you imagine a way in which I can express that while leaving room for anyone with a reason to do so can still choose to cite an older source?
How do you feel generally about the "five year" rule? What is your sense of how often medical information is generally revised and updated? 3 years? 5? 10? 25? Is it worthwhile to have a general rule, and encourage diligence to check for anything more up to date if for example someone cites a 15-year old source?
You say use "common sense on a case by case basis". The conclusions found with common sense vary a lot here. I do not feel strongly about what age makes a paper likely to be outdated, but I do feel strongly about saying something quantified. Blue Rasberry (talk) 18:33, 14 January 2015 (UTC)
Speaking to the general point, I'm uncomfortable with asserting that there is something so strong as a "five-year rule" with the associated (implied) rigid attitude; what we really have is a "five-year rule of thumb". It's one of several factors to be considered in evaluating the robustness and reliability of a source. A review published in January 2010 isn't magically 'good' this week and magically 'expired' a couple of weeks from now. I find that whenever we offer any contributing criterion as a proxy for publication quality that can possibly be converted into a binary test, we end up with the aggressive application of it in exactly that way, without nuance or balance.
I guess I get stuck on any rule that declares something to be absolutely true when "all other things are equal". All other things are never exactly equal (except, perhaps, in the aforementioned cases like Cochrane, where one version to the next may only differ in the publication date). I fear that we may find ourselves discarding high-quality, high-impact, seminal reviews by highly-regarded authors as they 'expire', to be replaced by low-impact, less-cited 'current' regurgitations by less-expert individuals who just needed a bit of CV padding. That is, if nothing significant has happened in a mature field, nobody worthwhile is going to write an important review—and indeed, recent reviews in such fields would be expected to overemphasize or overstate the importance of recent low-impact work just because there's nothing else to talk about. TenOfAllTrades(talk) 18:55, 14 January 2015 (UTC)
TenOfAllTrades I agree with what you are communicating in this paragraph and "rule of thumb" rather than "solid rule" is what we mean, then I would support that and efforts to make that more clear. I suppose this discussion contributes to establishing that this is not a hard rule.
For context, I was evaluating what to tell students who are using Wikipedia as part of the Wikipedia Education Program. See Wikipedia:Training/For_students/Medical_topics_2 which propagates the statement about 5 years. In teaching environments like this there has to be some balance in what is said and what the students are able to understand in order for them to know enough to give Wikipedia a good try. I do not mean to say that newer sources are better and am in total agreement with you about your critique of them. I am not sure how to teach or describe nuance or balance to this population except to say that I want to convey in about a sentence as much as possible about evaluating medical sources, and that I hope that with experience students would learn more. Any thoughts you have about that student guideline are welcome either here or on the course tutorial page. Blue Rasberry (talk) 17:46, 21 January 2015 (UTC)
Its going to vary a great deal by subject area. Fluoxetine has been out there for a lot of years, as have most of the atypical antipsychotics. No new trials are being run, or at least very very few. So a 7 or 8 year old review or meta analysis will be pretty much as up to date as a two year old one, as there really isn't much in the way of new data. And if you rigidly enforce a 5 year cutoff, only Cochrane will be heard from because they republish even when there is no new data. On the other hand, if you are talking about immunotherapy for melanoma, anything over 1 year old is hopelessly out of date. I don't think a context-independent cutoff will work here. Science doesnt move forward at the same rate in all fields. Formerly 98 (talk) 18:57, 21 January 2015 (UTC)
I actually don't have a problem with Wikipedia:Training/For students/Medical topics 2, given the specific way that page is likely to be used and targeted. As a high-level overview, expressed with some implicit flexibility ("... and typically not older than 5 years", my emphasis added) aimed principally at new student editors who are more likely to be adding (rather than removing) content, who are unlikely to be jumping directly into controversial and fringe topics, and who are unlikely to be coming to Wikipedia with specific, particular chips on their shoulders—it's reasonable and workable.
The usual sticky wicket I see is with editors who are here to push a particular (typically rather fringe) alternative POV. These editors want to use lower-quality recent publications in their field of choice because only true believers are left publishing in those areas; WP:MEDDATE is being used (overused, misused) to eliminate good-quality studies and reviews that represent the settled medical and scientific consensus. In the context of student editors and class projects, this likely won't be an issue. TenOfAllTrades(talk) 17:39, 22 January 2015 (UTC)
To the extent that it removes the "two to three years" idea, I think this is a small improvement. A further improvement might be removing the "newer is better" language. I don't believe that editors generally need advice on whether more complete sources (which is what we really mean by "newer") are better, so I believe that it could be safely omitted. WhatamIdoing (talk) 07:00, 23 January 2015 (UTC)
Retrieved from "https://en.wikipedia.org/w/index.php?title=Wikipedia_talk:Identifying_reliable_sources_(medicine)/Archive_10&oldid=1211784562"