Jump to content

Wikipedia talk:Identifying reliable sources (medicine)/Archive 15

Page contents not supported in other languages.
From Wikipedia, the free encyclopedia
Archive 10←Archive 13Archive 14Archive 15Archive 16Archive 17→Archive 20

funding of studies - background in WT:MEDRS

(note: i was OK with splitting this off into a separate section per this but not with hatting it. I unhatted here. Jytdog (talk) 17:53, 1 June 2015 (UTC))

i've been curious about the origins of the ""Do not reject a high-quality type of study due to personal objections to the study's inclusion criteria, references, funding sources, or conclusions." sentence. I imagine that was the product of a lot of discussion. Using Wikiblame i found that WhatamIdoing added it here in March 2010. She noted that she had made the change on Talk here; nobody replied.

I also looked in the history of WT:MEDRS for "funding" and for "COI". Here is the background of discussions about this issue.

  • Sept 2008 COI in secondary sources. Context there was papers about psych drugs in particular. Note the following comment (in the section right below that) about not casting hard and fast rules, but looking for what the consensus of the best sources say. (i would say, letting sources guide us and not looking for sources to "prooftext" what you already believe)
  • Oct 2008 Journals - raised as part of series of issues about the "industrial-medical complex" among others.
  • April 2009 A secondary source dilemma. Editor brings a concern about COI in sources. Results in no change. Similar answers as above.
  • June 2009. Casting it in stone section. Long post by an editor suspicious of many kinds of sources. Similar comments by MastCell as the year before, also similar comments from WAID, in reply.
  • June 2009 Assess evidence quality section first proposed.
  • Sept 2009 Pharmanoia brief discussion again of ruling out sources b/c of pharma or alt med funding
  • October 2010 section on this exact issue. An editor pushes very very hard to deal with funding of sources explicitly in WP. No changes to MEDRS as a result.
  • April 2011 Which RSMED "reliable source" to believe? Hmmm? - pharma-funded studies discussed. again. response was again - read a lot of secondary sources, use the best ones that reflect consensus
  • May 2012 The guideline should be changed - pretty rambly challenge. result was again, more or less: use the best sources you can find - read a lot of them, and no change
  • Feb 2013 Problems with secondary sources - addressed this head-on but with strange logic that b/c Cochrane doesn't take funding of primary sources into account, Cocharne is not reliable. went no where. Jytdog (talk) 00:39, 31 May 2015 (UTC)

independence

Directly related to this is the "independence" of a source from it subject.

Industry funding and ghostwriting of sources

The issue of source funding comes up regularly in medical articles. The concerns stem from the use of studies funded or ghostwritten by the pharmaceutical, biotech, and food-and-beverage industries.

The guideline doesn't mention ghostwriting, and of funding says: "Do not reject a high-quality type of study due to personal objections to the study's inclusion criteria, references, funding sources, or conclusions."

Basing articles on industry sources without alerting readers means we're presenting COI material in WP's voice. The situation is worse if the research papers were also written by PR companies. Leemon McHenry cites a c. 2002 survey that said there were 182 PR companies called "medical education and communications companies" (MECCs) active in the United States, charging industry (as of 2010) $18,000 to $40,000 per manuscript. These are placed in academic journals in the names of academic researchers.[1] This relates to the discussion (above) about impact factors too, because these PR companies seek to publish in journals with high impact factors.

The percentage of studies thought to be ghostwritten varies (50 percent is a widely quoted figure for pharmaceutical drugs). Most sources seem to agree that it's a problem. Ben Goldacre's Bad Pharma (2012) is worth reading. A 2014 review in the BMJ argued that "[e]vidence for the prevalence of ghostwriting in the medical literature is limited and can be outdated, misleading or mistaken." The article was written by a medical writer and funded by ProScribe—Envision Pharma Group, a company that offers these services.[2]

I have no solutions, but wanted to open this up for discussion. It would seem preferable to use independent sources whenever possible, and to leave out or at least flag the rest, but I don't know what percentage of sources that would affect. Sarah (SV) (talk) 20:42, 30 May 2015 (UTC)

  1. ^ Leemon McHenry, "Of Sophists and Spin-Doctors: Industry-Sponsored Ghostwriting and the Crisis of Academic Medicine", Mens Sana Monographs, 8(1), 2010, pp. 129–145. PMID 21327175
  2. ^ Serina Stretton, "Systematic review on the primary and secondary reporting of the prevalence of ghostwriting in the medical literature", BMJ, 2014; 4(7): e004777. PMID 25023129
issues of ghostwriting and funding are not 100% connected and we should discuss them separately. I agree that if a source is reliably identified as being ghost-written we shouldn't use it. Jytdog (talk) 21:32, 30 May 2015 (UTC)
The issues are very closely connected. We would also have to say what we meant by reliably identified. The ghostwriting might be signalled in the article by naming the ghostwriter, but not as an author, e.g. "X is thanked for his editorial assistance," and when you google X, it's an employee of one of these companies. Sarah (SV) (talk) 21:46, 30 May 2015 (UTC)
I think that what you're describing is very important and should be addressed. I'll need time to mull this over a bit. My head is feeling mushy today (and if you don't believe me, look at the post I just deleted). Coretheapple (talk) 23:33, 30 May 2015 (UTC)
(ec) I will note that there are a couple of different, sometimes-fuzzy concepts involved here. The BMJ article makes a not-unreasonable distinction between "ghostwriting", in which individuals who make substantial contributions to the writing of a manuscript (but who would not qualify for authorship under the ICMJE recommendations) are not acknowledged; and "ghost authorship", in which individuals who ought to qualify for authorship are not named as authors—or are omitted from mention in the manuscript altogether. The latter strikes me as a much more serious ethical breach (and likely much rarer) than the former, however there are discussions where I've seen both concepts lumped under the umbrella of "ghostwriting".
Beyond those two areas – which may have some overlap, or a fuzzy boundary, depending on how broadly one chooses to interpret the ICMJE (or other) authorship criteria – one can explore either end of the spectrum. At the deep(ly unethical) end, you've got the cases where someone who should be credited as an author isn't mentioned in a manuscript at all. (And the mirror case, I suppose—where someone who hasn't contributed substantially to the paper is improperly credited as an author anyway.) At the other end, you've got the sometimes-complex cases where mention of an individual who might (or should) qualify for acknowledgement – but not authorship – is omitted from the paper's Acknowledgements section. Sometimes this is sloppiness by the (usually corresponding) author; sometimes it's because (at least one of) the author(s) is a dick; sure, sometimes it's because the author(s) don't want to mention someone they've worked with; sometimes it's because a potentially-acknowledgeable individual has asked not to (or declined give permission to) be acknowledged.
Based on the exchange above, it looks like SlimVirgin may be using "ghostwriting" in yet another (particularly broad) sense—to refer to a paper where authorship is correctly credited by ICMJE guidelines, where the non-author individuals contributing to the manuscript are credited in the paper's Acknowledgements—but where one of those acknowledgements is of someone who was paid to draft a substantial amount of the manuscript text. And this is an area where things can get sticky. There are innumerable companies of wildly varying size, cost, and competence that offer to 'help' prepare scientific manuscripts for submission. (Any scientist whose email address has appeared in print will be familiar with the spam from the bottom-feeders in this class, but there exist more reputable firms as well.) Their aid can range from basic copyediting and formatting to translation, drafting of full manuscripts, mock peer review, and assistance with preparing response/rebuttal letters to peer reviewers. It's an interesting and complex question, but I'm definitely not willing to endorse or support any suggestion that we should throw out (or even flag or scarlet-letter) any paper which includes work done by paid writers (who are otherwise properly acknowledged, and where authorship is otherwise properly represented). Individuals who didn't help design, conduct, or analyze experiments; who don't get a final critical say and approval of the manuscript; and who aren't willing or able to stand up for the paper's contents and conclusions—shouldn't be credited as "authors" (in the sense as understood by scientific journals and their readers). TenOfAllTrades(talk) 01:07, 31 May 2015 (UTC)
  • Basing articles on industry sources without alerting readers means we're presenting COI material in WP's voice. Bottom line. We can argue about distinctions between ghostwriting and industry-funded studies, but let us simultaneously address the main issue: giving voice to CIO without alerting the reader. Just as news articles will often do, we should whenever possible flag industry-backed sources with a simple statement of attribution. This allows the reader to use their own discretion, and lets WP off the hook. petrarchan47คุก 02:40, 31 May 2015 (UTC)
P, would the same apply to this source, which is authored by members of ENSSER (an apparently funded by ENSSER - no funding is disclosed), an anti-GMO activist group, and which you appear to want to give a lot of WEIGHT to? The article publicizes their petition that there is no scientific consensus on the relative safety of eating GM food. (see the end of the article). (Additional note - the journal that published it, is on the list of predatory publishers.)Jytdog (talk) 15:06, 31 May 2015 (UTC) (strike wrong statement Jytdog (talk) 13:33, 1 June 2015 (UTC))
Apparently we aren't speaking of the same journal. Please strike your remarks. petrarchan47คุก 05:43, 1 June 2015 (UTC)
ack you are correct. mu apologies. struck that part. Jytdog (talk) 13:32, 1 June 2015 (UTC)
I thought your accusations of bias should also be struck. Of the 300 scientists supporting the document, half identified as decidedly pro-GMO, and the conclusion was that there was no support for claims that GMOs are dangerous. Doesn't sound anti-GMO to me. petrarchan47คุก 03:21, 2 June 2015 (UTC)
Jytdog, I'd like to keep this discussion civil and on track. It's an issue that everyone here has an interest in, not least because we're all readers as well as editors. The question is not what an author's bias is, but whether there was industry (or activist) funding and/or ghostwriting. Sarah (SV) (talk) 20:30, 31 May 2015 (UTC)
I wrote nothing uncivil. I agree with keeping things on track, very much. Part of my purpose was there, was to say that analyzing a source by funding alone doesn't get you too far. There are a number of things to look at. And most of all, we don't cherry pick. Jytdog (talk) 21:39, 31 May 2015 (UTC)
Also I don't think we should get bogged down in lengthy discussions of previous discussions of this subject. That's a fine way of starting down the path of doing nothing. Coretheapple (talk) 14:28, 31 May 2015 (UTC)
WP:RTA is standard practice. RTA =/= discuss the archives. Jytdog (talk) 14:48, 31 May 2015 (UTC)
Core points out that there are some practices that derail progress. There is much grey area between getting bogged down (for instance, starting new threads that divide the OPs message in two) and "discussing the archives" in the course of general discussion. petrarchan47คุก 22:25, 31 May 2015 (UTC)
  • Conflicts of interest are important to know about, but there is plenty of danger in taking this too far. Dr. Lisa Rosenbaum has had a good series in the New England Journal of Medicine this month about this: [1], [2], [3]. I didn't agree with all of it, but it's well-worth reading. NW (Talk) 14:58, 31 May 2015 (UTC)
  • Thanks for those links, NW. TenOfAllTrades, I don't agree with the distinction Stretton tries to draw between ghostwriter and ghost author. The point is that someone other than the named academics has written the manuscript entirely or substantially. It's the same situation as a Wikipedian being paid to add company-authored material to articles about that company. We don't regard that editor as the author just because he has to take responsibility for it. Stretton and the International Committee of Medical Journal Editors have defined author as the person who has final approval and agrees to be accountable. [4] That's never the ghostwriter, so that definition seems to airbrush the ghostwriter out of the picture. Sarah (SV) (talk) 21:16, 31 May 2015 (UTC)
Richard Horton, editor-in-chief of The Lancet, April 2015 points out that there exists a very serious problem with regard to accuracy in scientific literature today: The case against science is straightforward: much of the scientific literature, perhaps half, may simply be untrue ... science has taken a turn towards darkness. Another problem he points out is discussed here: Medical Journals Are an Extension of the Marketing Arm of Pharmaceutical Companies Plos.org petrarchan47คุก 22:25, 31 May 2015 (UTC)
Thanks, Petra, that's interesting. One thing we might consider is compiling a list of the best journals with good disclosure practices, and writing to the editors to ask for their help in composing something for this guideline. Sarah (SV) (talk) 22:32, 31 May 2015 (UTC)
Sarah, I'm not sure I'm fully understanding why you are bringing this up. Ghostwriting and issues of this sort come up most often in pharma-funded clinical trials, but we almost should never be citing those to begin with per WP:MEDRS#Respect secondary sources. The secondary sources are often written by persons with conflicts of interest, as are society and governmental guidelines, but again, see Lisa Rosenbaum above. I would be surprised to learn that ghostwriting affects review articles to any serious extent. Could you please clarify what change you are seeking? Thanks, NW (Talk) 23:01, 31 May 2015 (UTC)
It seems pretty shameless for Richard Horton to lament "studies with small sample sizes, tiny effects, invalid exploratory analyses, and flagrant conflicts of interest, together with an obsession for pursuing fashionable trends of dubious importance". After all, I think many people regard the Lancet, under his editorship, as one of the worst offenders in that regard. He is responsible for publishing (and, for an unpardonably long time, defending) the most harmfully untrue scientific manuscript of the current millenium: Wakefield's discredited MMR/autism paper. That paper combined all of the things he's complaining about: a small sample size, tiny effects, invalid analysis, flagrant conflict of interest, and science-by-press-release. It's like being lectured on how to avoid icebergs by the captain of the Titanic. But I digress.

Ghostwriting has been a significant problem in the medical literature. It is actually probably more prevalent in review articles than in primary research, because narrative reviews by their nature provide more wiggle room for subjective interpretation of data and thus are more amenable to spin. For example, as the Times reported in the case of hormone replacement therapy, "the ghostwritten papers were typically review articles, in which an author weighs a large body of medical research and offers a bottom-line judgment about how to treat a particular ailment." These days, most reputable journals ask explicitly at the time of manuscript submission whether there were any uncredited people who played roles in drafting the manuscript, and whether any medical-writing services were involved in the manuscript's production. I think this has helped to reduce, or at least disclose, ghostwriting, although it would probably be naive to think that it's a problem of the past. MastCell Talk 21:14, 1 June 2015 (UTC)

I'm not sure whether the Lancet is now considered "one of the worst offenders" in this regard based on the single discretion you note, or whether the extrapolation might be a fringe view. "Many people agree" insinuates it is not fringe, but I haven't seen support for this view in RS. From what I understand, half of all studies end up being retracted. If the Lancet is less credible than it once was, this should be written into (or somehow mentioned in) WP:MEDRS for clarity. petrarchan47คุก 22:04, 3 June 2015 (UTC)
My view of the Lancet is my own, and it's based not only on the Wakefield paper (although that's the most striking example). When I say that "many people agree" that the Lancet (under Horton) tends to favor papers which are splashy and attention-grabbing but not necessarily scientifically sound, that is again my opinion, informed by my real-life experience, so you're free to assign exactly as much credibility to it as you see fit (but I certainly wouldn't argue that it should be written into WP:MEDRS).

It is an important, and major, misconception to believe that "half of all studies end up being retracted". Retractions remain relatively rare, although they've increased in recent years for a variety of reasons. Exact numbers are hard to come by, but in 2011 an article in Nature estimated that 27,000 scientific articles are published every week, of which approximately 200 are ultimately retracted, suggesting that the actual retraction rate is somewhere around 0.7%, not 50%. I think you're confusing retractions with John Ioannidis' famous 2005 paper asserting that more than half of all published research findings are false. Ionannidis' argument is technical and statistically complex, but quite thought-provoking. MastCell Talk 22:24, 3 June 2015 (UTC)

Thanks, MastCell. It is an interesting (and hopefully relevant) subject. The 50% claim does not seem well supported. Here are a few tidbits I dug up on the issue:

"...we find that most retracted articles result from misconduct, and nearly half of retractions are for fraud or suspected fraud ... The recent increase in retractions for fraud cannot be attributed solely to an increase in the number of research publications: retractions for fraud or suspected fraud as a percentage of total articles have increased nearly 10-fold since 1975" PMC 3479492 (The Lancet is mentioned)

Committee on Publication Ethics, a multidisciplinary group that includes more than 9,000 journal editors, issued a statement suggesting a much broader potential problem. The committee, it said, "has become aware of systematic, inappropriate attempts to manipulate the peer review processes of several journals across different publishers."

"With researchers increasingly desperate for recognition, citations and professional advancement, the whole peer-review system has come under scrutiny in recent years for a host of flaws and irregularities, ranging from lackadaisical reviewing to cronyism to outright fraud." WaPo

Sometimes these retractions may be at the behest of industry, as is indicated here. petrarchan47คุก 22:41, 3 June 2015 (UTC)

Break 1

NW, McHenry discusses three kinds of medical ghostwriting. It's the second that would mostly affect us.

  1. Pharmaceutical companies hire medical PR firms to write up the results of clinical trials the company has funded. The academics listed as authors may have contributed to study design, etc, or may not. The real author is acknowledged as an editorial assistant or similar.
  2. Companies hire writers to produce manuscripts, often review articles, which they invite an academic to attach her name to. The academic who appears as the author will tend to be a "key opinion leader." Key opinion leaders are sought for "their ability to influence other prescribers." Adriane Fugh-Berman describes here being asked to do this.
  3. Companies encourage physicians to publish their positive experiences with certain drugs via one of the company's medical writers.

It's not only the pharmaceutical industry; biotech and food-and-beverage have come up too. Jytdog mentions that it might also be a problem with activist groups, though I'm guessing that's unusual because of the money involved. But regardless, the principle is the same. The question is: if we identify an article as ghostwritten/industry-funded (sometimes the funding is mentioned at the end), should we (a) refer to its material in Wikipedia's voice as if it were not so funded; (b) flag it in some way; (c) not use it? Sarah (SV) (talk) 23:49, 31 May 2015 (UTC)

Another option is simply to add that independent secondary sources are preferred. If an industry review article (assuming it makes the funding known) agrees with independent sources, it's easy enough to use the latter instead, and if it doesn't, that might be a reason to avoid it, or use with in-text attribution. Or we could include funding as a parameter in citation templates. That's less helpful to the reader, but not useless. Sarah (SV) (talk) 00:24, 1 June 2015 (UTC)
Sarah (SV), when you say that the ICMJE has "defined author as the person who has final approval and agrees to be accountable" I think that you're (I hope inadvertently) "airbrushing out" at least two of the important ICMJE criteria for authorship. The first two bullets are "Substantial contributions to the conception or design of the work; or the acquisition, analysis, or interpretation of data for the work" and "Drafting the work or revising it critically for important intellectual content." If someone is presenting themselves as an "author" when they haven't been a substantial contributor to the work's design or conception, or to the acquisition or analysis of the data, then that's a problem. If someone claims to be an "author" without having been involved in critical revision of the manuscript, then that's a problem. It's certainly not sufficient for someone only to have final approval and agree to be accountable, and neither I nor the ICMJE guidelines I linked to would suggest that it is.
We should certainly raise our eyebrows at papers where authorship is fraudulently attributed to individuals who do not meet the ICMJE (or other reasonable) guidelines. If there are legitimate, reliable, robust (in a WP:BLP sense) reasons and sources to support an assertion of fraudulent authorship of a publication, we should definitely flag such publications (and probably should strongly resist employing them at all). However, we shouldn't be jumping to suggest (or invite an inference of) misconduct or fraudulent authorship solely because a paper has private sponsors, or because a commercial organization was involved in the preparation of the manuscript. That's too broad a brush.
As an aside, I'll note that this issue is probably more fraught than it needs to be in part for inconvenient semantic reasons. As used by laypeople, the term "author" typically refers to one person (or sometimes two, and very rarely more) who sat down and wrote up the bulk of a manuscript, building it out of ideas (fictional or non-) created or assembled in their own head(s). We have a romantic picture of a lonely chain-smoking soul with a typewriter and a bottle of rye, or a cup of coffee and a laptop. Writing and rewriting and revising the manuscript is the biggest, slowest, and most essential part of the work.
In the context of the sciences and scientific publishing, being an "author" means something rather different. Actually writing the manuscript, while an obviously necessary step in the publication process, isn't the most essential thing. It's about who designed and conducted the experiments, who collected and analyzed the data, who drew inferences and formed interpretations and reached conclusions. The wordsmithing is important for communicating the scientific content effectively, but it's not the most central or time-consuming part of the work. The nature of scientific authorship is different from authorship in most other contexts, and we get caught up on the wrong point if we focus exclusively on who wrote the bulk of the text. TenOfAllTrades(talk) 01:12, 1 June 2015 (UTC)
  • TenOfAllTrades, re: your first paragraph and the list of ICMJE criteria for authorship: for an academic to be listed ethically as an author, the ICMJE requires only that they make substantial contributions to the design of the study or interpreting/acquiring the data (note that crucial "or"); draft or revise the article critically (note the "or" again: revision might involve no more than reading it); give final approval; and take responsibility for it. A company ghostwriter who interprets the data and writes the article doesn't count for them as an author. Sarah (SV) (talk) 01:31, 1 June 2015 (UTC)
I'm not sure what argument you're trying to make at this point. Is it that you think it's unethical for a ghostwriter to not be named as an author, or that it's unethical for someone who directs the ghostwriter's work to be named as an author? (Or something else?) Those are different issues, and addressing either depends a lot on the specific circumstances, contributions, and responsibilities of the individuals involved. The definition of a "ghostwriter" you're using isn't clear, either, and I'm concerned that you're perhaps referring to the least ethical situations (out-and-out ghost authorship and fraudulent authorship) and using that to tar every definition of "ghostwriter". TenOfAllTrades(talk) 01:54, 1 June 2015 (UTC)
The question I'm asking is how we should handle material where we find company sponsorship. Re: ghostwriting, again, it might help to look at McHenry's second example, which is the issue that perhaps affects us most, namely when a company writes a review article, then looks for an academic to put her name to it. The academic might end up having no input, some or a lot. The point for Wikipedia is that the content originated with a pharmaceutical or other company, and contains the message the company regards as important. There may be no indication of the real authorship, but the funding is sometimes acknowledged. Assuming we become aware of it, should we be placing company-sponsored or written material in WP's voice?
If we were to discover that a history of the war in Iraq had been written by PR people employed by the White House, then published in the names of apparently independent historians, we wouldn't add its material in WP's voice and not alert the reader. Sarah (SV) (talk) 02:26, 1 June 2015 (UTC)
So help me out here with that practical step, "If we were to discover" that someone claimed to be an author without actually being the sort of person who qualifies for authorship under academic standards. How exactly will we learn that? Are we just going to guess that any paper whose POV I disagree with is one of those?
Feel free to exclude papers that will be retracted because of this violation of professional standards, because we already want to exclude retracted papers. Just tell me how we're going to discover that there verifiably was an actual, bona fide ghost who secretly wrote a particular paper. WhatamIdoing (talk) 04:03, 1 June 2015 (UTC)
Yes, this appears to be the key sticking point. I personally am listed in the acknowledgements of a paper from a few months ago for my 'editorial assistance'. But all this meant was that I copyedited the paper twice. There is no indication of my affiliation or employer, and 'editorial assistance' is a broad enough term that could easily encompass 'copyedited and added references' to 'wrote a substantial portion of the paper'. No journal submission process I have been involved with has asked us to further detail why exactly someone is listed in acknowledgements instead of as an author – they're just barely even starting to ask which ICMJE criteria listed authors meet. I just don't see how this is a productive line of guidance to add. NW (Talk) 10:55, 1 June 2015 (UTC)
WhatamIdoing, if you have an example of papers being retracted because written by ghostwriters rather than the named authors, that would be helpful. The issue is funding, and that is often described in the paper. If neither ghostwriting nor funding are acknowledged, there's nothing we can do unless an RS draws attention to it. But I'm talking here about cases where the funding is acknowledged. Sarah (SV) (talk) 19:24, 1 June 2015 (UTC)
Here is a funny paper about ghostwriting: PMID 21473670, called 'Lack of involvement of medical writers and the pharmaceutical industry in publications retracted for misconduct: a systematic, controlled, retrospective study." written by... folks from ProScribe Medical Communications. Australia. Which is mentioned in this long comment on pubmed from ghostwriters. Jytdog (talk) 01:44, 2 June 2015 (UTC)
I am concerned that your comment puts some concepts side-by-side which aren't necessarily equivalent, and risks smooshing together or eliding some pretty important distinctions. I expect that we all agree that the case of "Company writes manuscript, then shops it around for a 'credible' scientist to sign as author" is an egregiously and unambiguously Bad Thing: a combination of ghost authorship (the individuals who conceived of and wrote the analysis and decided it should be published aren't credited as authors) and fraudulent authorship (the named 'author' didn't conceive of or materially contribute to the content).
But the term "company sponsorship" covers a much broader range of scientist-corporation relationships and writing situations, each with different expectations for authorship credit and (potentially) conflict of interest/competing interest disclosures. It doesn't strike me as reasonable to flag as suspect any paper where any of the authors (or non-author contributors) has a corporate affiliation or employer, or who has received any corporate funding—but where would you suggest we draw the line?
In any event, as WhatamIdoing and NuclearWarfare both note above it's not really practical or appropriate for us to play detective and pass judgement. You're asking us to put a label on real, mostly-living authors, declaring that they may have engaged in scientific misconduct. That's not cool, and it's getting into the territory of (mis)using Wikipedia to try to right great wrongs. If you have specific, articulable concerns about the authorship of particular publications, you need to go to the journal editors and make them correct the record. Wikipedia can then dutifully report on the misconduct. TenOfAllTrades(talk) 13:18, 1 June 2015 (UTC)

Just a quick note here, in general terms, from my perspective, as one who has paid close attention to the power/knowledge dynamic in fields where there are heavy vested interests, like fossil fuels in relation to climate science. There is a word, "frackademia", that exists for a reason. The reason is that heavy industry gaming of science to influence the production of knowledge is very apparent, and in some cases such as that of Willie Soon, evidence comes out that shows a clear line of influence, and yet there are surely many more cases in which evidence does not come out in a timely fashion, and yet there remains the smell of industry influence and bias. In my readings about glyphosate in regard to the agrochemical industry, i also detect a clear bias in many sources. I can see in many papers a strong agenda to show a particular outcome -- often to show that a chemical is safe, rather than to look at it without bias. On the other hand, there are also cases where the bias goes the other way, where it's clear that researchers are looking for damning evidence, to show that a chemical is not safe, or that fracking results in very high fugitive methane emissions, and that can also be called bias, but it's important to note that this phenomenon is not symmetrical. There is asymmetry, because industry with large fiscal vested interest has far more resources and a different sort of motivation to engage in biasing science. In summary, i think that it's good to look out for clear ghostwriting and clear fiscal influence, but it's also important to have a general nose in the air in regard to more subtle bias that can occur in science when there exists strong fiscal pressure to find one result over another. Just some general thoughts from an observer of the nature of power/knowledge. Hope it's helpful. SageRad (talk) 14:28, 1 June 2015 (UTC)\

PS -- I have found that often it's clear from the very introduction of a paper itself, what the bias of the paper is. It seems that industry-cozy researchers often describe a chemical in wondrous terms, for example, and then describe their methods and results after that glowing introduction. On the other hand, a more critical study often begins with a litany of the signs that the chemical causes harm, and cites papers to that effect. It's often very apparent just from the tone of the introduction itself, and i think that it behooves Wikipedia editors to be aware of bias and to truly do our honest best to assess reality by taking bias into account and using many sources. SageRad (talk) 14:31, 1 June 2015 (UTC)

Funding

  • The question is how Wikipedia should handle source funding. At present the guideline says (bold added): "Do not reject a high-quality type of study due to personal objections to the study's inclusion criteria, references, funding sources, or conclusions."

    Many studies have shown that funding influences conclusions. For example, a 2013 systematic review in PLOS Medicine found: "Reviews in which a potential conflict of interest was disclosed were five times more likely to present a conclusion of no positive association between SSB [sugar-sweetened beverage] consumption and weight gain than reviews that reported having no financial conflict of interest."

I assume we wouldn't use a tobacco company-funded review about beneficial effects of smoking, or at least wouldn't publish its conclusions in Wikipedia's voice. Writers have to be allowed to include funding as part of their overall assessment of whether and how to use a source. Therefore, I would like to remove the reference to funding from that sentence. Sarah (SV) (talk) 15:05, 1 June 2015 (UTC)
Thanks for the ping. We wouldn't use such a tobacco-funded study because it would be far outside the mainstream POV on smoking. Even if for some reason that source was important enough to warrant inclusion, then there would almost certainly be multiple reliable secondary sources disputing it, in part, on the grounds that it was funded by the tobacco industry. In that case, we would (and would have to) point that out to maintain NPOV. In this example there is no need for a new rulemaking that requires "flagging" sources in articles. But if you want to flag something for the funding source, there needs to be RS making an issue of that particular study on those grounds, otherwise it's OR and not neutral. Otherwise I'm concerned that pointing out funding sources without good reason could be used to insert bias into an article. Geogene (talk) 17:40, 1 June 2015 (UTC)
Hi Geogene, in-text attribution is common and adding funding wouldn't require a separate source (e.g. "A 2015 systematic review sponsored by the American Sugar Alliance concluded that ...").
The question I'm asking in this section is simply that we remove the words "funding sources" from the guideline sentence I quoted above, because we have to be allowed to take all factors into account when judging how and whether to use a source. The sentence is: "Do not reject a high-quality type of study due to personal objections to the study's inclusion criteria, references, funding sources, or conclusions." Sarah (SV) (talk) 17:53, 1 June 2015 (UTC)
Sorry for misunderstanding this proposal in my comment above. If I'm interpreting this sentence correctly, I don't agree there's a problem with it. I understand that a consideration of funding may go into determining the independence of a source as given in the RS policy; and that we prefer independent sources (noting that funding doesn't always compromise independence). But if a source is otherwise good on every account, and especially if it satisfies WEIGHT in representing more or less the mainstream view, then I don't see a problem here that needs fixing. However, if the sentence were modified to make my interpretation of it more explicit, such as by adding a "merely because of personal objections....", so it's clear that funding is not a non-issue, that I would not oppose. Geogene (talk) 22:04, 2 June 2015 (UTC)
nice to see a separate discussion on the funding question. In-text flagging is in my experience controversial and not settled (especially if there are not RS calling attention to it, that are cite-able). I'm of several minds about the proposal. The specific language is about "rejecting" (i.e. not using) a source due to personal objections about its funding. My review of the origins of the language and past discussions about this, makes me think that it was meant to prevent anti-cherry-picking ("cherry-tossing"?). We're not meant to do either cherry-picking or cherry-tossing based on personal preferences, but rather "assess" (per the section title in MEDRS) to identify high-quality types of studies, read them all, and summarize what they say. Instead of deleting reference to "funding sources", it might make sense to instead add something like: "However, a high-quality type of study may be rejected, if that study a) was funded by any group with an interest in the subject matter and b) if the study draws conclusions that fall outside of the consensus established by other high-quality sources and c) those conclusions are favorable to the interest group." The last one being important, since the concern is COI in the source leading to warped perceptions of what positions the relevant field holds ... so that warping would have to manifest itself by going against other sources, and everybody involved would have to be engaged with the literature as a whole. Something like that. That seems to address your specific concern as well as the deeper WP theme of resistance to any kind of cherry-picking or cherry-tossing. Jytdog (talk) 19:00, 1 June 2015 (UTC)
That's a separate proposal. I'd like to keep this simple, so for now I'm asking for consensus to remove those two words, because they appear to make WP say financial conflict of interest is never allowed to matter in medical articles. That would put us outside the mainstream. How we handle funding is a question for another section, because it will be tricky to find the right words (e.g. your three ands introduce problems). Sarah (SV) (talk) 19:14, 1 June 2015 (UTC)
I don't know what you mean about MEDRS putting us "outside the mainstream." What journals will not publish papers describing industry-funded research, or will not allow authors to cite industry-funded research? (the only example i am aware of, is that some journals will not publish tobacco-industry funded research. outside of that, what you propose seems far from mainstream to me. But i am interested to hear more. And am interested to hear what other WP:MED regulars have to say... Jytdog (talk) 20:44, 1 June 2015 (UTC)

To be sure I understand the views of the group, is it accepted that funding does sometimes affect the outcome of studies, reviews, etc., as is stated in this review [5]? Gandydancer (talk) 11:28, 3 June 2015 (UTC)

Yes, it's accepted that funding affects the contents of sources by making them WP:BIASED. But we don't reject high-quality types of sources because the authors are biased. An NPOV article is one that fairly represents all of the POVs in high-quality sources, not one that fairly represents only those POVs that are remaining after I've excluded any source that I believe is tainted or improperly conducted.
The problem being addressed here is partly due to the section heading (see discussions from ~2009). We have editors who see "Assess evidence quality" and believe that it's their duty as an editor to decide whether a source was correct, and then they adjust the article to match what's left after they've rejected all the "wrong" sources.
Let me give you a simple example: If a teenaged girl has a BRCA mutation, some docs are wary about giving hormonal contraception to her, because it may be associated with her developing breast cancer in her 20s (rather than in her 40s or later). We once mentioned this fact in a relevant article, as a fairly weak "Some physicians say" statement. It stayed there until an editor found the secondary source we cited, looked up the primary sources that it cited, thought about the sample size and the inclusion criteria, and decided that the physicians were wrong: In the opinion of the editor, the physicians shouldn't be considering BRCA status, and therefore the article must not mention the undisputed fact that some of them actually do what he believed they shouldn't.
IMO this example isn't important (we're not giving medical advice, probably nobody reading it will notice or care if it's not there, etc.), but this process is what we must not do. If a reliable source says something, then it's not our job to decide that the source was wrong. We should accept what the source says. This is a little complicated, and it has to do with who is taking action, so let me try it like a list:
  1. There's no requirement that any WP:VOLUNTEER use a given source. You personally may use WP:Editorial discretion to omit sources that seem wrong to you. You personally may refuse to cite any source on any grounds you like, including such arbitrary choices as using only sources whose titles begin with a vowel, if that's how you want to contribute.
  2. Everyone may always seek better sources.
  3. Everyone may always consider the best way to present a source. For example, sometimes a qualification like "In a 2010 study by Big Pharma" or "In a 2010 book by Andrew Activist" is ideal, especially if the material is disputed (in the real world). In other cases, you shouldn't, because WP:INTEXT attribution can be misused to marginalize mainstream POVs.
  4. But although you are never required to (and never should) make any edit that you personally believe makes an article worse, you also can't force other editors, who may hold other opinions, to go along with your preference to omit a particular source. In particular, you cannot force others to exclude sources that you dislike, if your choice is based upon "meta facts", like who conducted the study and how it was paid for. How this adds up:
    • If you don't choose to add a source (solely) because it was funded by Cynical Corporation (or similar issues), then that's fine—but you also can't prevent some other editor from using it. "Can't prevent" includes that you can't honestly claim that using the source violates any sourcing rules for Wikipedia. WP:BIASED sources are permitted, even though they need careful handling.
    • You especially can't prevent someone from using a better type of source than yours. It's no good saying "That meta-analysis was paid for by Cynical, so you are not allowed to use it; instead, we have to use this old n=3 case series (or in vitro study—we get that a lot around altmed, because clinical trials are "too expensive" for the multi-billion-dollar dietary supplement industry to pay for), because that is the only study that I'm sure wasn't published by someone with a conflict of interest".
Does that make more sense? The point is to go with the (apparent) better evidence, not the best evidence that's left after we exclude anything that we disagree with. WhatamIdoing (talk) 11:21, 4 June 2015 (UTC)
The concern for me is that as a reader, I appreciate for instance when a news source reveals the funding details of a study on which they are reporting. Simply put, I think our readers would appreciate it as well. A simple mention of funding source might be seen as condemnation of the study, however it doesn't strike me that way as a reader. People come here because they are hungry for information. Although funding sources don't necessarily tell us much with regard to the legitimacy of reported findings, it is a valid, neutral, and encyclopedic addition to an article, IMO.
Multiple studies conducted after the 2010 BP oil spill, fully funded by BP, came to roughly the same conclusions as independent scientists - none of it benefiting BP. The editors involved in the related articles noted funding sources as a matter of course, and in this example funding sources did not necessarily predict results. The most damning science did come from indie sources, and science claiming no harm was always tied to BP-funded science, however according to this example, the detectable effects of funding were far less egregious than even the most skeptical among us might expect. The extremes ("the Gulf is FUBAR" and "no harm whatsoever") accounted for only about 15% of the studies I've seen.
I'm wondering whether there is opposition to the idea of mentioning funding sources - flagging - as a matter of course. petrarchan47คุก 21:22, 4 June 2015 (UTC)
part of the issue there is that we generally don't write "according to a review published in 2015..." - it is a bad sign when articles devolve into that sort of thing. we generally just describe what MEDRS sources say in WP's voice. See WP:MEDREV. Jytdog (talk) 21:48, 4 June 2015 (UTC)
Yes, we do. That is exactly how I write, and I use WP's voice only rarely when reporting on studies or controversial issues. It is a bad sign when editors are diverging from this practice, as our job is only to summarize the best RS with due weight, and that includes the most important facts: date, source, the finding, and possibly the response to the finding. Perhaps this shows one problem with the influx of experts working on WP articles in their field - they are accustomed to authoring studies, not to dispassionately reporting on them for laypersons (WP readers). petrarchan47คุก 02:55, 5 June 2015 (UTC)

A few thoughts I haven't been able to add in the last week since I was traveling and just reading this conversation:

I don't see any agreement saying funding source affects outcome, and if you definitely won't see it from those familiar with how industry funding works in academia at least. Take my field for example. Some of the more applied field researchers do get funding from pesticide companies to test their products for efficacy and potential issues. Often times other companies are putting their products and funding in for competition at the same time to be independently vetted by the university researcher. Even if there was one funding source, it would not bias the outcome of that study. If that did happen, the other companies would know the researcher doesn't do trustworthy work. For them, that can mean a faulty study saying their product doesn't do as well as the competitor, or even not knowing about a bad non-target effect that that they would want to know about. You do see the close to finalized products making it into those studies. Companies don't waste money submitting junk products they don't intend to sell for independent evaluation. That is one of the huge reasons why industry-funded studies tend show "good" products because the company has already weeded out the junk and problematic stuff for the most part. Available funding can affect what gets studied, but that's very different from actual outcomes that do get scrutinized in the literature.

The above is really only addressing the red herring in the room though. That is really not what's truly being questioned here. In the case of academic researchers, we have satisfied WP:INDEPENDENT (there is a difference if it's not an academic researcher at a university but a consultant or member of the company, but that's a bit more ambiguous and not as common). What the funding source argument is used to mean by many editors is that there is something wrong with the study and it's conclusions. That means there is something that can be caught in the study methodology and results that either peer-reviewers or later publications will point out, and as Guy mentioned above, that's what we should be looking for rather than engaging in original research about the funding source that so many people seem to want to do.

On a final note, there is a red flag to indeed watch out for related to funding source. When we see the funding source listed in the acknowledgements, that is not intended to address COI at all. Instead, whenever someone submits a paper , they need to specify if the funding source played a role in the experimental design of the paper. For your independent academic researchers with industry funding, you won't see mention of that in the paper at all if they say the funding source didn't have say. Some journals do the opposite (though not as common) and will specifically say in that case that the funding agency wasn't involved. In that rare case that the funding source is involved in the design, that is something we do consider unreliable in academic research. It might still get accepted, but it will be red flagged. I mention this last bit because it seems some misinterpret funding source to imply this latter scenario when that specific case does get called out much more specifically if it does happen. Kingofaces43 (talk) 01:10, 5 June 2015 (UTC)

I think it's going to depend somewhat upon the field.
When we have reliable sources discussing a correlation between funding and outcomes (e.g., the BP oil spill), then I think it's okay to say "In a study conducted by BP" or something like that. But normally, I think it should be omitted. For example, I'd normally omit such claims from all new pharmaceutical drugs, because (as I understand it) it's typical for 100% of those studies to have funding and cooperation from the manufacturer. Before marketing approval is received, you pretty much cannot conduct a study without their agreement, because there's no other way to get your hands on the drug (and they're required by law to collect and report your results as part of their drug application, so they have to be sure that you'll do sound work and give the data back to them). Every bit of research in many articles about new drugs is going to have the manufacturer involved, but tagging "In a study funded by Big Pharma..." "In a review of studies, all of which had Big Pharma involved...", etc., on every sentence doesn't seem like a good idea. WhatamIdoing (talk) 05:33, 5 June 2015 (UTC)
I guess that's exactly the problem we are facing, Kingofaces43. If the funding source has got involved in the design, there's no way really knowing it since most likely, it wouldn't be spoken out. Therefore, certain caution should be exercised. As the saying goes: "He who pays the piper calls the tune." However, I agree with WhatamIdoing: medical studies are really expensive to conduct, and the reality is that the manufacturers tend to be involved in one way or another. It's just that if the results don't "please" the funding source, the researchers/institutions are less likely to attract equal funding in the future. Jayaguru-Shishya (talk) 11:37, 8 June 2015 (UTC)

It's not our problem.

We can't talk about COI in published articles unless it's addressed in the literature with at least parity of credibility. Example: every quackery shill on the planet claims that all findings contradicting their belief are funded by Big Pharma, usually with zero credible evidence. We don't do original research or novel synthesis. So, if there is documented evidence of COI, and this is noted in the original publications or in others with parity of impact, then we follow that (as we do, for example, with the refuted claim of a link between MMR and autism). If the claim is published by bodies of no evident significance, in low tier journals, and does not result in any retraction or comment in the original source, then we are entitled to ignore it. Guy (Help!) 22:11, 1 June 2015 (UTC)

Whoa, yes we *can* talk about it, and yes, it *is* our problem, if we're using unreliable studies to support claims. We have brains and we can use them. I'm WP:HERE, and i'm going to think and talk about undue influence in sources if i discern that to be the case. I'll use my mind and speak sensibly. You've set up a strawman. Not all who speak of these things is a "quackery shill". SageRad (talk) 05:56, 2 June 2015 (UTC)
We are not qualified to judge their reliability. See WP:NOR. We follow the sources - for example, if national scientific bodies say one thing and a single study says another, we go with the national bodies, because anything else would be placing our own personal preference above the judgment of specialists in the field.
This is especially important when we are sure in our own minds that the single study is right and the major bodies wrong. That is precisely the point at which we have to consciously check our own biases at the door. Guy (Help!) 07:55, 2 June 2015 (UTC)
I agree. It's not our problem. (And I've been guilty of rejecting papers because the statistical methods used are totally inappropriate. I've got to watch that, even though I could make an expert comment on such papers.) The key word in the guideline is "personal", not "funding sources". If reliable sources comment that the funding sources make the conclusions of the paper questionable, we can include that (quoting those sources) or use that information to determine whether the paper is "reliable" by our standards. If we, personally, believe that the funding sources make the paper unreliable, we may not use that information. — Arthur Rubin (talk) 19:06, 2 June 2015 (UTC)
Arthur, I encourage you to keep using your WP:Editorial discretion to choose, to the very best of your ability, the best of the high-quality sources available to you. Just don't stop other editors from using the ones that they believe are best, especially if "theirs" rank higher on the evidence scales than "yours". WhatamIdoing (talk) 11:28, 4 June 2015 (UTC)
When we are editing articles, our goal is, i assume, to reflect reality as well as possible, right? Editing an article exercises a lot of judgments by the editors, and if a particular source is deemed to be less reliable than others due to any reason whatsoever, i would expect that this is a reasonable topic of discussion and consideration toward the goal of writing the best article possible. Of course, if another source also comments on the unreliability of the initial source, then this is more evidence toward the unreliability of that source, but even without a source, i hope it's an admissible topic. I can see why you'd have caution against it, and i can see that it's apt to be an overused argument, because many people in controversial topics may be concerned about bias in sources, but i don't see how we can avoid this or rule this out completely as a topic to be discussed when need be. SageRad (talk) 21:47, 2 June 2015 (UTC)
No, the goal is to represent the mainstream/consensus view of reality without actually interpreting reality ourselves. Geogene (talk) 21:51, 2 June 2015 (UTC)
I agree with the premise that our goal is to represent the mainstream/consensus view of reality, based on reliable sources, but sometimes it happens that there is not a complete consensus on an aspect of reality, and competing perspectives do need to be addressed. One aspect of addressing this is the need to look at the credibility of the sources being used to justify claims by either "side" in a controversy, and one aspect of credibility of sources is sometimes the money trail. In cases where there is proven history of wrongdoing in the sense of distortion of science, as in the recent controversy of Willie Soon, for example, or the past controversy of IBT Labs, we need to be able to talk about the way that an agenda corrupted the science. These are cases proven in the past, of money influencing the reliability of sources of knowledge. However, we cannot just cast around an argument that *any* research funded by industry is necessarily biased. Anything claim to bias in research in this regard would hopefully be sourced decently in order to deserve any weight in a discussion. SageRad (talk) 00:17, 3 June 2015 (UTC)
Once there are reliable sources calling attention to the money trail, sure. Then we're obligated to cover it by NPOV. But I think the issue at hand is whether editors should be able to throw out an otherwise reliable source merely because it's funded by an entity they find problematic. I'm opposed to that, if the source is otherwise good. There are some things that Wikipedia just isn't able to protect itself from and one of them is what if enough reliable sources are all bought off or wrong or biased about something--in that case WP, as reflecting mainstream consensus, will inevitably follow. This systemic risk is much less of a threat than the intractable problems that would arise from letting editors do their own OR/investigative journalism in WP's voice. I think the proposal is WP:BEANS. Geogene (talk) 01:09, 3 June 2015 (UTC)

From that last reply, Geogene, i think we're in agreement. If there is reliable sourcing noting a funding bias possibility in a source, then that can be valid fodder for discussion about the reliability of the source, but we cannot just have the unsubstantiated allegations that any study funded by an entity is biased without evidence of some kind. However, i don't quite understand your reference to WP:BEANS. I think that it's best to have all our options laid out on the table, and know what's possible in editing Wikipedia. If one of those things is to note funding source biases if they're documented, then so be it. It will make a stronger encyclopedia in the long run. It may be a bit of a hassle, but that hassle is merely the reflection of the complexity of the world itself. I don't think it's suggesting some horrible practice like stuffing beans up one's nose (not recommended). SageRad (talk) 01:32, 3 June 2015 (UTC)

On second reading, perhaps i do understand what you mean about not having editors doing investigative journalism in WP's voice. Of course that makes sense. Sources must be substantial if someone is to make a claim that another source is not reliable due to funding issues. It's not the place to have original synthesis or investigative journalism being done. WP is a synthesis of knowledge from other sources. SageRad (talk) 01:37, 3 June 2015 (UTC)
What about sources that themselves mention a COI? Is that to be considered a reliable source? AlbinoFerret 22:54, 3 June 2015 (UTC)
I wouldn't think so. We would need an external reliable source stating the conflict is significant. After all, we are allowed to use biased reliable sources; it shouldn't matter that the bias is admitted by the author. — Arthur Rubin (talk) 01:36, 4 June 2015 (UTC)
AlbinoFerret, sometimes the most WP:BIASED source is the most authoritative one. The Coca-Cola Company has a serious COI with respect to their products, but if you want to cite what they're selling, then their corporate website is one of the best possible sources. WhatamIdoing (talk) 11:28, 4 June 2015 (UTC)
WhatamIdoing, that is often very true in terms of basic facts about products. However, i hope that a Monsanto-authored reassurance of the safety of Roundup herbicide would not be cited to claim that Roundup is safe, for instance. Interpretations are another matter, and are highly subject to bias of all kinds. I still hold that we as editors have brains and are expected to use them, and if a source smells bad, i will feel free to discuss the source, as well as to seek other sources that may comment on the source in question, but i will not feel obliged to hold my tongue on stating an issue with a source in a talk page. Is that what we're being asked to do by one side of this debate? SageRad (talk) 13:14, 4 June 2015 (UTC)
I think you may have misunderstood. You absolutely should discuss concerns with a source on the talk page until consensus becomes apparent. Geogene (talk) 18:19, 4 June 2015 (UTC)
Civil, collegial, collaborative discussion is good, but trying to force other editors to agree with your POV about the source is not. If the Monsanto-produced source is a higher type of evidence than the non-Monsanto source, then you cannot force other editors to omit it on the grounds that the better type of evidence was produced by a business that you distrust. You are never required to volunteer to use their sources; you are even welcome to politely and gently persuade people to prefer another source, or (if that fails) to recommend WP:INTEXT attribution and proper use of WP:YESPOV to provide contrasting viewpoints. But you cannot demand that editors use only lower-quality evidence because the higher-quality evidence was produced by the "wrong person". WhatamIdoing (talk) 05:39, 5 June 2015 (UTC)
Arthur Rubin Here is a hypothetical question. A critical review of a new (insert anything) is released. In the review a section titled Authors COI Statement is found. The statement sites that some of the authors hold stock in a competitors company, or have some form of employment there. Not just funding for the review. We would need to have a separate source? Shouldnt at the very least this be discussed on the talk page if that separate source is required for inclusion in the article of a COI and wouldnt that type of information lessen the likelihood of its use?. AlbinoFerret 14:20, 5 June 2015 (UTC)
Retrieved from "https://en.wikipedia.org/w/index.php?title=Wikipedia_talk:Identifying_reliable_sources_(medicine)/Archive_15&oldid=1136060011"