User:AxelBoldt/References don't increase Wikipedia's reliability

Adding sources and references to statements in Wikipedia is very useful, because it facilitates the readers' research. However, it does not increase the reliability or trustworthiness of Wikipedia in any way, and we should counter that misconception wherever it crops up.

Mistaken sources

We only allow "reliable sources" in Wikipedia, but reliable sources are frequently mistaken.

Virtually every peer-reviewed paper in mathematics contains some mistakes, and it wouldn't be difficult to enter all sorts of incorrect mathematical theorems into Wikipedia, carefully sourcing every single one of them with a peer-reviewed paper by an established research mathematician. Textbooks contain even more mistakes, and they are also considered reliable. For example, waged workers built the pyramids and Queen Victoria never said 'We are not amused'.

Results reported in the scientific literature are often later overturned or invalidated, for example if the experimental results cannot be independently reproduced or fraud is discovered. Such a discredited source clearly does not support the claims made, but the average reader without broad knowledge of the literature in the field will not be able to distinguish reputable from discredited articles. For example, most people still believe that unprotected intercourse is to be avoided because of the risk of sexually transmitted diseases, unaware of the fact that the anti-depressant properties of semen have been known for several years. Similarly, most laymen are not aware of the fact that the wearing of bras contributes to breast cancer.

Does the source support the given statement?

In many cases, the question whether a given source supports a given statement is highly non-trivial and can require substantial research.

The abstracts and introductions of scientific articles often contain quite far reaching statements; later sections of the article then usually give several caveats, qualifications, and alternative interpretations of the data, and typically call for future research to sort things out. Quoting a sentence from the abstract may be misleading if the source, taken as a whole, does not actually support the statement.

Selective quoting of sources can also be done maliciously, to support claims that the source's author would never have made in that form. When checking a book reference, one has to read the entire book and not just the page that's being referenced.

Sources that are hard to check

It is quite easy to manufacture bogus references to non-existing sources, or to sources that are hard to access and cannot be checked by most readers. This important point was already made in medieval times and has recently played a role in a German news scandal.

Statements changed after the fact

Wikipedia is not static. Statements, even if referenced, are often changed by later contributors. In many cases other editors do not or cannot ascertain whether the given source still supports the new statement, and so we can expect that over time fewer and fewer of the given references actually support the stated claims.

Missing statements

Even if every single statement in an article is properly referenced with a reliable and correct source, the article may still be extremely misleading and wrong. It is just as easy to lie by omission than it is to lie by making false statements. (Simple example: an article may correctly report that a person was convicted of a crime, but fail to report that the conviction was overturned on appeal three years later.)

Conclusion

If you haven't personally determined that a given source

• exists,
• supports the given statement when taken as a whole, and
• is not mistaken,

you should not place any higher trust in a sourced statement than in an unsourced one.

By extension, if you are not in the habit of regularly verifying sources in this manner, then the lack of a source for a given statement should not reduce your trust in the statement's truth.

Last but not least, you always must consult sources that are not given in the article to ensure that the article does not lie by omission.

Appendix: How to determine whether a given source is mistaken?

Leibniz dreamed of a general mechanical procedure that could be used to check the truth of any given proposition. Turing proved much later that such an algorithm cannot exist, not even if you are only interested in mathematical statements.

So you are out of luck; checking for truth is out. The best you can hope for is to determine whether a sizable majority of the people working in the field currently agree with the proposition under consideration. It's not a quick process tough. Here's what to do to:

• First you need to be familiar with the general terminology, assumptions and methodology of the field. If you don't know the standard interpretations of Hamlet, you can't evaluate statements about Shakespeare's work; if you don't know what "income elasticity of demand" is, you are in no position to evaluate claims in economics; if you don't know how a retrovirus functions, you cannot evaluate claims about AIDS. So walk over to your friendly neighborhood college library and borrow a standard textbook in the field, or watch lectures of an introductory course online.
• Then you need to read and understand the source in its entirety.
• Next you need to be familiar with the standard literature database in the field. The reference librarian of your friendly neighborhood college library can help you there. Most of these databases aren't freely accessible on the internet, but your friendly neighborhood college library pays for them and will let you use them for free.
• If the source you want to evaluate is a book, you need to find reviews of the book in standard journals. If the book hasn't been reviewed, chances are that it didn't have any impact and should be discounted.
• If the source you want to evaluate is an article, you need to find other articles that reference it. If there are none, chances are that it didn't have any impact in the field. Otherwise: did the referencing articles reproduce the original results, cite the original article approvingly, or criticize it?

Comments?

Please add your comments below.

The CITATION CABAL consists largely of Johnny-Come-Latelies to the Wikipedia project who missed out on the long, largely enjoyable work of writing articles and now spend most of their time generating additional tasks to be done. We must fight the CITATION CABAL. We must root them out and smoke them from their hiding places. -RatSkrew 04:22, 15 November 2006 (UTC)

Perhaps the Project should isolate itself? Just a thought inspired by your essay... --Santa Dog 72 22:12, 29 December 2006 (UTC)

I generally use citation needed when I, personally, want to know where they got a fact from. Usually because it looks interesting or surprising and I want to be able to read into it further. --tjstrf talk 12:39, 5 January 2007 (UTC)

The two examples of unreliable sources were rather topic-specific. What about the prevalence of mistakes in reliable sources used for literature topics, the social sciences, and the arts? I can see how the natural sciences and mathematics would, by their very nature as experiment- and peer-review driven disciplines, have published, conventionally reliable sources containing errors. What about the other disciplines who do not operate on this model? What are their specific issues in regards to this? -Fsotrain⁰⁹ 17:07, 5 January 2007 (UTC)

Well, yes, you can say this about anything that quotes its sources, whether online or off. However, referencing articles does make them more verifiable, which is equally important. -sthomson06 (Talk) 19:11, 5 January 2007 (UTC)

I'm in agreement with User:Sthomson06. Have you listed De scribere et veritas merely to illustrate the point that sources are hard to check? It seems that it doesn't exist. If this is your method of reasoning, I have to say that you have committed a logical fallacy. This argument is essentially "because I have listed non-exesitent/non-accessible reference, the use of references is not helpful". Even though you have demonstrated that it can be done, an important cornerstone of Wikipedia is to assume good faith. In particular, assume that editors , and that editors are not going to fabricate false references. The beauty of Wikipedia is that multiple editors are likely to contribute material, and even if one of them does mistakenly contribute a bogus reference, the references of the others should keep that balanced. Also, references that are difficult to check are just as valid as easy-to-access references. − Twas Now 17:44, 7 January 2007 (UTC)

In the words of the late, great Arte Johnson, "Very interesting... but STUPID."

Of course citations may be false and misleading, and occasionally intentionally so. And the inclusion of a citation does not per se improve the validity or importance of a written statement ("Let there be Light (17)" for example.) But conceptually it is antithetical for a reference work to make a stand against citations. It's just dumb. — Preceding unsigned comment added by Guard Pigeon (talk • contribs) 16:06, 23 January 2007

I've been rewatching the Sopranos recently on DVD.

Dr Melfi: Last time [stuff happened]. How are you with that?

Tony Soprano: I can't just turn off my feelings because you tell me they're a side effect of therapy.

Dr Melfi: I'm not asking you to shut off your feelings.

The author of this essay is not arguing in favour of a Wikipedia without references. The spirit of the essay is more nearly "who is watching the watchers?" If you hire a bunch of thugs to fend off the mobsters shaking down your construction site, you'll just find yourself shaken down by the new set of thugs, more lawless than the first. Citations have very little intrinsic value in the absence of effective scrutiny. The reference system as it presently exists is seriously flawed. It is also a work in progress. We should respect the citations that presently exist more as a gesture in the right direction than for their present value. I think in the long run what is required is a companion wiki that functions as a citation review, which can establish a permanent dialog (and a working consensus) of the strengths and weaknesses and misconceptions (in or about) the reference work. Scholarship is about leaving trails. Why should the Wikipedia adopt the rather clunky academic citation system, when a much more powerful notion of verification trails is built into the media itself? It seems to me Tony needs to work on his communication skills and not jump past the essential question because he lacks imagination. MaxEnt 01:24, 27 January 2007 (UTC)

---

Interesting points, but I probably disagree. In my experience, a demand for cites, especially in an article that already includes high quality references, is easily the best approach to take when confronting a POV pusher. It certainly makes Wikipedia more useful, as you state. In particular I've noticed some interesting, if minor, patterns in authorship when citations are well done, e.g. the regular Darfur BBC correspondent writes knowledgeable and skeptical articles, as opposed to some other BBC writers, who seem to either be writing from London or have spent two days in Darfur before writing superficial or breathlessly excited articles. Also, many articles I work on turn out to have contradictory sources. Cites allow me to come back later and evaluate which is the more reliable source, or if both need to be mentioned, without straining my brain as to where they heck I had gotten the info originally. - BanyanTree 19:47, 15 January 2007 (UTC)

References may not guarantee trustworthiness, but not having references guarantees untrustworthiness, and an untrustworthy encyclopaedia is not an encyclopaedia at all. --Sam Blanning^(talk) 23:37, 16 January 2007 (UTC)

• Template:Citecheck. Odds are at Wikipedia that when you discover a problem, someone has already created at least one solution for it. Durova^Charge 22:02, 17 January 2007 (UTC)

Nice essay. I agree with every word of it. Verifiability was always a noble idea, but anymore it (or rather, a skewed view of it) is promoted as the last word in article quality, causing a strong "form over substance" slant. The policy is still a good thing, but the culture surrounding it needs to change (though it won't at this rate). P.S. - I hope you won't mind if I link your essay from my user page. -- mattb 03:25, 29 April 2007 (UTC)

Yes, this is a good one. Strong support. Kayau Voting IS evil 14:33, 19 February 2010 (UTC)