Wikipedia:Wikipedia Signpost/2009-02-23/Philosophers analyze Wikipedia
Philosophers analyze Wikipedia as a knowledge source
The philosophy journal Episteme has published a new issue that focuses on Wikipedia from the perspective of social epistemology. It contains four articles that examine various aspects of Wikipedia as a source of knowledge, including one by Wikipedia co-founder Larry Sanger.
The issue is introduced by University of Arizona philosopher Don Fallis, who recently authored an article in the Journal of the American Society for Information Science and Technology, "Toward an Epistemology of Wikipedia", arguing that on the whole Wikipedia has good "epistemic consequences" and that its virtues outweigh its flaws. In his introduction on "The Epistemology of Mass Collaboration", Fallis presents some of the epistemological issues raised by Wikipedia's success—chief among them "whether large collaborative projects, such as Wikipedia, can be reliable sources of information."
Wikipedia and the Epistemology of Testimony
The first article, University of Memphis philosopher Deborah Perron Tollefsen's "Wikipedia and the Epistemology of Testimony", explores the concept of group testimony as a possible basis for understanding Wikipedia's authority. It builds on her earlier work, in which she argues that group testimony is fundamentally different from the testimony of an individual, since the testimony of the group itself may be different from the testimony of the individuals who make it up.
Tollefsen's conclusion is that some Wikipedia articles might be considered a form of group testimony, particularly when they are "mature" and represent the consensus of many editors and reflect the norms of the community, but others are better thought of as the individual testimony of their main author or authors. She goes on to characterize Wikipedia as "an immature epistemic agent", the claims of which—like a child's testimony—ought to be scrutinized carefully, rather than given the benefit of the doubt like the testimony of an adult. However, she finds that the traditional methods of scrutinizing face-to-face testimony do not translate well to Wikipedia and other virtual testimony. Instead, she argues, "receiving testimony from a source such [as] Wikipedia involves trusting not the man, but the system." According to another view of testimony, it is not the testifier's reliability but the testimony itself that should be scrutinized, by comparing it to other sources and to the "vast backdrop of beliefs the hearer has acquired" beforehand. This view of the epistemology of testimony is easy to extend to Wikipedia, Tollefsen argues.
The Epistemic Cultures of Science and Wikipedia: A Comparison
The second Wikipedia-focused article is "The Epistemic Cultures of Science and Wikipedia: A Comparison", by State University of New York at Oswego philosopher K. Brad Wray. In it, Wray considers Wikipedia as a community focused on inquiry and knowledge production, analogous to the scientific community. However, he draws sharp contrasts between the norms of science and motivations of scientists, on the one hand, and the norms of Wikipedia and motivations of its editors on the other. Although both science and Wikipedia are collaborative knowledge projects, they have very different "epistemic cultures".
Wray posits that one possible justification for trusting Wikipedia is an "invisible hand" argument: although no identifiable individual or group of individuals ensures the quality of information on Wikipedia, "the knowledge-market will take care of itself, and poor articles reporting false claims will be rooted out." According to Wray, while science does have a viable invisible hand, in the form of a reputation system that relies on peer-review, Wikipedians "lack the sorts of incentives that keep science in good working order", and face few or no consequences for mistakes.
Wray also explores what he calls the Wikipedia's "puckish culture", prone to gossip and practical jokes. He recounts the Seigenthaler incident, and contrasts it to the sober culture of science. In science, he says, "the closest incident to such a joke is the Sokal affair"; however, the Sokal affair should not be considered a joke, but rather a demonstration of "the editors’ appalling ignorance of science". This regrettable aspect of Wikipedia's culture, he suggests, might be absent if—as in science—"one had to wait months before one’s contribution is posted". Finally, against the argument of Deborah Perron Tollefsen's article, Wray argues that even when considered as a form of testimony, Wikipedia is a flawed source of knowledge, precisely because of the failings in its "epistemic culture".
Despite his negative assessment, Wray does find one ray of hope: "What Wikipedia can do for us is to draw greater attention to epistemology and its relevance to our place in the social world. Though we live in a time in human history when knowledge may be easier to obtain than ever before, we are in desperate need of means to sort and evaluate what passes for knowledge."
The Fate of Expertise after Wikipedia
| “ | Over the long term, the quality of a given Wikipedia article will do a random walk around the highest level of quality permitted by the most persistent and aggressive people who follow an article. | ” |
| — Lawrence M. Sanger | ||
A third article about Wikipedia comes from philosopher Lawrence M. Sanger—i.e., Wikipedia co-founder Larry Sanger, who is also the founder and editor-in-chief of wiki encyclopedia Citizendium. In "The Fate of Expertise after Wikipedia", Sanger explores the paradoxes and shortcomings of Wikipedia's relationship with experts and expertise, and suggests that Citizendium, a project that explicitly grants authority to expert contributors, is a better alternative.
Sanger describes Wikipedia's success as an egalitarian and open knowledge project, and then poses the question of how to reconcile the project's successes—both real and potential—with its lack of "any special role for experts or any expert approval process". One implication from such success might be that special roles for experts are not necessary in the rest of society either. However, Sanger shows this to be self-contradictory, in part because evaluating the success of Wikipedia requires expertise to compare it against, and in part because of Wikipedia's indirect reliance on expertise.
Sanger goes on to explore the actual roles of expert editors on Wikipedia, and whether Wikipedia itself could become an authoritative source without granting a special role for experts. He asserts that "Wikipedia is nothing like the egalitarian utopia its most radical defenders might have us believe", and that in practice experts are often given deference. This, according to Sanger, is the key to what success Wikipedia has had in creating authoritative articles on some topics. However, problems arise when such deference breaks down, as is likely to occur for non-technical topics. As an a priori hypothesis, Sanger suggests that "Over the long term, the quality of a given Wikipedia article will do a random walk around the highest level of quality permitted by the most persistent and aggressive people who follow an article." He argues that Citizendium's model, in which subject-matter experts are given final authority over content in their areas of expertise, can surmount such problems caused by persistent and aggressive non-experts.
Sanger's article has attracted some press attention and blog discussion, especially for his idea about the limits of quality on Wikipedia. It was discussed on Slashdot, although Sanger suggested on the Citizendium Blog that many commentators did not "RTFA". Sanger's contribution was also discussed by The Chronicle of Higher Education.
On Trusting Wikipedia
The final Wikipedia-focused article is "On Trusting Wikipedia", by State University of New York at Albany philosopher P. D. Magnus. Given the wide variability in article quality on Wikipedia, Magnus sets out to identify strategies for assessing reliability and to examine how well those strategies apply to Wikipedia.
Magnus describes five common strategies for assessing the reliability of other online knowledge sources, all of which fail to some extent when applied to Wikipedia.
- Authority may be a good basis for evaluating online sources when the individual or institutional authors have relevant connections or reputations; for the Wikipedia, this breaks down because of the anonymity of many authors, as well as the changing nature of articles that have been vouched for by outside authorities at a particular point in time.
- Plausibility of style—whether or not an author seems to understand the style and terminology of the topic at hand—can be a useful indicator of reliability; on Wikipedia, this is confounded by collaborative copy-editing, which may improve only the style of bad content or introduce implausible elements of style in otherwise good content.
- Plausibility of content—watching out for things that are obviously wrong—similarly fails on Wikipedia, because the most egregious errors, which might serve as a warning against additional subtle errors, are also the most likely to be corrected—leaving undetected errors behind.
- Calibration—testing a subset of claims against an independent source of known reliability to gauge a source's overall accuracy—is also difficult for Wikipedia content, since easily-checked claims are also the ones most likely to have been checked by other Wikipedia users, while harder-to-check claims may be less reliable.
- Sampling—checking any given claim against multiple other sources—can also sometimes be misleading with Wikipedia content, since Wikipedia is widely reproduced and changes frequently; two seemingly independent sources may both be derived from the same Wikipedia content and contain the same errors.
Magnus concludes that "teaching people to engage Wikipedia responsibly will require getting them to cultivate a healthy scepticism, to think of it differently than they think of traditional sources, and to learn to look beyond the current articles".
Discuss this story