Wikipedia talk:Factual review

This is (or was) a discussion page for factual review. It is intended as a venue for providing feedback on factual review, and all editors are invited to post here in accordance with policy.

The current discussion page for factual review may be found at Wikipedia talk:Factual review. There are no current archives.

---

Wikipedia is increasing fast in popularity, and is being used more than ever before. However, with greater reading dependence comes greater editor responsibility: all Wikipedia community members have an obligation to ensure that encyclopedic content contained within Wikipedia is as accurate as possible.

I recently created a proposed accuracy-reviewing system (at Wikipedia:Reviewing) which "restricted" participation in such reviewing to trusted "experienced, honest, and objective" editors. After a lack of community support for this system, I rethought my opinions on what is needed in a system for checking Wikipedia's accuracy, and concluded that restricted reviewing is not necessary, as I will explain in due course. In the factual review proposal that I now present, I have dealt with or defended all of the concerns that I know have been presented in regards to my old reviewing proposal.

Firstly, however, I will state that I believe Wikipedia:Peer review, Wikipedia's main existing "review" system, focuses mostly on technical aspects of articles; it certainly goes to little or no lengths to check articles for accuracy in an organised manner. My opinions on this matter are supported by multiple other community members; for example:

• RobinBK: "... perhaps there may be a systemic bias within peer review that focuses more on MoS structure than verified claims ..."; and
• Walkerma: "... [i]t seems to me that true peer review should be focusing more on the content rather than the technical aspects anyway ..."

RobinBK, referring to my old reviewing proposal and "peer review", said, "... [Y]ou don't fix a flat tyre by rebuilding the whole car." However, I do not believe that "peer review" is "flat" as such; I believe that it performs a different — and needed — task than the factual review system I propose. You don't fix a flat tyre by rebuilding the whole car, but you also don't try to rebuild a Grand Prix racing car to go four-wheel-driving on a riverbed or vice versa. Likewise, you don't try to build a Grand-Prix-racing-riverbed-four-wheel-drive vehicle; it is much better to have two separate cars for the two purposes. In other words, this is not about re-building, but about building a new and additional system to perform a separate task.

Therefore, I propose that Wikipedia:Peer review be moved to Wikipedia:Technical review in the future; this is a fairly major move, however, and I am in no hurry to rush the matter. GA and FA also provide basic reviews, which also seem to check articles solely for technical accuracy. (I have little experience in these areas, so I invite constructive criticism if I am wrong in some way.)

A factual review system is needed because (a) readers have the right to expect high-quality accurate content, and editors have the obligation to ensure that readers either get accurate content or no content, and (b) as previously stated, no existing active, everyday system appears to engage in checking articles' accuracy in an organised manner. Therefore, what is required is an open factual review system which does not focus on technical aspects of articles but instead on their factual accuracy issues, and this is why I propose factual review. "Factual review" is similar to "peer review" in many ways, especially in its openness to all Wikipedia editors, but differs in what it reviews: factual accuracy instead of technical accuracy.

Factual review, in order to be successful, needs:

• Lots of participation. Wikipedia articles are written about diverse, widespread, and varied subjects, and factual review needs participants who know such subjects well.
• A professional attitude amongst participants. Instead of arguing over accuracy aspects of articles, I expect all participants to discuss said aspects in a professional, collaborative, and cooperative manner, and I will make a large effort to ensure that I act in such a way.

My previously-suggested reviewing system aimed to assign "green ticks" to articles which had been reviewed and thereafter modified so that all errors located in the review had been corrected. I now believe that this is not necessary, since the flagged revisions MediaWiki extension, which will likely be introduced to the English Wikipedia in the future, provides at least two major functions:

• "default revisions", which would allow certain editors to set the default revisions for articles to display at for logged-out users; this would allow us to set up a stability system; and
• "ratings", which would allow certain editors to rate various aspects of an article's quality, therefore possibly replacing a need for "green ticks" on reviewed pages (of course, there is little point in saying, "This article's accuracy is 90%!", because accuracy issues should be corrected when found instead of merely being recognised).

Discussion on this page in regards to the flagged revisions extension and its features, including how factual review and flagged revisions could be integrated and work together, is mainly off-topic at this time. I look forward to participating in discussions surrounding flagged revisions, including its benefits for stability and ratings, and I hope that you will do the same.

This factual review system could serve, in my humble opinion, as a practical and useful procedure to check, in an organised manner, Wikipedia articles for factual accuracy. I invite your constructive input on this proposed factual review system.

Best and friendly regards,

— Thomas H. Larsen 08:09, 21 February 2008 (UTC)

Comments

This sounds a lot like what a Fact checker does for a magazine (and I prefer a name like "Fact checking" to "Factual review"). While this could be a quite positive contribution to Wikipedia, the devil is in the details and I have several questions.

1. Would this be a check of every fact or a random selection of facts? Every footnote / reference or a selection? Would you check that the reference cited exists or that it backs up the claim it is used for? What level of detail will be required?
2. At what point in the article writing process would this be done - GA? FA? anywhere? The reason I ask is that articles will change over time, especially as they are being improved, so this may have to be done more than once.
3. Since, in general, anyone can add anything to any article at any time, how would it be clear what parts of an article had been fact checked in the past and what had not?
4. On a related note, internet sources change over time too - links that worked when an article was written or passed FA or GA may no longer work a few months or years later. News items are especially prone to being available online for a brief period of time, then vanishing (and not being found in the common free archives). Would this be a check of the facts or of the sources, and since online sources change, how would you deal with that?
5. I see this as potentially the biggest problem - how do you fact check obscure print sources? I use several print references that are very hard to come by (not in most libraries, not online even with Google Books) and I run across similar sources in other articles frequently. How about subscription only refs (newspapers were mentioned above, but many scholarly journals are online only for subscribers)?
6. If you do not know about it, you might want to look at the Suggestions page of Did you know, where the hooks that are nominated are checked for length, time since creation / expansion, and that they match the citation provided. This is just checking one fact per article, but may give you some ideas.

Thanks for your work on this, Ruhrfisch _><>°^° 16:47, 21 February 2008 (UTC)

(Updated below.) Thanks for your comments; apologies for the following concise comments, but I only have 12 minutes of Internet time left on this library computer ...

Well, there are facts and there are facts. Obviously which facts get checked is going to be a function of the dedication of participating reviewers, and hopefully they will collaborate and cooperate. Ideally, the principle would be to check every substantial claim made in articles for correctness and full coverage of all available viewpoints.

Factual reviews could be carried out at any point, although I would support them being carried out before granting articles FA status. This is open to debate, of course!

Factual review would likely cover sources as well as factual accuracy of claims, although this might be better suited to a project such as WikiProject Fact and Reference Check.

It would be necessary for a lot of involvement in libraries, especially university libraries, to acquire copies of obscure resources.

Thanks again, and best regards, — Thomas H. Larsen 09:12, 3 March 2008 (UTC)

I have had a little time to form my responses to your questions in more detail (ahem), and I provide them respectively as follows.

1. Factual review would essentially be a check of all facts in an article for verifiability, neutrality, and non-original-research as per Wikipedia's three core content policies, Wikipedia:Verifiability, Wikipedia:Neutral point of view, and Wikipedia:No original research, respectively. If a fact was verifiable, neutral, and was not original research, then it would be "correct" for factual review's purposes. Neutrality also refers to adequate coverage of all available viewpoints. Thus, this has two main implications:
   • this would be a check of every factual claim in articles for correctness and coverage;
   • this would be a check of virtually every footnote/reference in order to check that articles complied with Wikipedia's verifiability policy, including checking that references existed and supported the claims they were used for; this might not be as difficult as it could sound.
2. Factual review could be carried out on any article at any stage provided the article was well-written; this as per Wikipedia:Factual review, "... [a]ll editors are invited to initiate reviews on well-written articles ..." I would also support factual review being carried out prior to articles passing featured status.
3. It would not be clear which parts of an article had been fact checked in the past and which had not, and I agree that this is somewhat of a problem. In the future, I anticipate flaggedrevs being used to provide a stability system and a rating system, and this might or might not provide a way to get around this issue. I invite any ideas you or others may have in regards to this!
4. Similarly, there is little or nothing that Wikipedia can do about the natural cycles of Internet sources — birth, availability, and death (or at least archival, and the archives are often placed somewhere so inaccessible or unfindable that they are impractical to find, or, even worse, stored in a system which requires retrievers to pay to acquire them). Wikipedia:Citing sources#What to do when a reference link "goes dead" says that "... [w]hen a link in the References section or Notes section [of an article] "goes dead", it should be repaired or replaced if possible, but the citation need not be deleted." Factual review would have no other alternatives than to do just that, at least as far as I can think. Have a look at the section I just referred to for more information.
5. A possible, although not elegant, solution to the "biggest problem" of checking articles' citations of obscure print sources would be to maintain a Wikipedia list of trusted editors who have access to certain journal subscriptions, rare books, and otherwise difficult-to-track-down print resources; when a reference to one of these resources needed to be checked, one or more of these editors could be contacted to check it on behalf of the community. Finally, a significant number of Wikipedians likely have access to a university library (I do, although as I'm not a student or staff member I cannot acquire resources on loan), and this may be another method of locating and tracking down elusive print sources. Therefore it is unlikely that checking rare print sources would be factual review's biggest problem, at least if such a list could be implemented.

By the way, I must apologise for a little handwaving: I previously said "... hopefully they [factual reviewers] will collaborate and cooperate ...", with which I dismissed the question of how editors would actually check articles without unknowingly unnecessarily overlapping their review work with that others or accidentally missing out parts of articles while reviewing. While I am entirely open to suggestions, one way that I feel the problem of organisation of article real estate could be overcome would be to (robotically?) list the name of each section in the articles on the respective factual review pages so that editors could assign themselves to factually review a certain section (or sections) of an article (of course, more than one person could factually review one section). In this way, the workload might possibly be diminished and editors might be more specifically suited to the content that they review: for example, in a theoretical article on pipes, two or three editors could review the "History of pipes" section, two or three editors could review the "Construction of pipes" section, two or three editors could review the "Uses of pipes section", and so on, ensuring that all sections were reviewed and were reviewed as well as possible.

Thank you again for your comments, and I wish you all the best, — Thomas H. Larsen 09:10, 13 March 2008 (UTC)

In my opinion, if Peer review, GA review, or FA review, due to a systemic bias, do not in fact produce factually accurate articles, then they should be made to do so through a greater emphasis on fact checking within those processes, rather than to create a separate fact checking system. Furthermore, factual review should take place on an article's talk page, among editors knowledgeable about the subject, rather than on a review page, among a group of reviewers who may not be knowledgeable about it. --Michael White^T·C 03:16, 22 February 2008 (UTC)

I beg to disagree. As noted above, there is little point in attempting to replace a perfectly good system that performs technical reviews with a sytem that performs factual reviews, because Wikipedia articles need both types of checking at different stages. In addition, it is not practicable for factual review to take place in the discussion pages of articles instead of on a widely-accessed venue; some articles receive little attention (did you know about and were you prepared to review Wikipedia's article on coccolithophores?). As per Wikipedia:Factual review#Initiating a review, "... [i]t is ... considered necessary to post a brief explanation and link of and to the factual review page from the article's discussion page ... [t]his allows interested editors reading the article's discussion page to comment on the review." Finally, factual review places reviews into different classifications, which have the potential to be divided into sub-classifications and even sub-sub-classifications if necessary, and editors can comment on reviews in classifications that they are knowledgeable in. As you undoubtedly know, Wikipedia articles' discussion pages are open to all editors whether or not they are knowledgeable about the respective articles, and, even though I dislike it, editors knowing little or nothing about articles still participate on these articles' discussion pages and try to act as Great Protectors of Wikipedia's Accuracy. Best and friendly regards, — Thomas H. Larsen 09:10, 13 March 2008 (UTC)

I'm on your side, Thomas, but there is a lot of attraction among Wikipedians to the messiness, the chaos, and I'm not unsympathetic to the majority view. If the idea behind changing the name Peer review to Technical review is that you'd like for all the peer reviewers to acknowledge that the end product is often not very factual (and by implication, not very good), so a separate project is needed...well, good luck with that. On the other hand, both the WP:Version 1.0 people and the Veropedia people are likely to be very much in your camp. And dealing with these problems from a Version 1.0 perspective overcomes the criticism that the facts could be changed the minute after they're all certified.

I notice you're into IT...wonderful, the IT articles in particular could use more fact-checkers. I'll be happy to lend my support, but I'm not sure how long I'm going to watchlist this page, so please give me a heads-up if I can help. P.S. I'm fact-checking in IT articles as we speak, in one of these damn windows... - Dan Dank55 (talk) 15:18, 28 February 2008 (UTC)

Just to clarify, I feel that "peer review" does a good job, but it is misleadingly named. Of course articles need to be checked for design and style issues — how they are written — but articles also need to be checked for correctness and coverage — what is written. Cheers! — Thomas H. Larsen 09:15, 3 March 2008 (UTC)

As I understand it, peer review is intended to do both. Perhaps peer review can be improved to check facts better, but I see no need to separate the two - they are different things, certainly, but they can be checked at the same time. --Tango (talk) 01:40, 21 March 2008 (UTC)

Not good

I don't really like this, it's unnecessary beuracraccy and no one writes false stuff anyway. Even backwater stubs get 2+ views a day and will be corrected--Phoenix-wiki 21:52, 24 February 2008 (UTC)

No one writes false stuff? Please tell me your kidding. Wikipedia has had incidents where false stuff has remained in articles for months on end until it was pointed out by outsiders. This seems like a great idea to me.--Urban Rose 23:10, 27 February 2008 (UTC)

Phoenix-wiki: I am afraid that you are incorrect in your statement. Let's not argue about this; ask any established community member whether some malicious editors — non-community-members — write incorrect stuff. Best and friendly regards, — Thomas H. Larsen 09:04, 3 March 2008 (UTC)

Operational

Factual review is now operational due to the fact that no objections were raised during the last ten days (to be honest, no participation whatsoever occurred during the last ten days, either). If there are any serious justifiable objections, feel free to revert my removal of the "proposal" templates and announcement post on the Community Portal.

I am writing a factual review HOWTO which I hope to upload in the near future; I invite all suggestions, comments, and questions that you may have.

Best and friendly regards to you all, — Thomas H. Larsen 09:18, 13 March 2008 (UTC)

Could this be merged with Wikipedia:WikiProject Fact and Reference Check? With Wikipedia the size it is, we really have to watch out for Instruction creep, redundancy, and duplication of efforts. The more pages/instructions that people have to read, the less likely they are to participate. I especially recommend a merge, based on the current lack of participation here (See the recent but DOA Wikipedia:WikiProject critical source examination). Thanks. -- Quiddity (talk) 19:41, 31 March 2008 (UTC)

what does "technical accuracy" mean?

""Factual accuracy" refers to the accuracy of what is written; "technical accuracy" refers to the accuracy of how it is written." This doesn't really clarify anything. Does "technical accuracy" mean following the manual of style or something?P4k (talk) 00:10, 28 March 2008 (UTC)

• It sure seems to me that is exactly what is meant: Facts = What, Tech = How. Example: an inexperienced user creates a small article and includes a reliable source in the form of a straight hyperlink. The user fails to include a stub template and the hyperlink should probably be a WP style inline reference using a {{cite}} template. Yet if every point in the article is in 100%agreement with the source then it is factually accurate even though a lot of work needs to be done to make it technically acceptable for WP. Low Sea (talk) 21:22, 29 March 2008 (UTC)
  • Low Sea said it perfectly. — Thomas H. Larsen 04:20, 7 April 2008 (UTC)

Fact Reviewing Board?

I don't understand who is going to verify and reference every fact cited in each WP article. I mean, is it going to be a full time job? How much is the starting salary? What qualifications are needed? What about discipline? JeanLatore (talk) 20:24, 1 April 2008 (UTC)

Anybody who wants to factually review articles will be welcome to, but they will not be paid. Qualifications are not needed as yet (if you feel they would be needed, you're welcome to suggest a way of verifying qualifications). Cheers. — Thomas H. Larsen 04:22, 7 April 2008 (UTC)

Note from Thomas, in regards to several issues

The issue of "know-it-all" users leaping into factual review as if they owned it could be a potential problem, but one that should be solved by an authoritive standard for factual review participaion which restricted participation of editors to subjects they were knowledgeable in. I know that actually enforcing this could be troublesome, but difficult editors are always going to be a plague on Wikipedia and we need to accept and work around this instead of stopping valid, useful, helpful, constructive contributors from checking articles effectively and efficiently on a standard, centralised venue.

The alternative to factual review — expert review — could be worse. A review system restricted only to experts would require a recognised board of people to ensure that they really were experts, and many questions would likely be raised about criteria for becoming an expert review, how expertise should be verified, who should be on the board, etc., creating a mess, if not bureaucracy, parallel to administratorship. If there is high support for expert review, I'd be happy to write, or to work with others to write, a proposed expert review system.

Therefore, debate in regards to factual review comes down to three main things:

• do we need factual review, and, if so, why?
• if factual review is needed, when and where should it be implemented?
• if factual review is needed, how should it be implemented?

My current proposal for factual review likely is imperfect, but imperfection can be subject to improvement. Do we need factual review, and, if so, should we improve my proposal, write a new one, or somehow merge it into an existing system such as "peer review"?

As a note on merging factual review into "peer review": I personally don't think that my current proposal of factual review could be constructively merged. Factual reviews would be so different from technical reviews that a compromised, "united" system would simply be destructive, inefficient, and ineffective: one example is the classification of factual reviews into different categories based on topic. Technical reviews can't be classified based on topic, since technical requirements do not differ (much) over various articles. Any editor, expert or not, can know that ideal articles will have an infobox, one or more images, etc.

As to merging factual review into WikiProject Fact and Reference Check, I don't have such a strong case against merging, although I feel that my proposal promotes a very organised, structural procedure of reviewing articles, while WPFARC is quite loosely organised. (If someone can convince the people at WPFARC to adopt certain aspects of my proposal, I'd be happy to merge. However, I do not have enough time on my hands at present to get involved in a possibly long, drawn-out, controversial discussion.)

Best and friendly regards, — Thomas H. Larsen 09:40, 10 April 2008 (UTC)

I have also strongly outlined participatory standards/expectations for potential factual review participants; read Wikipedia:Factual review/participants. — Thomas H. Larsen 09:43, 10 April 2008 (UTC)

It might be worth getting in touch with someone at, or creating something for, either Wikipedia:School and university projects and/or Wikipedia:Wikipedia Signpost/Series/Tutorial. Potentially a good source of minions volunteers. (I have no experience with either of them) -- Quiddity (talk) 04:55, 11 April 2008 (UTC)

Personally, I see a strong potential for school and university participation in factual review, and, if factual review is accepted by the community, I'd definitely support such engagement. — Thomas H. Larsen 01:02, 14 April 2008 (UTC)