Wikipedia:Featured article candidates/Featured quality source review RfC

Currently a nominated article may be declared to be of "featured" quality: through the featured article candidates page (FAC) or the equivalent page for lists (FLC). Both FAC and FLC require at least one review to address sources before a candidate can be promoted.

This RfC proposes a new, featured quality review process: featured quality source reviews (FSR). The sections below give details about the process. Please add comments to the discussion section, and indicate support or oppose in the poll section below. 08:11, 16 October 2018 (UTC)

FSR process details

Any article may be nominated, and any editor may review any nomination. Each article must satisfy the following criteria (parentheses refer to bullet points in the featured article criteria):

• (1c): well-researched: it is a thorough and representative survey of the relevant literature; claims are verifiable against high-quality reliable sources and are supported by inline citations where appropriate;
• (2c): consistent citations: where required by criterion 1c, consistently formatted inline citations using either footnotes (<ref>Smith 2007, p. 1.</ref>) or Harvard referencing (Smith 2007, p. 1)

Examples of source reviews as currently conducted at FAC: Sandringham House, The Cloisters, and Nihonium. Reviews that find no problems can be very short: an example is at the end of V. Gordon Childe.

Reviewers are expected to make it clear what criteria they have evaluated; a review may be completed by multiple reviewers, so long as both criteria are fully evaluated. Reviewers may either declare support or oppose, or wait for the article's editors to correct any issues before making a declaration.

A coordinator (there may be more than one) determines whether an article passes the source review. An article cannot pass unless at least one reviewer declares support. If more than one reviewer provides a source review, the coordinator determines if there is consensus to support. The coordinator may at their discretion ask additional questions of either the reviewer or the nominator, if they feel that either criterion has not been fully addressed. A coordinator cannot post a "Pass" or "Fail" for an article they themselves have reviewed. The initial coordinators will be the three FAC coordinators: Ian Rose, Sarastro1, and Laser brain. The community may choose new coordinators at any time.

FSR and FAC

One of the goals of the RfC is to require the source review for a featured article candidate before the prose/MoS review. If this RfC passes, the same page structure would be set up with nominations on subpages, transcluded to a nominations page, and Hawkeye7's FACbot will be set up to automate the closing of nominations. The FLC community may choose to make some or all of the same changes to their process.

Currently source reviews occur as part of every successful FAC and FLC nomination. Once FSR is set up and running:

• If an article is nominated at FAC without passing FSR, a coordinator would remove the nomination.
• FAC nominators would be expected to identify any sources that are new since the FSR passed. If these are too numerous, or support too large an amount of the article, the FAC coordinator may remove the nomination and require a re-review at FSR.
• Any new sources added since FSR or during the FAC should be reviewed during the FAC. The detail required for this review is up to the coordinators and will likely depend on the importance of the sources to the article.
• "Spotchecking" is FAC shorthand for checking some sources against the article text to ensure (a) that the text is supported by the sources and (b) that it does not plagiarize or too closely paraphrase them. Spotchecks are only required at FAC for first-time nominators and nominators who have been away from FAC for several years, although any reviewer is encouraged to spotcheck at any time. The same will be true for FSR.
• Any FAC reviewer is free to conduct additional source reviews at FAC.

Justification for a new process

Minimizing wasted reviews: The main argument for a separate source review process is that it is pointless to review an article for prose or MoS compliance if the text is based on poor quality sources, or does not reflect what the sources say. Sources should be evaluated first. Here are examples of FAC nominations that ultimately failed because of sourcing issues, but which absorbed much prose review time before the problems were identified:

• Bentworth, 2nd nomination. Six supporters, including some very experienced and capable nominators, before an oppose on sources by Brianboulton.
• Of Human Feelings, 1st nomination. Several reviews, including a long prose review, and a source review that did not include spotchecks, before an oppose on sources by Quadell.

Boosting source-review skills and input: Few people volunteer to do FAC source reviews. Many editors contribute prose reviews, but FAC also depends on a handful of frequent source reviewers. A critical eye for sourcing is a key skill for Wikipedians, and we should encourage more people to contribute in this way. Separating the source review process and requiring each FAC nomination to pass FSR first will make it apparent to FAC nominators that they must give back their time to both processes, by doing both source reviews and prose reviews. It will become much harder for nominators to argue that their prose reviews are sufficient contribution to the process. Sources are the basis of every Wikipedia article: evaluating them should be the most important skill for every content editor. Making a separate process for source reviewing will help prevent editors from considering sourcing as a minor detail: it will contribute to a cultural expectation that top-quality sourcing is required from everyone.

Alternative implementation

Some commenters have indicated they like the intention of this proposal, but are uncomfortable with the creation of a new process. Here is an alternative proposal, based on some of those comments.

• No new source review process would be created; instead, the source reviews would continue to happen at FAC, in the existing nomination pages.
• FAC instructions would change to state that source reviews are expected to be completed before other reviews are posted. A separate subheading could be used to separate the source review from the rest of the review.
• The source reviews would work just as outlined above -- one or more reviewers could contribute, and all of 1c and 2c should be evaluated. A declaration of "support" would be required for a pass, and the coordinators would make the pass/fail decision.
• As above, this proposal does not change what spotchecks are required -- changing that would require a separate discussion.

I will ping everyone who has already !voted to let them know there is a second option. Mike Christie (talk - contribs - library) 01:33, 22 October 2018 (UTC)

Poll for initial RfC

Support

1. Graham Beards (talk)
2. First choice. Mike Christie (talk - contribs - library) 10:44, 17 October 2018 (UTC)
3. RL0919 (talk) 18:13, 17 October 2018 (UTC)
4. I would also support something similar for good articles. (I am not watching this page, so please ping me if you want my attention.) wumbolo ^^^ 18:14, 17 October 2018 (UTC)
5. GreenC 18:18, 17 October 2018 (UTC)
6. Worth a try. Something that might help: A template {{source verified|~~~~|note=}} which does not display in read mode, to tag a source as having been checked by someone, based on their wikisignature ~~~~, which allows an editor to sign off a source as passing scrutiny, with automatically generated date, and space for a comment. This would allow checking of specific sources to be kept with the ref definition, making it easier to keep track of what has been done already, when it was done, and who did it. Makes it easier to check if anything has changed since the verification.· · · Peter (Southwood) ^(talk): 08:19, 18 October 2018 (UTC)

   I liked this idea so I went ahead and made it. See {{Source verified}}. Wugapodes [t^hɑk] [ˈkan.ˌʧɹɪbz] 08:50, 22 October 2018 (UTC)

7. Yes. It may not be perfect but can be refined from experience. Brianboulton (talk) 19:13, 18 October 2018 (UTC)
8. Makes sense; no point in reviewing the prose, if the sourcing is sub-par. K.e.coffman (talk) 00:53, 19 October 2018 (UTC)
9. Moisejp (talk) 14:18, 19 October 2018 (UTC)
10. Better to get this out of the way first. Kirbanzo (talk) 17:40, 19 October 2018 (UTC)
11. While not impressed by adding more process, supporting in the hope that this will lighten the burden on the few source reviewers we have; its tedious work and have seen burnout in cases; I know Mike is mindful of this and applaud him for this initiative. Also K.e.coffman is bang on, but of course sources matter more in terms of quality rather than in formatting style, which I see as necessary but in the scheme of thing, trifling. Ceoil (talk) 20:35, 20 October 2018 (UTC)
12. Kees08 Nominators can get frustrated when they have received several reviews but no source review, even though the source review is arguably the most important part and can take the longest. Front-loading that work will likely reduce frustration, and may highlight a need for more source reviewers. Kees08 (Talk) 22:46, 21 October 2018 (UTC)
13. There are several issues with GA and FA processes, and after reading some of the examples given, this does indeed seem to be trying to solve one of them. --Gonnym (talk) 08:17, 22 October 2018 (UTC)
14. Second choice. Prefer the alternate implementation, but could support either. --Jayron32 19:16, 23 October 2018 (UTC)
15. Either the original proposal or the alternative. Personally, I think that poor quality sourcing and source fabrication is a much greater problem than prose issues for FAs, and per coffman and others, there's no point in copyediting if the article needs to be rewritten from better sources. Catrìona (talk) 03:49, 24 October 2018 (UTC)
16. Support I am supporting, but if this fails, I still think it is fair to make this a recommendation. I might be missing something, but I think this RfC is misplaced, as forcing this as a "requirement" is likely to be controversial while strongly recommending this would probably have resulted in a highly supportive RfC. Many people in Wikipedia value choice and options, and while they will follow recommendations, they will push back against being forced to do anything. The recommendation can go into text immediately, regardless of this RfC, and build out examples, documentation, and a userbase of people who use the system as a choice. Another issue - within 3 years I expect new Wikimedia technology will appear to make this much easier in many cases. In anticipation of that new technology I encourage any discussion and development of source review now to build community culture and humanity in the process before technology becomes more influential. I support the concept of increased human review but codifying the source review process for humans would be difficult. There is a large Wikimedia project with a few hundred highly active participants and a few thousand participants total - meta:WikiCite - which is seeking to address the challenge of surfacing sources. Right now Wikidata itself is about 50 million items, with about half of those being source metadata. Managing source review is the major target of investment in Wikidata and fundamental to the advancement of Wikimedia content development. This RfC opened 16 October and now it is 11 November, and I seem to be the first person to mention either Wikidata or WikiCite in these discussions. I am not surprised because the Wikimedia community has extreme difficulty with the community management of conversations and news, so despite the huge amount of labor being directed toward that problem, thousands of other people who care about the issue intensely do not know the years of engagement or what where the development is happening. If there is anyone who would like to write an opinion piece about the problem at the center of this RfC for WP:The Signpost, then I could get WikiCite / Wikidata participants to talk through the steps toward resolving this issue. There are some things which Google and other search engines will not do, that some wiki technology and content can do, and a little technical support will make human discussion about source review much easier and satisfying. Blue Rasberry (talk) 22:47, 11 November 2018 (UTC)
17. Support as I think the reasoning given is sound. The only downside I can see here is that it creates a new process, everything else should remain the same. And if having that new process gives the whole thing a little more structure and allows for some important stuff to be reviewed before it is all shut down during a prose review then I am all for that. - adamstom97 (talk) 00:56, 12 November 2018 (UTC)
18. Support. Finding out whether an article accurately represents existing scholarship, whether it doesn't have significant gaps in its coverage, and ensuring that the sources are reliable and the article doesn't make conclusions not present in them, is fundemntally more important than checking its prose for readability or making sure it uses the right kinds of dashes. If there are concerns about this adding to bureaucracy, then the more sensible solution is to ditch the rest of the FA process and simply focus on the sourcing. And I don't find it particularly concerning that the present proposal would make it easier to review articles on topics where the sources are easily available online (like Hollywood films or medicine): we want to ensure quality, and if ensuring quality means waiting more then so be it. Wikipedia is a long-term project and the topics with preponderance of offline sources have long-term significance anyway, and whatever the case, there isn't much to be lost if an article has to wait for its star icon a little bit longer. – Uanfala (talk) 13:43, 12 November 2018 (UTC)
19. Strong Support I am Ling.Nut/Lingzhi/axylus.arisbe. DEAR GOD please separate the source review from FAC. Please look at the most recent Bengal Famine FAC's abhorrent excuse for a source review in the "Comments by SarahSV" section.... abhorrent gaming of Wikipedia's system, abhorrent subversion of everything that Wikipedia is supposed to be but is certainly not... Spinning source review off removes one avenue for gaming the FAC process. FAC. DOES. NOT. WORK. I spent years and years vehemently arguing that is does work, but I have been disabused of my wiki-religion. It. Is. Very. Game-able. Split it apart to decentralize the opportunities to do so. The FAC coords already have no power; the smaller purview they have, the smaller the negative impact of their powerlessness. If this had existed when Bengal FAmine went to FAC, it would have helped tremendously to diffuse the impact of gaming the system and reduce the opportunity to game the system... Bengal Famine was gamed because it had few reviewers; it had few reviewers because folks were intimidated; folks were intimidated because the scope under purview of FAC is too large.... Having said all this, I see no reason why source review must absolutely be first step. Could be 2nd. Could be concurrent. Modular is the key idea. FAC is NOT a test of article quality; it is a test of reviewer good faith. Having "source quality" and "prose quality" all contained in one process creates a bottleneck where bad faith editors can take advantage of. Splitting it out dilutes te impact, and puts more eyes on it.Reflets.dans.l'eau (talk) 06:59, 18 November 2018 (UTC)

    It's unclear why forking FAC into source review and content review subprocesses, populated (some would claim controlled) by mostly the same individuals would have any effect on this. Please explain why in one of the discussion sections below.  — SMcCandlish ☏ ¢ 😼  17:41, 19 November 2018 (UTC)

Yup. Corrupt in one step is still corrupt in two steps. Tony (talk) 11:20, 24 November 2018 (UTC)

Oppose

1. Perfect is the enemy of good, and this recommendation isn't good enough. If Wikipedians actually cared about accuracy they'd be checking sources, like they're supposed to for mere GA status. The problem is that we have too many wordsmiths and Grammar Nazis seeking therapy who think implementing MoS is the only important point. There's no professional ethic on Wikipedia. I'm not supporting a new process until it's rigorously designed and there's political will to enforce it. (I am not watching this page, so please ping me if you want my attention.) Chris Troutman (talk) 22:08, 17 October 2018 (UTC)

   Uh, that's kind of off-base in the latter part. Few cliques on WP have been more hostile to the MoS than the FAC crowd (in general; there are a few exceptions like Tony1, but he quit FAC). I'm also opposing this proposal but for different reasons (and one that we share, your GA point).  — SMcCandlish ☏ ¢ 😼  14:29, 19 November 2018 (UTC)

   I didn't quit: I was hounded and bullied off. Tony (talk) 06:05, 22 November 2018 (UTC)

2. per WP:CREEP. The proposal makes some good points but adding yet another process to the pile does not seem efficient. The most pressing issue for FA seems to be that its productivity has fallen below the basic level of one per day and so it's having to recycle pages. The process should therefore be made easier, not more difficult and burdensome. The systemic problem for Wikipedia is that it has accumulated a hard core of fanatical perfectionists who are trying to enforce standards that even professionals would find difficult to meet. The trend is to recreate Nupedia which failed because it was too difficult and intimidating. Andrew D. (talk) 10:11, 18 October 2018 (UTC)
3. A "separate source review process" is unnecessary. Editors should simply be encouraged to do this within the current process at WP:FAC. — Godsy ^(TALK_CONT) 19:12, 18 October 2018 (UTC)
4. Per Andrew Davidson. feminist (talk) 18:04, 19 October 2018 (UTC)
5. I think doing source reviews first is a good idea, but I don't think spinning it off to its own process is a good idea. Like Godsy, I think it would be better for this to operate internal to FAC, perhaps with section headers. New nominations are added to the "source review" section, and can only be moved to a "general comment" section once they've been properly reviewed. Wugapodes [t^hɑk] [ˈkan.ˌʧɹɪbz] 06:07, 20 October 2018 (UTC)
6. As an editor who has been reviewing FACs almost since the time I started working here and is a strong advocate of higher standards at our featured content processes, I'm quite surprised to find myself leaning towards opposing this proposal. However, I just don't think it is going to work as well as the proposers are hoping. Ten years ago, when FAC had a broader reviewer base, this idea probably would have worked fine and met the coordinators' goal of not wasting editors' time on poorly sourced articles. In 2018, though, we just don't have enough people who are actively looking at sources for this to be effective. As I write this, there are currently five articles near promotion that have had a source review requested for them, according to this page. All of these articles have been listed there for at least five days, and none has been fully reviewed yet. If we don't have enough source reviewers to handle this amount of work in a timely manner, how can this new process possibly scale when noms are coming in at a rate of almost one a day? And if reviewers do move to this process, are there going to be enough people to do the second-stage checks? Again, we as a community are running into our lack of reviewers at these processes, which causes issues like this. At some point we're going to have to do something about this on a broader level, but this RFC isn't the best place for that discussion. For this subject, if we agree that the idea of preventing poorly sourced articles from moving on is worthwhile, I'd like to echo a couple of others above in suggesting that this be put on the main FAC page in a subheading. The amount of reviewers might still be an issue, but I fear that adding a second process will fracture FAC, leaving both processes short of the activity needed for them to function. Giants2008 (Talk) 15:33, 20 October 2018 (UTC)
7. Good intention, poor implementation. We don't need a new process/acronym but could just as easily designate the first phase of FAC as requiring a source review to kick it off. Reviewers are free to review without a source review, but it wouldn't be recommended for the nom's stated reasons. Perhaps add a warning template to FAC preload text that the source reviewer can modify/remove when completed. czar 20:03, 20 October 2018 (UTC)
8. Too complicated. We have barely enough editors to maintain our current GA / FA processes let alone adding another one. Strong oppose. --Tom (LT) (talk) 07:54, 22 October 2018 (UTC)
9. Process needs to be simplified, not elongated. Fish+Karate 12:25, 22 October 2018 (UTC)
10. I understand the reasoning behind this, but it needlessly complicates various projects which rely on the already messy grading system. You can't fix a human problem with processes, and that fact is we simply don't have enough volunteers doing certain tasks. TheDragonFire (talk) 05:07, 1 November 2018 (UTC)
11. A two-step process for FA would significantly slow down the whole process of getting an article through the process, having to let an article sit around waiting for someone to review the sources without allowing the nominater to be alerted of prose or image problems. The article would likely sit for a while in FSR because, as established, there’s a shortage of people willing to review sources. Also, while an FSR is sitting, an unofficial discussion on prose problems is bound to be started on the article talk page, leading to three different reviews for the same article in three different locations. If we really want to bring attention to source reviews, I propose creating a new template, {{source review|done=yes or no}}, to be put at the top of an FAC talk page to let reviewers focusing on source reviews to easily know which article require one   User:Dunkleosteus77 |push to talk  21:08, 6 November 2018 (UTC)
12. Needless complexity. Tony (talk) 08:42, 18 October 2018 (UTC)
13. This would add another of delay, and create a systemic recentist bias as articles sourced to online sources would be signed off rapidly, but articles sourced to print sources would languish. As I've said before, I question whether source reviews are necessary in every case, given how often FAs form part of a series; if an editor has written Cheshire cheese, Mozzarella and Wensleydale cheese and the sources have checked out each time, I don't consider it a sensible use of anyone's time checking the sources when they nominate Cheddar cheese unless there's an extraordinary claim being made that raises doubts. ‑ Iridescent 10:10, 9 November 2018 (UTC)
14. My main concern is that this bottlenecks the FAC process. If reviewers have to rely on ancillary reviews to be completed before the main review, the backlog will grow even larger than it currently is. Aren't FAC reviewers supposed to check citations anyway? Shouldn't that have already happened ad nauseam in the Good article review process? SEMMENDINGER (talk) 22:31, 11 November 2018 (UTC)
15. While I know the proposal means well, people should already be checking citations for quality and accuracy during FA reviews and even GA reviews. This just needlessly complicates things. Snuggums (talk / edits) 22:48, 11 November 2018 (UTC)
16. Too much bureaucracy. A source review needs to be conducted at the same time as prose reviews, and vice versa. Imzadi 1979 → 23:13, 11 November 2018 (UTC)
17. Strongly oppose this mission creep and unnecessary additional bureaucratic step. Peacemaker67 (click to talk to me) 00:46, 12 November 2018 (UTC)
18. Normally not a fan of crying "WP:CREEP", but proposals like this are what that page was meant for. This will exacerbate the FAC backlog and will only add confusion to the process. -Indy beetle (talk) 01:04, 12 November 2018 (UTC)
19. Strongly Oppose - As a long time editor it seems to me that this proposal is an exemplar of the overly bureaucratic approach that has infected wikipedia in recent years. This proposal seems to focus on altering user behaviour to match what someone thinks users should do. Some people might be happy to do prose reviews but don't think they can do source reviews and that's fine. Each person has a different strength. If you try to get people to do something they don't want to do, or feel competent to do, they will probably just disengage altogether. Instead we should try to make wikipedia friendlier to those who do still contribute. Source verification has been weaponised in many articles and this totally puts people off. I'm not surprised people don't want to do it. Also it doesn't matter if the prose review gets done but then the source review fails. All that has happened is that an article has had its prose improved. The sources can be improved later. Morgan Leigh | Talk 01:43, 12 November 2018 (UTC)
20. Oppose per Godsy. Double sharp (talk) 01:55, 12 November 2018 (UTC)
21. Oppose: I don't see the need to put the source review first. If an article has near-FA-level, then the references are not usually the issue, in my experience. Jakob.scholbach (talk) 08:55, 12 November 2018 (UTC)
22. I can see the idea of requiring (1c) first, but I don't see why 2c (formatting of them) should be necessary before the regular nomination can start. Are users unable to judge the quality of an article if the date format in citations is not the same everywhere? --mfb (talk) 05:45, 13 November 2018 (UTC)
23. See below. I oppose both of the two proposals for similar reasons.  — Amakuru (talk) 11:32, 13 November 2018 (UTC)
24. Source review is hard work, and for technical articles it is hard enough to attract reviews at all. For sources not online it is asking a lot from a reviewer to go study in a library, possibly some distance away. There comes a point when sources can be taken on good faith if everything accessible checks out and the claims seem reasonable. SpinningSpark 19:20, 14 November 2018 (UTC)
25. This proposal seems to misunderstand what a FAC should be. FAC *is* the place to review the sources as well as the quality & comprehensiveness of the content, and the copyediting is more like a nice bonus. The copyediting shouldn't be the focus of FAC. All of that prose-spinning & fine-tuning, even when done with the best of intent, will be gone after a year's editing on many articles anyway, and even when it isn't, it's not clear if the end result is truly better (barring egregious cases). I applaud the spirit of the proposal, that quality of content is important, but moving that check out of FAC seems more like a step backward than a step forward. SnowFire (talk) 23:52, 15 November 2018 (UTC)
26. Oppose. I applaud anything that focuses reviewers' attention on sources, and I thank Mike for the work he's doing to draw attention to that. But I believe this takes us in the wrong direction by appearing to suggest that sources needn't be checked during the content review (which we shouldn't call a "prose review"; that's something else again). A content review should check the writing; whether the article is comprehensive; whether it strays into irrelevant detail; and whether it's neutral and representative of the mainstream secondary literature (majority and significant-minority views), or whether important sources are missing or not used enough. How else can the content be judged if not in relation to the sources?
    In addition, to set up a source-review process that must be completed first (whether on a separate page or, as proposed below, on the FAC page) will create a bottleneck, because we don't have enough editors willing and able to do comprehensive source reviews. Even during the pilot for this proposal, one article was left unreviewed and one source review appeared to have been abandoned. Reviewers will be under pressure to hurry up so that the content review can begin, which will mean source reviews will be cursory. Nominators already play reviewers off against each other and this will give them an additional supply: anyone who later criticizes the sources will be told "but the source reviewer says they're fine." SarahSV ^(talk) 00:42, 18 November 2018 (UTC)
27. +O. I have changed my mind. The problem here is "must be done first". I strongly support separating the source review and prose checks (fork), not both under FAC, to limit gaming. But outright requiring one to be done before the other is a wrong approach.Reflets.dans.l'eau (talk) 02:11, 19 November 2018 (UTC)
28. Oppose as redundant. Source review is already part of WP:GA. WP:FAC could simply require WP:GA first, [added after original post:] then do whatever additional review of sources FAC thinks is needed. I'm not opposed to FAC requiring that, or some kind of informal source checking, but am adamantly opposed to FAC instituting a new WP:BUREAUCRACY / WP:CREEP of any kind. It's already gone way too far in that direction. Worse, it's losing interested editors (both are review participants and submitters of candidate articles), so is turning more and more into a clique. We just disbanded WP:Mediation Committee because the community doesn't want a content mediation "board", yet FAC is basically proposing to create one. I do not at all trust a tiny group of editors focused on one process the rest of the project is increasingly disengaged from to determine what is or isn't good sourcing (not because of any particular editors involved, but because they are few – i.e., because of WP:CONLEVEL, WP:OWN, WP:VESTED, etc.). Source review is a continual community effort. GA and FA are nice milestones, and necessarily involve such checking, but it doesn't just stop there. Old FAs often have to sources replaced with better ones, and we cannot much control what is added to an article after GA or FA. This entire proposal seems predicated on the idea that article climbs to FA and is found to be great, and that's the end. WP doesn't work that way.  — SMcCandlish ☏ ¢ 😼  14:25, 19 November 2018 (UTC); clarified: 17:52, 19 November 2018 (UTC)

    Some followup discussion about the post above has been moved to the "Comment on recent opposes" section, below.  — SMcCandlish ☏ ¢ 😼  17:38, 19 November 2018 (UTC)

29. Although I am in favor of checking sources earlier in the process rather than later, not convinced that separating the process and transcluding separately is the ideal solution. Source reviewing can vary from checking formatting, which is important, to checking formatting and checking whether sources satisfy 1c of the criteria. Checking whether sources are the best that can be used is time-consuming, to some degree can require some familiarity with the article topic, and isn't always done. It can, however, be part of the content review, which I've done in many reviews, and doesn't necessarily have to be separated out as a separate process before the rest of the review is conducted. Victoriaearle (tk) 20:58, 24 November 2018 (UTC)
30. Unnecessary. Beyond My Ken (talk) 23:30, 8 December 2018 (UTC)
31. We need fewer processes, with fewer complications. I'd support one general review process for FA, GA, and ordinary articles for the improvement of sourcing or of anything else. Sourcing, as othrs have said above, is not distinguishable from content. Sources are reliable or not only in terms of what content they support and in what context. DGG ( talk ) 02:21, 13 December 2018 (UTC)

Neutral

1. Support in principle, oppose in practice. While I wholeheartedly agree with the general principle of needed more thorough reviews, I share the concerns of User:Giants2008 about lacking volunteers. I am also concerned about the bias that it would introduce. It would be much easier to get an article on (say) a Hollywood film reviewed than on a specialized math subject or a foreign subject that relies heavily on non-English sources. Maybe a start would be subheadings in FAC nomination page for "prose review", "sources review", "image review", etc. Renata (talk) 16:00, 20 October 2018 (UTC)
2. Like the idea, not so keen on extra layer of bureaucracy. Cas Liber (talk · contribs) 10:52, 11 November 2018 (UTC)
3. Inline citations are complex and fiddly for newbies. Without better tool support I'm very hesitant to support any such proposal. Ideally we need a script like the DYK article-length checker which automatically verifies that the article meets to technical requirements and/or highlights questionable things for manual checking. Stuartyeates (talk) 23:14, 11 November 2018 (UTC)

   Most elements of a FAC-level source check are not amenable to automation - you can flag dead links and the like, but not much more than that even on a technical level given the diversity of potential citation formats. Nikkimaria (talk) 23:18, 11 November 2018 (UTC)

   I'll settle for a decent automation tool for a single relatively-widely used citation format, because the diversity of potential citation formats is in fact a symptom of the problem. We need to be gravitating towards a single uniform style of citation and tool support for one of them would go a long way towards that. Stuartyeates (talk) 02:07, 12 November 2018 (UTC)

4. Re #1. above. Seppi333 (Insert 2¢) 23:45, 11 November 2018 (UTC)
5. Why not just slightly change the way FAN's happen instead of make a whole new process? --Tom (LT) (talk) 08:34, 12 November 2018 (UTC)
6. For now I'd support it as an "optional" process step that is encouraged but not required. Praemonitus (talk) 22:38, 12 November 2018 (UTC)

Poll for alternative implementation

Support alternative

1. Second choice. Mike Christie (talk - contribs - library) 01:33, 22 October 2018 (UTC)
2. This or bust. Better to modify the existing FAC process than to spin out a separate process. I've posted below about how a template toggle on par with {{editprotected}} could divide the two phases. czar 02:36, 22 October 2018 (UTC)
3. See my comment in the previous section. Wugapodes [t^hɑk] [ˈkan.ˌʧɹɪbz] 08:05, 22 October 2018 (UTC)
4. Fixing FAC is the better way to go. Chris Troutman (talk) 16:24, 22 October 2018 (UTC)
5. As commented below. Brianboulton (talk) 17:49, 22 October 2018 (UTC)
6. First choice. Implement one or the other, but this is the best option. --Jayron32 19:16, 23 October 2018 (UTC)
7. First choice. I certainly agree that sourcing is of the utmost importance, and this looks to me to be the simplest way to go about it. --Tryptofish (talk) 20:34, 26 October 2018 (UTC)
8. I don't actually have any experience with FA nominations (but wanted to express my support), so I have no preference for one option over the other. Jc86035 (talk) 17:04, 28 October 2018 (UTC)
9. I remain unconvinced that this will work in practice, but I support the idea of increasing the importance of sourcing reviews and trust that the coordinators will back modifications of the system as necessary. Giants2008 (Talk) 21:24, 28 October 2018 (UTC)
10. First choice. wumbolo ^^^ 19:23, 3 November 2018 (UTC)
11. I'm not wild about this idea (there's a high risk of it causing logjam-type problems, especially for obscure topics or topics where the references are hard for other editors to access), but this is the way to implement it. A separate process would be over-kill. Nick-D (talk) 06:34, 4 November 2018 (UTC)
12. An entirely new process would only make FAC more labyrinthine and hard to navigate; there's a reason why FAR and FARC were rolled into the FAC page, people simply don't find it otherwise. Dunkleosteus77's {{source review|done=yes or no}} option may be the simplest solution. FunkMonk (talk) 13:26, 9 November 2018 (UTC)
13. Am receptive to it. I can see issues as per Iridescent but think it has some merit. Cas Liber (talk · contribs) 10:55, 11 November 2018 (UTC)
14. second choice--Ozzie10aaaa (talk) 22:21, 11 November 2018 (UTC)
15. Much better choice. It can help more closely ensure citing is on par during article reviews. Snuggums (talk / edits) 22:49, 11 November 2018 (UTC)
16. Ideal solution to the problem IMO. Seppi333 (Insert 2¢) 23:46, 11 November 2018 (UTC)
17. Weak support. I see Iridescent's point about flipping the issue to little overall effect, but if something really must be done, then this is as far as I would go. Peacemaker67 (click to talk to me) 00:50, 12 November 2018 (UTC)
18. First choice. We do not need another process for this. Wikipedia already has a bajillion other processes and creating more just sinks us deeper in the quagmire of bureaucracy. No reason why we can't just make reviewers focus on sources first so a review can be quickly short-circuited if it's not up to par. However, if this doesn't get enough votes, I would still support something that gives sources more weight initially. Opencooper (talk) 09:28, 12 November 2018 (UTC)
19. Second choice. I hadn't bothered to list myself in this section when it was going unopposed, so taking the extra step now that it has. I think some of the opposes are not considering the asymmetry in how source issues affect prose. If key sources have not been considered or sources in use are not adequate, then significant passages may need to be revised or even new sections written. This means new prose checks will need to be done. In contrast, if the sources are all good and then someone finds issues in phrasing, MoS compliance, etc., it is far less likely that these will require new sources. --RL0919 (talk) 12:54, 12 November 2018 (UTC)
20. Support, this essentially is the same what I have suggested in my neutral !vote above. I think the lax language of "expected to be completed first" gives enough leeway to get rid of clearly not qualified nominations before going thru detailed source review (i.e. don't see a big problem in what Iridescent said). Renata (talk) 02:08, 13 November 2018 (UTC)
21. This seems a good solution. It is always possible to salvage well-referenced low-readability prose, however the reverse is not necessarily true. There's no need to be overly-dogmatic if a FAC clearly benefit from a different processing order. T.Shafee(Evo&Evo)^talk 10:35, 13 November 2018 (UTC)
22. Support, only choice. In addition to the previously discussed advantages, anything that might rebalance the review process away from line-by-line auditing of individual sentences (which should be done during a collaborative copy-editing process before FAC) is a win in my book. (A week ago I removed my earlier support from both proposals [1]; I'm putting this one back.) Outriggr (talk) 06:21, 18 November 2018 (UTC)
23. Support, this seems far more sensible. Chiswick Chap (talk) 13:01, 18 November 2018 (UTC)

Oppose alternative

1. Sorry to break the unanimity, but on consideration I can't agree with source reviews are expected to be completed before other reviews are posted. Sure, it means reviewers can waste their time reviewing an article where the sources turn out to be inadequate, but all this will do is flip the problem so that reviewers waste their time checking the sources on an article where the prose turns out to be inadequate. If I see an article at FAC which I feel is badly written and needs major work, it's just silly telling me I'm not allowed to mention the fact until someone else has spent time digging mouldering books out of libraries to review the sources. ‑ Iridescent 17:30, 9 November 2018 (UTC)
2. I agree with Iridescent; it might be good to call out the source review, but why waste reviewer time by forcing someone to wait to comment on the prose? Ideally, the two reviews should be carried out in concert, so that prose reviewers know the basic state of the sourcing, and that source reviewers know the basic state of the prose. Imzadi 1979 → 23:19, 11 November 2018 (UTC)
3. While much better than trying to create a new process, the requirement that source reviews are expected to be completed before other reviews are posted might inadvertently slow everything to a halt. All I see happening is people refraining to post prose reviews until someone else gets around to making a source review—not being motivated themselves (as intended) to give a thorough source review. Prose reviews on articles found to have unsuitable sourcing are not entirely "wasted" anyhow; they still provide constructive advice on how to improve a Wikipedia article. -Indy beetle (talk) 01:00, 12 November 2018 (UTC)
4. If there aren't enough people doing source reviews saying they have to be done before prose reviews will just stop people doing prose reviews. Prose and sources are each as important to the article as the other. You can't make people do stuff they don't want to do by making a rule. Instead find out why people aren't doing source reviews. Hint, its super hostile at Wikipedia these days. Morgan Leigh | Talk 01:51, 12 November 2018 (UTC)
5. Oppose per Iridescent. Double sharp (talk) 01:56, 12 November 2018 (UTC)
6. Tony (talk) 02:06, 12 November 2018 (UTC)
7. Oppose per Imzadi1979. Jakob.scholbach (talk) 08:58, 12 November 2018 (UTC)
8. Doesn't address the issues I have with the original proposal. --mfb (talk) 06:11, 13 November 2018 (UTC)
9. Oppose. I've watched this discussion unfolding, and like many I did initially think it might be a good idea. However, Iridescent's point is a very good one. Implicit in this proposal is that source reviewer time is less valuable than prose reviewer time, and Mike's assertion below they the two parts of the review are not symmetric further confirms that. I disagree. Time spent source reviewing an article whose prose is not up to scratch is still time wasted, and actually it's the prose that usually causes the failures rather than sources so wasted time might well go up rather than down. The two examples highlighted are from 2015 and 2013 respectively, which suggests this isn't a frequent issue. I suggest we keep the two processes in parallel, that's the most efficient and fair way.  — Amakuru (talk) 11:26, 13 November 2018 (UTC)
10. Oppose per Iridescent and others. No good reason to insist on a source review first. SpinningSpark 19:14, 14 November 2018 (UTC)
11. Oppose for the reasons I outlined above. SarahSV ^(talk) 00:50, 18 November 2018 (UTC)
12. Oppose per Iridescent and others, rather reluctantly. I was partly pursuaded by this thread on Iri's talk. Reviewer time is wasted if sources are rather obviously inadequate, but would now be if the prose is poor. Johnbod (talk) 19:50, 18 November 2018 (UTC)
13. +O. I have changed my mind. The problem here is "must be done first". I strongly support separating the source review and prose checks (fork), not both under FAC, to limit gaming. But outright requiring one to be done before the other is a wrong approach.Reflets.dans.l'eau (talk) 02:09, 19 November 2018 (UTC)
14. Oppose. This is a volunteer project. We don't need a bunch of WP:CREEP and WP:BUREAUCRACY telling us what we can review and in what order. If someone who's really good with layout or English or whatever wants to give their input on improving an article submitted to a self-selecting wikiproject of people gathered together to review articles, they don't have to wait for someone with EBSCO access to finish following up on source checking before they do what they're here to do. Frankly, things lilke this proposal and the other FA-related red tape are among the things that make FAC, and bothering with FA at all, unappealing to so many editors.  — SMcCandlish ☏ ¢ 😼  14:34, 19 November 2018 (UTC)
    Even more opposed to both versions of this proposal now that I realize this is just a extension of two years of "down with MoS" grandstanding by a few people at FAC (details here). If this kind of anti-consensus "lobbying" against guidelines and their applicability to "vested editors" is going to continue indefinitely, FAC should just be shut down. When multiple of its own reviewers think it's running off the rails, we have a real problem, and it isn't that source reviews need to happen in their own process.
     — SMcCandlish ☏ ¢ 😼  17:47, 19 November 2018 (UTC)

15. Oppose. Encouraging more source reviews is one thing, but requiring that source reviews come first makes the reviewing process needlessly inflexible. A. Parrot (talk) 02:06, 24 November 2018 (UTC)
16. Opppose per Iridescent and per my comment in the previous section. My fear is that this proposal will reward online sourcing (which raises the issues of whether 1c is satisfied), and will slow the reviewing process down. My biggest fear, however, is that sources won't be adequately vetted (which already happens to some extent), in a effort to keep things moving. Victoriaearle (tk) 21:03, 24 November 2018 (UTC)

Neutral alternative

1. No strong opinion either way. · · · Peter (Southwood) ^(talk): 04:53, 22 October 2018 (UTC)
2. I put this in Oppose above but I guess I’ll put it here too: I propose creating a new template, {{source review|done=yes or no}}, to be put at the top of an FAC talk page to let reviewers focusing on source reviews to easily know which article require one   User:Dunkleosteus77 |push to talk  15:45, 7 November 2018 (UTC)
3. I'm mostly neutral, tending to oppose the added process and time it takes to complete but supporting the increased visibility of needed source reviews. I think that Dunkleosteus77's comments have the most simple solution I've seen, so I would support any implementation of their template. Pagliaccious (talk) 18:02, 12 November 2018 (UTC)

Discussion

• See here for a note on how FLC currently assesses sources. Mike Christie (talk - contribs - library) 08:11, 16 October 2018 (UTC)
• In addition to the benefits a prior source review can bring to the FAC process, reviewing and improving the quality of article sourcing is beneficial in its own right. Even if an article never makes the leap to FAC, both readers and future editors will have a better, more reliable article to work from. In regard to the suggestion in the poll about doing something similar for WP:GA, there isn't anything in the proposed process that would prevent an editor from using it prior to a GA nomination, rather than going directly to FA (although it might be "overkill" since the FA sourcing standards are more stringent). --RL0919 (talk) 18:28, 17 October 2018 (UTC)
• Sounds like fact checking. Why not call it that? -- GreenC 20:11, 17 October 2018 (UTC)

  It's more than that. Here's a breakdown suggested by Factotem in a discussion a few days ago; I think this list, or something like it, would go on the instructions page for the new process.

  1. Technical checks for formatting (inline refs are consistent, p/pp, ISBN format, publisher locations, etc.);
  2. External link checks (including checking that ISBN/OCLC/etc. links lead to the correct editions);
  3. Verification that sources are both high quality and reliable;
  4. Survey of sources to ensure comprehensiveness;
  5. Spotcheck verification of sources for accuracy and plagiarism.

  Factchecking is similar to point 5 above, but the other points are required for FAC too. Mike Christie (talk - contribs - library) 20:23, 17 October 2018 (UTC)

A point of clarification, prompted by Andrew Davidson's oppose above. The proposed process does not add any new barriers to becoming a featured article. FAC already includes the proposed FSR as a sub-process. The goal of this RfC is to make the two processes consecutive rather than simultaneous. This should improve efficiency, because currently effort can be wasted on prose reviews of articles with poor sourcing. Mike Christie (talk - contribs - library) 10:51, 18 October 2018 (UTC)

• Myself, I'm more often reviewing DYKs and, for that, I use a checklist template: {{DYK checklist}} which helps ensure that everything is checked. I usually check everything rather than stopping when finding the first issue because it is inefficient to make a repeat review each time an issue is found. It sounds like we need a similar structured checklist for FA. Different reviewers might fill in different items in the checklist but it's not clear that strictly sequencing this would help. If no-one has done any fact or source-checking then a checklist would expose this and that seems sufficient. Andrew D. (talk) 11:36, 18 October 2018 (UTC)

  Checklists have been tried occasionally at FAC, and haven't caught on, I suspect because the community expectation is detail rather than summarization. However, even without checklists, it's usually easy to see if a source or image review has been done, so reviewers can avoid prose reviews if they don't see a source review. Here, for example, is a FAC with both source and image reviews clearly visible near the top. This is another current FAC in which the source reviews were clearly not done, but many reviewers decided to jump in and review anyway; the source review is near the end. So the lack of a source review is not stopping reviewers from reviewing, probably because they know that in most cases the sources are OK. That still leads to wasted effort in the cases where there are problems. Mike Christie (talk - contribs - library) 00:50, 19 October 2018 (UTC)

Why does this proposal combine source content with source formatting? Whether you should have a comma or a colon in a given spot in the bibliographic citation is a MOS question. Whether you should be citing that author at all is a completely different type of analysis, and needs editors with a completely different skill set (and probably domain-specific knowledge).

I think that items #1 and #2 in Mike's five-point list above (the "2c" items) should be removed from this process entirely. Everything that's being said about not wasting time evaluating whether the MOS is being followed in the article applies with at least equal force to wasting time evaluating whether the MOS is being followed in the citations. I don't want people to waste any time determining whether the citation to a garbage source is perfectly formatted. If we're going to have a two-stage process, all of the formatting should be in the second stage. WhatamIdoing (talk) 20:36, 18 October 2018 (UTC)

• Re: reviewers checking that the ISBN leads to the correct edition. You would have to get hold of that edition yourself, then check every single time that source was referenced to make sure the page numbers led to that edition. And you'd have to do that for every book. SarahSV ^(talk) 20:48, 18 October 2018 (UTC)

I think Worldcat world be useful for that, in all but a (tiny) minority of sources that may remain ambiguous. 21:07, 18 October 2018 (UTC)

This proposal is not about what checks should be made. That 5-point list details one editor's opinion of what a thorough source review should involve, whether the source review remains integrated as part of the FAC process or whether it is spun off into a separate process. But it's a fair point about the technical checks - that's more a MoS issue, more closely aligned with prose checks than source reviewing.

Not sure I understand the point SarahSV is making. ISBN link checks are simply a case of following the ISBN/OCLC/etc. link, usually to worldcat or GBooks, and making sure the edition details (such as year and publisher) listed on those sites correspond to the details listed in the article's bibliography. In my experience it is a common error for the bibliographical information to specify a year xxxx edition published by publisher A, but for the ISBN/OCLC/etc. link to lead to a year yyyy edition published by publisher B. It becomes a WP:V problem if the two editions have different pagination. Factotem (talk) 22:32, 18 October 2018 (UTC)

WhatamIdoing is right that points 3-5 in the list above are the ones that are important to check first. Those are the points that relate to the first justification given above, of minimizing wasted reviews. However, one of the impetuses for this discussion was the recognition that only a few people at FAC are doing the fairly tedious work in points 1-2. The second justification is partly about spreading the workload; FAC needs for that to happen, and including 1-2 in this process would make it more likely that more people would take it on. If this passes, I think any such list in the instructions should give the (1c) points first, and make it clear that the other points should be dealt with once 1c is seen to be satisfied. Mike Christie (talk - contribs - library) 00:43, 19 October 2018 (UTC)

I still oppose requiring everyone else to wait for someone to check whether all the citations have commas in the correct places.

Requiring proper formatting for citations is inconsistent with your own explanations for why this change needs to happen.

• Yes: It's pointless to review prose, comprehensiveness, neutrality, etc., if most of the sources are inappropriate. This is true even if, during the course of the review, a few things might be re-written or new sources added.
• No: It's not pointless to review the rest of the article if there's still a misplaced comma in one of the inline citations. In fact, it is a waste of scarce resources to do that, if that misplaced comma happens to be in a citation that will get removed in response to review comments, or if the whole thing is going to be dropped over dull prose anyway. In fact, it might be best to schedule that kind of 'technical' MOS review as the very last thing to be done, so that the few people who are willing to do it only have to do it once per article, and only on the fraction of nominations that are actually going to be promoted. WhatamIdoing (talk) 18:49, 2 December 2018 (UTC)

All FACs include source review, and source review is most useful as the first stage of the FAC process. Why doesn't that mean that source review should be the first stage of the FAC process rather than create a separate process? Seems like something I'm missing here? — Rhododendrites ^talk \\ 03:15, 19 October 2018 (UTC)

Enforcing that within FAC would lead to exactly this proposal, except that the source review process wouldn't be available to other articles -- as RL0919 points out above, an article does not need to be intended for FAC to benefit from a high quality source review. Mike Christie (talk - contribs - library) 09:57, 19 October 2018 (UTC)

I'm still having a hard time seeing a lot of benefit to making it a process separate from FAC. Certainly, yes, it's good to evaluate sources, but we have a volunteer problem with most formalized Wikipedia processes, so if most of the time it's going to be part of FAC, it seems better not to create parallel (even if often complementary) processes. For other articles, will there be some other sort of recognition displayed somewhere (like the FA star or GA green plus, but, I don't know, a purple book)? If that were the case, I would worry about indicating to readers a certain level of quality (what it would be taken to mean, I think) if it's entirely divorced from the quality of the article (i.e. a promotional/non-neutral start-class article with a long line of citations could presumably still qualify). — Rhododendrites ^talk \\ 13:24, 19 October 2018 (UTC)

I admit that (and I must have misunderstood, or missed an important piece of the discussion, perfectly possibly) I was under the impression that the source reviews were part of, but distinct from, the individual FA nominations—but not that they were separate from the FAC procedure generally. Completely outside the project? Reviewing GAs? ——SerialNumber54129 13:32, 19 October 2018 (UTC)

The primary intention is that this review leads to FA; it was conceived for that purpose. But if an article was source reviewed and for whatever reason didn't end up at FAC, it would still help improve the article. --RL0919 (talk) 13:48, 19 October 2018 (UTC)

A start-class article would fail (1c): "a thorough and representative survey". I think it's worth reiterating that this is not really a new process and doesn't imply additional work for anyone -- it's just splitting the existing process in two, so as to enforce the "1(c) first" approach, which might be hard to enforce at FAC. It's also worth remembering that source reviews and prose reviews are different skillsets, so it's a natural separation. Mike Christie (talk - contribs - library) 14:02, 19 October 2018 (UTC)

Thanks, both, that all makes sense, it was the mention of GA that tripped me up (@Mike Christie, start class indeed!) purely on the grounds of reviewers being thin on the ground as it is. In any case, I still support the proposal. Thanks for the replies! ——SerialNumber54129 14:14, 19 October 2018 (UTC)

• If people are struggling with English online sources, what will happen with articles that highly depend on off-line or non-English sources? No one will want to review them. Renata (talk) 14:47, 19 October 2018 (UTC)

Most book sources are to a large extent off-line. This is only an issue when it comes to spot checks for accuracy and plagiarism and is mitigated by the prevalence of GBooks previews. I don't think it's reasonable to conduct such checks for every single ref in the article. If it's possible to conduct such checks on say 10% of the refs, then that gives a reasonable indication as to how fastidious the nominator has been in the use of sources. Factotem (talk) 16:43, 19 October 2018 (UTC)

• What, in practice, would be the difference between this and passing the source-checking involved in becoming a Good Article? Would this proposal be functionally different from requiring articles to be GA's before they can become FA's? I haven't been systematically involved in either process, only helped out in a few cases, so I'm genuinely uncertain about this. XOR'easter (talk) 15:10, 19 October 2018 (UTC)
  • Is the difference the "thorough and representative survey of the relevant literature" part of criterion (1c)? It seems to me that this is harder to check than the rest, in that it can require more specialist knowledge. There are plenty of fields where I can understand a book you give me, and I can check that what our article says is really stated in the book that we point to, but I wouldn't be fit to judge whether our survey is "thorough and representative". XOR'easter (talk) 15:15, 19 October 2018 (UTC)

A "thorough and representative survey of the relevant literature" can be verified to a reasonable level of confidence even by a lay person, simply by googling the subject, both websites and Gbooks. The latter in particular should correspond fairly closely to the article's bibliography. If it does not, then it's a case of asking the nominator why not (and there may be a perfectly good explanation why not). Factotem (talk) 16:44, 19 October 2018 (UTC)

I suspect that for chemistry, physics, mathematics, etc., the default is not. XOR'easter (talk) 13:52, 20 October 2018 (UTC)

passing the source-checking involved in becoming a Good Article

There is no spotchecking requirement in the GA process. (not watching, please {{ping}}) czar 20:16, 20 October 2018 (UTC)

Czar, Wikipedia:Reviewing good articles says: "At a bare minimum, check that the sources used are reliable ... and that those you can access support the content of the article (for example, inline citations lead to sources which agree with what the article says) and are not plagiarized ..". SarahSV ^(talk) 21:09, 20 October 2018 (UTC)

The GA criteria dropped "factually accurate" as a requirement in 2013, so it lists only simple verifiability, not accuracy. czar 21:41, 20 October 2018 (UTC)

@Czar: GA reviewers are supposed to check that the sources support the text, at least for some of the sources. Sources are looked at somewhat for DYK too; the hook at least is checked. SarahSV ^(talk) 22:20, 20 October 2018 (UTC)

I've opened a discussion at the GA criteria talk page: Wikipedia talk:Good article criteria#"Factually accurate" czar 22:23, 20 October 2018 (UTC)

• I can understand the need for this proposal. I have this question: when splitting (separating) FCR out of the existing FAC process, do the FCR/sourcecheck requirements formally change? IOW does this source-check change in this "new" (separate) process? I have read the details like spotchecking getting more attention, but that is not a formal requirement change IMO. -DePiep (talk) 22:47, 11 November 2018 (UTC)

Poll option to implement this within FAC

Per this conversation with Giants2008, and because of several comments in the poll above (e.g. from Wugapodes, Ceoil, Outriggr, and Czar), I'm planning to add another option above, to implement the same idea but strictly within FAC. The key points would be

• Everything happens on the FAC nomination page with the FAC coordinators making the decisions
• A separate subheading appears on the FAC nomination page for source reviews, and the instructions specify that source reviews must (or should?) be completed before other reviews are begun
• A source review must gain at least one support to pass, and only the coordinators decide whether to pass it
• When the source review is passed the coords add another subhead for "Post-source review" or whatever we want to call it

I plan to add something like this tonight but wanted to give at least a few hours for others to comment first. I think I prefer the current RfC proposal, because it's a move towards establishing source reviewing as an independent and important skill, but this inside-FAC approach could work, so I would support it as a second choice. I think others might support it as first choice. Any comments before I go ahead? Mike Christie (talk - contribs - library) 21:00, 21 October 2018 (UTC)

+1 I'd recommend doing something like the {{editprotected}} template in which there is a big template (in default state) that says a source review is needed to move forward, and when a parameter is adjusted in the template, it can shrink in size and show a green checkmark that the rest of the review is ready to proceed (e.g., see the difference between these two templates). Or if it must be done through headings, have the template trigger them. As for better source reviews, I think we can give better guidance with or without a dedicated process. Our bibliographic guidance is abysmal. czar 21:21, 21 October 2018 (UTC)

Done; please tweak the wording if necessary. I will ping everyone who has already !voted so they can declare an opinion on the alternative if they want to. Mike Christie (talk - contribs - library) 01:33, 22 October 2018 (UTC)

Pinging: Graham Beards, Serial Number 54129, RL0919, Wumbolo, GreenC, Pbsouthwood, Tony1, Brianboulton, K.e.coffman, Moisejp, Kirbanzo, Ceoil, Outriggr, Kees08, Chris troutman, Andrew Davidson, Godsy, Feminist, Wugapodes, Giants2008, Czar, Renata. Mike Christie (talk - contribs - library) 01:38, 22 October 2018 (UTC)

• I'm fine with the alternative too - all I care about is making sure that we prove good sourcing for FAs beyond a reasonable doubt. Kirbanzo (talk) 01:41, 22 October 2018 (UTC)
• I prefer the original proposal, but this would be an acceptable alternative if the original doesn't get sufficient support. --RL0919 (talk) 03:53, 22 October 2018 (UTC)

*The four bullets above: that's how I assumed it would be. Looks good. Tony (talk) 05:53, 22 October 2018 (UTC) Support withdrawn Tony (talk) 06:04, 22 November 2018 (UTC)

• Agree alternative, per Tony. Brianboulton (talk) 09:35, 22 October 2018 (UTC)
• per Everyman. ——SerialNumber54129 09:44, 22 October 2018 (UTC)

Note: these opinions may be missed by whoever closes the RfC; I suggest posting your opinion in the second poll section above, as well. Mike Christie (talk - contribs - library) 10:21, 22 October 2018 (UTC)

Comment on recent opposes

Iridescent comments in their oppose that "I can't agree with source reviews are expected to be completed before other reviews are posted. Sure, it means reviewers can waste their time reviewing an article where the sources turn out to be inadequate, but all this will do is flip the problem so that reviewers waste their time checking the sources on an article where the prose turns out to be inadequate." This might be an issue for well-prepared articles (though those generally sail through FAC) but the two problems are not symmetric. Bad prose doesn't make it pointless to review sources, but bad sourcing makes it pointless to review prose.

The second justification in the list above, which was actually the initial impetus for this idea, is that we need to force attention on the need for source reviews, not just on the need for good sources. A handful of dedicated reviewers have carried that burden for several years now. We need a way to focus reviewer's attention on the backlog and get them to help, and making the two reviews sequential, without any change to the depth of review needed, will (I hope) achieve that. Mike Christie (talk - contribs - library) 00:42, 12 November 2018 (UTC)

I think part of the problem with getting people to do source reviews, aside from their tediousness, is that many people feel out of their depth doing them, especially in a topic area in which they have little familiarity. I, for one, would not feel comfortable examining sources on sports or science topics, but would be more than happy to look at the references for an African biography. Don't know if this has much water, but maybe we could compile a list of volunteers (like FAC mentors) who would be willing to give source reviews on certain topics, so the coords could ping them as needed. -Indy beetle (talk) 03:06, 12 November 2018 (UTC)

It's a skill I think we need to ask FAC regulars to develop, and it can be developed, with practice. Relying on a few experts seems both risky (as when Brian called a halt, a few months ago, to his involvement) and unfair to those experts. Mike Christie (talk - contribs - library) 03:22, 12 November 2018 (UTC)

I completely agree with this point, Mike, we should be encouraging all people to get involved with the process. Some sort of table indicating which FACs have had their review and which haven't would be great, with encouragement to go in and carry them out for each other as our FACs progress. The introducing of a new blocker in what is already a backlogged process is surely not the way to achieve this though, and I'd ask you to please reconsider your support for this proposal per Irisdescent's legitimate objections. Thanks.  — Amakuru (talk) 11:38, 13 November 2018 (UTC)

• Mike, to address one of your points at Iri's page, reviewers who try to do thorough source reviews may be regarded with mild distaste by other reviewers and open hostility by nominators, but at least multiple reviews are taking place concurrently, so things usually work out. But if you require that source reviews happen first, everyone else is going to have to wait until that reviewer is done. This puts tremendous pressure on that person, who will inevitably try to speed things up, and therefore won't go to the library, won't order the inter-library loans, etc. It's bad enough doing the work (it's very tedious), but to be criticized for it and made to feel you're holding everything up will be the last straw. If this passes, I expect that source reviews will be even less thorough than at present. SarahSV ^(talk) 03:29, 19 November 2018 (UTC)

The sub-thread below refactored out of the middle of the !voting section.  — SMcCandlish ☏ ¢ 😼  17:38, 19 November 2018 (UTC) A GA source review is typically (almost always, I'd say) a very different thing from what FAC wants. The number of editors engaged at FAC is small, which we/they are very conscious of, and regret, but it's not that small. 75 different editors posted FAC reviews in October, although just one in most cases. I'd suggest, if you want "a tiny group of editors focused on one process", try MOS talk pages. Johnbod (talk) 14:38, 19 November 2018 (UTC)

Guideline talk pages are not processes, they're talk pages, and all of them are populated only by people interested in the text of those guidelines. However, the MoS ones are actually watchlisted by far larger and broader range of editors than the typical guideline. FAC is an actual process for the entire project (albeit an unofficial, voluntary one). Your take on the GA to FA path does not agree with that of the community – we expect a smooth transition, not a conflict – and your "not what FAC wants" wording says a lot. Of course the FA criteria are higher, or we would just merge FA into GA. But the point is we already have a process for vetting the sources before it gets to FA. The solution is to improve that process, not add a new, redundant one. Frankly, Wikipedia doesn't care what WAC (i.e., the individuals who happen to be participating in it this month) "want" (more than what anyone else does). We have policies and guidelines, and processes with clear and simple procedural rules, for very good reasons. Adding another layer of bureaucracy to an already over-complicated review process does not serve the community's interests. Your 75 number is pretty meaningless in the face of 30,000+ editors per month, and over 3,000 "very active" ones (100+ edits per month). And most of your 75 only did one review; the particularly active reviewers are a much smaller number (only 10 who did more than three) – even smaller now, since the second-most-active one, with 17 reviews in one month, was recently hounded away by the a small but noisy cluster of the rest of them.  — SMcCandlish ☏ ¢ 😼  17:19, 19 November 2018 (UTC)

PS: I want to thank Johnbod, however, for demonstrating that my assumption was correct: This is really more anti-WP:MOS grandstanding. I figured that was the case when I saw that the rationale just couldn't help mentioning copyediting and that some of the defense of this is that such copyediting stuff is just a waste of time if so-and-so's sourcing concerns aren't answered first (which is patently untrue; our articles should not read like crap even if they still have a {{citation needed}} in them). While Johnbod was writing that, I got independent confirmation anyway: This monster of a thread [2] chased Tony1 (an MoS regular) out of FAC – where he was the second-most-active reviewer! The short version is that some people who flip out when their writing is criticized got mad at him for being critical, and for being critical of attempts to deny his reviews' validity/applicability, so those who focus more on sourcing review crucified him. This RfC is basically a bunch of grave dancing, and yet more tiresome anti-WP:MOS lobbying by an insular clique. Not nearly long enough ago, they basically had a massive shit-throwing temper tantrum, and even tried to declare themselves immune to MoS (with two separate proposals for their own "anti-MoS"). See this misleadingly named thread and all the drama that ensued in the course of it.
 — SMcCandlish ☏ ¢ 😼  17:35, 19 November 2018 (UTC)

More comments on opposing views

I've been the prime mover in this RfC, but it hasn't gone in the direction I thought it would. Some of the conversations about this RfC, both here and elsewhere, worry me. Some editors whose opinion I respect are opposed, and some editors who are regulars at FAC have not expressed an opinion. Not all the opposing arguments make sense to me, but I could just as easily be the one who's missing something. Perhaps there's an alternative approach that could gain more general support.

Here are three lines of argument that have been followed in the various conversations. For each of these I've tried to put uncontroversial points at the start of the list.

A. Difference between 1(c) and 2(c) reviews

1. Source reviews at FAC include 1(c) reliability/comprehensiveness issues and 2(c) MoS verifications (formatting, etc.).
2. If an article significantly fails 1(c), much of it is likely to have to be rewritten.
3. Prose reviews are wasted if the article has to be rewritten for 1(c) reasons.
4. A failure on 2(c) is usually fixable without much impact on the prose.

B. Resource constraints

1. Source reviews at FAC have disproportionately been done by a few reviewers.
2. We need more FAC reviewers to take on source reviewing.

C. Focus and quality of 1(c) reviews

1. FAC 1(c) reviews are often insufficiently thorough, unless the reviewer is a subject matter expert, or is unusually diligent in their source review.
2. Most FAC reviews spend much more time on prose than on sources.

WhatamIdoing's post today prompted point A4 above, which I think is uncontroversial; but the fact that uncontroversial points are being made as arguments implies to me that the RfC could be improved. Almost everyone who's commented on the RfC would surely agree that more thorough source reviewing, by more reviewers, is a desirable outcome. We can't increase the resources being devoted to FAC (and we don't want to drive people away, which some opponents have argued is the likely outcome of this, if it were to pass).

The original RfC proposed a separate process, because I wanted to establish a separate community of source reviewers, and elevate the importance of sourcing relative to prose. I didn't propose increasing the stringency of source reviewing because that seems like a separate RfC question, but I was hoping that if a source review process became established and attracted a community, we'd have the resources to increase the rigour of the reviews. As long as the process is embedded in FAC, I don't believe this will happen. Placing the review prior to the prose reviews seems natural if it's true that 1(c) failures do imply rewrites are necessary.

Would it make more sense to simply establish a separate (1(c) only) source review process, as in the original RfC proposal above, but delete all references to FAC? Articles could be submitted to it for 1(c) reviews; some editors might want to voluntarily use the process prior to going to FAC. Others, particularly those with a long track record in a narrow field, such as some of the MilHist editors, might skip it. This wouldn't help with points C1 and C2 above, but it might help establish a source-reviewing community. This would eliminate the bureaucratic "pre-requisite for FAC" problem that led to some opposes above. Would this be a better approach? Mike Christie (talk - contribs - library) 20:43, 2 December 2018 (UTC)

Have not !voted because I'm not sure the proposals so far will result in any real benefit to either source reviews or FAC. I do wonder, though, whether a separate, optional source review process, complete with its own star, might generate the benefits being sought here? It might give recognition to those articles that are very well researched but not quite there in terms of prose/MOS compliance and whose editors are not comfortable with the very high standards of prose and MOS compliance expected at FAC. It might be near par to FAC as a mark of quality but not supplant it as the pinnacle of quality. It could be optional as to whether the source review is completed as a separate process or as part of FAC; a successful source review conducted outside FAC could be accepted at FAC without further review (assuming, of course, that there are no significant changes between the two reviews). Such a system might create an environment that brings the same focus to source reviewing as currently exists at FAC for prose/MOS compliance reviewing, attract more source reviewers, and foster the development of more rigorous source reviews. Factotem (talk) 11:39, 3 December 2018 (UTC)

That's almost exactly what I was thinking, though I'm not sure a star is necessary. Pinging a sample of the opposers above whom I recognize as having experience at FAC: Iridescent, SarahSV, Amakuru, Imzadi1979, Johnbod, Indy beetle, Victoriaearle, A. Parrot: would this approach be better? Mike Christie (talk - contribs - library) 13:08, 3 December 2018 (UTC)

Disagree. I think a star would be essential to the success of such a scheme. It would imbue the source-reviewed article with a similar level of recognition and kudos one gets from seeing their efforts rewarded with a gold star. Isn't that one of the prime motivations for people to reach for FAC? Why not give the well-researched article a similar level of recognition? Factotem (talk) 13:25, 3 December 2018 (UTC)

Yes, that might work. Not sure about a star; certainly multiple supports would be needed for that (so like FA, not like GA/dyk). You'd need a couple of good people committed to getting the process off to a good start, and really a group of regular with some expertise in different areas. Johnbod (talk) 15:29, 3 December 2018 (UTC)

• How about implementing a system like the copy edit request page[3]? Where people can list their articles for a source review, to for example prepare for GAN and FAC, or just to improve/check the sources and their formatting with no further goal? And people could be encouraged to send their articles there, just like they are often encouraged during FAC reviews to send them to the copy editors. That would also make it seem more optional, and eliminate the issues brought up by most opposers here. FunkMonk (talk) 14:16, 3 December 2018 (UTC)
• I think fracturing FA into two stages would be a mistake, per Factotem. FunkMonk's proposal seems sensible, but I wouldn't be surprised if such a process developed its own backlog (and technically it could be a component of a peer review). However, I would like to draw everyone's attention to WikiProject Military history's A-class review process, which was modified a few months ago to include a mandatory source review. It's a shame the project is one of the only to have figured out what to do with A-class. For what it's worth, I think that the local initiative has at least some positive impact on the FA source review process. -Indy beetle (talk) 14:58, 3 December 2018 (UTC)
  • Well, what if A-class is a gap that this process could fill? Currently very few (only MilHist?) WikiProjects have an A-class process, meaning that the class is very underutilized (6.5K FA, 1.8K A, 31K GA), and that it's higher quality than GA despite not being a centralized process is rather confusing. What if we propose a project-wide A-class process whose criteria focus very heavily on the sourcing (while also maintaining adequate prose at or slightly above GA requirements)? This way we make A-class useful for a much wider range of articles, regularize the process, make it's relationship to GA and FA more clear, and incentivize source reviewing on the way to FA? Wugapodes [t^hɑk] [ˈkan.ˌʧɹɪbz] 21:59, 3 December 2018 (UTC)
    • So your solution to a lack of participation owing to editors being reluctant to involve themselves in an overly bureaucratic process, is to add another bureaucratic process? Nonsense like that is why I refuse to have anything to do with the MilHist project, which—despite the constant self-promotion of the clique who run it—I consider one of the most chronically dysfunctional parts of the entire Wikipedia project. ‑ Iridescent 22:10, 3 December 2018 (UTC)
      • I think I should clarify. I don't know how MilHist does A-class, I just know they have one. Whether this would work in a similar or different way to theirs isn't something I particularly care about. Rather, how to incentivize source reviews. As can be seen by this RfC, source reviews at FAC are not done as often or thoroughly as is needed. As can also be seen from this RfC, incentivizing source reviews by requiring them to be done first isn't well supported. So instead of doing nothing and hoping the problem resolves itself, I suggested a process focusing on sourcing as an A-class review process. It would be optional, just as GA, PR, or GOCE is, so it would address the concerns of those who don't want to break FA into two steps (like FunkMonk above). It would recognize both the prose and sourcing by the awarding of a particular quality designation like GA and FA so that it's more likely to be used and supported than something like GOCE or PR. It would be its own alternative to FA (given it's optional and an end-in-itself), so that those who want an alternative to the bureaucratic process of FA can craft and have one that would attract those disillusioned by the current FA process. It would make A-class useful to all the WikiProjects that aren't MilHist, and simultaneously make a project-wide class subject to project wide rather than WikiProject specific criteria. Whether the criteria or process look anything like MilHist or FA isn't necessary. If it seems like it wouldn't achieve any or most of those goals, that's fine, I'm not particularly wedded to this idea, but I don't think it should be dismissed out of hand because it could be bureaucratic or has the same name as what MilHist does. Wugapodes [t^hɑk] [ˈkan.ˌʧɹɪbz] 23:50, 3 December 2018 (UTC)
        • For the record, the only other WikiProject I know that has a formal A-class process is WikiProject Conservatism (seen here). It was created earlier this year, but after only reviewing two articles it, and the larger project, have fallen into inactivity since the departure of its patron, Lionelt. The criteria was not as stringent as MilHist's, but one of the reviews did incorporate a source review. As for Iridescent's comments, in my experience the A-class process at MilHist has been more helpful than unhelpful. -Indy beetle (talk) 02:46, 4 December 2018 (UTC)
• Mike, maybe it would be enough to separate the FAC review page into two sections: ==Sourcing== (or ==Sources and comprehensiveness==) and ==Writing and formatting==. Then people could review whatever parts they wanted, but it would be more obvious that the ==Sourcing== reviews needed to happen – and, well, if the over-stretched FAC regulars didn't choose to review the writing style until at least one source review was posted, then they're WP:VOLUNTEERS, and nobody can really fault them for their personal choice to wait. (Someone who wanted to talk about both could post in both sections: ===Source review by me=== and ===Style review by me===.) WhatamIdoing (talk) 01:07, 4 December 2018 (UTC)
• I think FunkMonk makes a good suggestion above: create a project to help editors review and improve article sourcing. If a key goal is to "establish a separate community of source reviewers", then that is the most direct way to do that, and it can assist FAC nominators (or would-be nominators) without creating any new mandatory FAC processes as such. I typically submit my articles to the Guild of Copy Editors before nominating them at FAC, but it certainly isn't a requirement. A sourcing-focused equivalent could turn into something very useful; if it doesn't, there's no direct harm to FAC. --RL0919 (talk) 01:59, 4 December 2018 (UTC)
• I like this section's direction. Wanted to add that all this talk of what is essentially an optional "bibliography review" process reminds me of what AfD should aspire to be. I imagine there could be future overlaps. Something simple like the GOCE request board style mentioned above should work as proof of concept, lest it end up like every other moribund WP initiative. If useful, I'd be willing to help setup/coordinate. Not watching the page but if you choose to go forward with this, let me know how I can help! Very exciting. czar 02:52, 4 December 2018 (UTC)
• A "Guild of Source Reviewers", on reflection, would not be a bad idea, as long as there enough people willing to staff it. -Indy beetle (talk) 18:31, 4 December 2018 (UTC)
• Mike Christie I'll try to respond to your ping. No, I don't think a separate process will help. What's needed, in my estimation, is a sustained effort of inculcating the expectation that articles submitted to FAC satisfy WP:FACR 1.c and that a "thorough and representative" literature survey has been undertaken using high quality sources. The problem, as I see it, is two-fold. First, it's easy to go to the web (and gets easier every year), vacuum up relevant sources, throw them together, claim they reach the bar of reliability, and submit a nomination without having checked scholarly databases, books, etc. The second problem is that it's not easy to oppose. What we get are articles with suboptimal sourcing and reviewers reluctant to oppose (and even the most skilled source reviewer isn't willing to walk the oppose gauntlet). The issue of 2.c can be equally thorny, for lots of different reasons I don't want to get into here (but am willing if you stop by my talk page), but just to give a single example there's uneven agreement re google book links despite the problems described in this excellent essay written eight years ago.
  A GOCE-type process might work, but I'm doubtful, because it might set up the same dynamic that happens with prose - reviewer points out prose issues, nominator claims it's been through GOCE so is "signed off". I don't have a single solution, to be honest. I spent a long time thinking about this RfC, vacillated between supporting it because I remember well the days when Sandy, Karanacs, Awadewit and others asked for a literature review and when I review the first thing I do is scroll to the bottom and check sources. I've done this with all your sf articles I've reviewed and it's my standard MO. But - I worried about articles using deadwood sources getting short shrift, Iridescent articulated that concern well, and I slid down to neutral. Then I worried about how very hard it is to oppose, and then foresaw a situation where source reviews will be done speedily to keep the process moving and reviewers won't oppose. Because opposing takes more guts and energy than it's worth. (Jane Austen is a good example, though the dustup there happened on the talk page.) In some ways your proposal doesn't go far enough - a good content review by necessity often requires looking at sourcing - and it goes too far by splitting the one process from the other. If that makes sense. Sorry for being long-winded. I don't stop by often, and am not always available for follow up comments, but do check my talk page. Victoriaearle (tk) 23:03, 4 December 2018 (UTC)

• Mike, I agree with Victoria about the importance of focusing more on (1c). The reality is that most articles don't undergo thorough source reviews. But if you don't review adherence to (1c), then you're also not checking (d): "it presents views fairly and without bias", because "neutrality" on Wikipedia means reflecting the majority and significant-minority positions in rough accordance with how those positions appear in the modern secondary literature. Checking for neutrality therefore includes making sure that the appropriate sources have been used, and that no important sources are missing or underused. These factors are almost never checked, because reviewers aren't experts and don't have easy access to the books.

So the question is whether we could set up a dedicated team of source reviewers who would be willing to train themselves how to do this well. Could we create a centre of excellence? What kind of editors might want to join it, and what incentives could we offer? For example, would Google Books be willing to offer 20 source reviewers full access to their online books? That would solve a lot of our reviewing problems and would attract volunteers. The Wikipedia Library could make sure they have access to all the journals. Where would we find potential reviewers? They would have to be people who enjoy checking things, not necessarily people who want to write, and they'd have to be willing to immerse themselves in something that could turn into a protracted process. Would Czar be willing to help set up a team like that? How many reviewers would we need? Is any of this realistic? SarahSV ^(talk) 05:28, 5 December 2018 (UTC)

I like your idea in general. I'd be willing to help in this as much as I can. Given this RfC I've tried to take on some more source reviewing, especially ones that involve offline sources since my institution is a member of Borrow Direct, giving me pretty wide access to books. I think a partnership with the Wikipedia Library would also be a very good idea, and maybe Google, though I'm not sure how that works. I also like it because it could help with sharing sources. If one reviewer needs access to a particular source that the WikiLibrary doesn't have but another reviewer does it easily facilitates sharing of resources.

As for how many would and if it's realistic, I'm not sure. I think Victoria rightly points out that part of the problem is how to even assess 1c. For example I'm in the middle of a source review on the Rawandan Civil War and while I feel fine interpreting the sources that are listed and checking them for verifiability, this isn't close to my field so I'm not sure what would even constitute a thorough representation of that literature. I think if this were to get off the ground, reviewers would either need to have some subject-matter expertise or we'd need to come up with some ways to clarify what constitutes a thorough representation. Wugapodes [t^hɑk] [ˈkan.ˌʧɹɪbz] 06:52, 5 December 2018 (UTC)

I'm pinging Jake and Sam of the Wikipedia Library (although I believe Jake's away at the moment). Jake and Sam, there's talk of setting up a team of dedicated source reviewers: a bit like the Guild of Copy Editors, but devoted to checking that sources have been used well. Source reviewing is hard, in part because of the difficulty getting hold of print and paywall sources and because it's so time-consuming. We're brainstorming whether we could put together a team of dedicated reviewers with excellent access to sources who would be willing to train themselves how to do this. Sam, one thing that would help a lot is if source reviewers had better access to online books. Do you think that Google Books or Amazon might give, say, 50 source reviewers full access to their online books? Given everyone's commitment to fighting "fake news" nowadays, this might be something they'd be willing to consider. SarahSV ^(talk) 19:08, 5 December 2018 (UTC)

While there are some cases that would obviously benefit from that proposal, be aware that this would probably make the systemic bias at FAC even worse than it already is. GBooks in particular isn't—and doesn't pretend to be—a "sum of all knowledge" endeavour, but is largely based on a handful of university libraries (Harvard, Stanford, U of Michigan and the Bodleian). As such it's hugely skewed towards US, and to a lesser extent older UK, publications, and towards those topics that the curators of those libraries have deemed worthy of collection. (The Bodleian is nominally a copyright library but in practice for reasons of space rejects most works that don't have relevance to courses being taught at Oxford; the other three just reflect the acquisition policies of their parent institutions over the years which in turn reflects what Harvard, U Michigan and Stanford happen to have been teaching and which authors had admirers on the faculty.) Indeed, some countries—France being probably the most significant—actively refuse to cooperate with GBooks. We already have a well-documented issue, both at FAC and on Wikipedia in general, of FUTON bias when it comes to sourcing; having a dedicated team relying on GBooks would create a clear cultural divide at FAC between those topics like US history and pre-1900 British arts that are covered in detail in GBooks, and the rest of the world.

There's also the ethical issue of whether we really want to set up any process that will make any part of Wikipedia reliant on the cooperation of one of the most notoriously unethical corporations on the planet (cooperation that they can withdraw at any time), of just what Alphabet would want in return, and of how the rest of Wikipedia and the WMF itself would react if they found out that FAC was unilaterally trying to negotiate deals with Alphabet, given that last time the WMF tried to cosy up to Google it plunged the Board of Trustees into open civil war. ‑ Iridescent 19:34, 5 December 2018 (UTC)

That Google Books isn't perfect doesn't mean it isn't amazing. If reviewers had access to that database, it would transform source reviewing. (And I don't understand what that 2016 WMF situation has to do with this.) SarahSV ^(talk) 20:35, 5 December 2018 (UTC)

• I've always found it strange how little bibliographic guidance we offer our writers, but perhaps that stems from us (mostly) not being professional librarians and not knowing any better. If you are writing about 1990s Rwanda, what databases and indices should you check for sources? When those catalogs are inevitably inaccessible, how can you get what you need? (Which Wikipedians can run the search for you?) I can picture "bibliography review" being something closer to a noticeboard, where editors raise a new topic, talk through which indices/databases they've referenced, and when outside their depth, a bunch of regulars ping specialists either for recommendations or actual retrieval requests. (Much of the latter can be referred to WP:RX.) Through the act of doing this over and over, I imagine we could build some decent bibliographic guides to check "before" posting. Some landed WikiProjects give this kind of advice, but mostly on free Internet sources, not, say, how to find book or album reviews from the 1960s.

Some kind of straw poll would be nice to gauge potential participant interest in this idea, if there is consensus to move forward.

Also this "bibliography review"... might be worth making slightly different in scope than a "source review" for FAC 1(c). The latter, like most article review processes on WP, is a kind of pillory that unearths what the article/sources lacks in order to meet the criteria, which the nominator is then tasked with correcting. Can't quite do the same with a bibliography if, by paywalls and physical restrictions, the nom simply can't do that legwork of correction. Does it make sense to keep that burden on the nominator and make the reviewers teach how to do lookup? Or should the reviewers become collaborators and just do the lookup themselves (and explain why they're using that database/index) as a collaborator, for the sake of efficiency? If part of the point of FAC is determining whether a nom has done due diligence with expected databases/indices, would it be enough for a nominator to return empty-handed when a bibliography reviewer recommends a specific search product? (Lots of questions here of how labor/burden would work re: instructing others how to do the lookup vs. efficiency of doing the lookup oneself—many trade-offs.)

My suggestion, as long as we're fleshing out how this could work, is to let reviewers be a resource on source breadth and when they consider their options exhausted, say so. They could assist in lookup and once whittled to specific citations, can defer more knotted discussions about those elsewhere (reliability to RSN and retrieval tasks to RX). This, however, doesn't solve the core of the RfC above: source review issues from close paraphrasing and text–source integrity, and the lack of these reviews leading to wasted reviews. Or is it sufficient, having restricted the FAC 1(c) source review to "claims are verifiable against high-quality reliable sources and are supported by inline citations where appropriate", having reduced the burden to check for "a thorough and representative survey of the relevant literature"?

Many other thoughts on the above suggestions but probably best to split out to separate discussions, if worth exploring. In brief, I think Google Books access is unlikely but I do think there is potential for outreach to library students, libraries that can offer fast and unlimited ILL scans, and editors who work in related areas. As for "They would have to be people who enjoy checking things, not necessarily people who want to write"—Sounds like gnomes might be converted but I, for one, would miss writing! 😢 Curious if any of the above resonates with anyone else.

czar 20:00, 5 December 2018 (UTC)

I wanted to get opinions from those opposed to the RfC on what could happen, and that's certainly been forthcoming. Pinging some of the supporters who may not have seen this thread: Brianboulton, Jayron32, Tryptofish, Giants2008, Nick-D, Casliber, Peacemaker67, Chiswick Chap: does any of the above sound better than the current RfC? Mike Christie (talk - contribs - library) 22:46, 5 December 2018 (UTC)

Thanks for the ping. I guess that my support was based principally on the importance of excellent sourcing, however we get from here to there. But as for how best to implement that, the honest answer is that I don't know. I do think we need to be careful about creating new work for which too few editors might volunteer, or that would just lead to an additional category of backlogs. Sorry I can't be more helpful. --Tryptofish (talk) 23:19, 5 December 2018 (UTC)

Ok, I worry about more bureaucracy. I think the main thing it needs is a culture change with source reviewing occurring earlier. When I get some free time I might oblige. I think the reasoning is sound. Cas Liber (talk · contribs) 23:34, 5 December 2018 (UTC)

• Thanks for the ping. It looks like we have some ideas to workshop through here. My idea is that maybe we should close this down as unclear consensus, and maybe take as much of the input from supports and opposes from all sections to workshop some better ideas at WP:VPIL to see if we can come up with some improvements to the process which could pass with broader support. In reading through some of the opposes myself, I have seen some points I had not considered in my initial supports, and think that we still have the opportunity to implement some useful improvements. --Jayron32 00:39, 6 December 2018 (UTC)
• Thanks from me also. There remain quite a few objections, so clearly some more discussion is needed to get people on board. As Jayron32 says, perhaps a neutral forum to toss the ideas about non-confrontationally could help move things onwards. Chiswick Chap (talk) 09:00, 6 December 2018 (UTC)
• Personally I think something like this is a great idea, and The Wikipedia Library would be happy to support it however we can. We actually put a little effort into trialling something related once with the Research Desk - intended to be a place where editors would ask for help identifying sources for an article (touching on the issues Czar mentions above). We didn't put a huge amount of effort into it so it didn't take off, but this is definitely something we've been thinking about. We also have a large network of librarians, particularly via the Wikimedia and Libraries User Group and campaigns like 1lib1ref that we could use to get librarian input and participation in source reviewing processes, if that was of interest.

@SlimVirgin: As for Google Books, we would love to get access to their full collection, however that seems exceedingly unlikely unfortunately, even for a small group of Wikipedians. Samwalton9 (WMF) (talk) 13:44, 6 December 2018 (UTC)

• The thought of creating "a team of dedicated source reviewers" sounds pretty pie-in-the-sky to me, I'm sorry to say. If it was that easy to gather interested source-checkers, we'd have enough interest in FAC source reviews that we wouldn't be having this RFC in the first place. In practice, it would probably be like one of the many WikiProjects with few interested editors. Giants2008 (Talk) 23:56, 6 December 2018 (UTC)

That hasn't stopped the guild of copy editors, though. Why do you think that functions pretty well? Like with the copy editors, a dedicated project would probably create a sense of purpose and community that doesn't exist when commenting on random FACs in an isolated vacuum, it would be done on their terms, and thereby motivate reviewers to work together and stick around. FunkMonk (talk) 00:19, 7 December 2018 (UTC)

Addressing the resource shortage issue mentioned below, I think a "GOSR" could also attract more source reviewers from elsewhere than FAC simply because, like the GOCE, it would not be a part/sub section of the FAC process. We could therefore draw on the probably many people who are interested in sourcing, but have no interest in the FAC process itself or feel intimidated by it (due to the time limit, grumpy/impatient nominators, etc.). This is of course very hypothetical, but I think it has a better chance than trying to rely solely on/maximise the use of the few source reviewers who are already regulars at FAC. If we don't advertise/define it as part of the FAC process (like the GOCE isn't), but still something that could be used to that end, I think it would seem more open and welcoming to non-FAC regulars. If it creates a bottle neck, fine, the GOCE does too, but people are usually willing to wait to get the benefits (source reviews in this context would be seen as something that would strengthen ones chances before FAC, like a CE, rather than a hoop to jump through during FAC), and the FAC list might get less swamped. FunkMonk (talk) 16:51, 9 December 2018 (UTC)

Be aware that GOCE is not Wikipedia's best-loved project; framing any proposed new structure in terms of a similarity to GOCE/LOCE is likely to repel as many editors as it attracts. (The three people most vocally complaining about GOCE in that thread are responsible for 119 FAs between them, and given that that thread was on a fairly obscure talkpage I assume there are many more who share the sentiment.) What we definitely don't want is for any proposed new reviewing system to replicate the issue GOCE has, of attracting a large number of people who don't really understand what they're supposed to be doing and consequently end up causing well-intentioned disruption. Both checking sources, and assessing the reliability of sources within the context of an individual article, aren't things that can just be done by working through a checklist (which is why you so often see the "What makes source name a reliable source for this article?" being asked at FAC). ‑ Iridescent 17:17, 9 December 2018 (UTC)

I have had my fair share of copy edits that made the prose worse (as the dodo FAC will attest; three different copy editors, two of which disappeared from Wikipedia entirely halfway through), so I'm certainly not trying to glorify that process, but I think the overall "assembly line"-like mechanism is pretty efficient. A major difference would also be that, unlike the copy editors, it isn't the source reviewers who will do the edits to the articles, but the nominator, so there will be few of the issues mentioned in that discussion. As far as I can see in the linked discussion, many of the arguments against the GOCE would also apply to CE suggestions and source reviews in general as they currently happen during FAC, the difference being that the nominators are expected to to the edits? FunkMonk (talk) 17:28, 9 December 2018 (UTC)

• Setting up a GOSR/guild implies organizational structure that this wouldn't need. A simple bibliography noticeboard with a high bar of new thread criteria would be sufficient (a detailed accounting of what indices/databases have been checked). This wouldn't "assign" one editor to the task, so makes it easier to disagree/redirect and lessen the impact of one's advice being bad. czar 18:52, 9 December 2018 (UTC)

• Also were there discussions of using bots/scripts to deliver basic source review advice? A good portion of this review (pulling from Iridescent's link above) could have been flagged by script without the editor needing to write it out. Tools would help with some of the aforementioned resourcing issues. czar 19:09, 9 December 2018 (UTC)

  I find this script useful; it's a little annoying as it usually leaves a few things highlighted in every article, but I've just learnt to ignore the red text if I'm not actually editing the article. Battle of Caishi and Are You Experienced are examples of FAs it spots problems on, though as you can see from the latter it highlights things that aren't problems too. Mike Christie (talk - contribs - library) 20:25, 9 December 2018 (UTC)

Recap and next steps

Suggestions from above:

• 1(c) source review process: Mike Christie, Johnbod; with star awarded: Factotem.
• GOCE-like process (Guild of Source Reviewers/GOSR): FunkMonk, RL0919, Czar, Indy beetle, SarahSV, Wugapodes
• A-class process across all projects to address source reviewing: Wugapodes
• Split FAC reviews into source/content sections and run them simultaneously: WhatamIdoing
• Inculcate expectation among nominators that 1(c) must be satisfied: Victoriaearle

There were negative or at least pessimistic comments about some of these options; I haven't listed those names.

Jayron32, who supported this RfC, suggests closing the RfC as unclear consensus, and I feel that's probably the best course. It looks like there is general agreement that something should be done about source reviewing, but no consensus what. Perhaps if a couple of supporters and I were to move to "Oppose" that would make it obviously no consensus on the RfC? I still think if it passed it would be a net benefit to the project, but I would rather take more time and try to find an approach that has broader support.

One constraint mentioned by a couple of commenters above is that the GOSR option (clearly the most popular so far) requires more resources, but the problem we're trying to solve includes a resource shortage. There's no indication of why a GOSR would give us additional reviewers. I framed this RfC so that it did not strengthen the requirements for source reviewing because of that issue; I was hoping that it would lead to a change in focus by developing a source reviewing community, and that at some not-too-distant date we'd be able to use that community to strengthen the source reviews themselves. Right now, asking for stronger source reviewing at FAC is just exhorting an understaffed group to work harder, and that has a predictable outcome. I like the GOSR idea myself, but I'd like to be convinced that Giants2008 and others above are wrong in their pessimism.

If we continue this discussion at another venue, where would that be? The idea lab at the village pump, as Jayron32 suggested above? Mike Christie (talk - contribs - library) 16:24, 9 December 2018 (UTC)

I think we can call this proposal a clear no consensus. While I agree implementing this would be a net positive, I actually really like the suggestions that have been raised so far and am grateful to the opposers who pushed the discussion in this direction. I think the idea lab might be a good idea, as would the Reliable Source Notiveboard and Wikipedia Library talk page. I'm partial to kicking it back to the FAC talk page, but no matter where we have it, I think posting a note to all those locations would be a good idea. Wugapodes [t^hɑk] [ˈkan.ˌʧɹɪbz] 23:56, 9 December 2018 (UTC)

I agree. Would you (and perhaps Czar) mind shepherding the next discussion, wherever it's held? I'd like to not be the one who starts the next attempt to get consensus on this. Mike Christie (talk - contribs - library) 02:24, 10 December 2018 (UTC)

@Mike Christie: I would be willing to help start the discussion, but am a little swamped at the moment so probably won't be able to devote much time to it until middle of next week. Wugapodes [t^hɑk] [ˈkan.ˌʧɹɪbz] 23:46, 13 December 2018 (UTC)

Looking around I found this failed proposal, Wikipedia:Sourcehelpers, which seems related, and also found Wikipedia:WikiProject Resource Exchange which might be able to help in the issue of increased resource need. Wugapodes [t^hɑk] [ˈkan.ˌʧɹɪbz] 00:07, 10 December 2018 (UTC)

Where do we stand now?

Would someone perhaps offer a resumé as to where we stand, now, after these various comments and !votes? Are we nearing anything conclusive (comments have somewhat dried up)? Brianboulton (talk) 16:41, 3 November 2018 (UTC)

The RfC has been open for over two weeks and I think we can ask for a closer now. I'll be interested to see how the close is worded; both options are supported but the one with fewer supports has no opposition, so I could see the closer picking either or even both. I'm off to the gym, and can't remember how one requests a closer, but if anyone wants to do it while I'm gone please go ahead. If not I'll figure out how to do it when I get back. Mike Christie (talk - contribs - library) 17:14, 3 November 2018 (UTC)

(Point of order) Per WP:RFCl, a formal request for comment (asI think this is?) is usually expected to stay open for 30 days, ad then a requested close can be made an WP:ANRFC. If the outcome's obvious, then of course it's a lot more lax; but as Mike Christie suggests, I think it's rather more opaque than that. ——SerialNumber54129 17:20, 3 November 2018 (UTC)

Just read that page. It says we don't need a closer if there's a reasonably clear consensus. I suspect the most natural close is that there is consensus that both options are fine and that it's up to the FAC community to pick one, perhaps with a recommendation to try the alternative first since it is simpler and unopposed. I had the alternative as my second choice but would be OK with that approach. If some of the early supporters of the first option -- Graham Beards, RL0919, Wumbolo, and K.e.coffman -- were to indicate they were OK with that outcome, then I think we could go ahead without waiting for a formal close. Mike Christie (talk - contribs - library) 19:10, 3 November 2018 (UTC)

I'm OK with that. --RL0919 (talk) 19:23, 3 November 2018 (UTC)

Pinging Winged Blades of Godric, who has—until now—little experience of FAC, but plenty of experience in RfCs  :) ——SerialNumber54129 19:27, 3 November 2018 (UTC)

Wumbolo has indicated support above. Mike Christie (talk - contribs - library) 20:12, 3 November 2018 (UTC)

Let this run for 30 days:-) No rush or so I presume.......

FWIW, if I am to close this RFC very today, I would close it in support of the 2nd proposal. ∯WBG^converse 05:52, 4 November 2018 (UTC)

Winged Blades of Godric, OK, you and SN54129 have convinced me. Would you mind coming back to close this after the 30 days are up? Around the 15th? Mike Christie (talk - contribs - library) 14:56, 4 November 2018 (UTC)

Re "there is consensus that both options are fine" and "I would close it in support of the 2nd proposal" – At this point, there's clearly a lack of consensus for either (though that could change).  — SMcCandlish ☏ ¢ 😼  18:31, 20 November 2018 (UTC)

Points to note

Following agreement and closure, whether or not after 30 days, on the basis of the present consensus the FAC instructions will presumably be amended to specificy that nominations now need to pass the sources hurdle before proceeding to the general contents review stages. This gives rise to a few possible problem areas:

• Not all FAC participants are assiduous followers of Rfc discussions, and old habits die hard, so expect some divergence from the new requirement, at least initially.
• If we are to avoid a permanent bottleneck at the beginning of the FAC process, rather more FAC reviewers than in the past will need to engage themselves in source reviews. I hope they are ready for this.
• If, say, reviewer A approves the sources and the review moves on to the content stage, does that close off all further discussion of sources? What if reviewer B has later reservations about the sourcing - are these objections considered "timed out", or do we reopen the source review stage? If the latter, in what sense have we moved on from the current system?

I think we need to clear in our minds, especially with regard to the third of these points. Brianboulton (talk) 13:04, 4 November 2018 (UTC)

We should consider advertising to the largest WikiProjects for feedback on the above RFC, which will help avoid item 1. --Izno (talk) 14:24, 4 November 2018 (UTC)

Good point - but it's 2 and 3 I'm more concerned about. Brianboulton (talk) 14:59, 4 November 2018 (UTC)

I agree with Brian on the first two points; I think in addition to the advertising that Izno suggests, some intervention by the coordinators will be needed, perhaps by temporarily collapsing or hiding non-source reviews to encourage/remind reviewers of the new sequence. As to the third point, no, it definitely does not close off further discussion. Right now, if an early source review fails to find problems, a later reviewer is free to dig deeper if they want to, and the same will be true under the new rules. Mike Christie (talk - contribs - library) 15:00, 4 November 2018 (UTC)

I see I missed your last comment, Brian; we'll have moved on because no content reviews will start till sourcing is validated. Mike Christie (talk - contribs - library) 15:02, 4 November 2018 (UTC)

I agree with Mike regarding point 3. Point 2 is concerning indeed. --Izno (talk) 16:03, 4 November 2018 (UTC)

I spammed the top ~200 WikiProject by watchers. That may change the sense of the discussion. --Izno (talk) 21:54, 11 November 2018 (UTC)

• For my final clarification: Once sources are deemed to be "validated", and the review has passed to the general review/content stages, new issues specific to sources cannot be raised within the general review. That is my understanding: Mike, am I correct? Brianboulton (talk) 14:36, 7 November 2018 (UTC)

  Up to the community and the coordinators to clarify, but no, I didn't think that would be the case. The way I've been thinking about this is that it would be exactly as if the source review just happened to occur first in every review -- reliability, formatting, and spotchecks if necessary. If that happens in the current setup, and a second reviewer finds issues, they're not barred from commenting, nor would they be barred from doing so in the new system. Does that seem reasonable? The benefits are that we'll filter out badly sourced articles up front (not a frequent issue, but it happens), and the repositioning of the logjam to the front of the process will (I hope) prove to be a stronger prompt for nominators to review other articles. Mike Christie (talk - contribs - library) 15:59, 7 November 2018 (UTC)

• Well, I hope it works as you envisage. My preference is for some formal sources validation sign-off, involving coordinator action, as a requirement before general content reviewing begins, or there may be little change from what we are doing at present. The issue of the logjam will only be resolved if sufficient editors are prepared to take a share in source checking procedures; this is a point that needs to be hammered home. Brianboulton (talk) 19:06, 7 November 2018 (UTC)

  The signoff is important; I forgot to include it in my previous post (I really shouldn't post from work -- I spend less time rereading before posting). Yes, as written, the RfC says source reviewers must explicitly support and coordinators must determine that the review incudes all of 1c and 2c, and must assess the article as passing the review before the next step begins. Mike Christie (talk - contribs - library) 19:17, 7 November 2018 (UTC)

• Brian, regarding your point 2 above, the people who supported this proposal will have to do the source reviews. Without them, this won't work. Re: your point 3, a source review won't mean the sources can't be scrutinized during the content review. Otherwise the content reviews would be prose reviews. Content reviews should look at all aspects of an article. SarahSV ^(talk) 01:33, 8 November 2018 (UTC)

• That makes sense. Brianboulton (talk) 15:54, 8 November 2018 (UTC)

• Mike Christie, as consensus isn't clear, would you mind asking for an uninvolved editor to close it after 30 days? You can request someone at WP:AN/RFC. SarahSV ^(talk) 03:25, 12 November 2018 (UTC)

  Sarah, I definitely won't be closing this. Winged Blades of Godric is willing to close (see section above), but we should probably let it go a little past 30 days -- we're getting new comments because Izno had the bright idea of notifying the top 200 Wikiprojects yesterday. Mike Christie (talk - contribs - library) 09:57, 12 November 2018 (UTC)

  Mike, the usual thing is to ask for a closer on WP:AN/RFC and they will decide when to close it. This isn't a straightforward one to close. A second option was introduced five days after the RfC had started; it isn't clear how many of those opposing the first also oppose the second; several people have said what their first and second choices are; and several of those who say they support the second option have left comments suggesting that they don't really support it. The closer will have to weigh all that up. SarahSV ^(talk) 19:05, 12 November 2018 (UTC)

  Mike is not objecting to have a 3rd party close it, only the timeline--30 days is a recommendation, not a requirement. --Izno (talk) 19:17, 12 November 2018 (UTC)

  I've gone ahead and asked for someone at WP:AN/RFC. SarahSV ^(talk) 19:51, 12 November 2018 (UTC)

  Is it appropriate to note as part of that request that we may want to wait beyond thirty days? If the notifications Izno left are still bringing in new commenters, that is. We got quite a few extra !votes as soon as the notifications went out. Mike Christie (talk - contribs - library) 22:03, 12 November 2018 (UTC)

  You can do that if you'd like to. Note that the bot will remove the RfC tag after 30 days. There are instructions at WP:RfC if you want to stop that: "To prevent Legobot's automatic ending and extend the RfC up to another 30 days, insert a current timestamp after the RfC statement, but before its original timestamp." SarahSV ^(talk) 22:26, 12 November 2018 (UTC)

  Mike Christie, I will note in passing that I did not vouch for any willingness to close this. I chose to opine that (1) the RFC ought not be closed with any immediacy and (2) the immediate state of consensus at that point of time; courtesy being pinged by SN54129. I periodically trawl through ANRFC and if I come across this, post the expiry, I might choose to close this; though. ∯WBG^converse 04:39, 18 November 2018 (UTC)

  WBG: agreed, you did not say you were willing to do so; I just assumed it based on the conversation. I still have no objections to you as the closer (I don't think you and I have ever interacted before) but I'm not clear if SarahSV does still object. Mike Christie (talk - contribs - library) 08:54, 18 November 2018 (UTC)

  Mike, no objection from me. SarahSV ^(talk) 22:49, 18 November 2018 (UTC)

  Thanks. WBG, just making you aware that there's no objection, in case you look at closing this. Mike Christie (talk - contribs - library) 23:00, 18 November 2018 (UTC)

Extended content

Winged Blades of Godric, I posted a request on WP:AN/RFC for someone entirely uninvolved to close this. One thing that concerns me is the hospital rankings RfC in June. That was an RfC opened by a paid editor (and it was obvious from the user name that it was a PR person), and it needed to be handled with extra care because of that. PR people wanting to remove rankings is possibly a red flag. Instead, you closed it with what was arguably a supervote. I say "arguably" because I can see why the close would have seemed reasonable to you. My main concern was what happened after the close. You overlooked the objections of one or two people who'd taken part in the RfC; you declined to overturn/re-open it when I requested (I hadn't taken part in it, because it wasn't even advertised on the talk page that had prompted it); and you ignored at least two pings about it from another editor weeks later. People who close RfCs are expected to be responsive to concerns, and this RfC won't be an easy one to close. SarahSV (talk) 05:08, 18 November 2018 (UTC) The above has been resolved (and was in part due to a misunderstanding and missed pings), so I'm striking. SarahSV (talk) 06:55, 18 November 2018 (UTC) SlimVirgin, Ignored 2 pings? I was missing pings, (a span back), but I don't remember whether that corresponds with the phase of the RFC. And, did you leave any tb-note over my t/p? I received a ping, wherein you hardly mentioned any good reason for overturn and I denied.Post that, I see you opened a detailed analysis at that very t/p but I missed the sets of attempted pings......Additionally, the RFC ran for the stipulated time and the behavior of the conflicted editor, was not egregiously bad, atleast as to the RFC itself. Nor were any concerns initially raised about any dubious motives. And, whilst optimal, it's seriously bizarre, if the closer has to stalk the edits of the participants to make sure that they are acting in good faith, informing all participants of any previous discussion et al. Anyways, I will take a re-look, very now. ∯WBGconverse 05:16, 18 November 2018 (UTC)