User:Looie496/Analysis of FAC
This study was motivated by my experiences at FAC both as a reviewer and as a nominator. My experiences gradually led me to believe that it would be very difficult for articles on topics with a large literature to pass -- mainly because of the way referencing is handled. This in turn caused me to wonder how frequently articles on important topics have succeeded at FAC. So I set out to do a bit of research. The findings show an interesting and significant pattern. Executive summary: From 2006 to 2008 articles on important topics were being promoted to FA status at a steady clip, averaging nearly one per week, but in 2009 the rate of promotion of important articles fell off a cliff, and through 2010 and 2011 the rate has been well below one article per month. The following table shows the data. Below I will explain how these numbers were calculated.
| Year | Number promoted | Per month |
|---|---|---|
| 2003 | 3 | 0.25 |
| 2004 | 21 | 1.75 |
| 2005 | 12 | 1.0 |
| 2006 | 36 | 3.0 |
| 2007 | 43 | 3.58 |
| 2008 | 33 | 2.75 |
| 2009 | 17 | 1.42 |
| 2010 | 5 | 0.42 |
| 2011 | 6 | 0.67 |
To obtain these figures, I first went through the list of featured articles at WP:FA, selecting the ones that seemed to me to be about topics of high importance. The selection was based on my own opinion, and other people might choose a few that I omitted and leave out a few that I chose: I don't believe the differences would affect the outcome. It is very important to emphasize that I selected articles without knowing the years in which they were promoted. Before starting this project I made a cursory scan of a few articles that caught my attention, and noticed that none had been promoted in the past two years; I then decided to treat the problem in a statistically valid way by blinding myself to the promotion dates until after the selection was made. The only way I know to access the promotion date is to load the article's talk page and click "show" on the article milestones -- I spent a couple of hours doing this for all 175 articles in my list, but only after I had put the list together. At the bottom of this page is a complete list of the articles I selected as important, and the year in which each was promoted. The list can be sorted by year if the reader wishes.
A few footnotes are in order. First, in 2004 there was an article category called "refreshing brilliant prose". Articles with that rating were later reassigned as FA, and I considered such articles to have been promoted in 2004. Second, there are about half a dozen articles that were promoted, then demoted, then promoted again later. In such cases, I used the year of the first promotion. A change to the second promotion would obviously decrease the numbers for early years and increase the numbers for later years, but the number of articles in this group is not large enough to make a meaningful difference. Third, since articles that have been demoted and are no longer FA do not appear at WP:FA, none of them have been included. If they were included, the result would obviously be to increase the numbers for the early years.
I will now discuss possible explanations of the sharp falloff in 2009 of promotions of important articles.
It might be suggested that the falloff occurred because editors were running out of important topics. That explanation might actually be valid for astronomy, where almost all of the central concepts are represented by featured articles. No other topic area comes close to that, though. I would estimate that there are several thousand topics I would have rated as important -- the 175 in the list are only a small fraction of that, and astronomy only accounts for about 20 of them.
It might be suggested that this is a result of the general decline in the number of editors. That phenomenon may be part of the explanation, but it can't account for the abrupt falloff in 2009 -- the shapes of the curves are quite different. My impression, although I have not tried to quantify it, is that the rates of promotion for articles on topics of minor importance do not show nearly as sharp a falloff -- if they show any falloff at all.
I believe that the only viable answer is that the policies at FAC changed in 2009 in a way that works against articles on important topics. I was not an active editor at that time so can't say from experience, but my impression is that the mechanism shifted from an emphasis on content and readability to an almost exclusive emphasis on nit-picking aspects of form, and above all to a rigorous demand for referencing of every sentence. Articles on important topics are generally much more work to reference than articles on minor topics, because the relevant literature is so much larger and because they demand a level of synthesis that makes it difficult to pin down each statement to one specific source.
Let me summarize the questions that I feel need to be addressed:
- Are these observations valid?
- Am I correct that a change in policies at FAC has caused the decline in promotions of important articles?
- Can we live with a situation in which FA status goes almost entirely to articles of minor importance?
- What can be done to fix the problem?
Followup
[edit]After discussion at WT:FAC, there are a few more points to be made. One is that I clearly missed a substantial number of important articles, such as Poetry or Funerary art. Thus the numbers given above should be somewhat increased, although I see no reason to think the pattern would change.
A useful counterpoint to the arguments here can be found at a draft essay by user:Grandiose called Wikipedia has come far, which can be found at User:Grandiose/sandbox. The essay includes a table of statistics showing 764 FA-class articles of Top importance, and 1244 FA-class articles of High importance. I can't come up with numbers quite that high doing a category search, but they are in the right ballpark at least. However, these ratings are assigned by WikiProjects, many of which deal with restricted topic areas, and at least half of the "Top-FA" articles would probably not be considered important in the broader scheme of things. I would estimate that we have at most around 300 FA-class articles of high importance by any reasonable criterion, and it is pretty clear that the number is decreasing due to demotions much faster than it is increasing due to promotions. There is no obvious reason to think that this trend will reverse without a change in procedures.
I should perhaps have tried to make a distinction between broad topics with a huge literature, such as poetry and atom, versus important topics with a relatively limited literature, such as Statue of Liberty. A cursory scan indicates that such a distinction would not reduce the amplitude of the pattern I have noted, and probably would amplify it.