User:Physchim62/Situation Normal: All FACked up

“ At the age of five, she left Armenia with her parents and her four-years older brother, Mikael, because… ”

That lovely approximation to the English language appeared in Susianna Kentikian, an otherwise perfectly worthy article about an Armenian–German boxer which was on the Main Page on 23 January 2009. It seemed strange that grammatical correctness didn't seem to be one of the criteria to select "the very best work on Wikipedia", so I checked, but there it was again in the version of the article which passed the self-appointed guardians of the purity of the Featured Article!

Now I'd be the first to admit that there's been little love lost over the years between myself and certain of the editors at WP:FAC. If you are a FAC reviewer and you're reading this essay, please be assured that I don't hold it against you as a person. It's just that I really do believe that you are part of a system – Wikipedia:Featured article candidates – that is worse than a waste of time. I really do believe that WP:FAC is detrimental to the project as a whole, and especially to the quality of its articles!

Anyway, to get back to the dodgy grammar problem, I pointed this out (not particularly tactfully, I must admit) on WT:FAC, asking what the regulars there proposed to do about the obvious quality control issue. After all, if WP:FAC is about anything at all, it's about quality control! I got several replies very quickly, most of which can be paraphrased as {{sofixit}} – the grammatical error that is, not the process that let it slip through. FAC reviewers are used to telling other editors to fix things, but I didn't feel like sorting out somebody else's mess. Anyway, I was interested to see how long it would take for somebody else to clear up the QC problem on one of Wikipedia's most visible pages that day. More than sixteen hours, as it turned out, a fairly debatable commitment to the quality of featured articles.

The undeniable commitment of the "FAC regulars"

That isn't to suggest that FAC reviewers are not committed to the FAC process, quite the opposite! They are typically just the sort of committed volunteers which would make any project proud. The problem is that there appears to be too much commitment to the process. I must emphasise again that I don't wish to criticise FAC reviewers as human beings, or as Wikipedia volunteers for that matter. They are innocent of any WikiCrime, victims of a phenomenon which is well known in the sociology of organisations: Parkinson's law, "work expands to fill the time available for its completion".

Work there certainly is at today's WP:FAC! Between the reviewers, the nominators and the editors who do the "administration" of the whole show, the process ties up the equivalent of about 30–50 full-time volunteers at any given moment. If you haven't been involved in an FA nomination recently, I can assure you that it can be a pretty arduous task. "Nominators are expected to respond positively to constructive criticism and to make an effort to address objections promptly." tones FAC, and criticism can easily reach tens of kB. Regardless of how much work you've previously done on the article, you'd be better off picking a week when you've not got much else on before making that nomination! It's worth noting in passing that it falls to the nominators to edit the article in response to the reviewers, in a reversal of the {{sofixit}}, be bold! philosophy which operates over most of the rest of the project: criticism is cheap when it's someone else doing the work. Reviewers, of course, and there are some exceptions among their number, would never have time to make all the corrections they demand: they have to dash off to go through the next nomination with their fine MoS toothcomb, and so the process continues.

When you get a fairly small number of people all working very hard towards a common goal, you get another interesting sociological phenomenon: that goal becomes very important to the people concerned, whatever it is, and even more so the harder they work towards it. The most effective group size for this sort of positive motivational feedback is about 20–30 individuals: it's no coincidence that this is about the size of a secondary school class in many countries, or of a platoon in the armed forces, or of a training squad in many team sports, or of a work team in many companies. It's also about the number of "regulars" at FAC, and indeed the number of "regulars" on many of the most active and productive WikiProjects. I'm not suggesting that there are any dark forces or bizarre initiation rituals at work on WP:FAC! This is pure human psychology at work. The key is to get the objective right at the start of the process because, once you have the team working hard towards that goal, it is the goal that will become important, not the reasons behind it.

The goal on FAC is to ensure that featured articles meet the featured article criteria, and its regulars work very hard at it. There are at least four semi-automated scripts which have been invented to do some of the work, so the human reviewers can work even harder at the rest or, in practice, work just as hard at ever more meticulous criteria. One-fifth of all featured articles promoted in 2008 were the work of just ten editors, most of them also reviewers for other FAC nominations. One editor managed to produce 22 featured articles that year, while also serving on the Arbitration Committee. Truly impressive stuff, I'm more than happy to concede, but it has its darker sides.

The reasons behind the goal of FAC are varied, and different Wikipedians would phrase them in different ways, but let's just get one thing clear, that there are reasons behind the goal! Ensuring that featured articles meet the criteria is not the end in itself, any more than, say, winning a football game is the end in itself. One of the reasons behind FAC is to ensure that the articles featured on the main page are of a high quality. Another reason is to promote article quality generally throughout Wikipedia. I'll come back to these two in a moment, but if you have any others, don't worry, I'm sure they're perfectly valid so long as they fall under the general (and hopelessly vague) heading of "improving the encyclopedia". And if you're an FAC regular reading this essay (1) congratulations for getting this far without throwing the computer out the window, and (2) allow me to repeat this very slowly to you:

ensuring that featured articles meet the featured article criteria is NOT the end in itself.

Does FAC promote article quality?

So does the existence of FAC promote article quality throughout Wikipedia? After all, FAC has doubtlessly led to some very good articles being written, and I'm sorry to say that some of those articles probably wouldn't have been written if it wasn't for FAC. But that is distinct from promoting quality throughout the encyclopedia. Let's take a closer look at the articles which pass the current system and, more importantly, why these articles pass and others don't (either because they fail FAC or because they are never submitted).

Featured articles are often (not always, admittedly, but often) concerned with small, well-defined and fairly obscure subject areas. The reason for this can be traced to the FAC review process. If the subject area is small and well-defined, it is fairly easy to show that you have covered it comprehensively, one of the Criteria. If it's fairly obscure, there are probably only a few accessible works that you can use as references, so you don't run the risk of a reviewer saying "but you haven't included X's take on the subject." So as long as the subject is not so small and so obscure that you can't write and illustrate a decent article about it, you will get a relatively easier ride of it at FAC.

But are these actually the articles we want to be improving on Wikipedia? Perhaps our readers are more interested in articles about wider subject areas and on topics they had heard of before they came to Wikipedia? How would we know? The Wikipedia 1.0 Editorial Team has conducted and coordinated an enormous amount of research into articles on Wikipedia: as one part of it, the various WikiProjects have been asked to assess the articles in their subject areas, both for quality and importance, and over 1500 WikiProjects participate (or have participated) in this two-fold assessment, which covers more than 1.6 million articles. Fully one-third of all featured articles have yet to be assessed for importance by the WikiProjects. Their subject matter is so obscure and insignificant that it falls off the radar of those editors working on specific subject areas! And yet such articles are specifically favoured (at least in practice) by the current FAC system.

Consider the fate of an article about a broader, better-known subject area: some of these articles do make it successfully through the FAC process after all. One of the risks that such a nomination will face is that someone will come along and say "it's not comprehensive, there's not enough about XYZ". Fine, maybe the reviewer is right. But what if the nominator thinks that the reviewer is wrong, that "XYZ" is simply not important enough to be in the article, that inclusion of "XYZ" would disturb the balance of the article or lead to undue weight on a given facet of the subject? Sometimes the reviewer can be persuaded of the error of their ways but, more often than not, the article will simply fail. So should the nominator make the addition, even though they believe that it makes the article worse? Should an "expert" reviewer from a WikiProject keep quiet about minor omissions of this type to avoid jeopardising the chances of work from their subject area appearing on the main page? I know I've kept quiet about such omissions in one or two chemistry FAC nominations in which I've participated as a reviewer. FAC certainly doesn't act as much of an incentive for improving articles on these broader topics, topics that the rest of the project considers important for our encyclopedic coverage.

What is "quality" anyway?

FAC doesn't handle subjectivity very well. It very much prefers to work with criteria which are absolute or objective, such as whether the references confirm to today's version of the MoS, or whether the external links work. If there's a difference of opinion, it is usually the person who objects to FAC status who will win, unless the criticism is really petty. To quote WP:FAC, "the main thrust of the process is to generate and resolve critical comments in relation to the criteria, and … such resolution is given considerably more weight than declarations of support." The current FAC system could not work any other way. But we are left with a paradox: we are, in effect, defining quality in terms of what can be objectively demonstrated, but we have no check that the objective criteria actually correspond to quality in any meaningful way. We can only say that FAC promotes quality if we define quality to be featured articles, and there is some evidence at least that that sort of quality is not what the rest of the project is really looking for.

On the perils of private ownership

What FAC certainly does do is make FAC editors feel good! After the long slog of writing a high quality article and the sheer WikiTorture of an FAC nomination, there's a wonderful rush of endorphins when (and if) your article finally gets promoted. I know, I've been involved as an editor on a couple of featured articles (not that that gives me any particular rights either way to comment on the FAC process). You feel proud of your little baby and it's brand spanking new shiny little star, ready to defend it against all the Big Bad Trolls lurking in the woods of Wikipedia!

But hold on a second! No editor actually owns an article on Wikipedia! Just about any article can be edited by anyone, that's a fundamental principle of the project and with good reason. I wouldn't say that the arduous nature of FAC was designed to act in opposition to WP:OWN policy, but that's certainly its effect. The shiny little star becomes a shield to an editor against unapproved changes, regardless of the pertinence or even desirability of those changes. "It's a featured article, it must be good!" Well no, as we've already seen, that phrase should really be "it's a featured article, it must be a featured article", but somehow that doesn't sound quite as good as a rallying call for would-be edit warriors!

Hold the front page!

Which brings us to the other raison d'être of FAC: to ensure the quality of articles which are featured on the main page. By and large, FAC does a reasonable job of making sure that bad articles don't get featured as "Wikipedia's very best work". If it didn't, it would have been done away with long ago. Usually, someone shouts "STOP" in time, before a really bad article gets featured. But at what cost? Thirty or forty volunteers tied up at any given time! Worse, great swathes of the encyclopedia effectively denied the chance to be considered "Wikipedia's very best work", regardless of their other merits, simply by the way the FAC process operates. And are some featured articles actually that good anyway?

I recommend you take a read through Toa Payoh ritual murders, which appeared on the main page on 10 February 2009. It's truly great writing, I'd say it's professional quality and indeed I think the authors missed out on the chance of some hard cash by putting it on Wikipedia instead of selling it to a newspaper or magazine as feature writing. But is it the tone we're looking for in an encyclopedia article? I'll leave it to you to decide: I'm easy with the idea of subjective quality criteria, and that means that not everyone will agree on what's a good encyclopedic article. But subjective quality criteria are not what we get from WP:FAC; instead we get a pretense of objectivity, of "absolute" quality, something which is simply unattainable.

If someone usually shouts "STOP" before a bad article gets featured, that is more by chance than by design. As FAC reviewers delve ever deeper into the arcanities of the Manual of Style, it is inevitable that they concentrate less on the wider view of article quality. As I've already mentioned, the decision-making process on WP:FAC, and particularly the short time for which nominations are considered, does not sit easily with problems that turn on shades of opinion: as a result, such questions receive less attention than maybe they should do.

The FAC reviewers of Susianna Kentikian didn't notice the obvious grammatical error at the start of the second paragraph because they were concentrating on other things. They were concentrating on the minutiae of style, not the wider question of grammatical correction. And because everyone was concentrating on the minutiae of style, no one noticed the mistake. It is a case in point of the practical advantages of taking differing criteria into account, but differing criteria lead to disagreements, and disagreements on WP:FAC lead at best to a delay in closing the nomination, more usually to the nomination's failing. They interfere with the steady process of ensuring that FAs meet the FACr, which is the goal and activity of WP:FAC.

What could be worse than featured crud?

A grammatical error in a featured article is an embarrassment, but hardly a catastrophe. On the other hand, if the quality control system is missing problems such as the error at the top of this essay, perhaps it's missing more important things as well… What would be a really bad article to put on the Main Page that might get through today's FAC process? I think most editors would agree that an article with serious BLP concerns should not be on the Main Page, but BLP is a notoriously contentious area of Wikipedia policy, exactly the sort of area which today's intensive FAC process deals badly with.

In the latter half of 2008, an article was featured on the Main Page which left me feeling very uneasy on reading it. I was sure that this was not the sort of article we should be promoting as "Wikipedia's very best work", but I wasn't quite sure why. Of course, I would have to be sure of the exact wording of the precise policy to be able to do anything about it, and then be prepared for quite a fight, because one thing that FAC does very well is making editors absolutely certain of the quality of their featured articles, especially on the day that they appear on the Main Page! So I left it at first, still feeling uneasy but convinced there was nothing I could do about it.

After a couple of hours' editing on completely unrelated subjects, it suddenly struck me what the problem was with that day's featured article. The article named three individuals who had been arrested on suspicion of a serious crime. But the individuals had been released without charge or acquitted in court: they were (and are) presumed innocent and there is no reason to link their names to the alleged crime (which might not have been a crime in any case). The article stated all of this, but it continued to use the individuals' full names for all the world to see, a clear violation of WP:BLP (the people concerned were not notable for anything apart from their arrest). Worse, in many jurisdictions – including the jurisdiction in which the three individuals live – it is illegal to publish personal details of specific, identifiable people (such as their full names, for example, or the fact that three specific, identified people had been arrested and later cleared) without permission or good cause, and Wikipedia didn't have a good cause in this case. This particular FA was not just slightly obsessive in the coverage of its subject (which it was as well), it was risking some pretty bad publicity for the whole project.

I tried to open a featured article review, but the "rules" state that you can't do an FAR within three days of the article appearing on the Main Page – Heaven Forbid that anyone should question the quality of a featured article just as it's at its most visible! I tried to bring the matter up on the article talk page – tangentially, so as not to generate the bad publicity I was trying to avoid – but the three lead editors of the article threatened a three-on-one revert war if I removed any of their sourced information, however insignificant it was to the subject matter of the article, because the article had gone through FAC that way and so it must be OK! In the end, I decided not to raise any more fuss: after all, why should I have to put myself out when the people I was most protecting from embarrassment – the article editors and the FAC reviewers – were exactly the ones who were being least helpful?

Wikipedia's had more BLP problems since then, so it should come as no surprise to anyone that there's another one lurking in its depths. If anyone wants to sort the problem out now, it's the article about the American teenager who is supposedly notable for having gone missing in a foreign country. If it were up to me, I'd just delete it: I don't see what purpose it serves the encyclopedia apart from being an FA. And I don't see why I should save the blushes of those who prevented me from sorting it out earlier.

In conclusion

Of course Wikipedia needs to have a way of rationing out the limited space on the Main Page, but that way doesn't have to be WP:FAC. Any new system should preferably involve a wider range of people (if possible in greater numbers) than currently contribute to WP:FAC. It should also be open to other valid criteria of what constitutes "Wikipedia's best work", rather than just blinding adherence to the Manual of Style. The MoS is important, of course, but it's not the only thing that's important or that the project as a whole wants to promote. It would help if the new system came as an extension of what Wikipedia editors are already doing, rather than a stand-alone system like FAC where the rules and usages differ greatly from the norms elsewhere on Wikipedia.

Today's FAC process monopolises the promotion of "Wikipedia's best work" while offering a false assurance of "absolute quality" and excluding many of the editors who strive to improve the areas that others have identified as important. It inadvertently promotes behaviour which runs contrary to some of the most basic principles of Wikipedia, and it uses up great volunteer resources for a product that's often of questionable value to the wider project. These are not just unfortunate accidents, they are the completely normal psychological consequences of the way the system has been set up and run. WP:FAC can't be patched up, say by adding yet another layer of complexity onto its already mind-boggling assessment procedures – it should be scrapped and replaced by something else. It's not simply that WP:FAC doesn't always work well, it's that it's completely FACked up.