User:GreatSculptorIthas/sandbox/Assessing Articles with Rater--A Detailed Guide

DRAFT!

There is truly no substitute for jumping in and assessing dozens or even hundreds of articles. That is the best way to truly learn how to use this tool. However, there are certain pitfalls which may be avoided by taking sound advice from someone who has already stumbled his way through that process.

There are three main tasks which you can (and should) perform with Rater:

1. Assessing article quality
2. Assessing article importance
3. Adding and removing WikiProject Banners

First, I provide some general tips for using Rater. Then I provide some basic guidelines for each task, so that you can get up to speed quickly and feel more confident about using Rater. Finally, I discuss some miscellaneous issues which may come up during your use of Rater.

Using Rater in general

I'm not going to give a guide to the interface and all the basic features of Rater. Those are all already available. Here I give a few tips on using Rater in any situation.

Shortcuts

Rater is best used with keyboard shortcuts. I use keyboard shortcuts all the time. It is my preferred way to do almost any task on a computer, so they come naturally to me. If you are less familiar with them, you may think you'd rather just click through the menus by hand. I'd like to encourage you to try the shortcuts for Rater.

Alt+5 will open Rater, and Alt+S will save whatever you have entered into it. These two make Rater so much more useful. It will save on your hand and your time, trust me. Remember that you can also set Rater to automatically open on unassessed articles in mainspace as well, saving you even more trouble.

While not an access key, pressing Enter upon opening the Rater interface will allow you to immediately type in a WikiProject name. This is very useful for adding new WikiProjects onto neglected articles quickly and efficiently. Press enter again, of course, to accept the suggested WikiProject banner.

There is one known issue with the Alt+5 access key. If you use the Vivaldi web browser or any other web browser with a Shift+Alt access key combination, you may face issues with the alt+5 access key used to open Rater. You can try importing the code directly to your User.js page and editing the access key to be a letter rather than a number. Short of that, I have found nothing else that works. I have, however, recently submitted a bug to the Vivaldi devs and this might get fixed. It's a problem with the browser, not Rater.

Using Rater on a large group of articles

Sometimes, you may come across a WikiProject which has been very neglected and which may have hundreds or even thousands of articles to be assessed for importance, quality, or both. This can seem quite daunting, but I have some tips to help you. First, for very large backlogs, consider looking at articles only in certain categories or task forces within that project. That may help break the task into more manageable chunks. I wouldn't recommend bothering with this if the log is less than 1000 articles or so.

Second, try sorting in Article Lists by quality and then alphabetically. Of course, this is impossible in virgin, totally unassessed articles, but when it does work it is very helpful. This will produce a list which is broken up two more times. All of the B-rank articles will be sorted by letter, and then all of the C-rank articles, and so on.

From this point on, you can simply work by letter. Open all of the B-rank articles starting with A in new tabs (Ctrl+click or middle mouse), work through those, and then return to the listing and go through a new letter. You can visually see which links are purple and use that to help you determine what you have already done. Refreshing the page will not usually work as these pages are only updated at certain times each day. If you are spending multiple days on this, though, it might make you feel better to watch the list shrink every day by refreshing.

Sometimes, a letter is really, really common to start articles with. When I did the backlog for WP:Biology, many articles started with B. Just move to the next letter, then. I did all the articles up to the first instance of Bio*. Then I did all articles starting with Bio in another set. Then I did the rest of the B articles.

Finally, sometimes you will just need help and that's ok. I made this guide in part to help you ask for help, after all! Make sure you link people to Rater and to this guide so that they don't think you are asking them to manually edit talk page sources for hours on end. This should drastically speed up the process and make people more likely to help you.

With a huge workload like the one I am currently facing in WP:Women scientists (almost 3,500 articles!), it might be best to get people to "adopt" a letter/handful of letters and be responsible for those. Such a system has already been devised for WP:BIO's gigantic backlog at their assessment page. Note the table. Again, the Article Lists pages don't update constantly and it would drastically increase overhead (and frustration) for every person to go through the same pages just to find that they have already been assessed.

Assessing article quality

Article quality is a key metric for WikiProjects. Not only does it aid in prioritization of tasks, it helps to quantify how much work needs to be done by a WikiProject on its articles as a whole. Finally, it can serve as a good way for editors to think about their own edits and workload. Moving an article from stub to C-class is a good feeling, and that sense of accomplishment is also a good stopping point for a busy editor. Now, let's start at the basics.

Every article is rated on its quality through WikiProjects, and technically, each project has its own rating. In practice, since all projects should be using the same assessment rubric, and since Wikipedia reports the highest currently-recorded quality as THE article quality, there really is only a single value in practice. It is strongly recommended that you always use the bottom button on the Rater interface to set the quality for all banners at once.

Using settings to help you

Go into your preferences and enable (!name). This will color article titles based on the article quality, and will also allow you to see the article quality written out in the header.

I also recommend one of the permanent header userscripts. I use one called Floadhead which runs pretty light as it is just a CSS tweak. This will make it easier to access the header and the Rater interface for those of you who don't like hotkeys or who don't always use them.

Judging article quality with the rubric

Article quality is judged against a rubric (!) which is common to all WikiProjects. In theory, you should follow this rubric. In practice, however, the rubric has some issues. First, it includes some ratings which you should not personally use. You should never rate an article as GA or FA on your own, unless it has been assessed to be one of those for one project and you just need to set it to that level for other projects. GA/FA quality is decided by a special process. Don't touch them. If ORES says GA/FA, you should probably rate it a B or C and consider sending it off to GA nominations.(!) Second, the A rating is really not used very often. Many projects do not use it altogether. It is so similar to GA/FA that it is practically redundant. I am 2000 edits in and I have not seen this one in the wild. Don't use it unless you know for a fact that your WikiProject uses it and wants to keep using it for some reason. It is my opinion that this rank should be removed entirely, but that is a topic for another day. Third, only a single article is provided as an example for each quality rank. However, quality should be seen as a range, not as a single level. Most of the listed articles are at the higher end of the range for their rank. Please do not think that all start-class articles have to resemble the one listed in the rubric. That is totally false. The good news is that the rubric performs best at the low quality ranks, and that is where you will do the majority of your work. ORES is also most accurate at these ranks, and you can lean on it more here. You should generally follow the rubric, but don't be afraid to go a rank up or down if you feel very strongly about a particular article. I have never disagreed with the rubric by more than a single level--if you do, you should think hard about what rating to give, or perhaps send it to someone else that you trust.

What if the rubric is a pain in the rear?

And what if the rubric doesn't answer your question about an article? Well, I go by my instinct, but I have done this literally thousands of times, so that won't work for you at first. I recommend you take a look at 5 qualities of each article. They are easy to scan for without reading line by line, and they are good indices for article quality. • Quality of the lead/lede--Read the beginning of the article to see what the wording is like. This is usually the best-worded portion of the whole article, so if it has problems, you know the rest of the article likely does too. • Length--Simple, just see how long it takes to scroll. You will begin to learn how long different ranks of articles are on average. • Breadth--Similar to length, scan the article and look at its table of contents. Does the article have several headings that present different aspects of the subject? Or does it have very few? Length together with breadth make for a better article in most cases. • Visual aids--Look for infoboxes, images, diagrams, equations, and anything else besides simple text. • References--Look at the reflist and look through the article. Are there a lot of in-text citations, or very few? Are there tons of external links but few internal ones? Those are things to consider. Finally, ORES receives an honorable mention. I always take the ORES rating into account, but I frequently disagree with it. However, it puts me in the ballpark, and that's what I really need.

Speed versus accuracy

It can be intimidating to do assessment on a grand scale. What if you are wrong about an article? What if you mess up? Well, it might not actually be a big deal. Mislabeling an article as start when it should be a stub is not good, and it is a very serious problem if you do it all the time. But if you make that mistake once in a while, it is likely not going to be a serious issue for anyone. Both ranks need serious work. However, ranking a C-class article as a B is a little more serious. There are not that many B class articles, and many people choose not to work on them much because they are "good enough." You should be more careful how you rank an article the higher you rank it.

Issues with ORES suggestions

Rater uses ORES to predict article quality and give you a good guess of what your article should be. It's often somewhat accurate, but it has some problems. Please do not ever trust ORES blindly. That is basically automated editing and it should be reported accordingly. Even then, it is your duty to check over automated edits. You are responsible, not Rater, and not ORES.

• ORES tends to use article length as a predictor of quality, but in articles with lists, lots of refs, or lots of para breaks, it may fail to understand that not all length measurements are comparable. • ORES will attempt to grade every single page you use Rater on, including non-assessable pages like redirects, disambigs, and categories. If it tries to call your list a C-class article, please do ignore it. • ORES tends to break at high article quality. B=class articles and even C-class articles are often estimated as GA or FA-class. Not only is it irresponsible to apply a GA/FA rating unilaterally, it's just not usually accurate when it says that. Generally, anything ORES/Rater calls GA/FA is most likely a B or perhaps a C rating. • ORES tends to generally overestimate ALL pages compared to the standard example articles on the assessment rubric. When presented with an article on the border between start and stub, it will pick start. Ask it about an article squarely within start, and it suggest a C. That does not necessarily mean it is wrong. As I stated in the last section, ORES does this because people don't really follow the rubric, and I kind of agree with them.

Assessing article importance

Article importance is the second most important task you will perform with Rater. It is also a bit more subjective and complicated. However, it is still extremely important, and you should not ever assess articles without filling out importance. Leaving fields blank makes more work for everyone else and is very poor practice. We should remember that importance means importance to readers, not to ourselves or to other editors. It is very tempting to rate higher importance on articles that we care a lot about, but we need to keep our biases in mind and try to avoid being too affected by them.

Displaying pageviews via the Xtools setting

Pageview counts are really the only numerical tool which can be helpful in assessing importance. By looking at the number of views of a page (and thinking a little about what might drive those views), we can often gauge the importance of a page. To use pageviews, one may click on the link in the left sidebar. However, I much prefer and advise the use of the Xtools page info tool. Go into your preferences and locate this setting (!name). Turn it on to receive page metadata below the title of each article. It is compatible with Shortdesc if you use that, thankfully. From now on, you will always see each article's pageviews for the last 30 days, and that information will be formatted as a link; click it and you may look back even further, up to 90 days.

Cutoffs

I advise you to come up with set values of pageviews which correlate with each level of importance. Unlike quality, which is graded against a rubric, importance is graded against other articles, and it should be graded in a logarithmic fashion. By far, the most common level should be Low. Fewer than a tenth of all articles should be graded Mid, and fewer than a tenth of those should be graded High. Top should be reserved for key articles in a project. I use cutoffs and discretionary ranges to do my assessment. For each level, I have a cutoff value which almost always puts an article in that level. However, I use a discretionary range to help me account for random fluctuations in pageviews. I don't require an article get a full 1,000 views in the past 30 days for a Mid rating--I am just as happy to issue Mid to an 800 view article, but I do give it a second glance first. I also keep in mind the project I am rating for--some projects are bound to have tons of views, and others are bound to have fewer. Importance is a metric which is only useful for doing work within a WikiProject, and you should consider that project before you rate.

Low--Below 500-1,000 Mid--Between 500 -1,000 and 8,000-10,000 High--Between 8,000-10,000 and (very roughly) 30,000-50,000 (depends heavily on WikiProject) Top--Above (very roughly) 30,000-50,000. Almost guaranteed above 50,000 (but not for every listed WikiProject)

Issues affecting pageviews

Many issues affect how many views a page gets. Generally, pageviews are spike at creation due to initial work/new page patrol, drop to baseline, and then slowly increase over an article's lifespan. A change in any of the issues listed here will usually lead to increased views. Keep that in mind. Articles may be underviewed if they have few links to them (check What links here in the left sidebar), if they are very old and abandoned, if they are in very few/very broad categories, if they are not a part of many WikiProjects, or if they are not slated for cleanup listing or any other type of work. If an article is a little below your cutoff but is also seriously neglected and hard to find, consider fixing the problems and rating it a rank higher than you would just based on the pageview count. Some articles have seasonal changes in pageviews--the classic example is that Shark gets more pageviews during Shark Week. Other pages have spikes due to outside media--I recently reviewed a biographical page that had over a tenfold increase in views due to the subject's death.

In general, you can usually trust the 30-day viewcount to be good, but if something seems off, like a random start-class biographical article having tons of views, check the longer term, do a little web search, and see if there is some reason. You don't need to make it a big project--just spend a minute or two on it before you assess.

Multiple WikiProjects

Importance should not necessarily be the same to every WikiProject. Unlike quality, importance is very dependent on the WikiProject. Some projects have more responsibility for the content of the article. Imagine Social darwinism being rated equally important to WP:Evolutionary Biology and WP:Sociology. Clearly soc is actually responsible for more of the content, and it should probably get a grade higher than evo. If I find an article with a lot of pageviews (say 4,000), I typically rate most of the projects as mid. If I end up rating a lot of the projects as low, it might actually be worth removing those projects and just making sure that the equivalent category is listed at the bottom. On the higher end of the spectrum, if a page is of top importance, that is usually only for some of the projects listed. The rest will likely be high importance. Again, if I see a project with low or mid importance on that list, it might simply be worth removing it (or not adding it if you are assessing from scratch).

Changing other people's ratings

When I first began assessment work, I would actually go into the talk page history and look to see how long ago assessments were issued to help me determine if a page is ready to be changed or not. I tend not to do that very much anymore--I think it is best to be bold and get the job done. If someone disagrees it can be easily changed, and that is the point. When it comes to importance, though, I give some deference to the editor(s) who came before me, and I usually split the difference. If I see a rating of high or top, and I want to rate low, I rate mid. I do that because this is not my Wikipedia and I do not OWN these articles. It is our Wikipedia. I don't know very much about most of the articles I work on and I feel it is best to assume that the other editors might know something I do not. I also often leave ratings of mid that I personally would not have given based on pageviews. Not everyone has the same criteria as me and that is ok, because mine are arbitrary anyway. I made them up completely.

Adding and removing WikiProject Banners

While this is the least important task you will perform with Rater, it is still very important. For brand new, totally unassessed articles in NPP or AfC, it is actually critical. Articles are only assessed through WikiProject banners, so every article should have at least one. Other than that and a few other cases, though, this is something you shouldn't feel like you must think too hard on.

Exploring WikiProjects with Rater

Rater will autocomplete a project name, so if you don't know if a project exists, just try typing it in. That will usually get you headed in the right direction, even if it has a slightly different name. It is always a good idea, though, to click on the link to the banner template page and then on to the project page to learn about its scope and goals and make sure it is the appropriate project. In many cases, you will come across a project which has more specific daughter projects. WP:Biology is a good example. Most articles belong in a more specific project (like WP:Genetics for example) and not in the parent project. In some cases you may have the opposite problem--you may need to place the article in a more general project, though I find that's rare.

Top priorities regarding banners

One can really go hogwild with this, and sometimes I do run into articles with 15 different banners. That is rarely helpful, and it probably wasted a good deal of time on their part. I try to limit myself to a few priorities when I add banners, and here they are: • Always add WP:BIO to biography articles, and always ensure that the living parameter and listas parameter are filled out. I have probably messed this one up before, but I try really hard not to miss this. BLP is of utmost importance to Wikipedia and we need to take biographical articles seriously. As a tip, if you need to fill out the listas and don't want to write it yourself, delete WP:BIO and add it again. It will fill out automatically in Rater! • Try to correct systemic bias. This sounds broad because I don't know who will be reading this and what they will be working on, but for me it is a very narrow thing. I work in science (biology specifically), so for me this almost always means adding WP:Women Scientists or WP:Women's Health to relevant articles. Women are not the only target of systemic bias, though. If you have the opportunity to add articles to WP:Kenya rather than WP:Africa, for example, please do so. We have terrible coverage of Africa, and as usual, the west treats it like it is one giant country rather than a collection of hundreds of different cultural groups. Many languages and countries are underrepresented on Wikipedia. I suggest making these kinds of projects a priority during assessment. • Other than that, I generally don't fuss too much about adding and removing project banners. If I add any, I often add one or two. I rarely add more than that unless it is a brand-new article, and even then it is rare. Removal is even more unusual. Sometimes it is really better to add a category at the bottom.

Irregularities, goof-ups, and head-scratchers

There are several problems which come up when using Rater, and they almost all have to do with the banners in general and not with the quality or importance parameters. I am writing this in May 2019, and by the time you read this some of these issues may have been fixed and new ones may have popped up. The best place to look is the documentation page at User:Evad37/rater. Someone may have already asked your question there. First, Rater does not catch banners which are not placed at the top of the page. If you get to an article from a project and you know it is listed there, or if it clearly has an assessment but you cannot see any WikiProjects, go inspect the talk page. As Rater users are not at all the only people who prefer banners at the top, I recommend you go ahead and move these first, then do your assessment as normal with Rater. It is a simple cut-and-paste procedure. Don't be daunted by the code! Second, some WikiProject banners break Rater or make it work a little funny. WP:Military History often breaks Rater completely and causes it to fail to load. When this happens, you should just assess it normally in the talk page code. All you have to do is type in class=start (or whatever rank you are giving) and importance=low (or whatever rating you are giving). Separate them from all other parameters with a pipe (|) character. Use another page's source as a template if you must--I have done so many times. Other banners just cause little hiccups. Maths Rating says it is not a normal banner because it is not, but you can and should still use it on maths articles. Some banners have all kinds of parameters that look really strange. Others may not have importance, or even stranger, they may not have quality! That is usually ok. Third, Rater sometimes just glitches out. If it fails to load and the article does not have WP:MILHIST in the talk page source, try refreshing and/or clearing your cache. If it still fails after both of those, throw Evad37 a line on the documentation page. Please include a link to the page and any error messages you receive. Fourth, Rater uses your browser cache as a sort of "I might need this data again soon" bin. That is usually a good thing, but sometimes we have to clean those sorts of bins out or we can't find anything at all! If you have been using Rater heavily (on dozens or even hundreds of articles), and you have some slowdown issues, I suggest clearing your cache. Fifth, I have heard that putting Rater at the top of your User.js page will help it to speed up. If it's your favorite userscript and you use it a lot, you can try this, but I have absolutely no proof that this works. It is unlikely to hurt anything, though.

Other work to perform while assessing

Article assessment is a Wikignome staple. There are other tasks which can be accomplished at the same time as article assessment, and which are also very important. Rather than going into great detail on each of them, I will present a simple list. (!add descriptions) • Short descriptions with Shortdesc • Categories with Hotcat • Cleanup templates • Wikilinks • Copy-editing While doing these may slow you down, remember that this is Wikipedia and there is no deadline. Those of you who feel that there is a deadline, keep in mind that these other tasks are actually more directly useful to readers than assessment. At present, I have a lot of assessment to do, so I mostly relegate myself to the first two in addition to the assessment itself. With my experience, I can finish most articles in 1-3 minutes, and the rest almost never take longer than 5 minutes.