Jump to content

Wikipedia:Undetected vandalism

Add links
Page semi-protected
From Wikipedia, the free encyclopedia

Undetected vandalism is vandalism that has remained on a page for a significant amount of time, often months or years. It is an enormous problem – perhaps the single most pressing problem – facing Wikipedia.

How widespread is it?

Donald Rumsfeld expounds upon the nature of Wikipedia vandalism.

There are known knowns; there are things we know we know. We also know there are known unknowns; that is to say we know there are some things we do not know. But there are also unknown unknowns—the ones we don't know we don't know. And if one looks throughout the history of our country and other free countries, it is the latter category that tends to be the difficult ones. - Donald Rumsfeld

Because undetected vandalism is, by definition, undetected, it is impossible to know how much there is. One fairly recent essay brings up the issue of low-traffic pages regularly attracting vandalism that lasts a couple days; if only that were the extent of the problem! But Wikipedia's records page paints a much more concerning picture. As of this writing, the longest time that pages have been vandalized without anyone noticing is:

  • Longest undetected vandalism to a page: 15 years, 29 days (4 January 2008 to 2 February 2023)
  • Longest undetected vandalism to a user talk page: 15 years, 82 days (8 October 2006 to 29 December 2021)
  • Longest undetected vandalism to a redirect page: 16 years, 6 days (12 February 2007 to 18 February 2023)
  • Longest undetected vandalism to a talk page: 19 years, 349 days! (20 April 2003 to 4 April 2023)

This should alarm you. More alarming is that this sort of undetected vandalism on Wikipedia is not:

  • Rare. The Wikipedia records page is frequently updated with even longer-lasting undetected vandalism – much of which is older than the average Wikipedia vandal.
  • Subtle. The vandalism detected is largely of the usual profane, scatological, insulting, memey, or even defamatory sort.
  • Contained to obscure or poor-quality pages, crufty lists, or other scarcely seen corners of the wiki. Vandalism has gone undetected for months or years on important and high-profile pages such as (but very much not limited to):

Good/Featured Articles

Vital articles

Other high-profile articles

Why is it bad?

The longer a piece of vandalism remains on Wikipedia, the more people have a chance to see it. A reader encountering vandalism that is profane, scatological, or otherwise prepubescent may be the single most embarrassing scenario for the encyclopedia. When the vandalism is insulting to or defamatory against a living person, it creates potential legal issues should they see it before we do and be unamused by the vandal's speculation about their eating habits or their mom.

Wikipedia is also heavily scraped by external sources, used to train AI, and reproduced in Google search engine summaries that themselves are heavily scraped, and cited by the media (whether or not they admit to it). This can give a piece of vandalism new life, as it leaps from blog posts to news articles to books and potentially even farther. The more a piece of false information becomes entrenched in external sources, the harder it is to identify its origin as vandalism – even if it seems "obvious," as what one Wikipedia editor identifies as blatant vandalism, another blogger might believe to be a quirky fun fact that readers will love.

Wikipedia's list of hoaxes is a compendium of such worst-case scenarios – as a hoax is just undetected vandalism at large scale. In one widely cited example, a vandal inserted hoax entries into an episode listing of the television series Street Sharks, inventing imaginary characters and episodes. These false entries made it into several articles that cited Wikipedia as a source, and were even referenced by the voice actors themselves!

How does it happen?

A lot of anti-vandalism work is done by recent changes patrollers. Some of this is assisted by monitoring plug-ins or automated bots. Some plug-ins use lists of "bad words" to filter edits by vandalism likelihood, but these are prone to false positives and false negatives. The best-known bot, ClueBot NG, does not, but instead applies machine learning algorithms to a dataset of known good/bad edits. Even so, according to its current settings, ClueBot only catches about 40% of all vandalism: less than half of it (that we know about). And since these tools are used to patrol new edits, they can do nothing about vandalism that already exists.

Vandalism edits can also be obscured by well-meaning editors:

  • Expanding a page: the whole reason we're here, but also something that can make edits seem more trustworthy due to their age.
  • WikiGnomes copy-editing the article or bots making routine, automated edits, making text seem more legitimate and less obviously amiss.
  • Anti-vandals reverting some, but not all existing vandalism on a page. This may happen, for instance, with rollback if two people have vandalized a page in succession, but only one is rolled back. It may also happen if a vandal has made several edits at once, some of which were fixed manually and some of which were not.
  • Administrators or bots merging, splitting, archiving, or translating articles, carrying the vandalism with it and making its original source difficult or, in some cases, impossible to find.

It is tempting to blame editor attrition for vandalism going undetected, as fewer editors presumably means fewer editors looking for vandalism. While this is plausible, there is really no way to prove it. The sheer amount of time that vandalism regularly goes undetected, however, suggests that this is not an overwhelming factor.

How can I find it?

A previously undetected gorilla

There are millions of articles on Wikipedia, all of which may potentially contain vandalism. Finding some is easier than you think. Read on for some tips that, if not guaranteed to produce results, will probably reveal more stuff than you ever wanted to exist.

Read your watched articles

As seen above, a surprising amount of vandalism goes undetected on high-trafficked, heavily watched pages – even Good or Featured Article caliber. Most Wikipedia editors don't rewrite articles wholesale, and certainly not in one go; instead, they tend to make incremental changes or additions to individual sections. Much like the invisible gorilla in the classic psychological experiment, even the most blatant vandalism can go unnoticed if a reader's attention is elsewhere.

If you're watching or actively expanding a page, take some time to read through others' edits to make sure nothing has gone awry. Don't assume a "typo fix" is OK – check it. Don't assume that tweak to the references is OK. Check it!

Read common vandalism targets

Some vandalism is done to random pages, but a great deal is targeted. Wikipedia has a list of most vandalized pages, but it is somewhat outdated, and most of those pages are heavily patrolled already. Pages that often harbor undetected vandalism include:

  • Any middle school, high school, college, school district, etc. – these receive a near-constant stream of vandalism for obvious reasons, and it can be hard for editors to keep up.
  • Any small-to-medium-sized town (larger ones are likely more patrolled)
  • Any children's author, YA author, or writer heavily anthologized in middle/high school, and their books
  • Celebrities, particularly current sportspeople

Search for common terms

To find the worst vandalism on Wikipedia, pretend you are a 14-year-old boy. What would you find hilarious to insert into Wikipedia? Profanity, gag names, Internet memes (especially older ones), insulting comments about your bros/your teacher/your crush, misspellings and non-English versions of any and all of them: the sky's the limit, the toilet's the nadir. If you'd prefer that someone else get in the internet troll mindset instead, several users have put together lists of bad words and/or regexes (example).

Once you pick a search term, you can either use Wikipedia's built-in search or search engines like Google – using a template like "site:wikipedia.org (potential vandalism term) -(term to exclude) -(term to exclude)" -- to find potentially vandalized articles. You may find it helpful to use both. A downside to this method is that it cannot catch subtle vandalism, only the blatant kind (and any subtle vandalism included along with it). But the blatant stuff is really bad.

Note: This work cannot be automated. This may be tempting, since some more common words/phrases will return thousands or even tens of thousands of results, but most of them will be false positives. There are few words used in vandalism that do not have other, legitimate meanings. The ass has been domesticated since Biblical times; at least one man held the title of the Farter; some things do indeed come in 69s or 420s. You will probably collect a long list of words/phrases to exclude for every term (e.g., "poop deck" for "poop") in order to narrow the list down to a (relatively) manageable couple hundred or thousand, without throwing out anything that actually is vandalism. But these still need to be checked one by one. After all, the whole reason the vandalism went undetected is because the automated tools didn't catch it.

Triage the edit history

Before reverting any edit, always check to make sure it is actually vandalism. Unfortunately, for undetected vandalism, this usually means scouring the edit history, which may have hundreds, thousands, or even tens of thousands of edits. Checking every single diff would be masochistic even by anti-vandalism standards, so triage extensively:

  • Use tools like WikiBlame to search for the offending terms. Caveat: For articles with large amounts of edits going back many years, this tool may be slow or time out.
  • Check the last revision on an edit history page first. If the vandalism still exists as of that version, you can move on to the next page, saving time. For more heavily edited articles, it can help to set revisions per page to 250, 500, or even 1,000 or more (by changing the parameter in the URL).
  • Look for edits that are roughly the same size as the text added/removed. For instance, if someone has simply inserted the word "crap" into an article, look for an edit that is +5 in size (counting the whitespace). Note that this may not always track it down, as sometimes vandals change other things, insert extra space, etc. in the same edit.
  • Look for editors who have only edited a page once or in one burst of small edits, or who use canned edit summaries.

A note on IPs

Not all IPs are vandals, but the vast majority of undetected vandalism is done by IPs or throwaway accounts. This is not an insult toward IP editors as a group, who do a lot of fine work (including anti-vandalism work). But the fact remains that when you see undetected vandalism, an IP probably did it. Thus, when triaging, make note of IP edits, or edits by throwaway-seeming accounts (no user page, few other edits, trollish-looking username, actually banned) and check those first. It will save you so much time.

Watch for red flags

Here are some small tip-offs that something questionable may be undetected vandalism:

  • Extraneous whitespace or line breaks; vandals will often hit Enter after their additions. Sometimes this extra space is only visible in the edit pane. (Example)
  • Broken headers, templates, refs, etc; vandals are often unfamiliar with Wikipedia formatting and will put their vandalism in places that will break it. (Example)
  • Words, especially in references or links, that are capitalized differently from their surroundings. (Example 1, Example 2)
  • Wikilinks that go to drastically different, less questionable places than the text. Vandals will often change the text but not the link – less rewarding, maybe? (Example)
  • Dubious-seeming information in a list or table that is out of alphabetical, numerical, or chronological order, as vandals often do not bother to rearrange the entries. (Example 1, Example 2)
  • Information in infoboxes that contradicts article text, or vice versa. Vandals will often change one, but not the other; alternately, well-meaning editors might fix one but miss the other. This is a particularly good indicator for census or school enrollment data, which can be tedious to fact-check. (Example 1, Example 2)
  • Questionable English text inserted into an article (non-enwiki), paragraph, title, or list that is otherwise in another language. (Example)

Look for the second instance

Editors and administrators do not always check vandals' other edits – even when that person has been banned! If you revert vandalism, undetected or not, check that person's edit history to see whether there's more. Note:

  • The "Reverted" tag is useful for triaging here, but it has false negatives, especially when the reversion didn't happen right away.
  • The older the vandalism, the more likely it is that their other unconstructive edits will have already been reverted or wiped away by rewrites.
  • A significant amount of vandalism comes from shared school IPs. The edit history most likely contains dozens or hundreds of different people, and just as not all IPs are vandals, not all people who edit at school do it to vandalize. Sometimes you'll see a spate of edits at the same time; these are likely to be from the same person and good candidates to check. (But not always – sometimes a couple kids edit at the same time, undoing and redoing or building upon each other's 'work.')

Don't forget other projects

Searching the wikipedia.org domain on Google will produce results across all projects except Wikidata. Here you will likely have to be even more judicious with excluding terms, as some questionable English words are perfectly normal in other languages, and some pages (especially images) appear across many different projects. Also, Google may rate-limit your results after a certain amount or periodically prompt you for a CAPTCHA, because searching Wikipedia for endless variations of the word "poo" is kind of a weird thing for humans to do. But this method does catch a lot that Wikipedia's search misses.

When you really can't find it

Sometimes you'll see an edit that is almost certainly vandalism but doesn't seem to originate anywhere in the article's history, or originated in a legitimate-seeming edit. A couple things might have happened:

  • It was an honest mistake or typo; when in doubt assume good faith.
  • The article was machine translated from another wiki, introducing errors. This kind of thing is unsightly and should be fixed, but it is not malicious vandalism. Note that if a machine translation has one obvious error, it probably has a lot more, and the entire thing may need to be rewritten.
  • If a page has been split or translated, or a talk page archived, the vandalism may have happened in the un-split, un-translated, or un-archived page. The only way to find it is to dig through those pages' history, too. It's helpful to put the diff of the vandalism in the edit summary once you find it, to prevent others from having to do the same digging. This can get incredibly tedious; sometimes it's best to just fix the vandalism without identifying the source, especially when it's obvious and/or easily fact-checked. (Example 1, Example 2).
  • If an image has suspicious-looking text, the likely culprit is vandalism to its Wikimedia Commons page. Content from Wikidata is incorporated into Wikipedia articles more and more; people vandalize this, too.
  • If revision deletion has been applied to huge swaths of a page's edits (and you don't have permission to view it), you may not find the source. Again, if you can fix the vandalism easily, be bold and do so.

Organize!

There are WikiProjects such as the Counter-Vandalism Unit that, while not as active as they used to be, still exist; however, they primarily focus on recent edits, not undetected vandalism. Significant support, at least in theory, for a task force or WikiProject targeting undetected vandalism, or at least better tooling. The best remedy against undetected vandalism is more people looking for it. If you're reading this, you can help!

Retrieved from "https://en.wikipedia.org/w/index.php?title=Wikipedia:Undetected_vandalism&oldid=1256770600"