Wikipedia:Community health initiative on English Wikipedia/Research on dispute resolution and harassment

Background research on dispute resolution and harassment

Analysis of the under-researched area of dispute resolution and policy enforcement is badly needed to inform discussions about new tool and potential policy changes. The first stage in this effort are two studies about English Wikipedia's Administrators’ Noticeboard/Incidents, and a survey about Administrator Confidence in managing cases.

English Wikipedia's Administrators’ Noticeboard/Incidents research

The first study is a quantitative data analysis of posts to AN/I and is currently underway. The second is a qualitative survey to be sent to English Wikipedians about their perception of and use of AN/I (November 2017).

Why AN/I?

The Administrator’s Noticeboard/Incidents page is one of the longest running and most-frequented forums for user disputes and problems on the Wikimedia projects. It has hundreds of archives and tens of thousands of threads, making it a rich source of data on how we manage user problems on the projects.

Why look at English Wikipedia only?

The Community Health initiative will be working with members of our many projects, not just those on English Wikipedia. The AN/I research is a good starting point, though, due to the age and high usage of the board. Many projects have not yet developed similar processes, and lessons learned from large projects can be very valuable to communities looking to expand and refine how they deal with problems.

What does the Foundation plan to do with this research?

This research will help inform the Wikimedia Foundation’s efforts to support community development, whether through software and technical improvements, or through discussions and proposals to improve processes and policies. The results of this data will be shared with contributors, and will help communities make data-driven decisions about how they develop their community norms and workflows.

Preliminary (experimental) queries run on the AN/I archives from April 1 to May 30 2017

Tracked in Phabricator
Task T171091

Experimental queries were performed in order to see if we can pull data to determine the number of case, number of cases resolved, types of case, number unique users participating, and other patterns such as the types of policy/guidance pages linked to.

According to the ANI archives, in April and May 2017:

533 total ANI cases, 315 of which were resolved (as determined by resolved templates)
1,008 unique usernames left signatures, 183 of which were admins.
40 cases included the keyword '3RR', 26 'COI', 18 'harassing', 14 'hounding' and 22 'boomerang'
Of the 183 admins, 111 participated in 2 or more cases, 24 participated in 10 or more cases, with the most active admin participating in 73 different cases.
68 users filed 2+ cases in this time period, with 5 opening 10 or more cases, with the most active opener opening 55 different cases.
Sunday was the most popular day of the week to post a new case to ANI.
The average time for a case (as determined by signature timestamps) was 41.4 hours, with the fastest case being opened for one minute (likely due to lack of discussion) and the longest running for 62 days.
The biggest case included 39 different users, 13 sub-sections, and 191 total signatures.
Top three policy/guidance linked pages were (in order of popularity): What Wikipedia is Not (linked to 59 times) Edit Warring (linked to 48 times) Here to Build an Encyclopedia (linked to 46 times)

Next steps:

Review the documentation about the type of data that can be pulled with queries.
Analyse the preliminary finding.
Share the preliminary finding with English Wikipedia community.
Discuss the types of data to pull in further queries.

English Wikipedia Administrator Confidence Survey

Tracked in Phabricator
Task T173536

The Wikimedia Foundation Anti-Harassment Tools Team is conducting a survey to create measurements on how to better support admins who are addressing disputes on Wikipedia. This includes measuring how well current tools, training, and information exists for admins in recognizing and mitigating things like sockpuppetry, vandalism, and harassment. This is specifically designed to learn more about and improve the admin experience. The survey ended on September 24th 2017.

Results

See Administrator confidence survey for a more detailed description of the survey, alternative methods of release of the results, or to discuss the results.

Administrators'Noticeboard/Incidents Experiences Survey

For more information, please see Research about Administrators' Noticeboard Incidents results