Wikipedia:Geo-targeted Editors Participation/report

Geo-targeted Editors Participation is an experimental project within the scope of Editor Growth and Contribution Program intended to increase participation of editors on English Wikipedia from underrepresented geographies where the most likely contribution language for editors is English.

From January to March 2013 WMF staff planned and executed an experiment aimed at attracting new contributors from the Philippines. Geo-targeted banners invited readers in the Philippines to a portal that encouraged them to edit Wikipedia article stubs on Philippines-related topics. This report summarizes our methodology and analyzes the outcomes and learnings from this experiment.

This experiment ultimately did not result in increased contribution to articles by new editors, so we do not recommend that you replicate it exactly. However, we do believe there are learnings from this experiment that could be useful for future experiments to build on.

Key learnings

1. Inviting contributions to a low-risk space (the user's own page) and surfacing a simple task with good step-by-step instructions did encourage people to edit. Out of each 5 visitors to the portal page, approximately 1 user added the userbox to their user page. If our goal were to get users to create their own page and add a user-box, this would have been success.
2. Inviting contribution by surfacing geo-targeted article content alone did not appear to be sufficient to motivate or help users make their first article-space edits, however. All new editors who joined, together, made only 6 edits in total to the article space during this experiment, and they made 0 edits to the articles we suggested. This was true even in phase 3 when we removed the user-box creation task from the workflow, and gave users the article-editing task first.
3. The difference in contribution to user-space vs article-space may be related to the fact that we provided simple, clear, customized instructions for completing the simple task of adding a user box but no instructions for how exactly to make edits to the suggested articles suggested. Previous research has shown that the standard Wikipedia interface also does not provide enough instruction about how to edit articles to new editors.
4. Providing suggestions via links to places users might go for help did not appear to sufficiently support or motivate these new editors to get involved. 50% of the survey respondents stated that they didn't look for help pages. Those who did view help pages nevertheless did not edit the suggested articles. It is possible that the calls to action to get editing help weren't strong enough, or it is possible that these help systems weren't targeted enough to meet the needs of this pilot group.
5. Sufficiently large sample sizes are important for measuring the outcomes of a pilot. Geo-targeted banners may not be an effective means to invite contribution if the overall number of users from a specific country on a specific wiki is not sufficiently large.
6. Wikiprojects can be great sources of content suggestion, although content suggestion alone did not appear to sufficiently motivate contributions. However, the Philippines WikiProject that we selected for this pilot had a fruitful list of stubs, pulling from that list was easy to do and gave us a lot of topic choices to serve new editors. Thank you WikiProject Philippines!

Background

This project was based on a few assumptions concerning the targeted region:

1. People are educated mostly in English, but their native tongue is not English
2. There is a high readership of English Wikipedia compared to other language versions.
3. Editors from the region are underrepresented on English Wikipedia

In this pilot project, we picked the Philippines, where an active editing community already exists, and where a survey of readers had suggested that being pointed at things to do would drive more people to edit.

The Team

• Haitham Shammaa did the bulk of the analysis and coordination work
• Heather Walls designed the landing and contribution pages
• Siko Bouterse and Asaf Bartov advised the project

Goals

The stated goal for this pilot project was:

• Develop viable strategies for getting more active editors from a Global South Anglophone culture, beginning with an initial pilot aimed at editors in the Philippines

Hypothesis

• Providing people with geo-culturally specific calls to action and activities to contribute to, along with explicit reference to existing help resources like the Teahouse, will encourage them to make more edits

Research

Readers

• 93,200 page views/hour from the Philippines (0.819% of total EnWp visitors)

Editors

• Active (5+) editors on ENWP from the Philippines: ~400
• Very active (100+/month) on ENWP from the Philippines: ~48

Survey

We ran a survey targeting readers of English Wikipedia in the Philippines during the period between 9th and 16th January 2012. 1382 responses were received. The full results can be found here.

Result from File:Philippines readers survey.pdf

Key findings:

• 81% of readers know that they can edit Wikipedia.
• 86% rate their English proficiency to be more than "good."
• Only 36% of readers had actually attempted to edit Wikipedia.
• Main reasons for not editing given include:
  • Didn't know what to edit (38%)
  • Editing interface difficulties (26%)
  • Not interested in editing (22%)
• Main requests for support from editors or potential editors include (multiple selection was enabled):
  • Having specific easy tasks to do on Wikipedia (63%)
  • Having friendly coaching from expert users (48%)
  • Having easier help pages (42%)

Framework

We divided the user's workflow into these three parts:

1. Invitation
2. Landing pages
3. Online Support

Invitation

Inviting new editors from the targeted regions was done using Geonotice, targeted for Philippines IP addresses. The message and design of the invitation was customized for the Philippines to make the invitation appealing for the target audience.

Two different banners were run:

1. Targeting readers: banner invited them to create an account or to log-in using their existing account to learn about contributing. (those who did so would then see the banner below.)
2. Targeting logged-in users: banner guided interested users to a customized landing page.

Landing page

Similar to the invitations, the landing page was also designed and customized to fit the target country.

The purpose of the landing page was to enable newcomers to participate in Wikipedia as smoothly and quickly as possible by surfacing articles about topics relevant to their geography.

Progress

Phase 1

A banner was displayed to logged-in users from the Philippines on English Wikipedia between March 1st, to March 7th. The text on the banner sent users to the portal page.

Phase 2

The second stage started on March 8th, where Another banner was displayed to only anonymous users from the Philippines on English Wikipedia. The text on the banner sent users to the account creation page. During this stage the banner that was visible to logged-in users was still on, so that a user might create an account based on the first banner's call to action, and then be directed by the second banner to visit the portal page to get started editing.

Third phase

The third stage started on March 19th, we changed the text on the banner for the logged-in users to send them directly to the topics page instead of the portal page. This change was driven by the fact that the number of users who access the topics page becomes smaller if they had to visit the portal page first. We wanted to see if users were more likely to edit articles if we did not distract them with editing their own user-pages first.

Data analysis

Accounts and edits

• Newly registered user accounts: 1526
• 190 users added the userbox to their userpages
• 53 users who registered during the campaign successfully made edits, mostly adding the userbox, and only 6 edits were in the article namespace.

Pageviews

Number of pageviews for both the portal and topics pages during the campaign.

The page view chart shows how the number of pageviews for both the portal and the topics pages changed during the three stages of the campaign. The graph indicates a maximum at around 300 views per day when banners for both logged-in and anonymous users were on. On the other hand, this number goes down to about 150 views per day during the first stage of the campaign when the invitation was visible to logged-in users only.

During the third stage and when the invitation banner was set to land directly on the topics page, although the number of pageviews for the topics page became higher, it was still less than the number of pageviews of the portal during the second phase. This might imply that users had visited the portal page more than one time.

User experience funnel

These two figures on the right show the user experience funnel during the campaign when the invitation banner linked to the portal page, and when it linked directly to the topics page. We can notice that the number of users goes down to about 1/10 from the number of impressions on the anonymous users banner to the number of users who actually registered an account. Furthermore, the number of users who clicked the banner to go to the portal page counts about 1/2 of the number of user who registered an account. This number goes to a lower proportion considering the number of users who visited the topics page, which are approximately 1/5 of the number of users who visited the portal page. On the other hand, the user experience funnel, when the invitation banner linked landed directly on the topic page, looks similar to the first funnel except that the portal page was eliminated, which drove more pageviews directly to the topics page.

User experience funnel when the banner landed on the portal page.

User experience funnel when the banner landed on the topics page.

Daily edits

Number of edits made to the all Philippines stub articles by day.

We measured the number of daily edits made to the list of offered articles on the topics page as a parameter to show the level of activity of all users. The graph on the right shows the number of edits made to all Philippines-related stub articles by day during the period of this campaign. Except for a few spikes (which are either due to weekend activity, or to article maintenance by regular editors), we did not notice an increase in the number of edits during the campaign which could be linked to the activity of users driven by the banners.

Although we estimated that about 200 users made edits either to their userpage, or to the article namespace as a result of this campaign, almost none of them made edits to the offered articles.

User survey

After the end of the three weeks campaign, we prepared a user survey targeting users who were registered an account as a result of this campaign. We have contacted 158 users who registered their email with Wikipedia, and asked them to fill the survey, however the response rate was also low here as there were only 14 responses.

The detailed survey report shows that although a large proportion of users noticed the banner, few users went to the portal, few of them accessed the topics page, and even fewer of them attempted any edits (which is another explanation of the user experience funnel). The main reason stated for not editing articles was because the users felt they had no information to add to the existing articles.

Possible future exploration

1. WikiProjects can be great sources of social support for editors. For this pilot, we provided links to Teahouse and help documentation and expected new editors to actively reach for help. This did not result in successful edits or interactions that we saw. If we were to do this project in the future, we might next try setting up a clearer framework to engage new editors in discussion with existing editors from the WikiProject. This would require having enough active editors in the WikiProject interested in onboarding new editors to work on the topics, and using calls to actions to set clear expectations for engagement between new and existing editors.
2. Suggesting articles for expansion may be too daunting a task for a new contributor's first edits in the article space. Perhaps trying a pilot with a simpler task, like copy-editing, would be a good test for the future (in a similar manner to this project).
3. While this experiment was clearly not successful in recruiting new editors, it may yet be interesting to explore in different contexts, e.g. smaller/younger Wikipedias. While WMF does not intend to attempt this in the foreseeable future, no special tools are needed to attempt this, and interested community members could design and run their own experiments along these lines. WMF would be interested to hear of the results, positive or negative, of any such attempts!