Wikipedia talk:Growth Team features/VPR proposal draft

This is a draft of a post for Village Pump (proposals). It is under construction, and open for edits or feedback at the bottom of the page. Thank you! -- MMiller (WMF) (talk) 04:31, 26 August 2021 (UTC)[reply]

Proposal to expand trial of Growth team features

Hi everyone – I'm Marshall Miller, the product manager for the Growth team at WMF. Over the past three years, the Growth team has been developing a set of features meant to improve the experience for new editors. The goal of our work is to help newcomers make successful first edits, instead of them leaving from frustration or confusion.

Following a VPR discussion in April 2021, the features have been applied to 2% of new accounts as a trial. The community members following along have agreed that the initial trial has had positive outcomes, and so we now propose increasing the share of new accounts receiving the features from 2% to 25% (with some caveats).

The Growth features aim to guide newcomers as explained on the project page. The most important elements are:

Newcomer homepage: a special page with useful actions that newcomers are encouraged to visit immediately after account creation.
Suggested edits: a feed of articles that have maintenance templates such as {{Advert}} and {{Underlinked}}. Newcomers can choose topics of interest to filter the feed, like "Sports", "History", or "Physics".
Mentorship: newcomers are assigned a mentor from a list of experienced volunteers. A pop-up form helps them submit a question to their mentor's talk page using a simple interface. Whereas Adopt-a-user is meant to be an enduring connection for a newcomer, "mentorship" in the Growth features is meant for quick questions.
Help panel: when the newcomer is editing an article, the help panel is like a collapsible mini-homepage, providing relevant help links and the ability to ask mentor questions.

The Growth features are now on 48 Wikipedias, including large ones like French, Spanish, Russian, and Portuguese. Evidence shows that the features improve newcomer engagement without burdening communities.

In April 2021 we discussed testing the features on the English Wikipedia at this page. We started giving the features to 2% of new accounts in June (about 2,000 new accounts per month), and posted data and results after a month. There were two main outcomes:

The "suggested edits" feature seems to be leading to many good edits with low revert rates. During the test period in June, the revert rate was 9% (with the baseline revert rate for newcomer edits that month being 27%). To date, 2,325 article edits have been made. Many users do dozens of these edits. You can see these diffs by filtering to the "newcomer task" edit tag in Recent Changes.
The "mentorship" feature yields some good questions and some low-quality or nonsense questions. To date, there have been 329 questions asked to the 22 mentors who have signed up. We're still discussing how best to scale up the mentor capacity and how to encourage high-quality questions.

The next step is to give the Growth features to 25% of new accounts for one month (about 25,000 new accounts). The exception would be the mentorship features, for which we have open questions mostly discussed here. We want to increase the share of newcomers getting mentorship to only 5%, so as not to overwhelm the mentors. We would run this phase of the test for one month, and then bring the results back here to decide whether to further increase the share of newcomers with the features. In looking at the data, we will think about these questions:

Revert rate of suggested edits. Are high-quality edits being generated without an undue burden on patrollers?
Volume of mentor questions. This would allow us to extrapolate how many mentors would be needed to handle 100% of new accounts, or how the mentorship features might need to be improved to handle increased volume.
Quality of mentor questions. Are there improvements we could make to the features to help newcomers ask useful questions?

We're interested to hear any questions, ideas, or concerns -- and we're hoping there's general support for moving to this next phase of testing the Growth features. We also are looking for about 20 more people to sign up as mentors to handle the increased volume of questions that will come in with the next phase. Mentors should expect to get 3-4 questions per week on their talk page. You can learn more about that and sign up on this page. Thank you! -- MMiller (WMF) (talk) 23:52, 31 August 2021 (UTC)[reply]

Discussion

A call to action at the end could cover the last 2 points.

As for the second point, I think we could stick with the three questions posed above.

And the first one, that's something we have yet to figure out ourselves... Sungodtemple (talk) 13:15, 26 August 2021 (UTC)[reply]

Good work. Just a few suggestions. Most will know what Growth team is, but perhaps link the first occurrence in your note to [[1]], for context? Also, as far as getting more mentors, perhaps a message at the Teahouse and Help desk, already filled with helpful volunteers. I wonder if it would be possible to have a short section at the Feedback request service sign up page linking to general info about other volunteering options, to include mentoring. Also, consider adding mentoring to the "Other noticeboards and assistance" section of Template:Wikipedia community. And perhaps clarify at the mentorship bullet how mentoring is different than Wikipedia:Adopt-a-user? TimTempleton ^(talk) ^(cont) 00:27, 1 September 2021 (UTC)[reply]
Thanks for checking things out @Timtempleton! I made the change recommended for clarifying the distinction with Adopt-a-user. I think after this gets some discussion at VPR, and if community members seem to be in favor, then I can go to the places you recommended to recruit more mentors. How does that sound? MMiller (WMF) (talk) 18:50, 1 September 2021 (UTC)[reply]
Sounds good! TimTempleton ^(talk) ^(cont) 19:46, 1 September 2021 (UTC)[reply]