Wikipedia:Feedback request service/2020 survey

Welcome to the 2020 Feedback Request Service subscriber survey! As you may be aware, a new bot has recently taken over the Feedback Request Service. To ensure that the FRS is best serving the English Wikipedia community, this survey has been created to request comments from FRS subscribers and other relevant stakeholders.

Each question below is intended to be a hub for specific discussion and comment around one topic. Please ensure that comments made under each question are relevant to the question at hand. Other discussion should go on the talk page.

Participation in this survey is entirely voluntary, but your thoughts and ideas are greatly appreciated!

Random selection

The documentation for the FRS states that users are "randomly selected". However, the previous implementation of the FRS bot appears to have simply selected all users who were eligible to receive the notification - that is to say, all of those whose limits had not been exceeded.

Should the Feedback Request Service

1. continue to select a random sample of users, of a random size, between 5 and 15 users;
2. select a pseudorandom sample of users of that size based on the last time a user was contacted (e.g. random, but then don't send to anyone who has received a message in the last n hours);
3. select a (pseudo)random sample of users, of a different size (please specify);
4. select all users who are eligible to receive the notification?

Alternative way of phrasing the question: leaving out (b) from above

Should the Feedback Request Service

(a) notify a random sample of N users, where N is

(1) randomly chosen to be between 5 and 15; [current implementation]

(2) randomly chosen to be between MIN and MAX inclusive; (please specify MIN and MAX)

(3) chosen by the editor who launches the WP:RfC or WP:GA nomination.

(b) notify all users that would not exceed their notification limit (e.g. if a user wants 3 notifications per month, at least 10 days have elapsed since their last notification)

--David Tornheim (talk) 05:03, 8 July 2020 (UTC)

Survey

• Option A (botop). A random sample of users ensures that different users are invited to different discussions, helping to slightly even out the self-selection bias inherent in people who've decided to sign up for the FRS. It also ensures that the FRS isn't seen as a "pile-on service", where users sign up to the service, and then overwhelm RfCs with whatever happens to be the dominant opinion only among FRS subscribers. Naypta ☺ | ✉ talk page | 10:45, 18 June 2020 (UTC)

Discussion

• (botop) Optionally, the functionality to set the minimum and maximum limits could be brought on-wiki; that is to say, a page could have the upper and lower limits set on it. This would have the advantage of allowing people to change the numbers easily, but could potentially attract vandalism, and may be an unnecessary step if a clear consensus is reached here. Naypta ☺ | ✉ talk page | 10:45, 18 June 2020 (UTC)

Messages to send

At present, messages are sent in order, and all are sent if they are able to be; that is to say, each RfC and GA nom that comes in is sent, so long as there are users remaining who have not yet reached their message limit for the month. This means that all messages are sent if a sufficient number of subscribers choose to subscribe; however, it also means that messages at the start of the month are inherently more likely to be sent out than those at the end of the month for small subscriber lists.

Should the Feedback Request Service

1. continue to send all messages as soon as possible;
2. (pseudo)randomly select whether to send any given message based on a probability of selection;
3. bias message sending towards the end of the month, to try and correct for the problem - that is to say, send more messages the closer to the end of the month it gets;
4. do something else entirely (please specify)?

Message limits

At present, message limits are set per month, with no specific limits set beyond that. This means that more users receive messages, and users who want frequent messages get them frequently; however, it also means that, in the case of a cold start, where the bot has been offline for many months, it is possible for a buildup to accumulate, which then sends up to the entire monthly allowance of messages to a subscribed user at once.

It has been suggested that a daily limit on the number of messages sent to an individual user should also be set, either automatically calculated or configured by the user.

Should the Feedback Request Service

1. automatically generate a daily limit calculated by the bot (please specify an appropriate calculation);
2. allow users to specify a daily limit as an optional parameter to the {{Frs user}} template on WP:FRS, and default to having no limit;
3. allow users to specify a daily limit as an optional parameter, and default to having an automatically calculated limit;
4. have no limit on the number of messages sent per day to each user, relying solely on the monthly limit?

Exhausted lists

At present, when an RfC or GA nomination is submitted in a topic area for which the FRS has exhausted its entire list of users - that is to say, there are no users remaining on the list whose sending limits have not been reached - the RfC or GA nomination is ignored. This means that feedback may be impaired for smaller lists, but it also prevents the problem where a queue of feedback requests builds up on a small list of users, causing the first of each month to result in a wave of notifications until the list exhausts itself again promptly.

Should the Feedback Request Service

1. continue to ignore RfCs or GA nominations where the list has been exhausted;
2. add such RfCs or GA nominations to a queue, and process them as a first priority if they are still open as soon as users are available again;
3. do something else entirely (please specify what)?

RfC information

At present, when invitations for an RfC are sent by the FRS, the invitations consist solely of a link. The FRS bot may be able to additionally send the contents of the RfC statement to users, to help them understand the context of the RfC without clicking on the link. However, this is not always reliable due to the nature of signatures being unpredictably constructed - Legobot too sometimes has trouble getting the relevant text and copying it onto its pages - and could cause huge amounts of text to be copied to people's talk pages in the event of someone not following the guidelines on keeping RfCs brief.

Should the Feedback Request Service

1. send the contents of RfCs to users, along with the links to the RfCs;
2. continue to send only links?

AfD nominations

Should the Feedback Request Service support AfD nominations, with users subscribing to categories listed at WP:DELSORT?

1. Yes, and I would subscribe to at least one category;
2. Yes, but I would not subscribe to any categories;
3. No.

RfA

Should the Feedback Request Service notify users of requests for adminship?

1. Yes, all FRS users should be notified of a new RfA;
2. Yes, there should be an opt-in heading for RfAs, but all users in the heading should be notified;
3. Yes, there should be an opt-in heading for RfAs, and a random portion of users should be selected from the heading in the same way as any other FRS heading;
4. No.

Survey

• Option D (no) (botop). The Feedback Request Service is built for specific feedback requests in areas that community users might not otherwise see, such as RfCs and GA nominations, which take place on specific pages, and aren't widely publicised often. RfAs are much more broadly publicised, so interested community members would likely see them anyway. Option B I feel is the best of the list of options here, but I'm not sure it fits best as a part of the FRS in the first place; a centrally organised mass message list may be a better idea. Naypta ☺ | ✉ talk page | 10:45, 18 June 2020 (UTC)