Wikipedia:Snuggle/Work log/Archive Apr13

Sunday, April 7th

I'm checking in again. I submitted the WikiSym paper on April 2nd, so I picked up the work of baking in the newcomer desirability system into Snuggle. I ran a few tests against Stiki's web API to get vandal scores for newcomers currently loaded into the Snuggle back-end. It turns out that building a system to requests these scores in real time is non-trivial. I hope to start generating these scores in the live system by the end of the week and letting you sort by them in the interface in two.

Now, let me make this clear, because I think that the awesomeness can get lost in the details: I'm building a system that will let you sort by the probability that the newcomer is trying to work productively. In a couple weeks, you'll be able to log into Snuggle and say "show me the newcomers who signed up yesterday and are doing the most productive work".

Now on the subject of new work, my hours to work on Snuggle are just about to open up dramatically. My contract with Google ends on April 19th. I've scheduled the last three weeks in May for Snuggle work. This means that you should expect some rapid improvements and a set of new builds leading to the release of Snuggle 1.0 on (or near) May 1st. I'm excited to boost Snuggle to front of my work queue from the very end.

Back to hacking. --EpochFail^{(talk • work)} 19:49, 7 April 2013 (UTC)[reply]

I forgot to share some of my research. The following two plots show the speed at which certain agents revert revisions in Wikipedia. This analysis helps us understand just how much faster User:ClueBot NG is than the average human and how much WP:Huggle speeds humans up. The metric time to revert represents the time between when a revision is saved and when it is eventually reverted. My sample was drawn from January, 2011.

Saturday, April 27th

Hey folks, it's taken me a little longer than expected, but I have the interface with STiki worked out and I'm generating "desirability" scores for newcomers. The back-end system is syncing right now. I'll have a new version of the interface up for testing on Monday. As soon as that is online, I'll send a notice the sign-up list. --EpochFail^{(talk • work)} 21:09, 27 April 2013 (UTC)[reply]

Monday, April 29th

New build: http://stat1.wikimedia.org:8081

As promised, I have a new build of Snuggle up for testing that demonstrates the desirability metric. Right now, you can just sort by the desirability of users, but this is really cool already. It looks like TeaHouse hosts and their bot are already doing a good job of inviting desirable newcomers to participate. When I sort by most desirable first (figure 1), the top of the list is nearly full of Teahouse invites (icon looks like this: TH). If I scroll down a little bit (figure 2), we see some users appear to not have a Teahouse invite. This could be an opportunity that Snuggle users can take advantage of. Note that, in the Feb. 18th update, I announces the "user menu" that allows you to invite newcomers to the Teahouse right from Snuggle.

In order to figure out which users were desirable, I also figured out which users were least likely to be editing in good faith. By switching the sort order, we can see the least desirable newcomers who joined in the last few days (figure 3). Note that most of these users' revisions have been reverted, many of them have warnings and a few have been blocked. A potential use case for Snuggle could be to identify those undesirable newcomer who have slipped through the cracks and are continuing to vandalize.

Figure 1. Users listed most desirable first (top).

Figure 2. Users listed most desirable first (one page down).

Figure 3. Users listed least desirable first (top).

Over the next few days, I plan to clean up some UI bugs. I plan to have a useful system for a set of alpha testers on May 1st. Please ping me if you are interested in being one of those testers. --EpochFail^{(talk • work)} 16:52, 29 April 2013 (UTC)[reply]