User talk:SandyGeorgia/ArbStats

Another space?

Hi Sandy, if you have time, can you have a look here? I've added counts for userspace edits for you, me and Tim (and Maralia). Would these be useful in the arb candidate listings? They show another dimension of the "communicates well" aspect.

If they look useful, then I have another question. Would it be more useful if the "own-space" pages were separated from the other-user pages in the most-edited-page section? Franamax (talk) 04:51, 22 November 2008 (UTC)[reply]

I missed this question until now: I'm not sure what you're asking? SandyGeorgia (Talk) 20:46, 24 November 2008 (UTC)[reply]

Are adding the user/usertalk counts for arb candidates a good idea? Do they help with analysis? (My listings, not the analysis you're undertaking here)
Looking at the most-edited-pages part, the own-space and other-user pages are interleaved. For yourself, your #1,6&19 most-edited upages are your own, the rest are others. I was thinking about separating the "own" and "other" counts - but since it's done that way in the "family" counts, it's probably not that important. Franamax (talk) 20:57, 24 November 2008 (UTC)[reply]

I'm not sure about that, because of different user talk conventions. For example, I keep topics together and always respond on my own talk page, and a lot of FAC "business" tends to come to my page. Other editors fracture conversations (Marskell and Tony1 for example), so their pattern would be different. For my purposes, I've found what I need to find with my data, although I would still like to see another measure of dispute resolution for the "tie-breakers". (I've got five definite supports per my rubric and knowledge of these editors, five definite opposes, and several undecideds in the middle: for those cases, I'm not sure another piece of data will help, as I've really got to go learn more about those editors, as I don't know them well.) Are you going to run the new candidates soon? SandyGeorgia (Talk) 21:42, 24 November 2008 (UTC)[reply]

I'll be back home in about 24 hours & will do it first thing. Depending on how many more candidates come in the last hour before closing... :) Franamax (talk) 23:19, 24 November 2008 (UTC)[reply]

ArbCand's are up-to-date on my end. If I don't see you making edits to this page in a while, I'll start adding them two-by-two. Can we agree that anyone with less than 2000 edits will be excluded (but noted as such below the table)? Franamax (talk) 21:14, 25 November 2008 (UTC)[reply]

If you have time to work on them, please go for it ! I'll be busy for quite a bit here ... yes, I left off those who have negligible edits. Thanks, Franamax! (Any further ideas on another ratio to measure dispute resolution activity?) SandyGeorgia (Talk) 21:30, 25 November 2008 (UTC)[reply]

I've been thinking over how to examine DR vs. drama. Nothing good yet, just some possible angles. I'll start a new section below with some thoughts, if any crystallize (or even if they don't). Franamax (talk) 22:07, 25 November 2008 (UTC)[reply]

Disussion

If you hold the view (as I do) that Tim Vickers represents the gold standard in excellence in balance between dispute resolution skills, policy understanding, civility, and mainspace editing, looking at the numbers this way shows that balance. Summarizing the numbers this way shows the editors top mainspace article, top WP (family) area of activity (for example, as expected, my contribs at FAC are disproportionate), and proportional time spent contributing quality articles versus engaging the Administrators' noticeboards.

Tim has healthy contributions to the Administrators' noticeboards, but he has contributed more edits to 9 articles (for more than 5,000 edits total) than he has to AN. He has 12 times as many edits to those articles as he does to AN. He has 2.5 times as many edits to his highest-edited article (Evolution) as he does to AN. This shows an editor with quality mainspace contributions, significant AN input, but a priority, commitment and balance towards mainspace while avoiding drama at the trainwreck that is AN.

We need ArbCom members who have demonstrated a commitment, first and foremost, to understanding how dispute resolution applies to building quality articles in mainspace, and understand article building skills. We don't need more career ArbCom members who don't know what content contributors deal with, who don't understand what goes in to building our best content, and who spend a disproportionate amount of their time rubbernecking the trainwrecks at AN/I instead of building articles. SandyGeorgia (Talk) 18:20, 20 November 2008 (UTC)[reply]

Your original idea, most-edited article:AN-edits ratio, seems to fail as a useful comparison - it's only identified outliers in the data. From what I've seen running these scans, I'd suggest using at least the top-5 article total as the numerator. Better yet would be top-20, since that is a very good cut-line for where the "long tail" runs out in mainspace. It's quite reasonable that a dedicated content editor would have 20 or so titles of interest, whereas the antivandals will have a different curve, with a lower front and a much longer tail. That will make the ratios more meaningful (Tim and Cas will just get bigger numbers, but the rest will get at least reasonable numbers. If you change the table and do a few at the top, I'll pitch in starting from the bottom. Franamax (talk) 07:45, 22 November 2008 (UTC)[reply]

I noticed that (which is why I stopped there, so I could reformulate), but had a busy day, and didn't get a chance to re-do it ... same for tomorrow. Looking for other ideas, too ! SandyGeorgia (Talk) 07:50, 22 November 2008 (UTC)[reply]

I'd suggest:

Top article --> FAs - replace with just Top 20 and Status, status in the form of "3F/2G/6B". (do A-class articles even exist??)
'Top WP Fam, Top Fam Edits, AN Edits - good numbers, leave as is, possibly more terse heading.
No. articles, Edits to those, Ratio 1 - this metric is not useful (right?) ergo - scrap it.
Ratio 2 footnote changes to "top articles" and uses top-20 count as numerator.

Other ratios could be of interest. Top non-AN family to AN-family? Utalk to AN? There are a lot of ways to measure the on-wiki efforts of an arb candidate. I agree with you that article focus is paramount, but participation in discussion and dispute resolution is very important for an arbitrator, and those metrics are much harder to obtain. Franamax (talk) 10:07, 22 November 2008 (UTC)[reply]

Took one day off and fell behind :-) I won't be able to resume this work until tomorrow. SandyGeorgia (Talk) 20:15, 22 November 2008 (UTC)[reply]

Sorry, not acceptable. We need you to work some overtime today. I am authorized to offer you double the usual amount you get paid to edit here. :) Franamax (talk) 20:42, 22 November 2008 (UTC)[reply]

Starting in on this. I can't do top 20, or I'll be adding numbers all day: I'm going to stick with top 5. ~~Once I get a structure in place, I'll temporarily split the table in two in case anyone else can work on the bottom while I work on the top.~~ SandyGeorgia (Talk) 17:58, 24 November 2008 (UTC)[reply]

~~Structure in place, I split the page temporarily in two in case someone wants to work on the bottom while I work on the top.~~ The first edit count (top article) is article edits only. The top 5 count is article plus talk, to measure editor interaction. We can figure another measure of dispute resolution to add to the chart once all of this data is in.

Done with the top, recombining to do the bottom. SandyGeorgia (Talk) 20:05, 24 November 2008 (UTC)[reply]

I've filled in the bits I could figure out for the missing entries. I left the FC bits out - I assume you know that what my tool reports has nothing to do with whether the editor actually achieved the rating. Franamax (talk) 09:22, 26 November 2008 (UTC)[reply]

Multiple accounts

Franamax, what about multiple accounts (see User:MBisanz/ACE2008, eg Privatemusings)? SandyGeorgia (Talk) 13:36, 25 November 2008 (UTC)[reply]

I was more worried about Secret, but I see now they've withdrawn. I was planning to hack in a merge between their (two, productive, non-overlapping, non-banned) accounts. I'm back now & will start updating candidates, I'll look at PM multiple accounts as a somewhat lower priority. Franamax (talk) 19:49, 25 November 2008 (UTC)[reply]

Franamax, Privatemusings is still footnoted as incomplete: is that correct? SandyGeorgia (Talk) 19:32, 27 November 2008 (UTC)[reply]

Yes, it's still incomplete. It looks like two of his formers listed by MBisanz have substantial edits. I'll list them separately on his page (they overlap and I don't feel like figuring out combined counting at the moment :). Franamax (talk) 20:17, 27 November 2008 (UTC)[reply]

Interesting

The results are a lot different than I would have expected. I'm even more impressed with Casliber and Risker—candidates who I already felt were qualified. When I last got figures for Charles Matthews (on November 1), he had 165,098 edits. It appears that you're counting both mainspace talk and mainpage edits under "mainspace edits." I don't mind at all; I'm very heavy in talkpage content disputes, but it should probably be noted. The fact the both main and talk edits are counted really emphasizes how low some of the figures are. Cool Hand Luke 18:23, 26 November 2008 (UTC)[reply]

I revised the figures for those exceeding Interiot's tool last night, using my own counts. I'll update them today some time. A few most-edited numbers changed also, which I will check as well. Franamax (talk) 20:35, 26 November 2008 (UTC)[reply]

Thanks for doing that, Franamax ... I just have not yet found the time to get back to this page, still a work in progress. Perhaps Friday. <eeeeek> SandyGeorgia (Talk) 22:17, 26 November 2008 (UTC)[reply]

Fascinating stuff. I'd love to see my own numbers, actually, but I'm not completely clear on the algorithm. I find myself surprised by the results for Charles Matthews, as I thought he had quite a few articles that had been rated. Is that a glitch or am I misunderstanding how this works? Thanks for compiling this. ++Lar: t/c 20:36, 28 November 2008 (UTC)[reply]

I'm also unclear on Charles Matthews numbers; I think Franamax had to jig them because of the 100,000 limit, so Franamax might explain. I'm flummoxed, but I doubt that anyone/hope that no one is going to be looking at evaluation a current arb based on these numbers. Their counts should be distorted by their RFAR contribs, as my counts at FAC are distorted. SandyGeorgia (Talk) 20:51, 28 November 2008 (UTC)[reply]

CM now has a semi-full count. The "career" listing still has counting overflows, only the last 30,000 articles CM has edited are considered. The "recent" list is fully accurate as far as edit counts go. However, read on:

First of all, there was a tiny little bug until last night, where I was only getting the first 10 categories on the talk page. I've just checked Charles' most-edited, of the 175 articles with talk pages: 0 had incorrect quality assessments, 5 had incorrect importance assessments (2 of which were missed conflicts), 12 total had >10 categories. I was really hoping I could slide that through without re-running everything, but there is probably a 3-5% error rate in article ratings (error to the low side). Red face, but all of CM's quality ratings are accurate.
Charles may well indeed have many more highly-rated articles. It's quite possible he does, and spent more time on producing them with less than 20 edits. Looking at just a tiny portion of his total edits, my feeling is that he's much happier quietly producing content than getting his content rated against any arbitrary standards. A quick view of article histories strikes me as having an awful lot of CM-"add ref"'s. This to me is at least as important as achieving a quality rating, and will be one of the targets of my next iteration of the tool.
General note: categories and wikiprojects are a big mess. My impression is that projects start off with great intentions, bots are enlisted, mass tagging is done, then almost everyone wanders off into the void. From my experience, advising someone to "ask at the relevant WikiProject" is often the same as saying "drop the subject, or wait a month and change it if you still care". Reviewing this, I've come across several examples where the projects and categories just don't seem right (not to mention editor's wonderful propensity to tag articles).
See also Cool Hand Luke's comments elsewhere about making a few quality edits, as opposed to many little nigglers. In the case of CM, having actually looked at a few of them, he's not a Huggler racking up the count - quality ratings or not, that's a major content contributor! Franamax (talk) 23:24, 28 November 2008 (UTC)[reply]

OK, so as not to have to undergo a major rework, can we just footnote CM somehow? I agree that article assessment outside of GA/FA is spotty to worthless, but FA assessments are always accurate, and GA often are. SandyGeorgia (Talk) 00:24, 29 November 2008 (UTC)[reply]

Editcountitis

Sandy, numbers only mean so much. It takes all types to make an encyclopedia run. I am not sure it is helpful for different factions within the community to start talking about "we" and "them". Some editors are great writers. Others like to format footnotes, design templates, help resolve disputes, or do whatever they can to improve the project. Yeah, AN/I is a trainwreck, but can you propose something better? People have problems and they are asking for help. Should we ignore them, or should we try to help? Jehochman ^Talk 19:04, 28 November 2008 (UTC)[reply]

I'll provide some context when I find time to finish my introduction. Yes, numbers only mean so much, but editors who have simply not contributed to article space are not what (at least) I'm looking for this year, as I believe it's important to balance some of the long-standing issues that have been raised relative to the composition of the current ArbCom. We need editors who understand the challenges of building a professional reference work and who are engaged with the community at the level where this work occurs. This may not be true every year, but it's an issue this year, as demonstrated by the problems that surfaced in ArbCom deliberations this year and the increasing disregard shown to top content contributors in dispute resolution processes. Re: ANI, "people are asking for help, should we ignore them or should we try to help": first, do no harm. There has been, of late, too much harm; we only have one chance per year to alter the tone and composition of ArbCom, and in the past, we've perhaps leaned too heavily towards career bureaucrats and too far from those engaged in the trenches, building top content, working with the community, undertanding the recurring issues. SandyGeorgia (Talk) 19:18, 28 November 2008 (UTC)[reply]

I agree with you that arbitrators need to be familiar with content creation, but even then, looking at edit counts can be deceptive. A hack like me may require 400 edits to get an article to FA. In contrast, User:Fritzpoll rewrote Robert F. Kennedy assassination to FA standards with many fewer edits. As for ANI, I agree with "do no harm". When people gang up and get ready to sanction an editor on flimsy evidence, I assume you'd support those willing to step in and say "hang on," such as User:Carcharoth (somebody who I will support). I think the numbers are very interesting because they tell voters where to look, but they still have to look. Voting on the numbers alone would be dangerous. Jehochman ^Talk 19:22, 28 November 2008 (UTC)[reply]

And, perhaps more than most Wiki editors, I'm very familiar with those editors who are capable of writing an FA with ten edits, and those who need 50 edits to destroy one paragraph; we have none of those examples in this sample of candidates, or I would highlight them with a footnote. Further, numbers are only one tool in the toolbox for evaluating candidates: I fully expect to support some candidates who fall outside of my concerns about top content contributions, and oppose some who fall within the range, based on other knowledge of those candidates. Those who fall at the extremes, though, need analysis and justification (for example, NYB was clearly an exception). If I know enough about a candidate to explain outlying trends, I'll do that, but when a candidate has neither contributed in the article writing trenches, nor shown me enough to know that s/he won't contribute to "more of the same" issues in the current ArbCom composition, I can't support. SandyGeorgia (Talk) 19:30, 28 November 2008 (UTC)[reply]

That's fair. Jehochman ^Talk 19:31, 28 November 2008 (UTC)[reply]

I haven't found the time yet to write my Support/Oppose rationale for each candidate (hard for me, since some are composed in my head but my prose stinks so bringing it to the page is harder), but looking at the numbers doesn't only apply to top content production: I'm also supporting a candidate who has relatively little non-mainspace dispute resolution editing because 1) that candidate has qualities needed to balance current ArbCom deficiencies, and 2) that candidate has shown me the ability to resolve disputes at the article talk level. And, there are candidates who focus almost exclusively on non-mainspace dispute resolution areas and all too often barge in to an area that they don't know or frequent with an incomplete and inorrect analysis of the issue, a lack of undestanding of the area they've surprisingly barged in to (likely contacted backchannel or on IRC, one of the big unresolved problems on Wiki), and draw completely faulty conclusions: that is the type of candidate that would further the problems in the current ArbCom, and marks the difference between an abstain or oppose from me. We don't need more backchannel cabals and sleuthing on ArbCom. SandyGeorgia (Talk) 19:45, 28 November 2008 (UTC)[reply]

Could you please add Russian submarine K-152 Nerpa to my stats as a footnote? I started it November 9. It ran on the home page "In The News" for about a week, and passed GA on November 24. Thanks. Jehochman ^Talk 20:43, 28 November 2008 (UTC)[reply]

Erkk - in progress, will redo all. As discussed elsewhere, article ratings are Sandy's province, but Sandy, if you think I should review your numbers, let me know. This is related to the >10-category bug, which should be low impact - but it's my bug. Franamax (talk) 23:54, 28 November 2008 (UTC)[reply]

You're going to re-do ... everything? <gulp> So I have to manually re-tally the top 5 counts? SandyGeorgia (Talk) 00:25, 29 November 2008 (UTC)[reply]

No, counts won't change. Just the assessment details, as in Top vs. Top*, and in Jeh's case, a Good instead of B-class. The <gulp> is all mine, it's strictly retrieving assessment categories from the talk page. The error rate is very low in any case, and only articles with talk pages having more than 10 categories. I'll check the impact and let you know when I'm done. Strict counting of FA/GA's shouldn't really have that much impact ('tho I understand you see it somewhat differently) - the essence remains encyclopedic vs. dramatapedic. No big deal, and nothing to do with any ratios you're using. DEFCON 1 (or whatever the lowest one is :). Franamax (talk) 00:53, 29 November 2008 (UTC)[reply]

Question

Does the mainspace % count talk edits? Because if it's not supposed to I REALLY shouldn't be at 72%, mine's closer to 32%, heh (I did too much assessment back in the day, so my talk % is 40%, which would explain 72%). Wizardman 20:34, 28 November 2008 (UTC)[reply]

I took that stat from here (mainspace career divided by GT edits); ~~I believe it doesn't count talk, but I could be wrong (Franamax should answer)~~. For the top five article edits, I had to manually tally talk plus article at top five for each candidate, and I indicated that includes talk. SandyGeorgia (Talk) 20:41, 28 November 2008 (UTC)[reply]

Yes, it does include talk. According to your individual page, your numbers in mainspace are Total Contrib's: 36689 (15757 page, 20932 talk) - Total pages: 29739, with GT edits of 50787. [1] I'll footnote that and doublecheck them all. SandyGeorgia (Talk) 20:44, 28 November 2008 (UTC)[reply]

Alright, just checking. I don't mind if the 72% stays, just wanted to make sure I knew why, thanks. Wizardman 21:01, 28 November 2008 (UTC)[reply]

If anyone thinks that column should be only article, we should discuss. Arguments can be made either direction. The "Top article edits" column is only article, not talk. SandyGeorgia (Talk) 21:02, 28 November 2008 (UTC)[reply]

One of the toggles in my program is "Combine talk spc", which I have always left turned on. I wrote this on purpose, observing that Interiot et al tabulate separately. The intent here is to recognize that the overall contribution to an article consists of both the content and the discussion. Especially in contentious matters, there could be six rounds of discussion before even a single sentence is changed in the article itself. In general the "headline" numbers are aggregate, the detail follows to the right or below. Franamax (talk) 23:37, 28 November 2008 (UTC)[reply]