Wikipedia talk:Review Board/Archive1

A modest proposal

The community has high expectations related to accountability and transparency of the Arbitration Committee and of the holders of privacy-related user rights (Checkuser and Oversight). Although historically a high general standard has been kept, a number of cases have caused concern and raised the community's interest in having an independent review body. Submitted here for approval is a tentative policy to establish a Review Board tasked with providing the community with the means to ensure those concerns are addressed now and in the future.

Who watches the watchers, indeed? Historically, the Arbitration Committee itself was entrusted with self examination. Arbitrators are amongst the most trusted editors, and have diligently attempted to ensure that community expectations were met; but are ultimately cursed with a view from the inside. The natural tendency to trust the people one works with day to day is indispensable for collaborative endeavors but if concerns arise about the conduct of such a user, then an investigation by their peers — however well intentioned — has a high risk of being unable to meet the wishes of the community for independent review. A clean break and separation with no day to day ties is better.

Ideally, then, the duty and responsibility of monitoring the editors we entrust with privacy related tools and with the handling of private information needs to be handed to a group that is entirely separate and independent. This group needs to hold both the community's trust and that of the Committee, for they need access to the most sensitive and private of information in order to be truly empowered to do their job.

Much of the detail of the proposed policy is open to discussion; the size of the Review Board, and the method by which the community will select the editors it trusts to be candidates to hold its seats in particular need to be hammered out. I, and others who have worked on this, believe that we will find in the founding of the Board an answer to the legitimate concerns of the past, and a solid way to move forward with greater accountability.

— Coren ^(talk) 23:31, 21 December 2008 (UTC)

Executive summary

This proposes the formation of a board of four editors who are entrusted with checkuser and oversight, and with privileged access to ArbCom communications, that has the mandate to investigate user complaints of CU or OS misuse, or Arb misconduct.

They are broadly under a communal mandate, and the result of their work is a report which may indicate they would endorse or certify an RFC/U. They also are tasked with general monitoring of standards and use of checkuser and oversight, and may intervene if they see substandard work or actions without support, in the logs. — Coren ^(talk) 23:46, 21 December 2008 (UTC)

Initial comments

The overall idea looks good, but there's some details that may need further work (roughly in order through the document):

• Why four members rather than three or five? An odd number would seem to work better with normal majority provisions; otherwise, we're effectively requiring 75% support rather than simple majority.
• "privacy related" applies less to considerations of Committee conduct than it does to the other two groups; perhaps a broader term could be used here?
• I'm not convinced that the "Background to the Board" section needs to be in the final policy; it would have merely historical relevance after adoption.
• Consistency between "Jimmy" and "Jimbo" would be good.
• As a matter of logistics, it would probably be better for the board to automatically be subscribed to arbcom-l rather than requesting items from it after the fact; we lack a good method for exporting only selected discussions from the archive, so the net effect will be the same regardless.
• Item #4 in the list of "rights" isn't really one, in my view; any user is free to consult with anyone who will speak to them.
• "above scope and criteria" - it's not clear whether this refers to the "Review Board" section, or the OS/CU material specifically.
• What's the reasoning behind having two votes, pre- and post-Jimbo? Given that the Board would provide copies of any report to him anyways, I don't see any reason for so much red tape.
• The nature of the communal vote to dismiss members needs to be clarified; I'm assuming that some quorum requirement is implicit, but it's never stated explicitly.
• I'm not convinced that Jimbo needs to retain the power to dismiss members; but if he does, it would need to be specified that such an action would not then be subject to Committee oversight (as other Jimbo actions are).

The entire document could also stand to get some copyediting—it's unnecessarily verbose at some points, in my opinion—but that can probably come later. Kirill 00:20, 22 December 2008 (UTC)

Thanks for the review. Some answers to your points:

• Four is a (semi) random number; I state it explicitly as a variable that needs community input above. I picked four because it seemed small enough to avoid creating a horde and large enough to allow for discussion.
• I'm open to rewording where appropriate; I think the intent is well communicated but what seems clear to me may be opaque to others. This draft is mostly meant to discuss the concept, I'm sure we can hammer out satisfactory wording while working on it.
• I think the background is useful because it codifies the intent and rationale; five or ten years down the line "why" we did this may not be as obvious as it once was.
• The two votes were in the case that Jimbo brings in important new information. I have no particular attachment to it.
• Yes, I would expect some quorum requirement as well, but I see that as one of the processes that needs wider community discussion — how to go in, how to go out.
• You have a point about Jimbo intervention, but I think more input is the key here to see what the rest of the community feels on that point. — Coren ^(talk) 00:31, 22 December 2008 (UTC)

Coren, two questions: first, can you say who wrote this, and also can you say why the board would be chosen by the community and the ArbCom. Why not just by the community? SlimVirgin ^{talk|contribs} 00:26, 22 December 2008 (UTC)

Slim, I would think ArbCom needs to be involved because there are times when ArbCom is in possession of information that the community lacks, and ArbCom can act upon it where necessary (often preventing embarrassment). -- Avi (talk) 00:32, 22 December 2008 (UTC)

Can you give a hypothetical example of what kind of information you mean? SlimVirgin ^{talk|contribs} 00:33, 22 December 2008 (UTC)

Others have explained it more clearly than I, but was thinking about sockpuppets of banned users. -- Avi (talk) 01:10, 22 December 2008 (UTC)

We should just force full disclosure of current and past accounts from anyone running as a matter of course, I think. Kirill 01:16, 22 December 2008 (UTC)

That may endanger the privacy of people with valid secondary accounts, unless the disclosure can be limited to one or two people. As of now, we have to identify our real life ID's to Cary Bass to get the CU/OS bit. Having to identify all accounts (former, present, etc.) to Cary is a viable option, but that is out of scope of this discussion already, I think. -- Avi (talk) 01:20, 22 December 2008 (UTC)

Simply put, because access to checkuser and oversight requires ArbCom approval. The best compromise to allow the community to pick its representatives while fulfilling this requirement was to allow the formation of a pool of candidates that are all acceptable to everyone, and to pick the members of the board from that pool. This way, only candidates the community feels will do the job right will be appointed, and the requirements for access to private information are maintained.

As for who wrote this draft, it was mostly myself and FT2 with some tweaks suggested by others. — Coren ^(talk) 00:38, 22 December 2008 (UTC)

Does the AC name prospects and the community then RfSomethings them, or the community RfSomethings people and the AC then picks from that? rootology (C)(T) 00:48, 22 December 2008 (UTC)

The community picks a "short list" of suitable candidates, and the AC picks from that list. — Coren ^(talk) 00:50, 22 December 2008 (UTC)

I don't see how we can have a board that has been vetoed by one of the bodies the board may have to investigate. As I see it, part of the point of a review board would be that it was entirely independent of the ArbCom and the body of checkusers in general. SlimVirgin ^{talk|contribs} 00:53, 22 December 2008 (UTC)

I'm not sure I understand your worry, SV. Since only candidates picked by the community can be selected, certainly any of them would have a community mandate. — Coren ^(talk) 00:58, 22 December 2008 (UTC)

I don't think there would be any significant problem with a pure community election, actually; there's fairly little chance of ArbCom knowing some damaging information but not having acted on it before it could become an issue.

We would probably need some additional specifications on account disclosure, and so forth, to prevent people running sockpuppets for the board, or failing to disclose banned accounts, or something of that sort; but that should really be in place for all high-level elections anyways. Kirill 01:03, 22 December 2008 (UTC)

(EC) It empowers the community somewhat, like a police review board, which is probably a bad analogy, but a close enough one I guess. If not every report is published in whole unless the AC decides as a whole, can a short version of the findings of any report be public for any actions taken as a result? I.e., a user losing some permissions, or having acted improperly? The permissions would be obvious since that can never be "secret", as them having it is public, but the latter is important, since an Oversighter or CU relies on the trust and support of their peers to be of value. rootology (C)(T) 00:54, 22 December 2008 (UTC)

Yes, every case should get a report. It may need to be redacted in rare cases, but should nonetheless provide enough information for the community to feel confident the job was done right. — Coren ^(talk) 00:58, 22 December 2008 (UTC)

Whether or not anything were published would be up to the review board, not ArbCom, incidentally. Kirill 01:03, 22 December 2008 (UTC)

Really?! Well, then I like this a lot as long as neither the AC nor Jimbo has **ANY** power to dismiss or sanction the review board or members for issuing a negative report--as long as no actual privacy is violated in the process. A hard-coded check like that to ensure separation of power, and as worded I love this, as I'm sure many would. Is that feasible? rootology (C)(T) 01:34, 22 December 2008 (UTC)

As written, the AC wouldn't have any ability to dismiss the board members in any case. I don't think it would be a big deal to not have any special Jimbo provisions here either. Kirill 01:39, 22 December 2008 (UTC)

Confirming I wrote it ruthlessly hardened on that point, to make sure as loophole-free as I could, that once they are appointed, they cannot be dismissed for a "negative report" or displeasing those they report on, or about, and I feel that is essential to hard-code into it. The reason for Jimboness is, suppose there is a member who does something dodgy, by the time the community argues and we find if its "consensus" or "no consensus" or 45% or 55% and what's enough... if there is a genuine concern then quick removal should be possible. I wouldn't want that power in the hands of arbitrators, or any arbitrator. Hence it's in there as a "Jimbo power", not planned to be used but in there against the eventuality. FT2 ^{(Talk | email)} 01:57, 22 December 2008 (UTC)

(edit conflict) Quick answers from me:

• Something like this should be small-ish, it doesn't need many people (WP:BURO). 5 seemed too large compared to need, 3 doesn't allow for 2 people to both be away or busy.
• Terminology and "background" section - easy to decide, not crucial. Usually policies have some explanatory element too.
• The arbcom lists discuss a huge range of highly sensitive issues. It's had legal cases, harassment and sexual matters, names of stalkers with police records, Mike Godwin's input, and the like, pedophilia advocacy information, in extreme cases. It might easily discuss one or more of the board in some matter. Very uneasy indeed about exposing that to any users who are at most going to want to see discussion related to a handful out of thousands of topics a year only. They will only need Arbcom emails to investigate a complaint about an arb (complaints about CU etc are basically "here's the check, you dig up any evidence you like, to demonstrate why it's valid"). When they need arb-l emails, out of 18 arbs, another 18 ex-arbs, and Jimbo, the emails will easily and quickly be found.
• Right #4 - the point was to emphasize "things the board may be expected to do", including reporting or inquiring further. Optional and possibly removable if redundant.
• "Scope and criteria" - wikilinked to clarify
• Two votes - when we reviewed the Checkuser and Oversight appointments last year, the users we had concerns expressed about had two votes. One was the routine "do we endorse this user" vote before presenting names to the community, the other was a "having had this concern raised and reviewed the evidence, are we still happy with them" vote. Both were valuable, to check how we felt after the concerns and examination. It was a good way to do it. "What do we feel, let's vote" and "having consulted with Jimbo, what do we feel now about it, let's vote".
• Quorum and removal - some tidying up is likely, yes.
• Why the community and arbcom? Arbcom is tasked as custodians and to safeguard privacy functions. For better or worse, they cannot be handed out to users (no matter how much the community loves them) unless we too are confident they are in hands we feel are some of the best on the project. Nobody can predict what means the community will use to choose its shortlist in future, so we cannot have any certainty that the users will all meet whatever standards may then exist for it. So for the foreseeable future, there needs to be some way to ensure that both goals are met -- the community gets users it has confidence in, and the arbitration committee gets users it can feel confident to give the tools to. In this model, the community chooses and lists for us, "here are users we would trust on the board". Arbcom then chooses only from within that list, the users it then feels most trusting of the WMF tools. Both get the core requirements met, that they need.

FT2 ^{(Talk | email)} 01:05, 22 December 2008 (UTC)

I'd appreciate if you'd stop the "best on the project" thing in describing the ArbCom. It's tired, it's worn, and it's false (no disrespect to any individual intended). The point here is that the community needs to pick the people they think could do the job. There's no reason or need to give the ArbCom any more power, and that's especially important given that the ArbCom may be the subject of a board investigation. SlimVirgin ^{talk|contribs} 01:11, 22 December 2008 (UTC)

(edit conflict) In regards to arbcom-l mail: on a practical level, it will be difficult, if not impossible, to provide all correspondence on a given matter to the board within a reasonable timeframe if they do not receive it to begin with; we have no good way of forwarding entire threads, so we'll be stuck trying to copy over hundreds of emails, which will lose threading structure and be error-prone. Worse, in the (extremely rare) case where this truly becomes necessary—where the community suspects impropriety of the Committee as a whole—correspondence forwarded after the fact will always be treated with suspicion.

Given a suitably rigorous selection process, I think it's better to just let the board have access to the unfiltered discussions. Certainly, we trust all the new arbitrators with access to it; I'm confident we can find a few more people who could be similarly trusted. Kirill 01:15, 22 December 2008 (UTC)

This is unlikely to be either as rigorous or as heavily polled as Arbcom elections by a long way. We have no say what level of quality may exist. Certainly we would not deem a user fit to have access to checkuser, oversight, all arbcom emails, on the basis of RfB alone, yet that presently is the next toughest standard below Arbcom election. We don't know what will be done here, and we can't presume at all, that it is comparable in depth or in communal spotlight and probing to the standard needed for Arbcom, or that the users it prefers will consistently for the future, be users who would meet our trust level for WMF tools and private information. Arbcom gets a thousand threads a year or more; only a tiny proportion will ever need to be reviewed. FT2 ^{(Talk | email)} 01:42, 22 December 2008 (UTC)

I would prefer to enforce the necessary level of rigor, in that case—and there's no reason that I can think of why we couldn't make this as rigorous as the ArbCom election—instead of relying on vetoes and reserve powers to compensate for an unacceptably lax initial selection. Kirill 01:44, 22 December 2008 (UTC)

How will you "enforce"? It's of necessity completely left to the community. Arbcom elections have been gruelling and very high profile. There is a limit how many high intensity spotlight processes of this kind one can run a year for the general community. Someone proposed a midyear arbcom election, and that would probably not be as rigorous and possibly less well attended compared to the main one (as others pointed out). How will we ensure these were equally high standard? And honestly, what's the point? There were maybe a handful of incidents this year, a tiny number of easily found emails out of ten or fifteen thousand. And they can ask any or all of about 40 people to forward them if needed. There's just no need. FT2 ^{(Talk | email)} 01:50, 22 December 2008 (UTC)

The proposal is still in its initial stages; I don't think it's unreasonable to assume that a selection method will be finalized before it goes live.

In any case, I don't really see the risk here. Clearly the people selected will have to be trusted to view any ArbCom discussions, because they will be authorized to ask for them, without having to explain their need, and we will be required to provide them. If we're going to trust them to have access to anything, then we might as well trust them to have said access in real time.

What's more, if they're to perform the same "monitoring of logs" function they have for CU and OS with regards to ArbCom conduct, then they pretty much need to see all of our discussions; otherwise, they will effectively have no logs to monitor, and be forced to rely on outside complaints about discussions that nobody on the outside really knows anything about. Kirill 02:25, 22 December 2008 (UTC)

Both good points and agree with them. However I still don't think we should pass full list access where probably 1/2 - 1% of emails in the year will be at all relevant. They also don't need list access to check oversight and checkuser. That's quite different in approach and will usually not even be an "arbcom matter". Oversight inquiry is "This diff doesn't have any obvious need, what's the need (plus evidence if applicable)". Checkuser iinquiry is "Please explain this check" or "Please explain why this check is one you should have done" and the response to both is an explanation plus whatever evidence if applicable. The checkuser is the person obligated to prove the check is valid. He may need to submit emails from Arbcom to do so, but much more commonly, it won't involve Arbcom emails at all, except to show due care, or whatever else needs evidencing -- in which case its up to the checkuser to collate whatever evidence he/she feels is needed. The Review Board wouldn't need Arbcom access for that purpose, as long as they can send the emails to any of 40 people or the list and ask "can someone confirm these". FT2 ^{(Talk | email)} 02:34, 22 December 2008 (UTC)

Kirill, this proposal badly needs a serious copy edit. It'd be appreciated if you wouldn't revert those changes while you make your own. SlimVirgin ^{talk|contribs} 01:18, 22 December 2008 (UTC)

I'm sorry; I was just trying to, y'know, copyedit the proposal. ;-) Kirill 01:39, 22 December 2008 (UTC)

Does this need saying at the end? "Any other removal may only be on an emergency basis, for good cause, with disclosure, and temporary until it can be reviewed."? I'm thinking of unfrocked abusive users, which has happened once now. Worthwhile, or "obvious and not needed"? FT2 ^{(Talk | email)} 02:48, 22 December 2008 (UTC)

Hm. There's a contradiction between the "Selection" and "Removal" sections; the former states that members may be removed by the board itself, while the latter only has communal vote and Jimbo as removal options. I'd suggest, if not allowing the board to remove its own members permanently, then leaving that as an option for temporary removal if it turns out that someone's access to restricted information needs to be cut off in an emergency. Kirill 02:52, 22 December 2008 (UTC)

Inconsistency fixed. Is the above text worth (or not) adding though? FT2 ^{(Talk | email)} 03:17, 22 December 2008 (UTC)

Second question - policy changes. Since this will be a significant check and balance (and may be in time our main one), and it contains some delicate balances between the community, the committee, WMF and its policies, and Jimbo, I'd propose it have a similar regime for amendment as arbitration policy itself (in effect it is overseeing Arbcom). That is, amend following wide consultation and agreement, and so on, same process as for Arbitration policy itself. Stability and "should not lightly be changed" measure. FT2 ^{(Talk | email)} 03:16, 22 December 2008 (UTC)

Added this briefly. FT2 ^{(Talk | email)} 04:08, 22 December 2008 (UTC)

Confusing sentence

I removed this as it's not clear what it means:

To investigate the "tiny minority" of truly exceptional circumstances (such as major self reversals or decisions that appear to be beyond plausible explanation), where concerns exist of such a nature that a finding of facts is exceptionally needed by the community.

SlimVirgin ^{talk|contribs} 01:07, 22 December 2008 (UTC)

This might probably have been worded more felicitously. The point was there there are some ArbCom decision that may leave the community at large going (pardon my french) "WTF?!"; when that happens, having an independent body allowed to look into this and emerge with a report (whether it's to say "That was all wrong" or "It was the right move") is the best thing for transparency.

I'd expect those cases to be generally few and far between, hence the colored wording. — Coren ^(talk) 01:21, 22 December 2008 (UTC)

(edit conflict) Midway through the year a user was stated to be given Checkuser, and this was then rapidly reversed. The post by an ex-arb got some comment.

I can imagine other situations where there is no "improper conduct" allegation, no tool abuse, but there was some disquieting incident that raised worries about something, and the community would wish the Review Board to fact find and confirm what the heck had gone on at Arbcom, to be sure of the reason and position. What you just took out, allowed for that. If there's a better less confusing wording, and you feel that might be a useful role to allow when needed, then it might be worth adding it (or something similar) back in. FT2 ^{(Talk | email)} 01:23, 22 December 2008 (UTC)

That moves it away from just investigating checkuser and oversight. Are you saying you want an oversight body for the ArbCom in general? SlimVirgin ^{talk|contribs} 01:23, 22 December 2008 (UTC)

And yes, of course. Not for every decision or every incident, but at least the community having the capability to look into it, if it does seem justified and is "one of those kinds of incidents". What else are we discussing? Watching Checkuser and Oversight and individual arbs, but if the committee does something really inexplicable, it wouldn't be able to be asked about it by the Board unless a specific arbitrator's conduct is improper? So yes, let's round it out and ensure we have in case of real need, a way to at least ask the Board to fact find on these kinds of truly exceptional cases. You remember the problem at the Ombudsmen, that their remit did not allow some stuff to be looked at. I want to ensure we don't accidentally have that same kind of problem. Hence that wording. FT2 ^{(Talk | email)} 01:29, 22 December 2008 (UTC)

(e/c) An example might be the OM case, where broken communication within ArbCom caused much drama and confusion. Having an independent body that could look into what happened and give a report would have been a Good Thing. — Coren ^(talk) 01:31, 22 December 2008 (UTC)

(I chose a different example because there clearly they could have justified it on the basis "there is a concern over improper action", at least in the early stages, when nobody knew what was going on. FT2 ^{(Talk | email)} 01:34, 22 December 2008 (UTC) )

I guess the distinction between the two points really just boils down to what the precise level of concern needs to be before the board will consider it, which may not be something we can pin down in advance. Kirill 01:40, 22 December 2008 (UTC)

Yes. But we don't need to pin it down precisely, to indicate roughly where it gets to be valid. FT2 ^{(Talk | email)} 01:44, 22 December 2008 (UTC)

(undent) There is an argument that, as trusted users, the board itself may make that determination. I would tend to allow the board wide latitude and discretion in determining what level of "community shock" warrants an examination of the circumstances. — Coren ^(talk) 01:48, 22 December 2008 (UTC)

Agree. If the remit permits it and gives rough enough guidance that they aren't plagued with "plz investigate abuz" every 3 hours :) then I'd trust them to determine it, and I'd abide by their determination. FT2 ^{(Talk | email)} 01:52, 22 December 2008 (UTC)

I've added a point to that effect in the spot formerly occupied by the "exceptional" statement for consideration. — Coren ^(talk) 02:01, 22 December 2008 (UTC)

Tightened a loophole; you picked up "Committee actions" but I think we also need to add "Actions by an individual user (an arb) that appear to be in their role as an Arbitrator". If it's looking like it may be an arb action, then its automatically fair game at that point, for checking if there is a serious concern. FT2 ^{(Talk | email)} 02:40, 22 December 2008 (UTC)

"Monitoring of CheckUser and Oversight" section

I'm trying to copy edit this section, but it actually needs a rewrite. Could the author explain what the point of it is, first? The board will presumably investigate as the board sees fit, and the standard it'll judge actions by is the extent to which they comply with Foundation policy. Does anything more than that need to be said? SlimVirgin ^{talk|contribs} 03:20, 22 December 2008 (UTC)

That's basically it, though most of the verbiage is to make explicit what the expectations are. I suppose including the relevant policies by reference would be sufficient, however. — Coren ^(talk) 03:23, 22 December 2008 (UTC)

... although mention of enwp policies (which go a bit beyond WMF's) and of "best practice" could probably bear mention as well. — Coren ^(talk) 03:25, 22 December 2008 (UTC)

Okay, I'll reword it that way. Any particular WP policies you have in mind? SlimVirgin ^{talk|contribs} 03:33, 22 December 2008 (UTC)

Basically:

• They will basically do ongoing review, checking, and quality control on CU/OS activity. So as they see fit they might look at CU/OS actions and check they are well documented, or randomly check them against the wiki RFCU case to see if they were well judged, or ask for the evidence of need if it's not obvious or they can't see how check A and B might be connected, or express a concern about COI, etc.

• In effect, they will be auditing. It should raise standards. But for the same reason they need to nudge people to improve. One or two missed descriptions are a nudge to try and avoid that, not the end of the world, if at the end of the day it looks like the checkuser did have good cause etc. But if it's a habit, then they need to nudge for change, and watch that it happens.

• The italic bullet lists are just examples of "things they might notice". It can be footnoted if needed, it's not core. I'll do that.

• The last 2 paras are to try and ensure they take account of any genuine CU/OS norms, not their own novel view on things, although to a high standard.

FT2 ^{(Talk | email)} 03:39, 22 December 2008 (UTC)

Had a go at refactoring and simplifying. FT2 ^{(Talk | email)} 03:52, 22 December 2008 (UTC)

And tried a bit more copyediting. FT2 ^{(Talk | email)} 04:03, 22 December 2008 (UTC)

"They are in no circumstances to use the functions themselves"

This won't work as written. The review panel should not be performing checks except for purposes of investigation, but there will be some reviews of checkuser cases where the review panel will have to be able to run checks themselves, not just look at log entries. Especially in complaints of misidentification, but also other complaints. For oversight, the content of the deleted revision is in the log, but for checkuser, not all the results are in the log, and only looking at the log could lead to wrong or incomplete conclusions. Thatcher 18:19, 22 December 2008 (UTC)

Thatcher, to prevent any possibility of misuse of the CU tool, the idea is being floated to restrict access to teh tool itself and just allow log access. Unless you believe that EVERYONE with the CU bit is in cahoots, the audit committee can find someone to run the check. -- Avi (talk) 18:21, 22 December 2008 (UTC)

See the discussion in the "Technical implementation" section (and elsewhere) below. -- Avi (talk) 18:24, 22 December 2008 (UTC)

You know how many actual checks it takes to run an analysis, right, and how much data is returned? You want to be sending emails (mostly via googlemail, which is hardly "high security") containing hundreds of kb of text and links? How about running recursive checks where you see something and can click right on the link to run the next level check? No, the fact that it is technically possible to separate the functions does not make it wise to do so. The problem you want to solve (presumably, checkusers being too chummy with each other to give an honest review) can best be solved by picking the right reviewers and telling them not to get involved in running routine checks of Grawp socks and RFCU reports. If the reviewer is the sort who would let personal feelings interfere in their performance and not recuse, then it doesn't matter which tools and emails they have access to and which they have to ask for help to get, they're just the wrong person. Thatcher 18:29, 22 December 2008 (UTC)

So what you are doing is just creating a new class of checkusers, who are also oversighters, who are also ArbCom. In that case, what is the problem with putting in a few more CU's/OS's and letting them self-police as they have always done? Yes, I do know how difficult it is to run these checks, but if we are concerned about misuse, then we need to be concerned about audit misuse too, otherwise you are just pushing the problem off to another level. Building in the safeguards now prevents us from having this conversation next year about the committee. What makes it wise is not the technical ability, but the check and balance on the audit committee itself. -- Avi (talk) 18:34, 22 December 2008 (UTC)(

But what safeguard are you actually building in in terms of reality and not merely appearances. What bad things can a panel member do with full checkuser access that he can't do with log-only access? Thatcher 18:42, 22 December 2008 (UTC)

Peer consensus

Can someone say what this means, exactly? "There is peer consensus (in which they have recused) ..." SlimVirgin ^{talk|contribs} 03:35, 22 December 2008 (UTC)

Basically, if one of the members wakes up one morning to find his other 3 very seasoned colleagues have emailed him to say "sorry, we decided 2-1 we don't have confidence in you and would like you to step down as a review board member", that's a bad sign and they probably ought to respect it (=peer consensus). They don't get to cast a vote "for themselves". FT2 ^{(Talk | email)} 03:42, 22 December 2008 (UTC)

Selection section

I removed two things: first, that any rankings given to candidates during an election may be ignored. It seems to me that, if the community is to choose candidates, then the rankings are important. Secondly, I removed that Jimbo can unilaterally remove people from the board. It should be up to the majority of the board or the voting community to effect removal, not just one person.

Is that really necessary? It seems pretty important to me that all the editors in the "pool" of candidates be considered on an equal footing and without regard to exactly how the community picked them (possibly at different times, and possibly with different levels of participation all of which make for poor and misleading comparisons). — Coren ^(talk) 05:04, 22 December 2008 (UTC)

Question: The section says that members could be removed by a 50 percent community vote. What is meant by that, exactly? How many people would have to vote before we'd assume over 50 percent to be a mandate to remove someone? Who would initiate such a vote etc? SlimVirgin ^{talk|contribs} 04:58, 22 December 2008 (UTC)

I think that needs wider community input. It was left fairly open for just that reason (just as the selection process itself was). The criterion for the community to withdraw its approval should be picked by more than a few editors hashing out text on this page, methinks. — Coren ^(talk) 05:04, 22 December 2008 (UTC)

Name

Should the name be changed to something less likely to be confused/associated with Wikipedia Review? Aiuw 04:53, 22 December 2008 (UTC)

Let's keep Wikipedia Review out of this. This has nothing to do with the site. rootology (C)(T) 05:06, 22 December 2008 (UTC)

I completely agree with you. This has no relation to Wikipedia Review, but the name Wikipedia:Review Board is very, very similar to the website. I don't think that it would be a major hassle to change the name to something more universal, such as the Ombudsmen Committee, or something of that nature. Aiuw 21:35, 22 December 2008 (UTC)

Number of members

I changed this from 4 to 5 to avoid stalemate, given that cases are to be accepted and decided by majority. SlimVirgin ^{talk|contribs} 04:59, 22 December 2008 (UTC)

Yes, 5 was my original number as well. — Coren ^(talk) 05:19, 22 December 2008 (UTC)

Makes sense. -- Avi (talk) 05:23, 22 December 2008 (UTC)

The reason for 4 was, five starts to seem a big committee. This should be small.

5 is a bit too big (WP:BURO) and 3 too small (if there is a recusal or absence). 4 allows one absence or recusal, or (with unanimity) even 2, and you still have a functioning group. FT2 ^{(Talk | email)} 13:10, 22 December 2008 (UTC)

If we expect splits on a regular basis, then an odd number is needed. Do we? I would hope not, or we will have chosen the members unwisely. The first round of ombudsmen numbered 2, IIRC. ++Lar: t/c 14:38, 22 December 2008 (UTC)

Rankings

The original text specifically stated that the rankings need not be followed. While Slim's version allows for that possibility, its restatement creates the connotation that the rankings should be followed, and it is the exception to not follow them. While this is a matter of importance, such a major change to the original proposition needs to be hammered out. -- Avi (talk) 04:59, 22 December 2008 (UTC)

The point of this being a proposal, and not a fully formed policy, is that people get to change it. Why would we not want to follow rankings? SlimVirgin ^{talk|contribs} 05:01, 22 December 2008 (UTC)

My specific concern is that as this relates to the possible release of personally identifiable information, at least with respect to Checkuser, there may be concerns that are known to ArbCom/WMF that are not known to the community as a whole, which is why I think Coren worded it that way in the first place. Coren? -- Avi (talk) 05:02, 22 December 2008 (UTC)

Actually, it's an interesting point. Why not hold the pool by rankings, say #1-10. If the AC doesn't like #2, they can go to #3, but must still take in the order the community gives to them? Would that be viable? rootology (C)(T) 05:05, 22 December 2008 (UTC)

That's an option, but it opens up ugliness in that there will be a hue-and-cry as to why #2 was not accepted. If there is no presumption of order, there can be no embarrassment for those not chosen. -- Avi (talk) 05:07, 22 December 2008 (UTC)

I don't know of a way around that. But if you put yourself up as a board candidate, I suppose you'd better be prepared for the option, if a ranking system is used. It would certainly keep people with anything to hide out of the mix, which would also be a good thing. rootology (C)(T) 05:12, 22 December 2008 (UTC)

Additionally, IF someone is bypassed, and the person calls Foul, it could similarly be reviewed by the Review Board that DOES get seated, to keep the AC honest. If someone is actually a Bad Person but doesn't want WP at large to know, it's instantly a check and balance on the candidates and the AC. If that happened, it would be invaluable to keep the whole affair clean. rootology (C)(T) 05:15, 22 December 2008 (UTC)

(ec) We need to move away from this idea that checkusers have access to information that places them beyond reproach. The reason we need this policy is precisely because there have been problems with the way the tool has been used. Checkusers are no more or less likely than other members of the community to make a good or bad decision about membership. We can't have a situation where they exercise a veto. That would undermine the board from the get-go. SlimVirgin ^{talk|contribs} 05:08, 22 December 2008 (UTC)

Slim, it is the ArbCom, not the CU's who excercise said veto, and as on EnWIki they are the ones who decide who gets the bit in the first place, it is not incomprehensible that it is so. So while a separate audit committee may be a reasonable suggestion, it needs to fit into the current framework, at least that is my opinion. -- Avi (talk) 05:13, 22 December 2008 (UTC)

(e/c) I've just answered it above. My primary concerns are more with the difficulty of ranking candidates fairly, but yes— I think the ability tactfully not pick a candidate for which the Committee have serious reservations about access to private data is also a factor. — Coren ^(talk) 05:12, 22 December 2008 (UTC)

Which is why the pool is a good idea, but the ranking, in my opinion, is not. -- Avi (talk) 05:13, 22 December 2008 (UTC)

Coren, what about letting the board review the bypass decision if the bypassed person calls foul? It would be about as transparent and clean then as is probably possible. If someone did have skeletons that would keep them out but they don't want people to know, it would keep untrustworthy people out that way, protecting the AC, and the community, while giving the community the chance to name people by rank, like we do the AC itself. rootology (C)(T) 05:17, 22 December 2008 (UTC)

Actually, I think that already falls within the remit of the Board (as a Committee action). I see no reason to exclude it either. — Coren ^(talk) 05:29, 22 December 2008 (UTC)

Oh, that makes it even easier then. Is it just a question of then of having a jury pool (RfWhatever) vs a ranked election (like the AC election)? rootology (C)(T) 05:31, 22 December 2008 (UTC)

Seems that way. The community decides on the pool, and ArbCom picks from them without resorting to any specific order. -- Avi (talk) 05:40, 22 December 2008 (UTC)

As Rootology says.

WMF (whether one agrees or not) says that at present, Arbcom are responsible for choosing who is trusted with the tools. Both tools allow access to significant private data, even with log access only. The community's concern is that members are people that have communal trust and are widely seen as independent. The Arbitration Committee needs to ensure it isn't put in a position of endorsing giving tools to people unless there is great confidence within the committee that they will use them appropriately only. The maintenance of a "pool" rather than a ranked list, fixes both of these. The community keeps a list of "people it would be fine with" and since all are trusted by the community to a high level, Arbcom is then free to choose any of those whenever an appointment is needed. FT2 ^{(Talk | email)} 13:06, 22 December 2008 (UTC)

Names of authors requested

In the interests of transparency, could we be told who the original authors of this proposal were, and who else read it before it was posted? I asked this earlier because, although Coren posted it, it did not look like his writing. I feel it's important to know the range of interests that are being represented here. SlimVirgin ^{talk|contribs} 05:04, 22 December 2008 (UTC)

Out of sheer curiosity, Slim, why does it matter? It appears that the wikipedia's community's interests are being represented here. Is there some sort of conspiracy I am unaware of? -- Avi (talk) 05:06, 22 December 2008 (UTC)

I have to admit of a certain unease about why the query as well. Glad it's sorted now but it just struck me as odd. If a proposal has merits, consider it. The source is less relevant than the merits. The query smacks of not assuming good faith. ++Lar: t/c 14:37, 22 December 2008 (UTC)

He already mentioned on this page at least one other author. Regardless, I (along with Avi above) don't see why it is important (but OTOH I don't see why it needs to be a secret - not that it is, but...). Surely you (and others) wouldn't oppose a proposal only because a certain editor or editors created it, right? - Rjd0060 (talk) 05:10, 22 December 2008 (UTC)

As I said, it's in the interests of transparency. The proposal was posted by someone who appears not to have written it, which is unusual. So I'm simply asking who the main author was, and who else saw it before it was posted, so we know where it came from. If it's not a secret, there's no reason to act as though it is. If it is a secret, that's worrying. SlimVirgin ^{talk|contribs} 05:11, 22 December 2008 (UTC)

I am the original author, albeit FT2 did help a great deal with wording revision. He has considerably more familiarity with the typical formulation of enwp policy and thought of a number of details and loopholes I had not.

The draft was originally worked on the AC wiki, from an original discussion on the mailing list started by me. Six of the arbitrators have commented on it. Does that answer your question? — Coren ^(talk) 05:12, 22 December 2008 (UTC)

So it was first created on the 20th, and hasn't been seen by anyone other than those on the committee mailing list, is that right? SlimVirgin ^{talk|contribs} 05:17, 22 December 2008 (UTC)

Well, being that it has now been seen by possibly millions of people, what does it matter? -- Avi (talk) 05:19, 22 December 2008 (UTC)

Actually, on the 21st, and yes, everyone who saw this before it got posted here is a member of arbcom-l or has access to the AC Wiki (which, as far as I know, is exactly the same list of editors). — Coren ^(talk) 05:23, 22 December 2008 (UTC)

Okay, thanks, Coren. SlimVirgin ^{talk|contribs} 05:44, 22 December 2008 (UTC)

Technical implementation

Since these users will want access to logs, but should not have the ability to perform the functions themselves, they can be assigned restricted rights instead of actual checkuser and oversight access:

$wgGroupPermissions['Reviewboard']['oversight'] = true;
$wgGroupPermissions['Reviewboard']['checkuser-log'] = true;

Would give members of a "Reviewboard" group access to the oversighted revisions and the checkuser log itself, without actual access to the tools. Oversight users also have "hiderevision" which grants the ability to remove revisions [1], and Checkusers have "checkuser" access, which lets them perform the queries[2].

If the community doesn't mind yet another usergroup, this could give the log access without granting access to the tools. HTH. Kylu (talk) 05:13, 22 December 2008 (UTC)

I think there is a general resistance to implementing new user right groups, but if that is acceptable that does simplify things. — Coren ^(talk) 05:15, 22 December 2008 (UTC)

THAT is a very good idea, Kylu, as it would allow the audit committee to serve its function without the risk of potential tool abuse. However that would mean that in the event of a possible misuse of the tools, they would have to approach other CU's or OS's to get the actual data if necessary, which I do not see as a problem. -- Avi (talk) 05:16, 22 December 2008 (UTC)

I think that a whole separate group should only be created if necessary (i.e.: if the community isn't comfortable with the current proposal). It would seem that the request to create a new group so that these "trusted users" can't actually run a checkuser or hide a revision is needless paranoia. But maybe that's just me. - Rjd0060 (talk) 05:19, 22 December 2008 (UTC)

Personally, I'd prefer another usergroup, which, IIUIC, just means a few more bytes on some server somewhere, then allowing the possibility of tool misuse. But I tend to lean to protection when applicable, so that may just be me. Also, if the entire purpose of this committee is to prevent misuse, shouldn't the committee itself be protected from potential misuse? $(Quis\;custodiet\;ipsos\;custodes?)^{2}$ and all. -- Avi (talk) 05:22, 22 December 2008 (UTC)

I'm just saying ... these 4 or 5 or however many users are supposed to be trusted. They realize that any use of the tools for any reason will result in their suspension from the Board. I'd have no problem with a new usergroup, but don't see the need. - Rjd0060 (talk) 05:24, 22 December 2008 (UTC)

So are CU's, OS's, and ArbCom themselves. The fact that we are discussing an audit board shows the possibility of misuse bothers the community, so it should be managed on this new group. In the US, where do you go to appeal a Supreme Court decision? The buck has to stop somewhere! Otherwise, we get reductio ad absurdum with the committee to review the committee to review the committee… -- Avi (talk) 05:28, 22 December 2008 (UTC)

New sets of rights are easy enough to set up nowadays, just create a global group, and put only en:wp in the list of wikis that are in the group, (at least at first) and then add the desired permissions, et voila. So I'm not sure I see creating a new group as a bad thing from that perspective. (however at present such newly created groups can only be granted to folk by stewards... for this particular set, that should be fine, it would not have a lot of coming and going). So that leaves the question of whether it's needed, or whether to just flip the CU/Oversight bits on for committee members. Personally, I think it's cleaner to do this, as it's a new role, than to reuse old roles. But this isn't the thorniest question that has to be solved so it's not a big deal. ++Lar: t/c 06:47, 22 December 2008 (UTC)

Woot! (I think is the internet expression!) A new usergroup that has log but not tool access? That works! However it's also (as Rjd says) completely redundant since any use of the tools is indefinitely logged anyway. It's easier to not use something if it's not presented to you, but a script could take care of that if needed. FT2 ^{(Talk | email)} 12:56, 22 December 2008 (UTC)

Did a quick test at m:Special:GlobalGroupPermissions. I didn't actually create the group but got as far as ready to press the commit button. Right now, access to the CU log, and ability to run checks already are separated. However apparently, the ability to delete revisions (accesible via an extra tab while viewing history), and the ability to view deleted revisions (accessible via Special:Oversight) are not separate (yet). As Kylu discusses, this probably takes either a config or a fairly straightforward code change. I'll search for, and open if needed, the applicable bugzilla. Also for reference the Ombudsmen global group (from which no wiki can opt out, I don't think) currently has and "Search deleted pages", "View deleted history entries, without their associated text" " View the checkuser log", "Check user's IP addresses and other information". (that is, they can run checks as well as see what was run), but they can't oversight things since they do not currently have "Review and restore revisions hidden from sysops". (ombudsmen have from time to time had the oversight permission flipped on somewhere temporarily to do things, the log at meta can be consulted to see that). CU and OV permissions themselves are not managed via global groups. Hope that's helpful information... ++Lar: t/c 14:16, 22 December 2008 (UTC)

Thanks Lar. The case with giving the Ombudsmen oversight has been (afaik) thorny, since the Ombudsman commission was formed to investigate checkuser issues, not oversight. One thing that occurred to me after a discussion yesterday is that the checkuser log does not show the results of a checkuser query. This may hinder users of the committee in investigating complaints. It's possible that they would be required to re-run checkuser queries to determine user intersections. (Caveat: While I've used checkuser elsewhere, if there are either Wikipedia-specific modifications or new revisions that correct this, I'd be unfamiliar with them.) For instance, if Peter says that checkuser Chuck blocked him for being a sock of Paul, the logs may be less than helpful for investigating this, since the results of the log would be needed. Perhaps in this case, full checkuser plus a bureaucratic solution ("The committee may perform checkuser queries only in response to complaints, and must keep a full, detailed public log of both the queries and results, only anonymizing IPs and unrelated usernames. There are no exceptions." &tc.) may suffice until a technical solution is found. Kylu (talk) 17:08, 22 December 2008 (UTC)

Kylu, firstly, the results of the checks are scrubbed after time, so if the check was performed long enough ago in the past, outside of off-wiki records, there is nothing access to the actual checking may do. Secondly, I am certain that all active CU's would be happy to supply the necessary results to the committee should that be necessary, so I still would prefer to have the new class only having log access for the committee. -- Avi (talk) 17:16, 22 December 2008 (UTC)

I'm aware, but thanks for the serious consideration! My thought was more along the lines of performing the same checkuser query as the original, assuming either the same editing habits of the query targets hold together, or if the complaint is brought in time before the results fall off the table. Another thought: Perhaps I've missed it, but has anyone given thought to granting these users access to the appropriate mailing lists instead of merely "To be shown all e-mail and chat correspondence that may shed light on a given matter" ? Kylu (talk) 17:41, 22 December 2008 (UTC)

That is an idea, but what just came to mind now is that access to those lists perforce allows the auditors access to that private information discussed on the lists and in the archives. If the idea is to prevent the dissemination of that information outside the people entrusted with it (CU/OS/or ArbCom) without cause, allowing open email list access goes against idea. -- Avi (talk) 17:57, 22 December 2008 (UTC)

Name (2)

Changed the name of this section from "Name" to "Name (2)" to avoid duplicate sections with the same name. Rjd0060 (talk) 05:31, 22 December 2008 (UTC)

Just a quick unimportant thought - can you call it something other than the Review Board? I don't want to call the panelists Reviewers. I prefer Auditors myself. --Tznkai (talk) 05:27, 22 December 2008 (UTC)

Audit committee sounds fine. -- Avi (talk) 05:29, 22 December 2008 (UTC)

Anything seems fine. I personally don't have a problem with the current one. - Rjd0060 (talk) 05:31, 22 December 2008 (UTC)

I have no particular attachment to the current name; it was picked mostly by virtue of not including "Oversight" in the name to avoid confusion. — Coren ^(talk) 05:34, 22 December 2008 (UTC)

Could we call it "The Oversight Committee to Check Users and Arbitrate Disputes?" -- Avi (talk) 05:39, 22 December 2008 (UTC)

Audit seems best, especially because Reviewers is likely going to be taken soon with the new Flagged Revisions thingy. NuclearWarfare ^{contact me}_{My work} 06:49, 22 December 2008 (UTC)

Its an accurate term but conveys too much of "official looking for faults" rather than "a collegial means of checks and balances agreed between colleagues". Hence not so keen on using the word "audit" in this process. FT2 ^{(Talk | email)} 12:54, 22 December 2008 (UTC)

Nerf Warfare Committee then?--Tznkai (talk) 16:30, 22 December 2008 (UTC)

Audit committee has a specialized meaning within the business world, as does an audit. Based on the proposed scope of the Review Board, associating it with audit may be confusing to people as to its role. It's easy to see how some people may believe that the review board is performing a financial statement audit of the foundation. Also, audit's tend to be involved the performing of work on a random test basis, not investigating specific privacy complaints. A name's an important thing, if we called the arbitration committee the trout slapping committee, people might get the wrong idea about its intended purpose. Fraud ^{talk to me} 22:06, 27 December 2008 (UTC)

Current state

Incidentally, the current state of the proposal (as of 05:41, 22 December 2008 (UTC)) is in perfect agreement with my vision for the Board. Thank you for all the help in cleaning it up and clarifying it. I'm going to bed soon, but I'll be back tomorrow to continue the discussion. — Coren ^(talk) 05:41, 22 December 2008 (UTC)

c== New sentence ==

I've added: "Candidates should be users in good standing who have not previously served on the arbitration committee, or as checkusers or oversighters."

Anyone having served in those roles could be seen to have a COI, so it's best to choose from an independent pool. SlimVirgin ^{talk|contribs} 05:50, 22 December 2008 (UTC)

I disagree. Making that statment implies bad faith about CU/OS/ArbCom that they would rather "cover up" for "one of their own" than have the project's best interests at heart. I resoundingly disagree with that. They should not be serving CURRENTLY nor have been removed under a cloud, but to disallow them at all is both going too far with assumption of bad faith as well as removing the most skilled people able to "sniff out" the issues. -- Avi (talk) 05:54, 22 December 2008 (UTC)

I dunno. Can you see a former Arb/CU/OS actually trying to run AND getting past the community...? rootology (C)(T) 06:00, 22 December 2008 (UTC)

Avi, the reason this policy has been proposed is precisely because it's inappropriate for people to investigate their own, as it were. If you resoundingly disagree with that, you can reject this proposal. It simply makes no sense to allow former ArbCom members to join. SlimVirgin ^{talk|contribs} 05:59, 22 December 2008 (UTC)

Please either remove it or leave it, but adding this makes no sense at all -- "Candidates should be users in good standing who are not currently serving on the arbitration committee, as checkusers or oversighters." SlimVirgin ^{talk|contribs} 06:01, 22 December 2008 (UTC)

Of course it does. Only sitting arbs/CUs/OSs are "their own." Now it's true that CU/OS bits are not subject to terms now, so the difference is moot. But as relates to ArbCom, I do not see there any problem with an Arbitrater emeritus coming in as "Special Investigator" if necessary. -- Avi (talk) 06:02, 22 December 2008 (UTC)

Someone who resigned from ArbCom last week shouldn't this week be regarded as suddenly no longer in COI. It also looks a bit silly to say members shouldn't be current arbs or CUs, because it's obvious. SlimVirgin ^{talk|contribs} 06:04, 22 December 2008 (UTC)

Slim, remember, the community has to vote people into the pool. If the community has no issue with any perceived COI from a now-resigned Arbitrator, why do you? Also, when it comes to wikipedia bureaucracy, sometimes the obvious needs spelling out. -- Avi (talk) 06:06, 22 December 2008 (UTC)

Yes, that's a fair point, but we've made mistakes before. I just think it's best to specify upfront that we want this board to be as conflict-free and as independent as we can make it. SlimVirgin ^{talk|contribs} 06:09, 22 December 2008 (UTC)

And we can do so without removing via fiat a class of people who may make excellent participants of this committee. As I said, there really is very little difference between our two proposals. It has to be a person, such as Thatcher, who has given up the CU/OS bit and is not a sitting arbitrator. I think those people should be allowed to be accepted by the community. I agree that everyone with an active CU/OS bit, and every sitting arbitrator, should not be on this committee. -- Avi (talk) 06:13, 22 December 2008 (UTC)

Agree with Avi. Limiting to only those who have never held the CU bit, for example, seems excessive. (also, what of experience on other wikis? Would holding the CU bit on, say, de:wp, preclude being on this committee?) ++Lar: t/c 06:43, 22 December 2008 (UTC)

We've had a problem for a long time in this community of members of a very small group of users being placed on ArbCom and given access to checkuser and oversight, simply because they're friends of people who already have it. This group has largely revolved around IRC. There are exceptions, of course, and there are also IRC regulars who are perfectly trustworthy, so please don't jump on me for making sweeping generalizations. The point I'm trying to make is that there's a perception abroad, rightly or wrongly, that access to the tools is unfairly controlled by a tiny number of people who are friends of each other.

With this recent election, that situation has finally changed. Jimbo's post-election statement indicates that he recognizes this, and has no problem with it. I'd like to see the process continue whereby elections for positions of responsibility are opened up to ever wider swathes of the community, where you don't need to have been (or become) an IRC regular, or be friends with the right people, or have the blessing of Jimbo or the ArbCom.

What I think would happen, if a former checkuser (for the sake of argument) were to stand for this board, is that the same group of users would come out in force to ensure his election, because he'd be someone they'd feel (again, rightly or wrongly) wouldn't spill the beans too much. It would also place the person himself in an awkward position, because we'd be asking him to act as though his friends weren't his friends; and no matter how independent someone feels they are, very few people can do that.

The fact is that, as Thatcher has pointed out, there is a degree of (arguable) misuse of the checkuser tool. All current checkusers are on the checkuser mailing list. Therefore, they could all be doing something about that misuse. If they're not doing something about it, it's perhaps because they feel constrained by the atmosphere of camaraderie. What reason do we have for supposing they'd necessarily feel less constrained one week, one month, one year after giving up the tool? SlimVirgin ^{talk|contribs} 06:50, 22 December 2008 (UTC)

Slim, I find that to be an example of an assumption of bad faith on the part of all current CU's and OS's. There isn't always a conspiracy around every corner, no matter how hard you look. And again, the community needs to vote these people into the pool, and it is ArbCom, also voted on by the community, who is making the selection from the pool. For your accusations to be valid, you need to believe that there is a vast conspiracy to rig both the ArbCom and Audit pool elections, and then have said ArbCom pick their "cronies". I do not think anything near like that is the case, IRC or no IRC. -- Avi (talk) 06:57, 22 December 2008 (UTC)

And, Slim, the reason "we have for supposing they'd necessarily feel less constrained one week, one month, one year after giving up the tool" is that people are assumed trustworthy and honest unless shown otherwise, it's called "good faith", and these former CUs/OSs (which outside of Thatcher barely exist) and former Arbs, unless removed for cause, have already demonstrated to be trustworthy and honorable. -- Avi (talk) 06:59, 22 December 2008 (UTC)

We need to move beyond the AGF meme. AGF is fine when we're editing or interacting casually, but it's no use when it comes to these positions of responsibility. We don't insist that people AGF instead of having jury selection done in the open. We don't criticize people for asking judges to recuse when there's COI because it shows bad faith in the judiciary. Let's grow up as a community, stop assuming good or bad faith, and put in place the kinds of checks and balances that people take for granted in real life. SlimVirgin ^{talk|contribs} 07:13, 22 December 2008 (UTC)

I actually agree with that. AGF only can take us so far. Nevertheless the restriction you propose is not one I can support. ++Lar: t/c 07:16, 22 December 2008 (UTC)

Slim, while AGF may only go so far, seeing conspiracies under every rock is inappropriate. There is no reason to think that someone who now has no access to the CU/OS/ARb information is irrevocably "damaged" because they once did. A valid conflict of interest exists with people who HAVE the bits now. No valid CoI exists with people who once had the bits, outside of some conspiracy theories. Let the community decide if they believe the former arbitrator is acceptable, Slim. What you are suggesting is not a protection against a conflict of interest, but protection against what you believe is a cabal, and that is inappropriate. That is a meme we need to move beyond as well, Slim. -- Avi (talk) 07:24, 22 December 2008 (UTC)

No one is seeing conspiracies under every rock. That's another meme: asking for safeguards is failing to AGF, and failing to AGF is like being a conspiracy theorist.

An example of my concerns is that, as things stand, an ex-Arb would be on the ArbCom mailing list. As I see it, there's no way that someone from that mailing list would be acceptable as a review board member. SlimVirgin ^{talk|contribs} 07:37, 22 December 2008 (UTC)

Being that one of the items under discussion is that the committee gets put ON the mailing list, that is somewhat confusing, Slim. If you were on it, you cannot get on it, but if you were not on it, you can be on it? -- Avi (talk) 07:42, 22 December 2008 (UTC)

That's exactly right. Each person allowed to wear one hat only. :-) SlimVirgin ^{talk|contribs} 08:45, 22 December 2008 (UTC)

SlimVirgin, there is a lot to respond to there, and it's late where I am. Let me just say that if you're at last coming out against cronyism, that's really great news. I hope this new view of yours encompass everyone's cronies, though. Including your own. ++Lar: t/c 07:02, 22 December 2008 (UTC)

I see both sides of this one, but tend to agree with SlimVirgin's assessment. You could hope that on a board like this the exes will become independent, but it would be hard to see them entirely in this way. More than good faith, the issues I see are 1.) true independence, 2.) an ability to evaluate any issue without concerns of past involvement via these positions, 3.) a lack of constraint based on the member's own actions or past immersion in the culture under examination. Certainly I can think of enough others who are qualified to occupy the position. I suppose the question may be whether enough are willing to hold the position, but if so, a group without history in these positions would seem far preferable. Mackan79 (talk) 08:31, 22 December 2008 (UTC)

The point about prior immersion in the culture is a crucial one. People actually don't know to what degree they're affected by this; they could be acting in the best of faith in thinking they can be neutral, without realizing that they're far from that. For example, if it's currently accepted that checkusers always do X, where X is strictly speaking against Foundation policy, anyone who currently has the tool is likely to see X as not a big deal. What we need are people not immersed in the culture to make that judgment. SlimVirgin ^{talk|contribs} 08:49, 22 December 2008 (UTC)

I don't have any issue with SlimVirgin's addition (provided it is worded in terms of "For the purposes of independence" or "To improve neutrality"). I think it may prove redundant as several point out. However, if it does have value, then the value is in spelling out a neutrality point, even if that point was never likely to be tested in practice. FT2 ^{(Talk | email)} 12:51, 22 December 2008 (UTC)

I'm fine with a requirement that there be a gap between AC/CU/OV and membership in this review board. 6 months, a year? Sure. But never? I don't think that makes sense, it cuts off too many wise and experienced folk from consideration. Previous membership should not be a requirement, (I can think of some great candidates for this board who are not and never have been any of these things) but it should not be a bar either. Also I think the pools are reversed. Instead of the community making a pool from which arbcom selects, it perhaps should be the other way round. And if arbcom doesn't give sufficient candidates, or refuses to pass a particular one, that's grounds for complaint, perhaps? ++Lar: t/c 14:23, 22 December 2008 (UTC)

Also whatever happens here with eligibility, I think the "experience on other wikis" question I raised above needs addressing... en:wp does not exist in a vacuum, and some definitive statement should be included to forestall later issues (I of course don't think that other wiki experience should be a bar, and am open on concurrency...) Also there was discussion of access to the Arbcom-l list. Discussion of access to the Checkuser-l list is probably also needed. ++Lar: t/c 14:26, 22 December 2008 (UTC)

I think a gap between release of the bit and access to the e-mail lists and appointment to the committee is appropriate as well, 3 months, 6 months, something reasonable, but a permanent ineligibility is too much. -- Avi (talk) 14:45, 22 December 2008 (UTC)

• The right people will be able to do the job whether they are one day or one decade away from actively serving (as CU, OS or Arb). The wrong people will never be right no matter how long a break they've had. Thatcher 18:13, 22 December 2008 (UTC)
  • semitechnical question: how would someone know enough about checkuser to be able to investigate its abuse if they have not previously had experience with it?DGG (talk) 03:18, 23 December 2008 (UTC)
    • There may need to be a provision either for training for reviewers who have not previously had the tools, or they will have to rely on the experienced CUs on the board to provide summaries and info for the non-experienced CUs. Thatcher 04:58, 24 December 2008 (UTC)

What if you set an upper (but not lower) limit on how many slots could be filled with past Arbcom/Checkuser/Oversighters? The limit could be expressed as X seats or X percent of seats. I think it could help balance SlimVirgin's concern about independence with the benefits of having some members with experience in the processes being reviewed. alanyst ^/talk/ 05:36, 23 December 2008 (UTC)

• Ex-arbitrators would probably not be good people to review arbitration committee internal matters and arbitrator conduct. Assuming that Arbcom itself remains in the remit of this board then any ex-Arbs should recuse from such matters. Otherwise, there should be no problem having ex-Arbs reviewing CU and OS as long as they aren't buddies with the subject of the case, in which case they would also be expected to recuse. (Here we should not assume the other members of the board are potted plants, they will know when one of their own should recuse.) And note that removal of Arbcom itself from the jurisdiction of this board is under discussion. Thatcher 05:02, 24 December 2008 (UTC)

WP:BUREAUCRACY?

I'm not sure I see the use of this. Some people are bound to distrust the review board. Are we going to put in place people who review the reviewers and what about those people themselves? Looks like an endless circle of reviewers to me. Since arbitrators serve limited terms, I don't think this is useful or productive. If there's something serious, someone should appeal to Jimbo. -- Mgm|^(talk) 05:54, 22 December 2008 (UTC)

Jimbo mandated a host of changes to privacy and oversight of how things are done on his talk page when he named the new committee. This proposal is crazy and new, because its the Arbs basically saying "Sure thing, and we'll put the community in charge of keeping us in check." This is the furthest from a bad thing as is possible, and not even the most pro-pro-pro-pro Divine Rights of the Arbitration Committee sort of person can complain, since it came from Jimbo, and this is what the Committee we elected chose. rootology (C)(T) 06:04, 22 December 2008 (UTC)

Apparently he does, but arbcom members making unpopular decisions may well be making the right decisions. I suggest that rather than building a process for unpopular arbcom members as Jimbo put it, to make a recall process for members that actually do something wrong (COI, bad editing practices, or the mentioned opaque voting). If we were to allow forcing the resignation of unpopular arbcom members people with a chip on their shoulder could disrupt arbcom and kill the process. - Mgm|^(talk) 08:16, 22 December 2008 (UTC)

I think the issue is basically that you don't want to give completely unlimited leeway to a decision maker just because an issue involves private information. Traditionally, we've had the same people deciding that something doesn't need to be heard publicly, and then also deciding the merits of the issue. Or we have private rooms, where single individuals are able to look completely unilaterally at private information, with little or no meaningful review or evaluation of their actions. Wikipedia is based on the merits of transparency, but in these situations there isn't any, which suggests a particular need for some type of uninvolved review. Mackan79 (talk) 08:44, 22 December 2008 (UTC)

The balance of power in the proposal is quite carefully considered. The community chooses those it trusts, and then Arbcom chooses from within those the community trusts, whoever it feels safe to give privacy access to. The review board prepares a report that it alone decides circulation of (obligatory to Arbcom and Jimbo) and which names findings, recommendations and reasoning, but itself does not mandate what shall be done (I'd expect them to be taken seriously though and it includes making a finding someone should go). Review board members once appointed cannot easily be "fired", and especially, Arbcom, Checkusers and Oversighters have no ability to speak on this. Jimbo as back-stop is fine by me. Jimbo as actual "doer" is not because I can't see him watching and keeping an eye on things, performing the detailed investigatory work, or having these areas under his watch generally, which is what's needed. FT2 ^{(Talk | email)} 12:46, 22 December 2008 (UTC)

See above, I think the pool ordering is the wrong way around. Arbcom first, community veto (with a possible "Hey arbcom, explain why not this person" feedback loop) rather than other way around. ++Lar: t/c 14:35, 22 December 2008 (UTC)

That way it's basically "arbcom's nominees who the community gives a nod to". Not what's needed. On this, you want as much independence as possible - the pool should be the community's open choice of nominees out of which Arbcom then picks users that it feels confidence in, under the WMF mandate to responsibly allocate WMF tools. FT2 ^{(Talk | email)} 16:00, 22 December 2008 (UTC)

Yes. Active CU's are now chosen by ArbCom and the community has an opportunity to give their input. The audit/review committee should be the other way around, with ArbCom selecting from a pool of community pre-approved auditors. -- Avi (talk) 16:03, 22 December 2008 (UTC)

I see your point but I fear ArbCom blocking folk who are perfectly suitable. That actually can happen either way round, I'm afraid, so I don't know how to resolve it. Perhaps the process needs to be entirely independent of ArbCom then, but that's not my preference. By and large I'm in the "I trust these guys (AC) mean well. Although maybe sometimes they didn't do the right thing, it wasn't out of venality" ++Lar: t/c 16:41, 22 December 2008 (UTC)

All candidates in the pool are "considered suitable by the community". Whoever is chosen will be someone trusted by communal voice for the job, regardless. FT2 ^{(Talk | email)} 16:51, 22 December 2008 (UTC)

Agreed, but as this body may have to review ArbCom itself, I think propriety is better served by letting the community speak first. -- Avi (talk) 16:57, 22 December 2008 (UTC)

Case handling nits

In general I like this quite a bit. I've tightened up what I saw as some perceived nits in the case handling section... acceptance is a majority of the total (alternatively, it could be a majority of the quorum, if desired) and recusal should happen as early as possible (possibly before acceptance) and in any case, changes the effective majority. Revert me or discuss if you think I erred... these seem fairly uncontroversial changes. ++Lar: t/c 06:41, 22 December 2008 (UTC)

Outright Strong Oppose

Yet another bureaucratic process.   «l| Ψrometheăn ™|l»  (talk) 08:03, 22 December 2008 (UTC)

Agree. Stifle (talk) 10:42, 22 December 2008 (UTC)

Agree. Necessary bureaucracy, though. Our current democratic systems of oversight aren't working. AGK 11:51, 22 December 2008 (UTC)

• People who missed it should read my essay User:Thatcher/Quis custodiet ipsos custodes which describes exactly why there is currently no meaningful and effective method of monitoring checkuser and oversight performance. I never contemplated reviewing Arbitrators, though. Thatcher 18:04, 22 December 2008 (UTC)

Police, a court system, and fuck me sideways, a sort of IAD. Still, it at least provides another forum for drama for those who enjoy it, naming no names like. I prefer my idea better. Prevent power blocs by rotating power. A House of Lords was a terrible idea. A select committee is probably worse. Given that it's likely to consist of the same people as the nobility, well, the problem's obvious.Grace Note (talk) 09:16, 29 December 2008 (UTC)

Alec's Guiding principles

Glad to see so much work on this already! Good job all..

Thoughts:

• 1. Maximum Independence of the Review Board. Basic organizational dynamics. You need as much social and organizational distance between oversight and those under review. That means Jimbo and Arbcom need to have as little as possible to do with this process. If money were no object, this is the kind of thing we'd hire and outside firm for. Right now, that says direct-election-by-community to me. But we might also consider asking the German wikipedia to aid in the selection process somehow.

• 2. No New Jimbo Mojo. This is a standard argument I make. As Wikipedia becomes more and more powerful, we need to be divesting power from single individuals. Sooner or later, Jimbo is going to be the subject of a major PR scandal, go all Caligula on us, or unexpectedly die. As time continues, the probability of one of those events occurring approaches certainty. WMF, Arbcom, and the Commmunity needs to shoulder as much of the decision making as possible. There's no harm whatsoever in Jimbo being a leader of the community, just as there's no harm in that special je ne sais quoi that New York Brad has. But we need leaders, not rulers-- and until we get home rule, everything could still fall apart.

• 3. Lack of Community Confidence Currently, it seems like the review board only reports on abuse of power. I suggest this is only one of two potential reasons that a CU/OS/ARB/ADMIN might need to be removed. The other would be a crisis of confidence. There is some level of community having "no confidence", lying somewhere between 50% and 100%, where somebody should step down even if they sincerely didn't do anything wrong. You have your work cut out for you, trying to decide where that line should be, when and how such confidence should be assessed-- but as long as you're going to consider the removal of authorities, you might want to be considering this too.

Good good work getting a jump on this discussion. Never put off to June what can be discussed in December! :) --Alecmconroy (talk) 09:57, 22 December 2008 (UTC)

Instead of asking the German Wikipedia alone, perhaps ask the five largest projects to elect one person each who is fluent in English and not a significant contributor to the English Wikipedia? A side benefit to this would be to defuse some of the anti-Enwiki sentiment in other projects. Kylu (talk) 02:28, 23 December 2008 (UTC)

Even better! There's nothing special about the German Wikipedia, after all, aside from its size. --Alecmconroy (talk) 17:31, 23 December 2008 (UTC)

Oppose... but one comment

I oppose this proposal as yet another piece of useless bureaucracy, thoroughly redundant to the Ombudsman Committee authorised by the Foundation (which monitors these tools for breaches of the privacy policy). However, I should point out that with recent improvements in CentralAuth, the system that manages our global logins, the Stewards now have the power to slice and dice permissions for any wiki in whatever fashion required. So it would be easily possible for them to create a new global user group which grants access to the checkuser and oversight logs without access to the functions themselves. See documentation. Happy‑melon 11:45, 22 December 2008 (UTC)

But they can only manage global groups? Ruslik (talk) 11:53, 22 December 2008 (UTC)

See above. CU can be sliced/diced that way already, and a global group that has JUST en:wp as a member wiki can be created, but it may take a small config/code change to slice/dice oversight. There are two permissions in the code, but I think global groups manages both. Happy to be found out wrong of course. I may actually create such a group and carry out a test, on some random wiki, have to consult with other stewards first. ++Lar: t/c 14:28, 22 December 2008 (UTC)

Unfortunately this isn't so. The Wikimedia Ombudsman does not monitor any of these except specifically, breaches of privacy policy. There is nobody and no person, at WMF or anywhere else, who currently can investigate or watch these things except Arbcom itself. FT2 ^{(Talk | email)} 12:36, 22 December 2008 (UTC)

• People who missed it should read my essay User:Thatcher/Quis custodiet ipsos custodes which describes exactly why there is currently no meaningful and effective method of monitoring checkuser and oversight performance. I never contemplated reviewing Arbitrators, though. Thatcher 18:04, 22 December 2008 (UTC)

What happened to 'trust'

I thought the guiding principle of user rights on Wikipedia was always that they are 'no big deal', just a few extra functions given out to those trusted not to abuse them. Now, that was intended mostly for adminship, but there is no reason to abandon it wholesale. Here we are, saying that we don't trust the people who are checkusers and oversights (and if so, why are they checkusers/oversights) and moreso we trust the people we propose to watch them even less, saying that they will be removed from the 'Review Board' if "They perform a checkuser or oversight operation for any reason". Why would this ever be a good idea, it is simply far over complicated bureaucracy. Prodego ^talk 13:42, 22 December 2008 (UTC)

To address your two primary points: I don't think that the existence of the board would be to say "We don't trust AC/CU/OS editors", but to provide the means by which the community at large can be confident that the trust is well-placed. Like it or not, those bodies are the police, lawyers and judges of our community and confidence in them is necessary for everyone.

As for the "no perfoming of CU or OS", it's strictly a question of separation of powers. The board is meant to be basically immune to being ousted by the groups they watch— certainly this means that they should not be in a position to break the very rules they are meant to monitor compliance to! That's why I'm also in favor of creating a user right group that gives access to the log without the tools— that removes the question entirely. — Coren ^(talk) 16:09, 22 December 2008 (UTC)

An extra access level sounds helpful. I think (and I could be wrong) that setting up such an access level for viewing the oversight log would be quite easy, since Special:Oversight (the log itself) is actually seperate from the revision hiding too, Special:HideRevision. The checkuser log Special:Checkuser/Log is actually integrated into the checkuser tool so might be a bit harder, but I'm not a developer so I don't know. Certainly anyone who has this right should be trusted too, since they'd be able to see all the sensetive information.

Also, I agree with Coren that the existance of this board would not say that we don't trust oversight/checkuser users anymore than the existance of the Arbitration Committee says that we don't trust administrators/users, which obviously isn't the case. :-) --Deskana (talk) 16:13, 22 December 2008 (UTC)

Assorted edits

I checked what had been added and removed. Most copyediing was good; a few babies got mislaid with the bathwater:

• Explicit "if a query arises in routine patrolling, they may investigate"
• Added back that cases "may be considered in light of other cases, or incidents; that is, a pattern of insufficient incidents may add up to evidence of a concern. The board may reject cases which appear after a brief check to have little likelihood of an adverse finding." [allows quick judgement of tendentious or trivial claims]
• Added (avoidance of doubt) key points that don;t speak to neutrality: "Bringing an issue to board attention, proposing the opening of a case on a matter, and minor matters that would leave the member uninvolved for administrative purposes, are not by themselves, grounds requiring recusal"
• Added "Cases are private and decorum is expected until concluded" and ability to enforce.
• Added back "Jimbo Wales and the Arbitration Committee shall both receive a full copy and may at the board's discretion be given updates or other information during the case, and may take action at any stage". Disclosure that far should be automatic.
• "Report should be clear as to..." -- broadly speaking "facts" aren't enough. Too many such cases may hinge on misunderstandings, opinion, and judgements. They need making clear so that the report won't be taken as one sided.
• Jimbo informed but also, his comments noted.
• A years break after any stint (ensures no "appointments for life")
• Clarify the "no rankings" part better
• Specify when appointment is formal
• Remove duplication of removal criteria
• Reinstate "Jimbo can remove"; unless and until we resolve the concerns to the point we don't have this escape route for Arbcom we probably should have it here too.

FT2 ^{(Talk | email)} 14:01, 22 December 2008 (UTC)

The writing is unnecessarily wordy and unclear. If this is going to become policy, it has to be streamlined. You also seem to be trying to tell the board in some detail what it may and may not do, whom it may and may not consult.

Some of your additions are problematic for other reasons. For example,

"In case of doubt review board members may consult other enwiki checkusers, oversighters, or Wikimedia Foundation Ombudsmen, to determine whether or not others would view the incident as within their understanding of the policy."

It seems to me that either there's no point in saying this, because it's obvious that board members can consult whomsoever they choose to consult; or you're trying to pave the way toward establishing current checkuser practice, which is sometimes in clear violation of the Foundation's policy, as some kind of baseline or best practice. SlimVirgin ^{talk|contribs} 14:09, 22 December 2008 (UTC)

I'm looking at this proposal and thinking "where are the real problems going to be in practice". Wishing for "streamlining" is good, but we found communally that just saying "be neutral, don't be a dick, ignore all rules" doesn't work well on this scale. The main grey areas really do benefit from being mentioned, and there really are points above that are important. I take the point of keeping it short where possible; those additions were not vast in size but I think are important. FT2 ^{(Talk | email)} 14:30, 22 December 2008 (UTC)

To SlimVirgin: "Clear violation" of foundation policy may be a bit of a rhetorical reach. Clearly there is some gray area here, and some allowance for difference of opinion is needed in cases where different reasonable people draw different (reasonable, but different) conclusions. Such cases can be lessons to us all for the future, but if there is a gray area, trying to fence it precisely can have a chilling effect. I think you present a false dichotomy with respect to FT2's edits and motivation for them. Perhaps the matter just needs clarifying, without impugning motive for them. Please assume good faith about people making suggestions here. ++Lar: t/c 14:33, 22 December 2008 (UTC)

There is no rhetorical reach at all. Checks are being conducted for no reason. Checks are being conducted in content disputes. See Thatcher's examples, and there are many others. The problem is compounded by Jimbo's feeling that checks should be conducted more or less at random. The bottom line is that checkusers should adhere to the WMF's policy. SlimVirgin ^{talk|contribs} 14:50, 22 December 2008 (UTC)

And in the vast majority they do, Slim. Out of the hundred, if not thousands, of checkusers run every month, which means possibly tens of thousands per year, how many outright violations have there been? How many "grey-zone" checks have there been? I do not have any hard statistics, but I'd say that the checks have been around 99.99% compliant, which is about as good a track record as can be expected from human beings. Improper checks, should be and need to be, corrected and prevented, but, in my personal opinion, you seem to be under a definite misconception as to the frequency and severity of the actual issues. -- Avi (talk) 15:23, 22 December 2008 (UTC)

SlimVirgin: There is indeed a gray area here and you need to acknowledge that there may be differences of opinion about validity. I'd point you to a certain case which you were involved in which made that point to you rather clearly. Don't try to retry that case here, and derail what is shaping up to be a good, positive step forward for this community, please. ++Lar: t/c 16:38, 22 December 2008 (UTC)

Quit the tiresome personal comments. SlimVirgin ^{talk|contribs} 16:41, 22 December 2008 (UTC)

My apologies if you find them tiresome, but I find them germane, and you need to internalise them, as apparently you have not. Speaking of tiresome, I found the entire case tiresome, frankly, as well as your approach to it... If you've turned over a new leaf, great but stop trying to fight old battles over again, OK? There IS a gray area around which checks are valid and which are not. Accept that, and make sure that the proposal accounts for it, or this will have a chilling effect to the extent that no one will be willing to run any but the most obvious checks. That's unacceptable, I think, as it will hamper normal operations in addressing even routine socking and block evasion. ++Lar: t/c 17:03, 22 December 2008 (UTC)

No one's in a position to say how many are non-compliant, because no-one's done an audit. What we do know is that there has been misuse, and that nothing is done about it when it's pointed out. SlimVirgin ^{talk|contribs} 15:33, 22 December 2008 (UTC)

They do. Whether or not one personally feels a checkuser can check up the work of another checkuser, that is the WMF requirement and norm, and largely so far as I can tell, they have sought to do so. FT2 ^{(Talk | email)} 16:05, 22 December 2008 (UTC)

I can see SlimVirgin's point. It was carefully worded. Nothing in that item says they must conclude anything or must agree with it. It says, "before assessing fully, identify what others see as a norm too". It is relevant. The views formed on whether an action was in good or bad faith for example (as opposed to right/wrong), might profoundly change depending if other checkusers also feel it's questionable or wrong, or if it turns out it's normal usual practice as they understand it too. We don't want a "hanging mob" issue, we do want to ensure standards are improved -- but adoption of widely held consensus standards per other checkusers shouldn't result in scapegoating. If there is doubt, members should check and identify what others feel are the norms operating in practice, as part of their work. It may well be the norm needs criticizing, more than the user. FT2 ^{(Talk | email)} 14:41, 22 December 2008 (UTC)

One thought. A number of these are more of the nature of "guidelines for the board". We have guidelines on many policies, to supplement them - deletion guidelines for admins, arbitration guidelines, wheel war examples, NPOV FAQ, and where there is no guidelines page (such as BLP or verifiability) often the policy is long or includes examples lists itself. So I'd be tempted to see some of these as "best practice or expectations of the board" more than "policy about their role". Might that provide a route to keep what's useful but also streamline the policy? FT2 ^{(Talk | email)} 14:57, 22 December 2008 (UTC)

I wouldn't mind having an explanatory page, so long as there's nothing that implies what checkusers currently do is okay or "best practice," or consistent with community "norms," because it begs the question and misses the point that the Foundation has a checkuser policy. SlimVirgin ^talkcontribs 15:03, 22 December 2008 (UTC)

Indeed. It should not imply that any given norm or action is okay. It should simply assert that information and comment from other checkusers/oversighters, should be obtained in cases of doubt - as evidence what's going on, as evidence if there is a systemic problem, as evidence if a WMF requirement needs more vigilance, whether it was a good faith following of a bad norm or a bad faith action, or whatever. The case may in fact highlight a systemic problem with a norm, and if it doesn't spot that and just describes it as the action of one user, then the opportunity to correctly describe the problem and fix it more generally, may be overlooked. FT2 ^{(Talk | email)} 16:32, 22 December 2008 (UTC)

"In case of doubt review board members may consult other enwiki checkusers, oversighters, or Wikimedia Foundation Ombudsmen, to determine whether or not others would view the incident as within their understanding of the policy." Actually the may was my addition, it originally read should. I support Slim's revert - it was bolder than I dared. I was trying the lightest of touch to remove FT2 making explicit prescriptive requirements of the board - surely they should proceed in whichever manner they think best? I'll be blunt and raise a concern here - the review board is hopefully an initiative to enhance community confidence regarding the use of checkuser and oversight. Given there are, as far as I'm aware, still unresolved and fairly high profile concerns regarding the use of oversight in relation to you FT2, do you think that editing the ground rules of a body, which might be dealing with such a complaint, is a good idea? --Joopercoopers (talk) 15:24, 22 December 2008 (UTC)

Actually, I think this speaks to transparency. I see no attempt here to weaksauce the board, and I would expect that particular oversight would have been found to be a Bad Thing by the board then if it had existed as well. — Coren ^(talk) 15:44, 22 December 2008 (UTC)

(edit conflict) JC - Yes, I do. My use of WMF tools is close to exemplary (as any checkuser/oversighter can check); I also checked a few weeks ago my own compliance with norms of documentation in the logs, to ensure others can recheck my work - it's close to 100%. (To be exact, I think my compliance is roughly 99.9%+; and both of the exceptions from June were evidence-able by other means.) Right now that's about the highest level of compliance of any major enwiki user of those tools, as best a cursory glance shows. I'm also a fairly seasoned developer of policy. Check out the list, it's linked on my user page. (2008 work is on the talk page.) Bluntly on something like this I endorse thoroughly the idea, and my aim is to ensure it delivers, and is not belatedly found to have any flaws that give rise to disputes by accident. I would have no hesitation in having the whole of my use of WMF tools, or indeed my every action as an arbitrator, reviewed with evidence by such a body, and if this policy passes and the body forms, it is welcome to review mine first (including OM and including every other matter) if anyone wishes and if they wish to accept. Any arbitrator, checkuser, or oversighter, should be able to make this commitment at any time, and with peace of mind. FT2 ^{(Talk | email)} 16:21, 22 December 2008 (UTC)

I'm not ready to write a blank check to any board at all, (it's still possible that the selection process definition could come out in a way that empowers the same bad old cliques to "pack the board" to settle old scores), but under any reasonable selection process, I too am willing to make such a commitment. As always. That doesn't mean endless witchhunt enabling or endless reasking of questions already answered, mind you. ++Lar: t/c 16:38, 22 December 2008 (UTC)

I also feel uncomfortable about that, but was worried about raising it. FT2 said he had no time to address the oversight issue until after Christmas, which is fair enough and it's why people have waited. Until that issue is addressed, it's perhaps not entirely appropriate for him to be writing a policy proposal about checkuser and oversight review, especially not one that aims to be so prescriptive. SlimVirgin ^{talk|contribs} 16:14, 22 December 2008 (UTC)

If the content is good, why does it matter if FT2 had input? It is not as if you anyone is unaware of what is being written here and does not have the ability to comment. -- Avi (talk) 16:20, 22 December 2008 (UTC)

(And thanks for the few days SV, much appreciated, and not overlooked.) FT2 ^{(Talk | email)} 16:25, 22 December 2008 (UTC)

SlimVirgin: You say "perhaps not entirely appropriate for him " (FT2) "to be writing a policy proposal" with regard to FT2's input. I think the same could be said of you as well, if it came right down to it, so perhaps that's not the best approach for you to take. You yourself have, after all, been chided in some quarters for giving the appearance of rewriting policy while embroiled in some controversy or another related to the very policy you were working on at the time. Fair impression or not, it's a view some hold. So my suggestion is to focus on the content, not the source. If this is good policy and workable, great (and I'm in general liking what I am seeing here so far, despite having some differences)... keep helping make it better without casting aspersions, please. ++Lar: t/c 16:32, 22 December 2008 (UTC)

Lar, a number of your posts to this page have taken a poke at me, and for no reason that I can see. Please stop it. I have no involvement in checkuser issues, except that I've been trying since 2006 to have something like this set up. Hence my presence on this page. SlimVirgin ^{talk|contribs} 16:37, 22 December 2008 (UTC)

Strikes me that you seem to be taking pokes at others, notably FT2. I am far from FT2's biggest fan but I'd say you should stop that. Look, we both want some kind of change here... and we mostly agree about what. Where we disagree is around the margins, not the main thrust. Stop trying to settle old scores (IMHO) and I'll stop calling you on it. But I'm not the one that raised a case about CU actions in the forum of public opinion and then got sanctioned by ArbCom about how they did it. I'm the one who suffered through a rather unpleasant and protracted case (delayed not by me, I complied promptly) and then was found by ArbCom that my checks were within acceptable bounds. Just try to remember that, please. All that said even though I did nothing wrong in deciding to carry out that check I've tightened up my personal standards anyway, because I took to heart that improvement is always possible. Set that all aside. Let bygones be bygones. I will if you will... this change is more important than either of us. ++Lar: t/c 17:16, 22 December 2008 (UTC)

I've watched you do this countless times to me and to others. I am not settling old scores, and if you think I am, please spit out which ones. But do it elsewhere please e.g. my talk page. As for the case that concerned you, it was a farce held in secret. It's exactly that kind of process the creation of this board might avoid in future. Now, I am asking you for the third time, please stop making personal comments about me. SlimVirgin ^{talk|contribs} 17:20, 22 December 2008 (UTC)

Peace. Let's take this elsewhere, as we agreed offline. This is too important to the community for us to derail it with a particular case. (A case we both surely can agree could have been handled better, and perhaps would have been, had this community already had this Review board at the time). Peace. As I say, we disagree about details, not the basic idea here. ++Lar: t/c 17:56, 22 December 2008 (UTC)

Wikipedia is a kingdom of ends, and the signature at the end of each line is a convince - not what is important. What will forever matter more than who.--Tznkai (talk) 16:34, 22 December 2008 (UTC)

Appropriate scope?

At first blush, I like the gist of this proposal, but there is a wrinkle I feel needs an iron. My apologies to the community for going on at such length.

As written, this proposal aims to create a group which would exercise an oversight role (not to be confused with Oversight) with respect to the actions of Checkusers (CUs), Oversighters (OSers), and Arbitrators (Arbs). I am concerned that creating a group with such overarching responsibility might be trying to solve too many problems at one go.

There are fundamental differences between the ways that CUs and OSers exercise their powers versus Arbitrators. CUs and OSers generally work as individuals. They may take on investigations and engage in actions of their own choosing, almost entirely at their own discretion. They may report their actions, or not, as they see fit. By the nature of their work, detailed descriptions of their work and the evidence on which they base their actions must remain private. CUs and OSers are appointed, not elected. While they are generally trusted members of the community, they are not required to demonstrate that trust in a formal process before assuming their posts.

Arbs, meanwhile, are generally not free to take independent action. Powers are delegated – by Jimbo and the community – to the Committee as a whole, not to individual Arbitrators. Decisions are taken collectively, and must be endorsed by a majority vote. The evidence and statements which they consider are generally public, as are their deliberations. Final decisions – to have any useful effect – must be announced. (Further openness is assured by the statements Jimbo made a couple of days ago.) The members of ArbCom have been selected through an election and approval process involving extensive discussion, harsh scrutiny, and massive community participation.

CUs and OSers generally perform a narrow set of well-defined tasks. Their actions are governed by specific policies and fairly-detailed guidelines. The expectations of the community are reasonably clear about how CUs and OSers should go about their business.

On the other hand, Arbs are expected to deal with virtually every type of serious and complex user conduct dispute. Issues go to the ArbCom only after the community has demonstrated a failure to resolve them by any other means. In even the best circumstances, at least some parties come away from Arbitration at least a little bit unhappy. The tools which the Arbcom employs are not primarily technical in nature (though many Arbs do have access to CU and OS tools and logs.) The tasks which Arbitrators perform defy the easy description of those done by CUs and OSers. The clear boundaries by which CU and OSer conduct is regulated are absent for Arbs.

This is a problem which is implicit in the proposal as it is presently drafted. Unlike the clear, easily-summarized CU and OS policies cited in Wikipedia:Review Board#Monitoring of checkuser and oversight, our Arbitration policy is only mentioned once as a throwaway line at the very end of the proposal. Reading the Arbitration policy, one notes that it is peppered with phrases like 'general guidelines', 'may make exceptions', 'at their discretion', 'common sense', 'usually observed'. The ArbCom is generally given extensive leeway to operate with maximum flexibility, for the good of the community.

This proposal offers a specific mandate to Reviewers to examine the conduct of CUs and OSers with an eye to their complaince with the m:privacy policy and m:checkuser policy. On the topic of Arbitrators, the proposal only offers that the Reviewers will investigate 'improper conduct' by Arbs or the Committee, and that they will examine actions 'which have caused significant community concern'. On this basis, they will be able to recommend 'removal of a user from the arbitration committee'.

While I think that it would be fully appropriate for such a Board to be able to examine the conduct of an Arb with respect to his use of the Checkuser or Oversight tools, expanding the mandate to include revisiting ArbCom decisions and unseating Arbs is both overreaching and unnecessary. The Board should have a clear and focused role examining the use of tools which are not readily monitored by the community at large. Attempting to tack on what amounts to an ArbCom Court of Appeals and a Request for DeArbcomship is...inelegant, and likely to sink this proposal in its entirety. TenOfAllTrades(talk) 16:34, 22 December 2008 (UTC)

There is a valid case for creating two different entities for both task, and you made it quite eloquently. However, there is also a case to be made for a single body, even if one of the tasks is defined (by necessity) more nebulously than the other:

• Philosophical scoping. This body is tasked with "Please watch out for us in regard to things we cannot check on our own."
• Reduction of bureaucracy. This proposal adds yet another "class" of users, a set of right, a selection process. I think it's a necessary evil, but I wouldn't like to do the same thing twice.
• Overlapping coverage. All three groups (CU, OS and Arb) maintain some level of privacy because of the nature of the information they deal with; and often the acts of one are based on the findings of the others. Not all cases, certainly, but there is a significant amount of entangling involved that is best handled by the ability to look at the "big picture"; all three at a time.

My thoughts, anyways. — Coren ^(talk) 16:43, 22 December 2008 (UTC)

I'm not sure that I'm convinced by the 'philosophical scoping' argument. While the ArbCom may consider evidence that is not publicly released, and their deliberations are not open to the public, in nearly all cases both their findings of fact and their remedies are published. Typically the bulk – and usually the entirety – of both evidence and party statements are posted right out in the open. The level of public participation and public exposure of Arbitration proceedings is quite different than that seen for Checkuser and Oversight actions.

Another concern I would have about a Review Board with a very open-ended mandate (investigating Arbcom actions which caused 'significant community concern'?) is the potential for what amounts to 'constitutional crisis'. The Review Board would not have the power to dismiss a sitting Arbitrator, only to recommend that one be unseated. What happens when such a recommendation is issued? I'll have to check my Cabal Handbook, but I think one of Jimbo's reserve powers is to remove members of the ArbCom with whom he is unsatisfied. Presumably the Board (of the Foundation) also has the de jure power, though I doubt that they want to get involved unless it is unavoidable.

What takes place at this stage is likely to be an unholy mess. The 'easy' outcome is to take the recommendation, and get Jimbo to implement it. In that case, the Review Board's authority is bolstered by Jimbo's credibility. Even then, however, there will likely be a vocal minority demanding to know why a tiny committee gets to axe Arbitrators who were legitimately selected by the community in open elections.

The 'hard' outcome is where Jimbo disagrees with the findings of the Review Board, and decides not to fire the Arb. Arb candidates undergo by far the most rigorous, time-consuming, involved, open selection screening on Wikipedia. When selected by Jimbo, they are clearly among the most respected, widely-acceptable, experienced candidates for the job. Firing an Arb is a difficult process, and requires – or should require – an assessment that their continued presence on the Committee is harmful to Wikipedia. If Jimbo chooses not to follow the recommendations of the Review Board, does it become a lame duck? Do all the current members resign, as Jimbo has essentially rejected their judgement? Do their past and future judgements fall into question? Is it likely to damage our one genuine safety valve (Jimbo himself) by sucking him into the fray of wikipolitics?

I'm also not clear on the purpose of the statement that the Review Board would investigate 'significant or habitual improper conduct by arbitrators or the committee as a whole'. As the proposal stands, the Review Board only has the power to issue recommendations to start RfCs and to strip Arbs, OSers, and CUs of their positions. The remedies avaiable to the Review Board don't seem to mesh with the scope of their investigation. What constitutes improper conduct of the committee as a whole? If it is the intent of this proposal to create an Court of Appeal (because people want to have another go at the Giano mess), we should be honest about it. If the intent is to compel the ArbCom to remedy some procedural or other issues, then let's be clear about that. I think it would really help if someone could provide some sample situations (real or hypothetical) which might require this type of investigation — or it should be dropped from the proposal. TenOfAllTrades(talk) 19:22, 22 December 2008 (UTC)

I certainly never contemplated having review authority over Arbcom. I'm not sure this would work. Hopefully Arbitrations cases decisions are not reviewable, so what's left? Being slow? Being obstinate and holding up discussion on the mailing list? Five people vote to remove someone who got 100 votes for office? Thatcher 20:01, 22 December 2008 (UTC)

At this point, the proposal is open for rework. I'm not particularly attached to the combined duties and while I think there is value in having a single point of contact when one has worries about improper handling of something behind the scenes, I'm also not willing to sacrifice both functions by demanding they remain tied.

There is still relatively little community-at-large involvement in the crafting of this proposal, and a wider perspective would be required before we settle on a strict scope in my opinion. I think segregating is premature, and arguments can still be made both ways, but let's wait for a while before we go down one specific road? — Coren ^(talk) 20:34, 22 December 2008 (UTC)

I am concerned that this "solution" for reviewing the Arbitration Committee and its individual members has not crystallised from within the community; the recent elections brought forth successful candidates who were all over the map with different proposals to address community concerns about the committee, and a review panel wasn't on the radar scope of many people. On the other hand, the proposal for a review panel to look specifically at CheckUser and Oversight functions seems to have broad support throughout the community, both by the response to Thatcher's essay and in the selection of successful candidates. I would rather see the two concepts split off, and proceed with the CU/OS review panel now, with further discussion within the community about how best to proceed on the issue of Arbcom members. Risker (talk) 20:45, 22 December 2008 (UTC)

Given that it is eminently possible and reasonable for the community to decide, as it visits the issue, that the monitoring of ArbCom activity does best fall to that same board I have no objection if we reworked the current proposal to exclude that aspect of the task for the time being. My only worry is that this discussion then imperatively needs to occur before the community picks suitable candidate for this board as ex post facto expansion of their role puts into question their selection. — Coren ^(talk) 21:39, 22 December 2008 (UTC)

Powers and other concerns.

I couldn't find a section to put these concerns in, so I've started a new one.

• Unlike other Wikipedia permissions, panelists immediately lose checkuser and oversight permissions upon the end of their terms.
• I recommend staggered six month terms. I suspect this job will be hard, boring, and have a subsequently high burnout rate. There isn't a lot of "skill" here, just a lot of "legwork." so the terms can be short.
• Panelists should not be removed by other panelists - if one panelist is raising a stink over what they think is an impropriety that panelist needs to be protected.
• A majority of the panel should pass most business, and the vote totals at least should be publicly accessible.
• Panel should be empowered to direct a steward to remove permissions by unanimous, on wiki, signed vote. The matter can then be fast tracked to ArbCom for review.
• ArbCom oversight is trickier - who does the panel refer the matter to? Again, unanimous vote to remove - then fast track to Jimbo's desk perhaps.

--Tznkai (talk) 16:44, 22 December 2008 (UTC)

Agreed with reservation: I don't feel there's quite enough abuse to investigate to merit six-month terms. If we have that much rights abuse from a group of forty individuals in a span of six months, there's more of a problem than any panel of five can handle. Also, I agree with your opinion on the removal of panelists but we still need a removal process. I don't have any ideas that are corruption-proof to deal with that, however. Kylu (talk) 17:24, 22 December 2008 (UTC)

On arbcom oversight, I'd suggest the review board be empowered to immediately remove an Arb from active duty pending further review. Active duty could be restored by WMF, by the review board, by community reconfirmation, or with the review board's consent, the remaining members of Arbcom. --Alecmconroy (talk) 17:22, 22 December 2008 (UTC)

On arbcom oversight, I think that should require a unanimous vote to remove an arbitrator, temporary or otherwise: think about it this way, the review board is overturning a (semi)-democratic process - if you can't get five people to agree to that, you have no business doing it.

As to the panel I think six month terms are required to build up a rapport with the committee, and if the past is any indicator, what will happen is there will be three months of nothing - followed by a month of flooding complaints.

Keep in mind, we create these powers under the hope/expectation that they will never be used the mere presence of little o oversight should correct what errors have been cropping up.--Tznkai (talk) 17:41, 22 December 2008 (UTC)

No need - this board will carry a lot of weight in its findings. If it makes recommendations, they'll be taken seriously by their nature, and acted on. FT2 ^{(Talk | email)} 17:39, 22 December 2008 (UTC)

there are two weak needs, which combine to make it necessary for the panel to have some sort of statutory power. First, it is conceivable that the panel, in its investigations will discover an Arbiter, checkuser, or oversighter is obviously unfit for duty - a compromised account for example - the panel needs some sort of timely but reversible mechanism to prevent harm. Second, part of the need of this review board is to provide transparency and assurance to the community, and people tend to put more stock in statutory power than the power of persuasion. See my above comment on actual use of these proposed powers.--Tznkai (talk) 17:57, 22 December 2008 (UTC)

I tend to agree, going against a recommendation to remove privileges from a user by the board should be, at best, a Big Deal and the presumption should be that their word is binding. — Coren ^(talk) 19:24, 22 December 2008 (UTC)

Grandfather clause

We should decide whether we want to add a grandfather clause, e.g. that the board may investigate complaints about issues only from the date the board is created. Or do we want to allow complaints about checks that were performed before the board was set up? Or only complaints where the alleged misuse came to light after the board was set up, but where the incident itself may have taken place earlier? SlimVirgin ^{talk|contribs} 16:48, 22 December 2008 (UTC)

Well, I don't see why we should limit the ability of the board to look into the past, but it needs to be relevant and not just digging for random crap. How about only complaints of alleged misuse involving someone who holds the right to this date? It's pretty much moot otherwise anyways— there is nothing to be gained but drama by looking into the behavior of somebody who doesn't hold the right and no longer has the possibility of misusing the tools anymore, is there? — Coren ^(talk) 16:59, 22 December 2008 (UTC)

Maybe trust the members to make the decision? They surely won't want every possible claim back to 2003 re-raised, and they are seasoned users. A formal line wouldn't hurt, since there are any number of old wounds some users may wish to dig up. The main reason a grandfather clause might be seen by some as helpful is to draw a clear line and make sure it doesn't suddenly go back to revisit old closed or stale matters in a seemingly vindictive or whimsical way.

(For the record I have no problem with my own actions - any of them - being scrutinized this way; any arbitrator, checkuser or oversighter should be able to say so and have peace of mind. I therefore abstain on this point below, with an understanding that if such a clause were decided I would voluntarily not utilize any protection it might provide. That's my own personal stance though, not really relevant to the discussion.) FT2 ^{(Talk | email)} 17:01, 22 December 2008 (UTC)

Let the panel create and publish a rule of thumb - "We consider investigating complaints more than a year old to be difficult because of the problems with deteriorating memories."--Tznkai (talk) 17:18, 22 December 2008 (UTC)

Yes, though the difficulty with a limit of a year is that it further encourages CUs not to tell people they've been checked. Also, Coren, limiting it to those who currently have the tool wouldn't really limit it at all, as most people who have it keep it, even if they don't use it. I'm fine with not limiting it at all, by the way, but I was thinking it might look fairer if we have some kind of grandfather clause. SlimVirgin ^{talk|contribs} 17:27, 22 December 2008 (UTC)

A year was just an example - my point was that the panel should decide themselves - but tell everyone exactly what they've decided.--Tznkai (talk) 17:52, 22 December 2008 (UTC)

Okay, I'm fine with leaving it up to them. SlimVirgin ^{talk|contribs} 17:58, 22 December 2008 (UTC)

I'm open to any and all my actions being reviewed by a properly constituted board. As far back as necessary. I'd hope they were reviewed in the context of what was understood and accepted at the time... I would not want to insist that every other CU do so if they did not wish to (but would hope they would). SlimVirgin puts the alternatives well, it's more than just a yes or no. I think another factor is the technical issue... information eventually is not available any more. In part that's by design. (why that is, is a large topic) Also recollections fade, mail gets archived/erased/destroyed, etc etc. Not from malfeasance necessarily, but who here has all the email they sent in 1985? 1995? 2005?... so some reasonableness is needed. If the community will accept a finding of "we understand the concern but this was 3.5 years ago and the records are not amenable to us making crisp determinations any more, but here's some thinking we had" I don't think a grandfather clause is needed. But I'm not opposed. ++Lar: t/c 18:01, 22 December 2008 (UTC)

Source of policies

To the extent that the WMF and local policies differ, both should be referenced. Thatcher 17:55, 22 December 2008 (UTC)

Which relevant local policies differ from WMF ones? SlimVirgin ^{talk|contribs} 17:59, 22 December 2008 (UTC)

I'm not sure if there are any significant difference or not. I mean comparing WP:Checkuser with m:CheckUser policy and WP:Oversight with m:Oversight policy. It is certainly the case that the local checkuser policies on DE and JA are more restrictive of what kinds of checks can be run than the blanket Foundation policy. I have not compared the four pages to see if there are any significant divergences in our case. Thatcher 18:08, 22 December 2008 (UTC)

I don't think our pages say anything different. The checkuser page even says it's not allowed to diverge. SlimVirgin ^{talk|contribs} 18:17, 22 December 2008 (UTC)

I think they've differed in the past although I can't say exactly/definitively how. Perhaps we should not have our own pages instead just have a copy/paste and a crosswiki link. I think maybe one point where they once differed was in who had access to Oversight, en:wp allowed steward access, which was subsequently removed (FT2 and I worked on that removal IIRC). ++Lar: t/c 18:22, 22 December 2008 (UTC)

Actually, I think that the local policies are allowed to be more restrictive than the WMF ones, and I beleive that is the case with our checkuser policy at least: last I checked, "fishing" isn't forbidden by the global policy but is in ours. — Coren ^(talk) 18:38, 22 December 2008 (UTC)

Some clarification (but not general disagreement) I'm not saying they're not allowed to diverge to be more restrictive. Just that if they don't, it may be more clear to point (with a copy for convenience) than to maintain almost identical but perhaps subtly different policies... It's being asserted ours aren't substantially different. I know for a fact other wikis are different. As for fishing, we may be explicit but I take "There must be a valid reason to check a user" to mean no fishing. Commons uses a straight passthrough to the global policy but I turn down requests on non fishing grounds. The Meta indicators page even has the not for fishing template. (of course that's because I cribbed it from en:wp). ++Lar: t/c 18:48, 22 December 2008 (UTC)

Five people???

Five people are going to monitor Checkuser, Oversight, and all the Arbitrators? Will they be subscribed to Arbcom-L? In a case like OrangeMarlin, where FT2 posted a bunch of decisions that he thought were backed up by discussion on Arbcom-L but where the other Arbitrators backed away from it on-wiki, will the review panel be tasked with finding out whose fault it is? Only 5 people to monitor 40? Thatcher 17:58, 22 December 2008 (UTC)

We could perhaps add that the board can take additional members from the pool if they feel the workload is too heavy. SlimVirgin ^{talk|contribs} 18:00, 22 December 2008 (UTC)

Monitoring will hopefully be a very, very boring job, as most of the time, most of the people with the tools are using them properly. In the cases such as the OM issue, the job of the panel will be ex-post-facto fact-finding and recommendations, not necessarily a priori controls, unless you are suggesting all ArbCom decisions be funneled through this committee, which would be disasterous, IMO. -- Avi (talk) 18:02, 22 December 2008 (UTC)

I agree with Avi's statement and Slim's limitedly. Lets consider five members the pilot program - and if history is any judge, the panel may quickly get its own clerk pool unless that course is deliberately avoided.--Tznkai (talk) 18:03, 22 December 2008 (UTC)

Yes. Keeping the number always odd perhaps? However as someone said above, if there is enough work to keep 5 people thoroughly busy we have a problem that's rather large. These folk would not have regular CU/OV/AC duties to distract them. They have to be subscribed to arbcom-l (and possibly to checkuser-l as well, and maybe even oversight-l ) as discussed above, for archive accessibility reasons. BTW: Is this today's fastest growing WT: space page? ++Lar: t/c 18:05, 22 December 2008 (UTC)

Can you comment above, here, or in a new section after refactoring, as to the privacy issues that may be involved with open access to the email-list archives that I suggested above (at the end of the Technical implementation section)? -- Avi (talk) 18:15, 22 December 2008 (UTC)

Make that the section below. -- Avi (talk) 18:19, 22 December 2008 (UTC)

Even 4 would be enough to do the job. It's not a difficult task. FT2 ^{(Talk | email)} 08:14, 23 December 2008 (UTC)

Access to email lists

From Kylu: "Perhaps I've missed it, but has anyone given thought to granting these users access to the appropriate mailing lists instead of merely "To be shown all e-mail and chat correspondence that may shed light on a given matter" ? Kylu (talk) 17:41, 22 December 2008 (UTC)"

From Lar: "These folk would not have regular CU/OV/AC duties to distract them. They have to be subscribed to arbcom-l (and possibly to checkuser-l as well, and maybe even oversight-l ) as discussed above, for archive accessibility reasons.…"

Question from Avi: Access to those lists perforce allows the auditors access to that private information discussed on the lists and in the archives. If the idea is to prevent the dissemination of that information outside the people entrusted with it (CU/OS/or ArbCom) without cause, allowing open email list access goes against idea. Thoughts? -- Avi (talk) 18:18, 22 December 2008 (UTC)

• The review panel can not meaningfully act without full unfettered access to the mailing lists and their password-protected archives. Going through a gatekeeper could compromise their ability to investigate independently. If you are concerned with too many people having access to sensitive information, don't create a review panel that will increase that number by 5. Thatcher 18:22, 22 December 2008 (UTC)
  • Nod. Either we trust the review panel or we don't... I think maybe we're getting wrapped around this (and the exact permissions) too hard, and need to try to unwind from it for a bit. Let's suss out what the board is empowered to do, the access and permissions required will follow. 18:24, 22 December 2008 (UTC)

I am, personally, in favor of having members of the board automatically subscribed to the relevant mailing lists, of only so that they have the full corpus to dig in locally if needed. I understand FT2's point about the fact that almost none of the contents being relevant, but the point is that knowing which is the "good part" in advance is impossible and picking the right bits post facto will never appear to be truly transparent (and is subject to omission or error). My original pre-pre-draft had specified access to the list. — Coren ^(talk) 18:32, 22 December 2008 (UTC)

Nod. In legal cases, "discovery" often requires wading through a great deal of stuff, and it's hard to predict in advance what will or won't be useful. ++Lar: t/c 18:37, 22 December 2008 (UTC)

One alternative: Give them access to the archives temporarily and when needed, but don't subscribe them to the lists themselves. Obviously these folk will be required to identify to the Foundation, so perhaps the Board will be willing to authorize something unique? Kylu (talk) 02:56, 23 December 2008 (UTC)

One of the two lists is not archived. But to underline the point, if it's a matter of justifying a check or oversighting, then list access isn't needed, because the question won't be "what went on at Arbcom". It will be "please supply any evidence of your choosing (from the list or from anywhere else) that this action was reasonable". From experience, that will almost never be an arbcom email list matter. FT2 ^{(Talk | email)} 08:15, 23 December 2008 (UTC)

But there are people who want this review board to investigate and report on Arbitrator personnel matters as well. If the board's remit includes arbcom, then it needs access to Arbcom-L, and not just through a gatekeeper. Thatcher 14:30, 23 December 2008 (UTC)

Checkusers who can't checkuser and Oversighters who can't Oversight

What problem are you trying to solve to by saying that the review panel must never use the tools, going so far as to create technical solutions to enable them to see logs but not act? This is the key question. To me, it seems pointless. The idea is that the review panel should not be routinely running checks or handling oversight requests. But it is certainly the case that some checkuser investigations will be impossible to run just from the log, and difficult if not impossible to ask another checkuser to run checks and send the data along. (Security, recursive checks, etc. One month I ran 900 logged checks for 70 RFCU cases, that's a average of 12 each. You think the panel can function effectively having someone else run those checks, and you think googlemail or IRC are secure?) Plus, having someone else run checks raises gatekeeper and independence issues. Or, if no other active oversight people are on line and there is an urgent need to remove something dreadful, why not allow the panel members to act? They need to be independent, not vestal virgins. That comes from picking the right people, not from how you fill their toolbox. A biased reviewer will be biased no matter what tools you give them. So I ask again, what problem are you trying to solve by giving the panel crippled access to the mailing lists and a crippled toolbox that can't be better solved in some other manner (like picking the right people)?

• More succinctly, what bad things can a review panel member do with full access that he could not also do with log-only access? Thatcher 18:43, 22 December 2008 (UTC)

The problem is that appearance of conflict of interest is indistinguishable and just as bad as actual conflict of interest. By having a group that cannot perform the actions they are mandated to oversee is that there cannot be conflict of interest. — Coren ^(talk) 18:45, 22 December 2008 (UTC)

• Other than the appearance, what bad things can a review board member do with full access that he could not also do with log-only access? Protect friends? Target enemies? Hide wrongdoing? Blow molehills into mountains? Act for political or personal gain rather than the good of the encyclopedia? What? Thatcher 19:04, 22 December 2008 (UTC)

It would seem you mainly need to clarify just that they'll only use the tools in furtherance of their specific duties as members of this review board. I assume you're acknowledging the problem otherwise, which wouldn't be abuse of the tools, but that their right to use them generally would defeat their purpose of providing a disinterested review. As long as that safeguard is explicit, I think people would trust that the limitation would be followed; if not, one way to be more explicit would be to say they're only to use the tools in in furtherance of an investigation formally undertaken by the board (if that's the intended method of review, though I'm not sure it was intended to be for all three types). Mackan79 (talk) 04:05, 23 December 2008 (UTC)

Thatcher, in that case, all we need are a few more CU's and let the CU's be self-policing? If the only difference between the CU and the Auditer is that the Auditer should not be using the tools unless asked, I think we'll just have this issue all over again, unless the CU's now have to monitor the Auditors' usages, to ensure that THEY are not abusing their privilege, and we get into a vicious spiral. By separating the powers, we prevent a class of "Super" CU's or OS's who themselves are just as open to abusing the tools as anyone. Lastly, wasn't the idea of having ArbCom appoint the CU's after intense vetting to ensure that, as you say, the "right people" are picked? -- Avi (talk) 18:47, 22 December 2008 (UTC)

Jeepers Christmas, by all means let's avoid the appearance of a conflict but ignore the potential for actual conflicts. Let me drop a few names here, nothing personal intended. SlimVirgin has expressed a strong interest in these issues, do you think she would have no conflicts of interest as long as she has a crippled toolbox? Suppose FT2 decided to resign as Arbitrator and run for the Review Board? How about Privatemusings or Proabivouac? Better to prevent conflicts by picking the right people in the first place. If you don't think the community can pick 5 good people, or at least 4 good people who would kick the 5th off the panel if he turned out to be a bad egg, then you have no business pretending that your idea is any better than the current system. And Avi, let's be serious for a minute. If some Review Panel member started running routine checks for vandals or RFCU, it would be obvious to everyone who can see the log, that person gets asked to either cut it out or resign from the panel to become a regular CU. Thatcher 19:01, 22 December 2008 (UTC)

And yes, if the CUs and OSs would be self-policing then we would not need this panel and I would not have written my essay. But they aren't, and adding more bodies won't change that. You need either an internal cultural change that embraces an internal review process that is much more adversarial than current mailing list chatter, or you need a new panel. Thatcher 19:01, 22 December 2008 (UTC)

And, I reiterate, if we are creating a new panel, shouldn't we put in protections from the get go? Otherwise, why don't we strip the bit from everyone who has not had community input to their CU bit and start over? -- Avi (talk) 19:07, 22 December 2008 (UTC)

I carry out auditing functions as part of my RL activities. I have full access to the same information and tools as those whom I audit. The reason is that I must be able to demonstrate that, when I (the independent auditor) carried out the same task, I got the same result. That is simply a routine and normal part of auditing. While I might be persuaded that this isn't necessary for Oversight, as the oversighted edits will be visible in the log, I would fully expect that someone auditing CheckUser would verify the results of checks to ensure that the actions taken in response of the checks were in keeping with the actual results. Risker (talk) 19:27, 22 December 2008 (UTC)

That, actually, is a very good point I had not considered. Would it remain reasonable to restrict the reviewers to running checks that have already have been ran? — Coren ^(talk) 19:32, 22 December 2008 (UTC)

As long as "check" is interpreted broadly to mean "incident" or "case" and not "logged check." If I'm looking to see if Smith and Jones are the same person, CU might disclose dozens of IPs between them, depending on what ISPs they use. I might pick 3 or 4 to check more deeply. A reviewer should not be restricted to those same 3 or 4 IPs, but should be permitted to run whatever checks are needed to pursue the broad objective of deciding whether the check was appropriate and whether I got the correct result. Thatcher 19:45, 22 December 2008 (UTC)

(<-) I figured that uninvolved CU's would be glad to supply the necessary data to the auditors, but if that is calling the entire audit into askance, I can understand that. However, what steps can we take to ensure that these new auditors are not abusing their tools? (Quid ipsos to the nth as it were)? Are we to have the CU's monitoring the CU log to see if the auditors are running checks? What is the procedure to remove an auditor based on CU's checking the logs? Unless we build something into the process, we are just transferring the issue at hand from CU's to Auditors. For example, even now, what is the procedure should we find out that an Ombudsman has been running multiple improper CU's? (Note: that has never happened to my knoweldge.) -- Avi (talk) 19:58, 22 December 2008 (UTC)

Oversight is broadly, self-checking. Checkuser is not (you often need the result of one check to see if the next was merited). If there is an investigation, then it would be rash to assume the result where this isn't needed. So some checking is likely to be needed.

Given that the original checker should be able to explain why those checks were done and the cause-effect connections between them, checks to validate those specific claims would be reasonable. A simple precaution if needed would be that no member performs a checkuser action, without beforehand emailing the other members to disclose the check/s to be made and what evidence it is intended to recheck (and perhaps, getting at least one concurring email back).

Importantly, the purpose of such a check is not to test if A and B really are socks, or whatever. It is to check whether the claim of the checkuser that check A reasonably led to check B or action C, is evidenced. So the members should not be seeking to "prove the point" of the original check. Their job is to check "was the evidence there as claimed", for example that this IP appeared in that user's edits. The check either supports the claim or doesn't. FT2 ^{(Talk | email)} 08:30, 23 December 2008 (UTC)

This point should be hammered home on the page somewhere, I think. It is entirely possible to be both perfectly justified in a check and also completely wrong about them being sockpuppets. --Deskana (talk) 08:33, 23 December 2008 (UTC)

I think one of the purposes of the panel should be to handle complaints that CU results were reported inaccurately, that could either be a good faith mistake or some kind of misconduct. In many cases the board would just go to checkuser-L and say "recheck this case please". But a CU who gets results wrong and obstinately refuses to ask for cross-checks is just as much of a problem (if not more) as someone who peeps people out of curiosity. Thatcher 05:06, 24 December 2008 (UTC)

Address actual failings

When setting up the Review Board, it is most important to address the actual problems that are interfering with adequate review of users with Checkuser and Oversight access, and Arbitrators delivering proper rulings. We need to worry less about satisfying appearances of impartiality. Instead address the problems that are stopping the current system from delivering adequate reviews and sanctions. Unless these items are addressed, then the new Review Board will soon not be seen as impartial because it will also not give a satisfactory response to legitimate concerns. FloNight♥♥♥ 19:12, 22 December 2008 (UTC)

Indeed, but how well do you feel the current proposal does address this? — Coren ^(talk) 19:20, 22 December 2008 (UTC)

My concern is that issues will not be fully investigated even if complaints are made or problems are noticed. There needs to be some type of transparency about the filing of a complaint, whether the concern is raised by an user or a member of the board. FloNight♥♥♥ 19:27, 22 December 2008 (UTC)

I don't think we're at the point where looking at the day to day procedural aspects of the board is productive, and I would expect the original batch of members will want to have their own say, but I would expect some sort of RFAr noticeboard where concerns can be raised and responses given would be most likely. — Coren ^(talk) 19:30, 22 December 2008 (UTC)

I anticipate that many of the complaints will come by email since they involve privacy related issues. There must be a tracking system for each incoming email with a method to describe the disposition of each complaint. Privacy issues might stop an user from complaining themselves. This issues is extremely problematic. It is possible that investigating the problem will expose an user to more invasion of their privacy than actually occurred in a bad check. We need to be sensitive to this issue as we write proposals that will allow for more people to look at private information. Tracking methods need to be by transaction number rather than by name of complainant. FloNight♥♥♥ 19:48, 22 December 2008 (UTC)

A system of ticket numbers does sound like a good idea. On the implementation side, the OTRS system might be a tool to consider; but again I think we're putting the cart before the horses. Having the body with the authority and means set out needs to come before procedural details, methinks, and I would expect the newly-formed board might have a good thing or two to say about how they feel is the best way to do their job? — Coren ^(talk) 20:26, 22 December 2008 (UTC)

Yes, I understand that the details of how to track the complaint is not best decided right now. But the issue of tracking the original complaint needs to be addressed as I see this as a key concern. FloNight♥♥♥ 21:02, 22 December 2008 (UTC)

• Look, no one is ever going to find that in the ongoing ethnic conflict between the greens and purples, Arbitrator Smith has aided the greens by improperly influencing debates and cases, because it doesn't happen. The internal problems on the committee are things like, "The committee had nearly reached consensus in the case RFAR/Greens and Purples when Arbitrator Smith suggested a series of new proposals that derailed the consensus and bolixed up the committee for a further two months." The Review Board does not have the remit to challenge case decisions, right? So what exactly are they going to do about Arbitration matters? Can 5 people vote to remove someone whom 200 people voted to elect? Are you going to issue letters of censure to people who don't keep up with the mailing list or who ask irrelevant questions or who habitually sidetrack debates? I certainly am not clear on how this panel can have any meaningful role in auditing Arbcom. Thatcher 19:42, 22 December 2008 (UTC)
  • An independent investigation of complaints about an arbitrator that provides a written finding and recommendations will be a helpful first step. Then a method to execute the recommendations through Committee or Community action is needed. For small issues then the written feedback will hopefully be enough to solve the problem. For larger concerns, then more appropriate sanctions might be needed. In either case, it is wrong to ignore reoccurring personnel problems that are interfering with Committee work. FloNight♥♥♥ 20:06, 22 December 2008 (UTC)

Flo, I'm not following your point about tracking the original complaint. Could you explain it some more? I can see that complainants will lose some of their privacy, which is unfortunately inevitable. But you have concerns beyond that, is that right? SlimVirgin ^{talk|contribs} 03:37, 23 December 2008 (UTC)

Binding?

First off, let me start by saying I think this is an excellent idea and should help restore trust in our current Arbitration, CheckUser, and Oversight process.

Second, I have a question regarding the boards final decisions. Say the board as a whole decides that ArbX has somehow done something wrong and needs to be removed from his position and both the oversight and checkuser flags removed. The current wording is a bit ambiguous as to what the next step is, and how binding/enforced their decisions are. Does the board simply make a recommendation to Jimmy, or does it have the power to contact a Steward and have ArbX's rights removed? Will the Ombudsmen have any say, or do they just need to be contacted? Tiptoety ^talk 22:39, 22 December 2008 (UTC)

Ideally if it were a checkuser or oversighter, they would present their findings to Arbcom or the Ombudsman Committee, and if it were an Arb, to Jimbo or the Ombudsman Committee. Not sure what other forms of review we have, since we wouldn't want the RB ordering someone de-checkusered and then Arbcom ordering them re-checkusered. MBisanz ^talk 22:47, 22 December 2008 (UTC)

Arbcom issues should be public statements to both the Arbcom and Jimbo (that way the member can be removed if needed), or if there is systemic corruption issues, the Board itself. The Ombudsman Commission would only be involved if it's a checkuser-related issue that our local board could not address. (The extent of their remit, as the Ombudsman is not appointed to oversee Arbcom activities per se.) Kylu (talk) 03:00, 23 December 2008 (UTC)

Well if Arbcom was leaking private information sent to its mailing list, or is arbitrators were misusing CU results, that probably would fall in the Ombudsman role, no? MBisanz ^talk 03:44, 23 December 2008 (UTC)

To be specific: If Arbcom were leaking private checkuser-related information (either self-discovered or via non-arbitrating checkusers) the Ombudsman get involved. From a strict Foundation perspective, while the Ombudsman commission may be invited to review oversight-related activities, there has been (to my knowledge) no expansion of the commission to overview the abuse of Oversight or other permissions. See: m:Ombudsman commission, foundation-l 2007-Nov, and wmf:Resolution:Ombudsperson checkuser. Nothing else.

Now, for clarity, I would assume that any use of the checkuser extension itself (and other extensions giving similar abilities) would fall under the strict remit of the commission: Boardvote, which is a separate extension, gives checkuser-like results, and would rightfully be within the scope of the Ombudsmen, for one example.

Having checked the global rights log, I see no instance of where the Ombudsmen global group has ever had access to either oversight nor its peripheral functions. [3].

So, no, I firmly believe that only cases where checkuser or functionally similar rights are abused should and do fall under the remit of the Commission. That is not to say that oversight is not in need of such review, but that's not my decision to make. A member of the Board of Trustees or the Checkuser Ombudsman commission should probably be consulted for final word. Kylu (talk) 06:04, 23 December 2008 (UTC)

• The Ombudsman commission only gets involved if there has been an improper release of non-public information. What we are talking about here mostly won't involve that. If the review board recommends removing checkuser or oversight access from someone for cause, that recommendation will be as binding as Arbcom wants it to be. Several arbitrators seem to be broadly in favor of some kind of review mechanism, so presumably if Arbcom approves of the review board, they will at least take the recommendations seriously. A recommendation to remove an Arbitrator will have to go to Jimbo who can be bound or not as he chooses. I don't think there is a way to make it more binding, but certainly if Arbcom accepts the process they will need a darn good reason to ignore the result. Thatcher 14:37, 23 December 2008 (UTC)

Larger Arbitration Committee, not endless bureaucracy

What is needed is a much-expanded, rotating Arbitration Committee that is truly representative of the community—twenty to thirty people at a minimum would probably be a good number. Such a Committee, if diverse and broad, could act effectively as its own "check". – Thomas H. Larsen 01:54, 23 December 2008 (UTC)

Great, but what for ...?

Don't mean to sound negative, but I'm not exactly sure what this proposal is for. The current proposal gives arbitrators choice of membership, introducing the personal considerations of individual arbs, such as fear, into the selection algorithm. Obviously that's not very helpful for a body whose avowed purpose is to protect the community from potentially bad arbitrators. Secondly, there's nothing specific about powers, so it's not clear that the Review Board can actually do anything without the approval of Jimbo or the arbs. Ultimately what's being proposed is probably no more than some method of legitimizing some complaints over others (ones endorsed by the arb-selected board) and a group of people, potentially arb-groupies because of the selection algorithm, who are added to the list of people who can watch the logs (though they are specifically charged with this). I guess the one thing in its favour is that potentially it can grow in power and become, through change, a rival to the power of the arbcom/ Deacon of Pndapetzim (Talk) 02:03, 23 December 2008 (UTC)

Creepy?

I can cope with a panel to oversee checkuser and perhaps oversight. Arbcom arguably has too much work to do to do this properly. But I'm struggling to understand why the notion of an arbcom appeals board is being taken seriously. If some vocal vested users don't like arbcom decisions because they don't go their way, they jump about and try to convince people that the system is evil, so now they get a new place to appeal to. Well when that rejects them, won't they just call it corrupt too? We have a panel of 15 people chosen to arbitrate, can't we just keep it simple?--Scott Mac (Doc) 02:41, 23 December 2008 (UTC)

I agree with that, particularly as to the currently described purpose to "oversee the actions of the Arbitration Committee in general." If review of ArbCom were pursued, one situation where I could at least potentially see value is with private proceedings that by their nature don't allow the usual community oversight. But even then this would involve one panel second-guessing another (unlike the first two categories of review). Perhaps the other situation would be where an individual arb is accused of some form of serious COI or corruption, or some other serious impeachable-type issue, where it's thought that a review board may have more legitimacy to review than the ArbCom itself, and therefore might satisfy an inquiry without a case needing to be brought publicly. In that case, we would broadly be talking about "investigating policy breaches by community trust-holders in cases where the community is not able to oversee." If that is the intent, however, it would probably need to be made clearer. Mackan79 (talk) 04:39, 23 December 2008 (UTC)

Doc, the people who suggested the arbcom oversight angle were Coren and FT2 (and whoever else read the first draft on the arbcom mailing list). These are not users who want to convince us that the arbcom is evil.

Mackan, I agree about the private proceedings. It would be very helpful in such cases to have a truly independent person overseeing who says what. SlimVirgin ^{talk|contribs} 04:52, 23 December 2008 (UTC)

Yes, but I think Coren and FT2 are unwisely kneejerking to unreasonable attacks by vested users grinding axes.--Scott Mac (Doc) 09:08, 23 December 2008 (UTC)

Not interested in vested users with axes here. It's not a factor. The basis is the perfectly reasonable observation that standards will be higher if a small team exists tasked with a more critical review of actions and with an investigatory role, than is the norm at present. FT2 ^{(Talk | email)} 08:18, 24 December 2008 (UTC)

In my opinion, any review of Arbcom should be restricted to a meta level: not to serve as an appeal of any decision or a re-examination of its particulars, but a look at any procedural irregularities such as miscommunication, undue haste or slothfulness in handling a case, neglect of duties, improper disclosure of private information, improper concealment of things that ought to have been made public, etc. The review board could go as far as to strongly recommend that the Arbcom revisit a case or hear an appeal if it feels the outcome was compromised by the procedural problems, and could even have power to require Arbcom to vote whether to do so (??), but should not supplant Arbcom in its function as a dispute resolution body. alanyst ^/talk/ 05:52, 23 December 2008 (UTC)

terrible idea

I would be totally opposed to any notion of an arbcom appeals board. Because every case against vested users will be appealed on some technicality, and if the vested users are kicking up a stink (as they do), it will be very tempting to the Board to review things. Political pressure will be put on the Board - and this will encourage more shouting, and more delays for new considerations and more scope for drama at the behest of the same minority of soap opera players that already disproportionately drag down dispute resolution resources. This will also have a chilling effect on arbcom's already emaciated willingness to act against vocal vested users. If we really must have this bloated infrastructure, let it be limited to checkuser and oversight allegations and any "high crimes and misdemeanours" regarding individual arbs. (Although I think we've had precisely TWO instances of alleged misuse here in the last year - do we really need elections and more paperwork just to cover that? It is a lot of community resources with very little gain for the encyclopedia?) As to a review board looking at proceedural failures, what the heck is the point in a ruling saying that a case dragged on for too long? a) it will be obvious that it did - the community knows b) What's the remedy? Overturn it? Pour tea for the parties? - you can't force volunteer arbcoms to move faster and impeaching them and electing a new one won't work either (every new arb says he'll speed things up, nearly all fail). I have to say I think this is a really bad idea from start to finish. I see no gain in term of the project's goals. This is all about ego and "narcissistic injury".--Scott Mac (Doc) 09:08, 23 December 2008 (UTC)

On the selection of the Review Board

Is there specific thinking behind the idea that only a pool can be elected, but those actually serving have to be chosen by the Arbcom. As this destroys its independence, is there some rationale that would make up for it? The proposal seems to be saying that the community can be trusted to elect arbs, but cannot be trusted to elect arb reviewers. This would be an odd thought indeed. I should note that the wording of this specific part of the proposal is not clear and it is difficult to understand what is meant. Deacon of Pndapetzim (Talk) 02:54, 23 December 2008 (UTC)

I can think of two reasons. First, the election of Arbs is a long, drawn out affair with large participation. Secondly, this garantees that both criterion are met: only people trusted by the community can be on the board (because Arbs must choose only from that pool), and only people trusted by ArbCom (as the privacy policy demands) will get access to the private data. — Coren ^(talk) 03:30, 23 December 2008 (UTC)

Where does the privacy policy say that, Coren? SlimVirgin ^{talk|contribs} 03:32, 23 December 2008 (UTC)

Sorry, that's the Checkuser policy. — Coren ^(talk) 03:47, 23 December 2008 (UTC)

Thanks. That was added in March this year by Jdforrester. [4] Before that, it simply said checkusers need the approval of the ArbCom on some wikis, and before that, it said that the ArbCom may select checkusers. Given that this point is in flux, and that it was added by someone with an arguable COI, I think we can safely override it, if that's what people want to do. Alternatively, we can stick to the "approval" concept that existed before James's edit, namely that the community elects the review board, which the ArbCom is then expected to approve, unless one of the members is known by the ArbCom to be clearly unsuitable, in which case we'd have a veto situation, which they'd be asked to use only in cases of very serious concern. SlimVirgin ^{talk|contribs} 04:31, 23 December 2008 (UTC)

No, you can't. As a practical matter, no steward will assign checkuser and oversight rights on enwiki without Arbcom approval. The way to change that is not to fight over the wording of the policy, but to get together a process to demonstrate broad community consensus to change the way things are done, something at least as broad as the creation of PROD, or deciding whether to adopt flagged revisions, for example. Thatcher 14:40, 23 December 2008 (UTC)

I think there's a minimum expectation that introducing something like an RB would change the way "things are done", and would need the approval of one steward, arbcom's overlord Mr Wales. Deacon of Pndapetzim (Talk) 14:47, 23 December 2008 (UTC)

Arbcom (or Jimbo) could choose to tell the stewards to grant CU and OS to the winners of an independent election, or they could hold out a veto/approval power over the election, or they could appoint based on elections that were purely advisory and non-binding. But the community can not assign CU and OS by inventing a new elections process in the face of Arbcom and Jimbo disapproval. Thatcher 14:56, 23 December 2008 (UTC)

Is there a particular reason to be thinking about a community versus Jimbo struggle here? Deacon of Pndapetzim (Talk) 15:06, 23 December 2008 (UTC)

I don't think so, except the same people who would object to Arbcom picking reviewers would probably also object to Jimbo picking them. The answer to your original question is that Arbcom controls CU and OS and the community can not easily override that. Therefore, the review board either must be approved by Arbcom in order to have CU and OS, or Arbcom at least needs to not object to whomever is selected. Since Arbcom presumably would object to certain individuals (say, someone with a history of "outing" editors on other web sites), this implies less than full independence. You can have the selection be entirely dependent on Arbcom (and/or Jimbo), or mostly independent, but not fully independent. Thatcher 15:50, 23 December 2008 (UTC)

Coren, if the RB suggestion has a serious prospect, the current wording of CU policy page is obviously not a big deal, and if anyone tries to make a big deal of it [for whatever reason] it can be changed accordingly. Arbs become arbs (with their cu, os and judge dredd powers) de facto because they are elected ... therefore if it's good enough for arbs it should be good enough for arb reviewers, and there's no need for an extra - redundant in fact - rubber stamp. Both ultimately have the same basis for their authority, and there's no objective reason at this point to declare the arb elections "more valid" than the arb reviewer elections. And, OK, Coren, here's a situation to consider:

• a charismatic user, say it's Giano, get's elected as an arb reviewer. He finishes top of the whole "pool" by a large margin. The ArbCom decide, without giving any public reason, that they just prefer in their fickleness/wisdom [no-one can tell] Ncmvocalist or Ryan Postlethwaite, who finish 10th and 9th respectively. What kind of effect is that gonna have? The Giano user can say he has as much legimitimacy (which is ultimately what elections are about) as any arb. The process will - rightly - have no credibility.

I find this part of it very difficult to make sense, either in principle or as a practical suggestion likely to improve anything. Deacon of Pndapetzim (Talk) 03:56, 23 December 2008 (UTC)

Deacon is right. The rankings issue needs to be addressed. I can see a role for the arbcom in reviewing the elected members to make sure there's no-one privately known as a serial sockpuppeteer, or known in real life as a fraudster. But egregious cases of that ilk aside, if the arbcom overlooks candidates the community has elected in favor of someone who attracted fewer votes, but who is liked by the committee, the integrity of the board will be undermined from the get-go. SlimVirgin ^{talk|contribs} 04:58, 23 December 2008 (UTC)

I'm in agreement with Deacon and SlimVirgin here. There is a valid concern that the review board is not independent of ArbCom enough, given the rather large role ArbCom plays in the selection. (whether selecting candidates pre community vote or selecting from a pool of community voted candidates, it's a large role) My suggestion to reverse the order from how it is written doesn't solve the problem, because this is fundamentally a separation of powers issue. In the US, the Supreme Court (which acts as a review, final appeal, etc. and thus is at least weakly analogous in needing separation of powers) is independent of both other branches, even though nominated by one branch and confirmed by the other, by virtue of outlasting tenants. Appointments are for life and typically last over several Administrations.

For good and valid reasons above the terms here are to be limited, and there is to be a requirement not to serve consecutively without a gap. So "life" isn't a valid separation power. I think only direct election will serve... ArbCom should have no veto at all. (raise issues during the election perhaps, but not a final say so. Trust the commmnity)

Someone pointed out that only ArbCom approved/selected people can have access to CU/oversight data. That's not exactly true, if ArbCom passes a motion saying "we approve of this process and candidates so elected are authorised by us to have access to the data" that's sufficient. IMHO of how the policy is worded (at least one ArbCom has devolved CU election to the community, by explicitly saying "We devolve CU election"). ++Lar: t/c 05:19, 23 December 2008 (UTC)

I agree that the selection process should be independent of ArbCom. Cla68 (talk) 05:23, 23 December 2008 (UTC)

If the review board is tasked only with checkuser and oversight auditing, wouldn't that be an extension of the Committee essentially? Besides, we've been suggesting very short term limits - I don't relish the idea of constant elections - let alone the campaigning that will likely happen.--Tznkai (talk) 05:32, 23 December 2008 (UTC)

If Thatcher is to be believed, and I believe him, we need more more eyes over the oversight and checkuser processes. I don't think the campaigning that may occur is any big deal. Let 'em campaign. Cla68 (talk) 05:38, 23 December 2008 (UTC)

Actually, Deacon, I think you're presenting us with a false dichotomy based on several dubious presumptions. In your example above, you're presuming the election by acclamation of a candidate that ArbCom would be — let's say "hesitant" to pick — in first place (which is possible, but unlikely). You are presuming that ArbCom would then skip all the way to what would have been the bottom of the list. You're presuming that those other candidates, despite having also been picked by the community, would somehow become inadequate to hold seats. You're presuming that dissent or being vocal would somehow garantee that a candidate would not be picked out of the pool. None of those presumptions are probable, and all of them being true is... fantasy.

The particular example you have chosen to pick is interesting. Giano is certainly a vocal dissenter, and indubitably holds quite some amount of popularity as a strong proponent of transparency. That would give him quite a bit of support. He also has a very large (and, judging by the 2007 ACE, somewhat larger) number of detractors. I would guess it to be likely that he be picked to be a candidate for a seat, but I would be very much surprised if it was by a significant margin over any other candidate. It's also interesting in that I wouldn't bet long odds against ArbCom picking him from the pool. I would be opposed because I have serious concerns about his willingness or ability to keep private matters private; but I am one of seventeen voices, and I expect he can find support within ArbCom as well.

You appear to be arguing that having ArbCom pick apointees from a pool entirely and exclusively composed of candidates selected by the community as acceptable suddenly magically taints the candidates into being unacceptable by simple ArbCom involvement. Unless you think being crazy enough to be willing to sit on ArbCom is a transmittable disease, I don't see how that can be justified. — Coren ^(talk) 06:45, 23 December 2008 (UTC)

Coren, when did magic come into this? ;) This is about separation of powers and, above everything, not giving the personal desires and fears of those being reviewed a big place in the selection alogorithm of reviewers. Did you ever watch the British tv show Yes Prime Minister (or Yes, Minister)? If you did, you may recognize that what you are proposing is almost like something Sir Humphrey would have been proud of. Give the community the illusion of choice while making the real choice yourself. Arbs get their authority from community election and the rubber stamp of JW, RB members should either be the same or come from some process more independent or otherwise there's just no point. Deacon of Pndapetzim (Talk) 09:13, 23 December 2008 (UTC)

If a user is not trusted by the community, or is considered "weak", then they should not be in the pool anyway, full stop. If that's the concern then fix the process by which users are deemed acceptable to the community and added to the pool. That process is 100% "owned" by the community. FT2 ^{(Talk | email)} 02:24, 24 December 2008 (UTC)

This thing will endanger people's privacy/'safety'

By it's very nature won't this involve people who ask to be reviewed sharing their IP or whatever details with five or whatever more people than those who already may have abused that trust. The fewer people that have people's personal details the better. , the more people have them the more risk there is of something/someone going wrong, privacy being compromised and possible further risks. At least the checkuser ombudsman seems sort of external even though it isn't and is made of en.wiki users. Sticky Parkin 03:12, 23 December 2008 (UTC)

You may be interested in the suggestion described in Alec's Guiding principles above. Kylu (talk) 03:26, 23 December 2008 (UTC)

Only Checkuser needs reviewing

Per my outright oppose several sections above, I think that only checkuser use has a need for review atm. I'm certain I myself have been a victim of checkuser abuse by none other than Raul654 whilst in a dispute with him, but at this time no one cares because nothing suggests he released any confidential information, to my knowlege despite numerous requests it was never followed up.

So I support review of checkuser usage, that can be done by making a new group (Reviewer) with the checkuser-log right, I think arbitrators should also have the checkuser-log right.

I outright oppose review of arbcom as WP:BEUROCRACY and un required. Im yet to hear about oversight abuse.   «l| Ψrometheăn ™|l»  (talk) 06:16, 23 December 2008 (UTC)

A few points:

1. It's quite possible to abuse oversight. Get into an argument with someone over them calling you a "pernicious twit" and it escalates from there, you battle, it goes to dispute resolution, and that person's friend (who has Oversight) removes the edit, so it looks like you started the argument. Quick off-the-top-of-my-head example. Others can devise more if needed, I assure you.
2. Most of the arbitrators have the checkuser right, which includes checkuser-log access.
3. it's spelled "Bureaucracy".

Thanks. Kylu (talk) 06:44, 23 December 2008 (UTC)

Lets try and use existing examples IE: is there so much oversigsht abuse that warrants a new system that will just make more Bureaucracy?   «l| Ψrometheăn ™|l»  (talk) 06:49, 23 December 2008 (UTC)

Part of the problem of oversight abuse is that the examples aren't there to point to. It takes someone with oversight access (or at least, access to the logs) to actually determine that it's happened. I do know of actual examples of oversighted edits that I do not think qualified under our policy, but I'm not going to discuss those specific examples. I'm quite sure others can supply them if needed. Can you provide an example where having someone review the logs of oversight actions for abuse when it's suspected would be detrimental in a manner that could not happen with the current system? Kylu (talk) 06:59, 23 December 2008 (UTC)

we are not talking about some*one* here, we are talking about an <sarcasm> almighty <end sarcasm> review board. We have things here on an as needs basis, do we really need to add the pressure to oversighters of a board reviewing thier every move when quite clearly oversight abuse is not a prominent problem? And the whole idea promotes another rung in the "Power Ladder"   «l| Ψrometheăn ™|l»  (talk) 07:06, 23 December 2008 (UTC)

(edit conflict) (a) How would you have any idea how much abuse of oversight (or lack thereof) happens? (b) There have been very recent revelations about clear misuse of oversight in order to help win an election. Any and all arguments you appear to try to be making are rather hollow. --MZMcBride (talk) 07:13, 23 December 2008 (UTC)

Although I disagree with MZMcBride on his exact interpretation, I agree with him that there are real concerns with oversight usage that could have felt the benefit of a board like this. Indeed the very fact that I disagree with MZMcBride is why this board would be so useful; they could impartially investigate the matter. --Deskana (talk) 07:17, 23 December 2008 (UTC)

While you assert that it's not a problem, you don't provide evidence, whereas others may disagree. Also, if it's not a problem, what's the harm of giving them the ability to make sure it doesn't turn into one? I'll let others take over this discussion: I don't think your opinion has a basis in fact at this time, sorry. Kylu (talk) 07:10, 23 December 2008 (UTC)

The oversighters are innocent until proven guilty are they not, surely any examples of abuse would be dealt with severly and do we not elect these people ourselves, so look at what your saying closeley as it bordlines on bad faith does it not? If there's frequest abuse you show me evidence of *recent* or extreme abuse and ill reconsider my POV. Again, if its not required don't implement it. we dont have new oversighters that often, so those already there a deemed trustworthy.   «l| Ψrometheăn ™|l»  (talk) 07:21, 23 December 2008 (UTC)

Try reading what me and MZMcBride said. We're talking about one specific case that I have knowledge of. I believe he has heard of more that I have not. --Deskana (talk) 07:23, 23 December 2008 (UTC)

I did read it, So do show me this ONE example of abuse, I'm dying of anticipation.   «l| Ψrometheăn ™|l»  (talk) 07:28, 23 December 2008 (UTC)

There you go. Of course, you can't see the diff because it's been oversighted and there's no way for anyone to un-oversight it. So I guess we'd need some select group of about five people who are appointed to deal with these sorts of complaints, eh? --Deskana (talk) 07:39, 23 December 2008 (UTC)

The problem is not necessarily that mistakes or abuses are common or have happened at all; the problem is that if they were to occur or have already occurred, there's no established procedure for investigating them and no independent body to do so. Suppose that there have actually been no cases of checkuser or oversight misuse and the Arbcom were mistake free. It would still be valuable for an independent group to certify this fact. Very valuable, in fact, because of the confidence it would help instill among community members. I see no reason to require evidence of abuse of these privileges before a review mechanism is put in place. alanyst ^/talk/ 07:34, 23 December 2008 (UTC)

Doesn't arbcom already handle complaints of abuse though?   «l| Ψrometheăn ™|l»  (talk) 07:36, 23 December 2008 (UTC)

Read the above sections, and stop being lazy. Your answer is up there and it is not our jobs to repeat ourselves because you can't be bothered reading. Infact the answer to your question is in the very first post on this page, so it's not like you've even attempted to read it before diving in headlong and making a fuss. --Deskana (talk) 07:39, 23 December 2008 (UTC)

WP:AGF, perhaps I have read the above section but want clarification. Lets look at this from another POV, to many cooks spoil the broth, can the reviewers be trusted? whos going to review the reviewers of the....?   «l| Ψrometheăn ™|l»  (talk) 07:43, 23 December 2008 (UTC)

On IRC, you've explicitly told me you've not read the section by stating quite strongly that you should just make the changes to the page without reading the talk page. So by assuming good faith I would be assuming you lied to me? AGF is quite possibly the weakest defense ever too, read this. Not that you will. --Deskana (talk) 07:49, 23 December 2008 (UTC)

O RLY? The link you gave me shows nothing, surely if it as abuse of oversight it would have been undone now, its not that hard you know :) Or even better tell me the title of the page as it could be anything. and im well aware of AAFF.   «l| Ψrometheăn ™|l»  (talk) 07:54, 23 December 2008 (UTC)

(unindent) Again, you are clearly unaware of what you're talking about. A developer was contacted to unoversight the edits in question and the developer said it was not a technical possibility. And no, I will not reveal oversighted information to you. Which is why people want this review board to exist. That you're still missing this when it's been quite explicitly stated to you... --Deskana (talk) 07:58, 23 December 2008 (UTC)

I have a better idea, alter oversight so the oversighting can be undone by developers ;). Thats the way it should have been made   «l| Ψrometheăn ™|l»  (talk) 08:08, 23 December 2008 (UTC)

Then what? Follow up every complaint by unoversighting the edits? Are you aware of the possible legal repercussions for the WMF if the developers of a WMF site start unoversighting libellous information? We still need some form of review process that doesn't involve unoversighting edits for review. (I also assume that you retract your insightful and knowledgable comment that "it's easier than you think" to unoversight things given that it's apparently currently not even possible to do so?) --Deskana (talk) 08:13, 23 December 2008 (UTC)

'Reviewer' group is reserved for WP:Flagged revisions, so, please, propose another name. Ruslik (talk) 08:22, 23 December 2008 (UTC)

Wikipedia was initially designed as an open system. In its ideal ethos, anything private is a detraction from that, and is a real risk long term to neutrality and content (censorship sneaking in via the back door). Anything that places secrecy as an acknowledged "undesirable but necessary thing to be treated with caution and minimized" is probably good. I also happen to like anything that will push for higher standards and this in my view, will. These are functions most users cannot check. While there are a dozen or more people who can check these things, a specific and tasked group with separation of powers, used to create "non-acting checkers", will (in my view) result in improvement to the project outweighing any bureaucracy.

It would be nice if "the wider checkuser community" could do this role, or Arbcom. For that purpose I pushed hard in 2008 for wider participation by non-arbitrators and non-admins in many processes. Some of those came to nothing (view-deleted) and some succeeded (checkuser selection), but in this area, the existing checks and balances probably are insufficient. Stronger checks and quality controls are probably therefore a good thing. FT2 ^{(Talk | email)} 08:43, 23 December 2008 (UTC)

• I'm not sure about Promethean's argument "I was wrongly checked therefore only checkuser needs to be monitored." I know of one oversight matter that was a definite mistake and two matters that were probably not mistakes but where the people who knew about them were unsatisfied by the assurance from the oversighter that it was ok and where a more transparent review and report process would have made a great deal of difference. Thatcher 05:09, 24 December 2008 (UTC)

A fair rankings notion and method suggestion

Sample process, since people keep talking about it, and since we'll be picking the "pool" by elections anyway.

1. Election happens.
2. We vote. We rank the Top X number of users. I recommend a simple system like I used here. Very hard to game since you are ranked by overall performance. How simple is it? That took me all of about 4-6 minutes to do with a calculator and Notepad.exe on my computer. But ranking methods can be sorted out later--it just needs to be firm and fixed.
3. We figure out the community-mandated Top 10, let's say.
4. The AC needs to appoint, let's say, 5 people per year.
5. They review the Top 5. They like 1-4, and approve them for privacy access and the Review Board.
6. For whatever private reason, the AC passes on prospect #5.
7. The AC is now required to move on to #6. Passes the AC vetting? Welcome aboard. No? The AC is now required to move on to #7, then #8, then #9, sequentially based on the ranked order the community of editors decided upon, until the pool is exhausted.
8. If the AC somehow exhausted the pool, then something is either seriously screwed up with 1) the AC; 2) the vetting process for private data being employed--wouldn't it be the same one that the AC themselves get, and if not, why not?; 3) all the candidates that ran.
9. This (to me) prevents anyone from having weight if they call "foul" on the Committee bypassing any specific person--the AC is still then required to take the next person we offer. Any theoretical political bypassing could very, very easily blow up in one's face, if you skip over a suspected tiger for a suspected milquetoast, and get a crusader. Or in baseball terms, you can pitch around the #8 batter, only to have the #9 pitcher then smash a ball over the fence.

My thoughts on why it's good:

• This gives the AC the ability (which is required for privacy policy vetting) to weigh the prospects.
• This firmly lets the community decide who watches the watchmen.

What downsides are there to such a straightforward system? This is the idea that's been rolling through my brain, that I suggested with poor wording above in a jumbled threaded conversation. I'd be curious for Arb thoughts on this. rootology (C)(T) 08:09, 23 December 2008 (UTC)

Wouldn't work. Quick reasons -- can you see the drama and strife when Arbcom isn't happy with #3 and passes them over, but isn't able to publicly say why? (Similar to Jimbo deciding that an 85% candidate was going to be skipped at arb election.) And the unfairness of the taint of it, "arbcom must know something". The assessment of 2008 Arbcom when this was examined was basically, that if ranking order was taken account of, then effectively either there would be no ability to choose the best or avoid concerns, or there would be no effective ability for that within a very short time of a year or two. The alternative was, the community names all those it would trust, and any of whom could do the job. If they wouldn't be trusted, couldn't do the job, or would be later perceived as "weak", then they shouldn't be in the pool in the first place. FT2 ^{(Talk | email)} 08:48, 23 December 2008 (UTC)

Personally, I think the drama is a big factor to avoid, yes, but fairness to the candidate is even higher on my list. Anyone known to have been explicitly "skipped" by ArbCom would fall under an unjustifiable pall. — Coren ^(talk) 15:24, 23 December 2008 (UTC)

Committees out the wazoo

Why are you proposing giving power to a select group of people as a solution to the problems caused by giving power to select groups of people? -- Gurch (talk) 14:29, 23 December 2008 (UTC)

For the same reason many nations have elected to put more than one body in the "path of power", whether commons/senate, executive/legislative, etc. One serves as a check against systemic problems and individual abuse in the other. — Coren ^(talk) 15:28, 23 December 2008 (UTC)

This is not a nation, though, it's a wiki, and unlike most nations, we have the option of not using committees at all -- Gurch (talk) 18:32, 23 December 2008 (UTC)

I practice? No. It's nice and idealistic to believe so, but we've grown past the size where a reasonable chat around the village pump is sufficient. — Coren ^(talk) 21:58, 23 December 2008 (UTC)

Contributors have been banned on a number of occasions following discussion at the administrators' noticeboard. More complex sanctions tend to be avoided, which is a good thing in my opinion, but they also happen sometimes. Administrators issue blocks all the time. Administrators get their rights removed by stewards in emergencies or voluntarily in response to pressure from the community. What does the arbitration committee do that couldn't be done elsewhere by a larger group of people without so much individual power? -- Gurch (talk) 22:45, 23 December 2008 (UTC)

Anything that involves significant privacy issues, and matters where the community is divided to the detriment of the project and no ready other solution seems to exist. The former isn't long term able to be handled by the public community (by its nature), the latter we may learn to. However as of 2008/09, both are problems where an idealized solution is too far away and cannot just be waited for in many cases. FT2 ^{(Talk | email)} 02:15, 24 December 2008 (UTC)

On eligibility

This seems still an open question... I'd just like to point out, that if former CUs/OVs/Arbs are not allowed on this review board, then: Thatcher is ineligible. Mackenson is ineligible. NewYorkBrad is ineligible. Among others. That seems a pretty strong argument against not allowing alumni on. Trust the community to make the right decision. ++Lar: t/c 15:18, 23 December 2008 (UTC)

Not letting alumni in (especially for the inaugural run) has a lot of benefits as it pushes the panel away from any systemic biases that say, the CU group as a whole have. The catch of course, is the loss of technical knowledge. With Checkuser little o oversight especially, I think having someone with CU experience is a good thing - what is important though, is that any appointed members be sufficiently separated from the daily use of the permission.--Tznkai (talk) 16:04, 23 December 2008 (UTC)

I agree with Lar. The difference between preventing active Arb/OS/CU and former Arb/OS/CU is small, and the downside of permanent prohibition is great. Secondly, if the reviewers have to have the full CU/OS bit for auditing purposes, why can't current CU/OS operators should be acceptable if they agree to stop using the bit for regular checks? Unless the idea is that once someone gets the CU/OS bit they are "contaminated" and "part of the problem". In that case, however, receiving the bit as an auditor has the same contagion. I find it somewhat difficult to believe that the unofficial title one has when a steward flips the bit makes a significant difference. It is the character of the operator that matters, and, hopefully, the community can determine that. -- Avi (talk) 16:07, 23 December 2008 (UTC)

• You want to appoint people to review Checkuser who have never used the tool and who are not allowed to use the tool except for the investigations themselves (even you even give them the tool at all, that is). That's like hiring someone to audit your business accounts who has never taken an accounting course or learned to use a calculator, except for whatever on-the-job training they can figure out on their own. This is going to be amusing. Thatcher 16:33, 23 December 2008 (UTC)

• • If you are talking to me, originally I thought that log access was enough. However, after reading the arguments here, I've changed my mind, and agree that the impartial auditing would require full access to the tool, so what I would like is that the auditors to be former CU's, be very knowledgable as how to use them, but not use them for any other purpose than auditing to minimize the risk of having the same problem that exists now (no check and balance on them). This woul dbe very similar to how WMF Ombudsman serve now. What would you like to see? -- Avi (talk) 16:38, 23 December 2008 (UTC)
    • I've already agreed that the review board should not get involved in routine use of the tools. Board members shouldn't have to be former checkusers or oversighters, but those folks should not be barred, either, and at least in the first year there should probably be a minimum of one experienced CU and OS on the board (assuming you can find any to run). And with checkuser (maybe less so or not at all for oversight, I don't know) there should be some provision for a "ride-along" for board members who have never had checkuser access before to run a couple of checks off RFCU in consultation with a current checkuser, just to see what the results look like and how the tools work. Thatcher 16:46, 23 December 2008 (UTC)
  • So, it seems, you, I, and Lar agree on this. OK, I think we have a quorum. . -- Avi (talk) 16:49, 23 December 2008 (UTC)

I posted in an earlier section an idea that I'll slightly amend here. In order to balance out experience with independence, mandate that at least X seats must be filled by people with past experience in the functions being reviewed, and at least Y seats must be filled by people who have not had those roles and might feel more freedom to look at things critically. Thoughts? alanyst ^/talk/ 17:05, 23 December 2008 (UTC)

Personally, I'm uncomfortable with mandating anything like that. While it may be preferable to have a mix, mandating that mix removes flexibility that may be needed in the future. I know that currently, in order to get the CU bit as a dedicated CU, as opposed to a member of ArbCom, you already have to demonstrate a certain technical expertise and knowledge about IPs, proxies, etc. If the same standards are applied to the prospective auditors, together with some mentorship from an experienced person, I would think that sufficient. -- Avi (talk) 17:12, 23 December 2008 (UTC)

Brief thoughts

Very briefly, I feel that creating the RB as a de facto appeals board when you don't like what the AC decides is a monumentally bad idea. It has the distinct possibility of creating more drama than it solves, which we most certainly do not need. I've been a bit critical of arbcom over my time here, but the way to affect change is through the elections, not through creation of yet another layer of bureaucracy. In my view, there are two real choices we can make:

1. Keep the current arbcom system, and elect people who will bring the sea change we need;
2. Scrap the current system and begin again.

Neither of these involves adding another layer of status to an already status-laden project. Given that adminship has already become a "big deal", notwithstanding what it was supposed to be, we can't afford to be adding further levels of class structure to our project, which is what this feels like to me. SDJ 19:00, 23 December 2008 (UTC)

I think this is the best I've heard so far. – Alex43223 ^{T | C | E} 06:39, 26 December 2008 (UTC)

Suggestion

"Review board members will regularly check the checkuser and oversight logs." This seems impractical and impossible and burdensome and practically invites cronyism of the sort this policy was intended to subvert. Why can't the review board just be ordinary unskilled users, chosen on the basis of decency and honesty (however you do that), to provide a backstop against bad practice and misuse. The only skill required would be understanding of the relevant two policies (checkuser and oversight, of which oversight seems the simpler). That is how such functions work in other organisations. Peter Damian (talk) 21:35, 23 December 2008 (UTC)

The thought is that since some checks will have to be re-run, skilled users have an advantage. -- Avi (talk) 21:39, 23 December 2008 (UTC)

Most of the discussion on this page bears on this question in one way or another. I know it is a lot to read. Thatcher 21:40, 23 December 2008 (UTC)

[To Avi] Why? You are talking about checkuser rather than oversight, correct? Peter Damian (talk) 21:40, 23 December 2008 (UTC)

Yes, Peter, specifically CU. -- Avi (talk) 21:44, 23 December 2008 (UTC)

On CU as I understand it, the skills required are not primarily technical, but more on deciding, on the basis of edits, whether account X originates from the same person as account Y. Correct? Peter Damian (talk) 21:48, 23 December 2008 (UTC)

Yes, although an understanding of IPs, proxies, forwarding, etc. is helpful. Personally, should this get off the ground, I'd think that the first iteration should have at least one experienced CU to help educate people. Also, realize that as these people will not be doing CU checks often, the learning curve may take a bit longer, although access to checkuser-l will help to have a pool of people whom to ask. -- Avi (talk) 21:57, 23 December 2008 (UTC)

[To Thatcher] I have attempted to read it, most of it is garbled and makes little sense. Care to present the key issues in a short set of bullet points, anyone? Peter Damian (talk) 21:43, 23 December 2008 (UTC)

Peter, start at my essay User:Thatcher/Quis custodiet ipsos custodes first. Thatcher 21:49, 23 December 2008 (UTC)

I read it last week, and I just read it again. This is not 'bullet points' i.e. a summary of the key issues in a clear and understandable format. Peter Damian (talk) 21:57, 23 December 2008 (UTC)

[edit] The problem with checkuser, unlike oversight, is that there is no clear one or two points of policy that any ordinary person can check. I'm still not sure what checkuser even is, let alone on the circumstances where it might be appropriate, or not. Peter Damian (talk) 22:00, 23 December 2008 (UTC)

[edit] I've taken a quick look at Wikipedia:CHECK. I am right. The policy is unclear and confusing. Are we talking about a tool which allows one to see IP addresses? The policy suggests that, but then talks about other methods. The practice of monitoring 'behaviour' concerns me greatly, as you know. But how is an ordinary person to decide whether monitoring 'behaviour' was appropriate or not? Peter Damian (talk) 22:04, 23 December 2008 (UTC)

m:Help:CheckUser is a better explanation. In a nutshell it allows the CU to see information about the IPs associated with a named editor and the names associated with IPs, including individual edits. As the main purpose of CU is to prevent abusive sock/meat puppetry, the behavioral analysis skills that go into successfully identifying sockpuppets are applicable to CUs as well. And no, the CU bit does not return information such as age, height, weight, net worth, and favorite colors, much to the relief of many . -- Avi (talk) 22:13, 23 December 2008 (UTC)

Don't worry; IPv6 will (finally) support favorite color and evil bits. TenOfAllTrades(talk) 00:02, 24 December 2008 (UTC)

Peter, I'm on my way out but let me try and do this in 2 minutes.

1. Review of whether edits are properly oversighted requires the ability to see the oversight log. Otherwise, the reviewers would be relying on the oversighters to provide them with information, in effect, making them gatekeepers of investigations in which they might be the target. A much bigger independence problem than the potential of cronyism.
2. Review of whether checkuser results are reported accurately requires the use of the checking tool and the ability to use it, to double-check the results.
3. Review of whether checkusers are being run for legitimate reasons requires access to the logs and sometimes access to the checking tool itself. Most checkusers are smart enough not to tell someone, "Guess what, I checked you because you pissed me off" so most cases of suspected "checkuser abuse" will probably arise from people looking at the log and seeing suspicious checks or patterns of checks. If the reviewers themselves don't have access to the logs then they will be relying on the checkusers themselves as gatekeepers again.
4. Access to checkuser and oversight is limited due to privacy issues at the Foundation level.
5. Bad reviewers (cronyism, etc) will be bad reviewers and good reviewers will be good reviewers no matter what tools they have. You need to pick good people. Thatcher 22:45, 23 December 2008 (UTC)

There's ways of determining a user's color scheme (look at .css pages in the user's userspace) and system information (user-agent string returned by the browser, usually specifying OS, version, and at times hardware information) so while we don't know someone's favorite color and net worth, we could make an attempt at a guess... wait, you were kidding? Kylu (talk) 00:28, 24 December 2008 (UTC)

[Further summary for Peter Damian and other newcomers who don't want to read 200kb of discussion.] An editor with CheckUser can enter a user name and get a list of all the IPs they have recently used, and enter an IP and get all the editors who have used it recently, and their edits. It is useful to identify sockpuppets and discover hidden accounts created by vandals. The rules say it is to be used "to prevent disruption" and should not be used "for political purposes" for example. The rules are vague but most people have a sense that if I legitimately think you are up to no good I can probably check you but if I am just curious about where you live or work (or worse, I would hold it against you if I knew) then I should not check. Oversight is the ability to hide an edit so thoroughly that it can not be seen by most editors and can not be restored by anyone. Some people think these tools are used abusively; my own review suggests a very small but non-zero number of possible concerns. The Checkusers and Oversighters could police themselves but usually do not, hence the proposal for an independent review panel. Thatcher 02:18, 24 December 2008 (UTC)

Presumption of public reports

Justice must be seen to be done, as they say. It seems reasonable to add a presumption that as much information should be reported publicly as possible without compromising users protection under the foundation's privacy policy. So a finding that "user:Sneezy's checkuser of user:Dopey was not within policy for xyz reason" should be publicly reported, whereas "user:Sneezy checkuser of user:Dopey found a 95% likelyhood the user was Roy Disney" clearly shouldn't. I can see little reason why at least the recommendations shouldn't be publicly available in all cases. Any other caveats? --Joopercoopers (talk) 02:41, 24 December 2008 (UTC)

Wording:

"All cases are to have a report written at their conclusion. Reports should be written clearly, so that any uninvolved, experienced editor can understand the complaints, the evidence, and the recommendations. The reports are distributed or published at the discretion of the board, in compliance with the Foundation's privacy policy but with the presumption of public availability."

Picking up on the the evidence above, can we clarify what evidence will be available for uninvolved, experienced editors in CU cases? --Joopercoopers (talk) 02:52, 24 December 2008 (UTC)

You're going to run into a number of problems here. Do you want to publicly name and shame CU and OS who make well-meaning mistakes? Name and shame first offenders? I think if there is a serious offense that is referred to arbcom for removal, that should be made as public as possible consistent with editor privacy. I think cases should be publicly summarized and accounted, but the review is not supposed to be a pillory you know. And a lot of the evidence is going to be private. If a CU could get publicly shamed for an innocent mistake, how many people do you think would want the job? Here are some reports you are likely to see, when do you think the parties should be named and when do you think, out of courtesy, respect or privacy, we should not?

• Case 2009-01 User Adam complained that he had been inaccurately confirmed as a sockpuppet of User Bob and that the checkuser involved refused to reconsider. The board has reviewed the findings and determines by a 5-0 vote that Adam is unlikely to be a sockpuppet of Bob. The checkuser agrees with the board's findings and has agreed to ask for cross-checks from other checkusers when questioned about his findings.
• Case 2009-02 An oversighter improperly removed 3 edits made by User:Cheryl to User talk:Don. The edits appeared to contain Don's phone number but the oversighter realized after removing the edits that it was an April Fool's joke. The oversighter has apologized.
• Case 2009-03 A member of the review board noticed in the log that Checkuser Evan ran a check on User:Fran that seemed to have no legitimate basis and initiated an investigation. The board voted 4-1 to accept the checkuser's explanation that certain comments made by Fran appeared to constitute a threat to create sockpuppets to influence a debate that Fran was losing.
• Case 2009-04 A member of the review board noticed in the log that Checkuser Gene ran a check on User Hank that seemed to have no legitimate basis and initiated an investigation. The board finds as follows: Gene and Hank were involved in a dispute (passed 5-0); because of the dispute Gene should not have run the check but asked for another checkuser to evaluate the situation (passed 5-0); the board does not accept Gene's explanation of why the check was run (passed 3-2). The board voted on the following actions: A reminder to all checkusers not to check editors when you are involved in a prior or current dispute or your impartiality could otherwise be questioned (passed 5-0); Gene is reprimanded (failed 2-3); Gene is referred to Arbcom for consideration of removal of checkuser (passed 3-2).

One can imagine all kinds of scenarios. I would argue that only in case 4 should the name of the checkuser be disclosed, and then only after Arbcom has had a chance to deliberate and make a decision. (But even if Arbcom voted to let Gene off with a warning I would like to see the panel publish its findings). Minor findings should not result in public condemnation unless it is repeated offense by the same person. Thatcher 04:53, 24 December 2008 (UTC)

The board will be pushing for higher standards. That means that it will be critical of users in cases where in fact, nothing was wrong, but things could have been done better than they were. For this to be seen as helpful and mentoring, it's best it has the ability to decide what to publish, what not. As well, a lot of the time even hinting at a case - its parties or the related pages - will be enough for others to locate the private information removed or the accounts protected by privacy in the first place (that's exactly why none of the oversight log is public). if genuine fault is found, then it has discretion on where to publish. If Arbcom and Jimbo are informed and the board feels inadequate action has been taken, then that discretion also includes publishing to the community. FT2 ^{(Talk | email)} 08:26, 24 December 2008 (UTC)

@Thatcher - I largely agree with you in principle. "Do you want to publicly name and shame CU and OS who make well-meaning mistakes?" I see no problem in naming CUs and OSers who make well meaning mistakes. The 'shame' and your later 'condemnation' are your own rather subjective assessments of what publishing results of the board might be to users. Newbie admin's actions can be reviewed by anyone inclined to look with a quick reference to their contributions and logs - some latitude is given to their early performance and the large body of admins provides a good check and balance to right any wrongs that may occur. I'd suggest that the community should extend similar tolerance to CUs and OSers, and CUs and OSers should not see investigation as shameful or condemnation but helpful review. So to take your cases:

• 2009-01. A checkuser is found to not reconsider after a reasonable request and asked to get feedback in future from other CUs - For a newbie this should be no big deal, he shrugs and changes behaviour. If the guy's been in the job for 3 years and racked up 30 findings of a similar nature, the behaviour isn't being modified and the community should be aware.
• 2009-02. Ok, so an OSer removed edits which turned out to be unproblematic. We may applaud the OS in question for his vigilance and good faith removal at the time, we might also ask in other circumstances, that the edits be restored as a result. I fail to see why public reporting of this is a bad thing.
• 2009-03. An investigation decided that Evan is an honest Joe. That's a good thing, we've got a CU we can trust and that's on record. --Joopercoopers (talk) 12:43, 24 December 2008 (UTC)
• 2009-04. No, I believe the community should be told the results of the review boards investigation, once it has concluded and then the Arbcom case should proceed. (BTW will arbcom then hold its own evidence phase in these cases or will it take the boards results at face value?) I'm unclear why you think these matters should be shuffled behind closed doors, to spare the blushes of users acting inappropriately.

I agree repeated minor findings are just as important as major ones in establishing patterns of trust and improving standards, but to say isolated minor findings shouldn't be published seems wrong - so user:thatcher is having an off day and slips up a little but no real milk is spilt - so what? If there's plenty of findings that say you're a stand up guy, the board's not going to be recommending any sanction.

@FT2 - Re. Oversight - how can 'nothing be wrong' but 'things (need to) be better' at the same time? I don't see the review board solely as a means to mentor individuals - its also about public trust in our institutions and perhaps even you are ready to conceed now that the "trust us, we've got it all in hand" sandwich we are required to swallow on a regular basis, rather works against public trust. It requires us to assume everyone is lilly white all the time, which as long as there are people in the world, nobody is going to believe. So checks and balances are required and the public needs to be involved and informed, not scorned. This is the danger of having this review board's ground rules, being written entirely by CUs and OSers - I suggest you concede some of this stuff to community concerns. As to hinting at cases, the board will have to be careful about its wording, but that's implied in the caveated presumption. --Joopercoopers (talk) 12:42, 24 December 2008 (UTC)

By only listing complaints, you create a false picture. If (say) a grocery store was required by law to post all complaints it received no matter how petty, it would likely affect their business, and even then the complaints would be balanced by their publicity department talking about the good things they do and balanced by the fact that you shop there every week and love them and have never had a complaint. Here the relevant balancing data is hidden, and I think posting glowing monthly reports about how wonderful the CUs and OSs are is likely to be viewed with a certain cynicism. Ultimately the decision about how much info to release about each case should be up to the member of the board. Thatcher 14:55, 24 December 2008 (UTC)

The board is going to decide whether or not it accepts cases for review? So the truly petty won't be looked at, and the tweaking, proding and improving can and should be public - I'm not keen on your grocery store comparision - better is the function of the Independent Television Commission/Ofcom whose activities have yet to dissuade people from watching television or TV companies from making programmes, but have sanctioned companies when they step over the line, or cleared them when they have not. Publicly I might add, and usually requiring those companies to publish apologies - I'm not going that far :-) --Joopercoopers (talk) 15:06, 24 December 2008 (UTC)

Indeed. Aside from the biasing effect of only reporting negative reviews, it also leaves the community somewhat out in the cold about what – if anything – the Review Board is up to. (While my Latin isn't up to the job, I'll ask in English — Who watches the watchers who watch the watchers?) If the Review Board just says, Don't worry, everything we've looked at is fine, does that mean that everything is just fine, or does it mean that the Review Board just isn't doing very much? With bodies like the ArbCom, much of their work product is highly visible; this isn't the case with the Review Board.

I don't know precisely what metrics the Board should report on, but some thoughts come to mind. At a minimum, there should be a regular (quarterly?) report summarizing their efforts. That report should include information like the number of direct requests or complaints they received (public and private); the number of complaints that they felt had a basis for followup (some will be obviously spurious); and the disposition (in summary form) of the cases that they investigated. There should be a similar summary of cases which they chose to investigate on their own initiative. (Some of these may be cases where trouble is suspected, others may simply be random spot checks.) While it may not be proper to publically disclose, the Board should report to Jimbo and/or the ArbCom on how their checks were distributed across CUs and OSers, and be prepared to account for disproportionate attention given any CU or OSer. Level of participation (or lack thereof) of each Board member should be reported to the community.

On the other hand, one does not want to err on the side of too much disclosure. The Review Board will be entrusted with sensitive information, the irresponsible release of which might be harmful to Wikipedia. Even where the Board adheres to the letter of the privacy policy, releasing too much data about the dates, times, and types of CU actions may inadvertently hand a great deal of help to repeat vandals and sockpuppeteers. TenOfAllTrades(talk) 15:28, 24 December 2008 (UTC)

There has been a fair amount of noise about whether the review board audit panel (I really prefer that name instead) will report publicly or privately. The simplest answer is both. A simple summary (something like X complaints Y went past preliminary investigation Z resulted in reprimands. Case 2009-4 has been submitted to the committee with recommendations (Passed 3-2). Case 2009-5 was investigated and no violation was found (Passed 5-0) Case 2009-6 was investigated has been submitted to the committee (passed 5-0) and with recommendations for a speedy permission removal (passed 3-1-1) There were N checks this quarter, and K sockpuppets blocked as a result.) The Committee would receive a much longer detailed report with the usernames of the involved users, the exact time frame on the complaint, and similar details. After ArbCom makes their decisions, they can publish whatever details from the fuller report they wish to, especially in cases where the Checkuser's names if they have been cleared in a public complaint.

Another thing to keep in mind is the panel will quickly evolve in its first few months of operations: it will likely have to deal with a backlog of complaints now that more people suddenly have an outlet they'd like to use, it has to determine its own voting procedures, divide up how it wants to do the work, create internal procedures and otherwise work out the kinks. The policy is more about creating the general remit of the panel, its powers, and its selection - the details are important, but we have to trust that in our selection procedure, we'll select people who can create those details responsibly as long as they are given a clear remit and powers.--Tznkai (talk) 21:40, 24 December 2008 (UTC)

Why? Why should the independant review board delay disclosure of its finding, until Arbcom have decided whether or what action should be taken? Why shouldn't the subjects of reviews be named? If we're writing ground rules here, why should the principle of public reporting to the greatest extent possible be ignored? --Joopercoopers (talk) 21:29, 24 December 2008 (UTC)

Alternative draft - limiting review panel to auditing CheckUser and Oversight tools

I remain concerned that the combination of overseeing CheckUser and Oversight tool use (which is review of primarily technical tools) and overseeing the Arbitration Committee and its members (which is primarily an exercise in policy application and human resources management) is not a natural fit. While I can see the purpose of overview of the Arbcom and its members, I am not persuaded that this is the process that has been envisioned by the community as an entity. On the other hand, I think that this process is pretty close to what the community is looking for when it comes to CheckUser/Oversight tool use.

With this in mind, I've taken the liberty of using this proposal as a baseline, and have done some reworking to remove references to the board/panel reviewing Arbcom itself, and have expanded some of the sections relating to CU/OS auditing. It is available for review and comment at User:Risker/Audit Panel (a different name to prevent confusion). I will leave it to the community at large to determine whether this might have a better chance of acceptance. Risker (talk) 04:25, 24 December 2008 (UTC)

For clarity, do you mean solely the review of Arbitrators in their role as Arbitrators, or in their use of CheckUser and Oversight as well? Kylu (talk) 05:07, 24 December 2008 (UTC)

Anyone who has CU and/or OS is subject to have their use of those tools reviewed. The issue is whether or not the review panel should be able to examine Arbs as Arbs, including looking at the internal workings of ArbCom, and decide whether particular Arbitrators should be removed. I know Jimbo asked for a recall process but I think it is too early to fold Arbcom internal monitoring into this proposal. (If 16 Arbitrators can't figure out that #17 is screwing things up and ask Jimbo to remove him, I don't see that turning it over to a different committee of 5 will be any help.) Thatcher 05:18, 24 December 2008 (UTC)

I concur with Thatcher about all users with CU/OS having their use of those tools subject to audit and review of the panel, regardless of whether or not they are also members of Arbcom. Risker (talk) 06:04, 24 December 2008 (UTC)

(e/c)I think the fork linked above is a better plan - we have no idea what the work load of auditing/reviewing/oversighting CU and OS will be, and that task is much higher priority.--Tznkai (talk) 06:12, 24 December 2008 (UTC)

I agree, let's keep this to OS and CU monitoring - Jimbo's suggested "creation of a method for the community recall of unpopular ArbCom members" and other Arb issues will just muddy the waters and isn't necessary to set up reasonable, independent review of the use of tools. --Joopercoopers (talk) 14:05, 24 December 2008 (UTC)

For the record, I am entirely unopposed to restricting our first pass as this to CU and OS auditing. I will support the more limited proposal as a good first step if the community prefers that. — Coren ^(talk) 15:05, 24 December 2008 (UTC)

I would much prefer the review board to be only for checkuser and oversight. Can we copy Risker's draft to the project space if we're going to go with that scope? Vassyana (talk) 10:46, 26 December 2008 (UTC)

Per this discussion I've removed the following from the list of functions for now:

1. Investigating allegations related to any improper conduct by arbitrators or the Arbitration Committee as a whole; and
2. Examining actions undertaken by the Arbitration Committee or any of its members, whether or not related to the use of checkuser or oversight, which have caused significant community concern.

--Joopercoopers (talk) 15:51, 24 December 2008 (UTC)

• If (1) is removed, wouldn't the remaining functions just simply duplicate the Ombudsman commission? - Mailer Diablo 17:38, 24 December 2008 (UTC)

My understanding (please correct me someone if I'm wrong) is that a. This is an en. local initiative which is either in addition to or will replace the en. ombudsman functions b. it includes monitoring of oversight rather than just checkuser. --Joopercoopers (talk) 17:49, 24 December 2008 (UTC)

The Ombudsman Commission investigates specific allegations of breaches of the WMF privacy policy, and limits itself to that one function; it is also a creature of the Foundation. They act only on complaints. The proposal here (as currently formulated) is specific to English Wikipedia and is intended to audit and review the activities of individuals granted Oversight and CheckUser permissions on this specific project, and to investigate any type of irregularity relating to the use of these two tools. The mandate is considerably broader. Risker (talk) 17:53, 24 December 2008 (UTC)

Yes, WMF Ombudsman are solely tasked to look into issues regarding the release of private information. This would be a local (enwiki) group tasked to look into the usage and interpretation of tool results, even if no private information was released. -- Avi (talk) 18:34, 24 December 2008 (UTC)

I wouldn't rely on the WMF Ombudsman. The remit is limited and fragmented. What the Ombudsmen have is that if a WMF privacy policy breach occurs, they can probably propose direct action at WMF level. But we have the same ability here. Nothing against the Ombudsmen, but they don't actively check usage as we might wish, they don't have the scope to check all aspects as we might wish to be checked. I don't mind the minor duplication of responsibility involved in providing more checking and reassurance for the community, and patrolling these functions in a more dedicated manner. FT2 ^{(Talk | email)} 18:50, 24 December 2008 (UTC)

I agree with Risker above. However we should look into merging this with the Ombudsman Commission. Otherwise, people will get turned down for going to the wrong place. --Apoc2400 (talk) 10:56, 26 December 2008 (UTC)

Apologies for the IP edit, Apoc2400, but I'd have to point out that as the Ombudsman is a Foundation mandate and not based on community decisionmaking, it's simply not going to happen that we'll merge this idea with the Ombudsman. Firstly, we'd have to persuade the Board to agree and implement the change (and there are many, myself not included, who feel that the Board has moved away from its ties to the community). Secondly, this proposal as amended includes five locals to ensure proper use of both rights and only on this project, whereas the commission has three to enforce our privacy policy for all projects. Thirdly, having two "enforcement departments" allows them to watch not only the checkusers and oversighters, but eachother for any potential complicity. While the positions are sensitive, I think it's reasonable and responsible to have two distinct groups that can watch for abuse. Thanks. :) 207.145.133.34 (talk) 14:19, 26 December 2008 (UTC)