Wikipedia talk:Counter-Vandalism Unit/Academy/Archive 5

This is an archive of past discussions about Wikipedia:Counter-Vandalism Unit. Do not edit the contents of this page. If you wish to start a new discussion or revive an old one, please do so on the current talk page.

Archive 1

←

Archive 3

Instructors

Does anyone have a great idea to get more instructors? We seem to be having an overload of students... Dan653 (talk) 01:50, 28 August 2012 (UTC)

No ideas I'm afraid. In my honest opinion, the overload is the result of an overload of active campaining for students before the CVUA has the capacity for for them. This is why many SME collapse - they advertise too much on the back of their initial success and then fail to meet customer demands. I've noticed also that some instructors appear to be leaving. There may be other reasons for all this. I notice also that new users are getting bombarded with templates such as welcomes and invites to join the TeaHouse, and various other projects. This albeit well intended in-your-face recruitment would scare me off if I were new here. It's a bit like all the junk mail you get from local organisations when you move into a new neighbourhood. Kudpung กุดผึ้ง (talk) 05:20, 28 August 2012 (UTC)

Agree with Kudpung on this - I think that, along with this new "redesign", we should work make finding more instructors a priority. One of the easiest ways to do this is just by looking though the STiki leaderboards, as a start. Theo polisme 11:05, 28 August 2012 (UTC)

I'd recommend chatting to those who've successfully graduated... perhaps have it as part of their congratulations "Have you considered training editors yourself?"... Worm^TT(talk) 11:25, 28 August 2012 (UTC)

Hm, odd, I thought someone had already mentioned that... but yep, agreed. Theo polisme 11:26, 28 August 2012 (UTC)

I think you're right... someone did ;) Worm^TT(talk) 11:29, 28 August 2012 (UTC)

Hehe, must have been your alter ego. But in all seriousness, I think that a) perusing various anti-vandalism leaderboards/projects/member lists as well as b) inviting graduates is a good plan. Theo polisme 11:37, 28 August 2012 (UTC)

I remember that post quite clearly, that's why I have made so many good faith comments that I felt advice was not being taken. I fully realise that the time of most of the members of the CVU/CVUA is pretty fully committed to RL, but perhaps they could watchlist all the project's pages and eachother's talk. It shouldn't be too difficult - they seem to be able to follow everything I post on Wikipedia ;) Kudpung กุดผึ้ง (talk) 08:46, 29 August 2012 (UTC)

Mentors

Per Trusilver should we go from instructors-->mentors? Dan653 (talk) 01:03, 29 August 2012 (UTC)

Good idea! --Tito Dutta ✉ 02:34, 29 August 2012 (UTC)

There is a possible problem here, in that "mentoring" is often used to mean something slightly wider than counter-vandalism instruction. So for example, if instructors were actually called mentors, then imagine the possible situation in an ANI thread. An editor is facing an indefinite block (for, say, repeated image copyvio), and it's agreed that instead of being blocked, they should agree to a mentoring arrangement. "Oh, but I've already got a mentor from the CVUA", they say. Much confusion all round. If "instructor" isn't inaccurate or misleading, it might be as well to keep it. --Demiurge1000 (talk) 08:55, 29 August 2012 (UTC)

Demiurge has a good point... Hm. Anyone else have another way of looking at it? Theo polisme 11:04, 29 August 2012 (UTC)

How about teacher or tutor? Personally, I rather like trainer (a student might then be referred to as a trainee), but I'm also aware that "trainer" has another meaning in British English. Cheers! ➢ Bgpaulus <small-caps>(WORDS & DEEDS)</small-caps> 22:45, 29 August 2012 (UTC)

I'm British, and have used "trainer" to refer to footwear a great deal, but we don't confuse the two - "trainer" does not seem incongruous to refer to a person, if that's the context it's in. "Teacher" would be more problematic, as some trainees might be teachers in real life, and some trainers might be pupils in real life. I also still don't see the problem with "instructor". --Demiurge1000 (talk) 00:01, 30 August 2012 (UTC)

So we now have an option of: ~~mentor~~, teacher, instructor, tutor, trainer. Dan653 (talk) 00:13, 30 August 2012 (UTC)

I believe leaving it as "instructor", at least for now, is the safest bet. Theo polisme 00:21, 30 August 2012 (UTC)

Turkey

Hey, we must be doing some good because the turkish wiki now has a CVU too [1] :D Dan653 (talk) 00:15, 30 August 2012 (UTC)

I was proud of us... until I checked the page history and found that it was created in 2006. Oh well... :) Theo polisme 00:21, 30 August 2012 (UTC)

You had to check the page history :( rats :P Dan653 (talk) 00:22, 30 August 2012 (UTC)

Instructional Methods

Having borrowed heavily from ItsZippy's content (thank you), and as someone who develops training programs for a living, I believe we should have a minimally acceptable defined outcome that must be achieved in order to consider someone 'graduated'. We do have the "Required Lesson" section on the Instruction Methods page, but there's is not (to my knowledge) any mechanism in place to actually be sure that the enrollee has achieved all those lessons. At present, there are new instructors coming into the program (some of whom graduated only the day before they become an instructor) and they're left a bit to their own devices as to how to achieve the results we seek. As noted above, there's a sense that we need more instructors, and if we get them, I think that situation will only grow. In most cases, I suspect new instructors will likely follow the techniques that were used with them, but I'm also concerned when I see comments such as that of Trusilver above stating that there's little perceived value in the Academy. Whether it's a final exam or specific tasks that every enrollee must complete successfully I believe it's in our best interests as an Academy to have specific standards of achievement in order to consider oneself "graduated".

I can't remember the instructor from which I got the inspiration (though I thank them!) but I also create a page where the enrollee and I can have direct conversation about their tasks, progress and standing. This is also where I ensure they can perform the tasks that are required of them and I suggest they make this available to the admins reviewing WP:PERM when they request rollback permissions as evidence of their ability. If you'd like, have a look at the academy pages I create for my enrollees here, here or here.

Does this revamp present us the opportunity to standardize the meaning (and therefore improve the credence) of actually having graduated? Just a thought... happy to help in any way I can. Thanks. Vertium ^{When all is said} _{and done} 19:20, 31 August 2012 (UTC)

I agree wholeheartedly with what you're getting onto here. We've traditionally left the curriculum quite open - and making some strict(er) end requirements seems like it would make a lot of sense. I think that requiring a final exam would make sense - but if others have any other suggestions, I'm quite open. I do think this revamp provides the opportunity to solve lots of the problems that have plagued the CVUA - and that's exactly what we need to be doing! Cheers, Theo polisme 19:35, 31 August 2012 (UTC)

Prescribing how instructors are to teach their students does seem to go a little against the move towards a more open and accessible CVUA that we are trying to develop. Having said that, having desired outcomes might help with consistency (and move towards resolving the issues Truesilver raised above. I would want to keep any prescription to a minimum, perhaps just something that says "by the end of the course, a CVUA student should be able to show that they understand/are able to...". We should allow instructors to work as they wish, and if we see instructors who perhaps need a little support themselves, we can offer that to them. I wouldn't want to see 'required lessons', nor a standardised graduation exam, but saying what we expect graduates to know and be able to do would be helpful. ItsZippy ^{(talk • contributions)} 19:42, 31 August 2012 (UTC)

I like your points. Consolidating the WP:CVUA/IM page into some concise information about what students should learn would make sense. Theo polisme 21:03, 31 August 2012 (UTC)

I completely agree with not wanting to be overly prescriptive, but I also know that there's a talent to teaching that isn't universal, and purely from an instructional design perspective (yes, I'll admit I do have that bias), you'd always want your learning objectives to be SMART - specific, measurable, attainable, relevant and time-bound. So, you might build an objective that says, "Upon graduation from CVUA, the student will have demonstrated their ability to consistenly differentiate good faith edits from vandalism." The question that now invites is, "How will we measure that". If you want to leave that open to each instructor (many of whom will be new), then we'll kind of take our chances with the credibility the academy brings forth and I don't think we'll do much to quiet the criticism, but I'll absolutely go along with consensus on that issue. All that said, I don't quite see how the establishment of success measures goes against being open and accessible. Those do not seem to be at odds with one another to my thinking. There's nothing in a competency-based training program that precludes a wide recruitment of candidate students. It does however, give the students a clearer understanding of exactly what they're signing up for. I've seen conversations on the enrollment page where the enrollees have to ask what CVUA is all about. I think it would actually benefit the academy to be clearer up front about what will be expected of them. This is particularly true if we're going to recruit a lot of people to the academy and make a lot of new people instructors. It will actually lead to greater ease in recruiting instructors if we provide the tools to ensure the goals of the academy can be met. I think it would be prudent to remember that there's a difference between completing a curriculum and proving you learned something. Any true academy should be able to illustrate the abilities their graduates have achieved. Thanks for the opportunity to have a dialogue here! Vertium ^{When all is said} _{and done} 21:13, 31 August 2012 (UTC)

What you say makes sense, Vertium; I wasn't saying that your ideas are necessarily over-bureaucratic, just that excessive bureaucracy is something we are trying to avoid and so, if we are to look at something like this, we will need to do it as un-bureaucratically as possible. Perhaps a brief overview of what we expect graduates will have learnt somewhere would be good (can tell the difference between good faith and vandalism, can give appropriate warnings, etc); perhaps someone could create a draft somewhere that we can look at (I would myself, but it's getting late). I think monitoring instructors is a related but separate issue, which we can do quite informally. Any experienced instructors who are willing are able to watch other instructors, and can give advice, or offer to informally monitor less experienced instructors. I don't think we need a process for monitoring, provided that some of us are willing to spend a bit of time watching how others are delivering their courses. ItsZippy ^{(talk • contributions)} 21:53, 31 August 2012 (UTC)

I'll put some content out here for discussion over the weekend. It's not so late here right now, but it's been a long week, so I'm headed for a little R&R tonight. Thanks! Vertium ^{When all is said} _{and done} 23:31, 31 August 2012 (UTC)

I also agree completely, but I think the solutions are not so easy as a minor change in methods. As I mentioned to Worm That Turned a couple weeks back, I feel that the biggest thing this project suffers from is a lack of credibility. I think it seriously needs to start feeling less like a social club and more like an actual project. One thing that I would like to stress is that the instructors themselves need to be practicing what they are teaching. A cursory review of the current roster of instructors shows me that a FULL 50% of instructors have not done any significant vandalism reversion in the last 10 days, despite being active on the project. more than 25% of the instructors have not done any significant vandal whacking in a month or more. I have a very hard time taking anyone seriously who pretends to be teaching antivandalism, but doesn't seem to trouble themselves with actually participating in that pursuit... especially when it's someone who is supposed to be coordinating the project. I understand completely what Itszippy is talking about in wanting an open and accessible CVUA, but I feel that it undermines the legitimacy of the project when the bar is set so low for instructor qualifications that it's a tripping hazard. I will once again make clear that I think this project has potential, but not in its current incarnation. Trusilver 02:00, 1 September 2012 (UTC)

I have said I won't be chiming in with any more criticisms or suggestions on the CVU/CVUA talk pages (you all know why), and I won't be, but in the interests of transparency and to demonstrate that some experienced users are genuinely concerned in the very best interests of the Wikipedia as a whole, I'm linking to this thread. And CVUers, please don't take it the wrong way. Kudpung กุดผึ้ง (talk) 05:49, 1 September 2012 (UTC)

Since I'm relatively new to the academy, it's probably not relevant that I add to any conversation about how things have been done historically nor how things came to be where they are (nor do I know why Kudpung won't be "chiming in with any more criticisms or suggestions". What I've read above and in the conversation shared by Kudpung seems to indicate that overall (with exception, of course) there's a fundamental belief that the academy can be valuable to the project as a whole. What's not helping it are some concerns and doubts. As for the concern over "credibility", I'd like to understand the source of this concern. I don't find the percentage calculations of someone's recent reversions a meaningful indicator of credibility because I know that many (myself included) have a finite amount of time to donate to the project and if we're teaching others, contributing to AfDs, participating in RfAs or simply editing articles in order to improve them, we're not spending that time fighting vandalism. Signing up as an instructor for the Academy was in no way intended to convey that fighting vandalism is the only activity in which I plan to engage. Further, I'm disinclined to debate exactly what constitutes a "significant" amount of vandalism reversion as I can say for myself, I don't have to fight vandalism every day in order to know it when I see it and help others learn to fight it. People are donating their time and expertise, so let's just say "thank you" to them and look for ways to improve outcomes rather than take people to task for the past. If there are specific issues leading to a credibility concern, let's put them on the table and discuss them. Consensus is the watchword here, so my personal request is that we get specific with what needs improvement and drive to some form of consensus about achieving that improvement. I'll live with the outcome, but from what I can gather, we seem to be swirling around historical criticisms, and I'd genuinely appreciate just fixing what's broken and moving on. I can tell you this from a professional perspective: without having defined instructional objectives and a consistent approach (e.g. curriculum, rubric, etc.) to achieving them (with certain tailoring allowed and encouraged within guidelines), then having someone graduate doesn't really tell others exactly what they're capable of doing. Which means that it doesn't really mean anything. If we want an academy where its graduates have a sense of both accomplishment and recognition, there has to be some rigor. If not, we're just a loose group of people helping others learn more about fighting vandalism. If that's all we want, that's fine too... but just understand the implications. All that said, let's do what needs to be done and once consensus is achieved, let's all not chime in with further criticism... only suggestions. Thanks. Vertium ^{When all is said} _{and done} 18:09, 2 September 2012 (UTC)

Thank you, that was very helpful, and I agree with a lot of it. I do believe, as I think you do, that measuring the outcomes of the CVUA will be useful to us, and help us get better (as well as gauge whether the academy is successful). I found Truesilver's analysis above unhelpful, as he was not comparing like with like - it might be worth comparing how well students did at anti-vandalism before they entered the academy, and how well they did afterwards. That would show whether or not our efforts are working, and would also help us highlight which areas of anti-vandalism we need to improve teaching, and also which instructors need support. I think a statistical analysis like that would be a good place to start with improvements, as it would show us what needs to be improved. I would be happy to get started on this, but I don't have a great deal of time to contribute to the project. Would others be willing to help me? ItsZippy ^{(talk • contributions)} 20:32, 2 September 2012 (UTC)

Thanks Vertium - my thoughts echo much of what Zippy said. I agree that some sort of analysis would make much sense - and would be happy to do it. I think we need to nail down what specifically we're looking at/for: AIV reports? User warnings? Revert counts? Accuracy before and after? Theo polisme 20:42, 2 September 2012 (UTC)

Vandalism reverting accuracy would be the obvious place to start. We would need to randomly select (and I mean randomly) a pool of students who have graduated; we would then look at their vandal reverting accuracy before hand (take a sample of 50 vandalism random reverts from before they joined the CVUA, say), and then do the same for 50 random reverts after they graduated, and see if they improved or not. If that starts to give us useful results, we can then begin to look at other metrics. ItsZippy ^{(talk • contributions)} 21:13, 2 September 2012 (UTC)

I think the main point of this whole conversation is: When a student graduates what should they know? A side point: Maybe the above could be the next VS? Dan653 (talk) 01:08, 6 September 2012 (UTC)

Instructor requirement

Hello everyone. I respect everyone's opinion and would appreciate your comments. I think there should be one at least one requirement to be an instructor... Rollback. We (or at least I) want new users to learn correctly (and Trusilver's comments above show that this may be more of an issue than we thought originally), especially since mistakes are permanent. I think it is best for the student to have an instructor that has first hand experience with Rollback. Editors who have the rollback user right have shown that they can identify what is, and what is not, vandalism - and have access to all the tools a student might also want to learn about (for example, Huggle). If a student graduates and is granted rollback and has questions about rollback, Huggle, Igloo or anything else, we need instructors that are able to answer those questions instead of people who just redirect new users to guides.

Opinions, questions, objections? -- Cheers, Riley Huntley _talk 06:08, 3 September 2012 (UTC)

No objections. Vertium ^{When all is said} _{and done} 20:42, 3 September 2012 (UTC)

No objections either. --Activism1234 20:44, 3 September 2012 (UTC)

We have just started a trial which is specifically designed to see if having no bar to becoming an instructor (including requiring rollback) will work. Do you think we could wait until the trial is over, before rushing to change it? ItsZippy ^{(talk • contributions)} 21:12, 3 September 2012 (UTC)

Editors should use their own judgement before putting themselves down as a Counter Vandalism Unit Academy instructor. Rollback; a right which converts two clicks into one, should not be a requirement. There might be some good reverters who just don't want to convert two clicks into one, and can tell the difference between what is vandalism, and what is not vandalism. In my experience, Rollback is often handed out due to about 50-100 reverts, which isn't much, considering I've done over 2,000 reverts. Therefore, I Strongly Oppose this. Thine Antique Pen (talk) 21:17, 3 September 2012 (UTC)

Closed until the trial is over. It makes no sense to talk about requirements, especially when we just did away with all of them for the time being. Dan653 (talk) 23:59, 3 September 2012 (UTC)

Oppose There is a discussion at Wikipedia Talk:Rollback regarding that twinkle users might not need rollback.--Anderson - What's up? 01:57, 6 September 2012 (UTC)

Discussion has been closed for the time being. -- Cheers, Riley Huntley _talk 02:08, 6 September 2012 (UTC)

Just to let you know

I'm afraid I will be leaving the CVUA at the end of the month; I'm going to university at the end of September, so I don't want to be committed to too much. I intend to continue with my current two students until they graduate, but will not be taking any more on from now. I'm sorry to leave just as the academy is beginning to change and improve; I wish you all the best with it, and will still be around if you want to contact me. Thanks. ItsZippy ^{(talk • contributions)} 18:05, 5 September 2012 (UTC)

Best of luck at university. You've done much for the academy (and WP), so I hope you'll return and contribute as time and your studies permit. All best wishes... Vertium ^{When all is said} _{and done} 22:37, 5 September 2012 (UTC)

Good luck at the university, all the best. Dan653 (talk) 01:05, 6 September 2012 (UTC)

Some things for you to ponder

As I noted above, I promised I'd take a stab at this and I thought I'd have more time, so this isn't as complete as I would like, but I don't want to slow down the conversation. In fact, I rather live by the motto of my 10th Grade English Teacher, who told me there was "no such thing as good writing, only good re-writing!", so I invite your markup (or complete re-write) on any of this. I just want to get the ball rolling and focus on the outcomes.

Introduction and Learning Objectives

(this should be on the enrollee page, so they know what is expected of them)

The Counter Vandalism Unit Academy is a structured educational program to help new vandal fighters become more knowledgeable and consistent in the battle against vandalism. While each enrollee will be assigned an instructor to guide them through the learning path of the Academy, we strive for consistency and excellence from the Academy’s graduates and therefore, we have established these objectives for your learning as part of the academy. At the conclusion of your learning within the Academy, you will be able to:

Define and describe how to identify and revert vandalism, differentiating such vandalism from good faith edits made within the wiki;
Revert vandalism using a variety of tools;
Determine and apply the correct warnings on vandal’s talk pages when you discover vandalism;
Report repetitive vandals on the appropriate admin noticeboard for blocking actions; and
Constructively communicate with other editors should they question your reversion activities.

Rubric

(This is for the instructors, in order to grade their enrollees and certify that they've achieved the necessary qualifications to graduate. This is a FIRST DRAFT - so mark it up any way you like!)

Competency	Fail	Pass
Knowledge and Understanding of Concepts Able to correctly define and describe: How to identify vandalism The importance of assuming good faith Differentiate between vandalism and good faith edits The purpose and correct use of warning templates The best way of dealing with trolls	Cannot describe these concepts at a comprehensible level. Unable to differentiate the difference between vandalism and good faith edits Does not understand the key purpose of reverting vandalism	Can concisely and elaborately describe each of these concepts. Can describe the proper use of warning templates and how to correctly make an AIV submission.
Critical Thinking Able to review historical reversions and identify whether such reversion were correctly categorized	Correctly assessed fewer than 85% of the examples provided by their instructor.	Correctly assessed at least 85% of the examples provided by their instructor.
Communication Able to effectively communicate with other editors regarding reversions	Communicates inconsistently with editors via talk page comments or in response to editors who question or challenge their reversions.	Communicates in a polite and professional manner and avoids biting other editors and harsh comments.
Application Effectively applies the concepts and tools of vandalism fighting in a productive and proficient manner.	Cannot consistently revert vandalized pages correctly. Is inconsistent in use of warning on user talk pages or uses incorrect warning templates Has reverted fewer than 50 instances of vandalism in the two weeks prior to graduation - or – has reverted more 50 instances of vandalism with a high degree of incorrect reversions.	Demonstrates proficiency in performing reversions, either manually or with a tool (Twinkle, etc.). Consistently leaves the correct messages on vandal’s talk page explaining the reversion, and is able to further explain the vandal their reasoning behind the reversion if prompted. Has accurately reverted at least 50 instances of vandalism in the two weeks prior to graduation.

Some tools I've used

Again, I borrowed a lot from ItsZippy and have added a few things myself. I include these here only to offer these to anyone else who'd find them helpful or to come up with a superset of examples from which the instructors could borrow for each of their students. I believe that mixing-and-matching is a good thing so I'm always on the lookout for more examples.

Knowledge and Concepts Question: Please describe below the difference between a good faith edit and a vandalism edit, and how you would tell them apart.

Question: Please describe the various warning levels for vandalism and when it's appropriate to use each.

Question: How would you personally determine the difference between a good faith user asking why you reverted their edit, and a troll trying to harass you? What are the appropriate actions for each?

Question: When is it appropriate to submit an editor to AIV?

Critical Thinking The following diffs illustrate various reverts that have been done by other editors or rollbackers. Please indicate whether you agree with the revert and its categorization as either a good-faith revert or vandalism. Also, please explain your rationale for your decision. (Note to instructors - a couple of these that I selected are deliberately vague as I really wanted to know the thinking that my student was putting into their decisions. We will have to agree on the correct answers and apply this consistently. If additional diffs are added to this list, the correct answer must be provided as well so we're all being consistent.)

Reversions classified as "Good Faith edits":

Reversions classified as "Vandalism":

Communication - done primarily through audit

Application - done primarily through audit

then, this task: Please find and revert three examples of good faith but unhelpful edits, and three examples of vandalism. Please warn the editors with the correct template and give the diffs of your reverts below.

Good faith

Vandalism

Again, not trying to force this specific content, but also didn't want to suggest that we have such tools without contributing. We can adapt/change it any way you like. Thanks! Vertium ^{When all is said} _{and done} 22:12, 3 September 2012 (UTC)

I'm actually impressed with the Rubric. It conveys the essence of what someone should know coming out of this program. The only thing I would change is to include an expectation of 90-95% (I think it should be 95, 90 would be acceptable) correct reverts in the two weeks prior to graduation. That seems like a high bar, but I firmly believe that anyone who is misidentifying vandalism more than one out of every ten attempts probably shouldn't be doing recent change patrol to begin with. Overall, though, it's a good step in the right direction. Trusilver 23:07, 3 September 2012 (UTC)

Like I said, a starting point. As an aside, I would always have the score be rather high to pass, but I am sensitive to the fact that we're not dealing with a Harvard or Oxford education here, so I'm quite open to the bar being set by the consensus of the group. Thanks for taking the time to comment. Vertium ^{When all is said} _{and done} 01:16, 4 September 2012 (UTC)

I've got a bit more reading to do and a bit of thinking, so I can come up with a few suggestions on this, but overall, I think it's excellent. It takes out the hierarchy and bureaucracy and allows human judgement to rule. Vertium, you've done a fantastic job. Worm^TT(talk) 07:48, 4 September 2012 (UTC)

Vertium, this is - as WTT said - quite fantastic. I have a few comments/ideas that I'll make momentarily.. Theo polisme 10:58, 4 September 2012 (UTC)

Your kind words are appreciated. I'm quite anxious to hear your comments and ideas! Vertium ^{When all is said} _{and done} 20:40, 4 September 2012 (UTC)

I have made annotations, comments, and changes to the rubric and to the "some tools I've used" sections.. more to be had, as I think of them. Thanks - Theo polisme 01:17, 6 September 2012 (UTC)

Good stuff. Keep it coming! Thanks! Vertium ^{When all is said} _{and done} 17:12, 6 September 2012 (UTC)

Question Is this mandatory? And if mandatory, should it be followed word for word, or used as a guide? Dan653 (talk) 02:56, 8 September 2012 (UTC)

Wikibreak

I just wanted to let you know that I am taking a (short) wikibreak, and am also stepping down as coordinator of this project. I wish you all the best. Dan653 (talk) 21:07, 17 September 2012 (UTC)

A Note: As I have full confidence that the arbitrary role of "coordinator" will be abolished with the reform, I think it is purely procedural for me to remove the "coordinator" label from my name - as I'm really just another editor trying to make Wikipedia better through the counter-vandalism training program. Theo polisme 00:47, 19 September 2012 (UTC)