Wikipedia:Wikipedia Signpost/2008-09-15/Dispatches
Dispatches
Dispatches: Interview with Ruhrfisch, master of Peer review
Wikipedian Ruhrfisch has been responsible for many positive changes at Peer review (PR) and helps make sure every peer review gets some input. He has also nominated 12 Featured articles (FA) as of September 2008, many dealing with Pennsylvania State Parks, creeks and covered bridges as well as other topics. Ruhrfisch is an example of an editor who not only contributes to Wikipedia's top content, but has also worked to help other editors produce better articles via Peer review.
You first edited under your username on July 8, 2005; what drew you to the project?
- Like many users, I first edited as an IP, intrigued by the fact that I actually could edit. After a few months, I registered to participate in an AfD I came across via "Random article". Although the article was eventually deleted and I was accused of being a sockpuppet, I was hooked. More importantly, I am thankful for the kindness of Jwrosenzweig during all this – I doubt I would have stayed had it not been for him.
- As for what drew me, I had done some things on the internet before and found interesting websites, but many of those efforts had disappeared – the fact that Wikipedia was here to stay was attractive to me, even if my edits were changed or removed over time. I remember making a map fairly early on and being told to make sure it was correct, as it was as close to immortality as I would know.
How did you get involved in Peer review (PR)?
- When I was preparing to nominate an article at Wikipedia:Featured article candidates (FAC) the first time, I submitted it to peer review. AndyZ ran the semi-automated peer review (SAPR) on it and then there was nothing. I made a plea for a review and AndyZ kindly came back and reviewed it too. The process was very helpful and when AndyZ took two wikibreaks I did the SAPRs for him by hand (I have the script). This was tedious, so AndyZ graciously allowed me to use the faster AZPR account, and I have done almost all of the SAPRs since September 2007.
- In December 2007, Peer review went to a different system thanks to Geometry guy and Carl (CBM). Before this requests were listed by hand on one page, now VeblenBot does it and sorts them by one of ten topics. Allen3, who had faithfully archived peer reviews for about two years, stopped at this point, so I took over. Between adding the SAPRs and archiving, I saw how many requests got no real response. This bothered me – at least 40 percent of peer reviews were archived with no or very minimal comments beyond the SAPR, frustrating editors trying to improve articles.
- On February 24, 2008, I started a backlog list for peer reviews that were at least a week old and had no responses beyond a SAPR – an idea stolen from Good article nominations (GAN). This helped, but most backlog items were still archived without a response, which was worse in a way. So on the Ides of March 2008 I proposed that all PR requests get a response, which was met with positive feedback. Many editors have pitched in to do reviews, and on May 14, 2008 PeerReviewBot took over archiving peer reviews (thanks again to Carl), and has done a great job. Neither the bot nor I have archived a "no response" PR request since March 15.
As noted here and known by anyone who participates in peer review, the process (like many others on Wikipedia) is somewhat limited by a lack of active participants (in this case reviewers.) From your experience, has this always been the case?
- Yes, as long as I have been involved in Peer review there has always been a lack of reviewers. In August 2007, a thread on the PR talk page called the process dead for just such reasons. Things have gotten better since the December 2007 introduction of the list of reviewers by topic, which allows editors to ask for help from someone in their broad topic. These volunteers are a credit to Wikipedia (and I cannot thank them enough). Since reviewers are the limiting reagent in the whole PR process, we have tried some tweaks to help relieve pressure on them: no more than one peer review request per user per day (a user once made ten requests in one day); no more than four open PR requests per user; and a two-week waiting period between peer review requests for a given article.
- The ultimate solution is to convince more users who request a review of any kind to also do reviews in return. FAC, FLC, GAN, and Peer review all face versions of the free rider problem. People need to be aware that if they have an article being reviewed, they owe the process at least another review, and usually more. If an FAC gets six editors weighing in, that nominator should weigh in on six other FACs at some point; free riders who do nothing in return make the load even heavier on those who do their part.
Let's talk about peer review more. You touched on what you think can be done about improving peer review above. What, if anything, do you think puts people off of peer review (aside from the "work" aspect), and what could be done to cause some sort of positive change?
- Many new editors worry they are not experienced enough to do a peer review, or about making a mistake. Peer Review is a place where it is easy to start small: any constructive comments are helpful and you can literally leave one sentence and help improve an article.
- Expertise can be a problem for both PR requesters and PR reviewers. Some requesters ask for very specific items (a made up example – "Will all particle physicists fluent in Latvian, Urdu, and Cantonese check the translations please?"). Some off-the-wall requests get very specific answers, but perhaps WikiProjects are better suited for that. Many reviewers won't comment on articles outside of their areas of interest and/or expertise either. I think reviewers can comment on any article as an article – does it meet the manual of style, does it flow well, is it understandable, is it referenced? A person unfamiliar with a topic may be the best reviewer to make sure the article puts things into context for the reader and avoids jargon.
- Listing several things the requester already knows are wrong with the article can also put off potential reviewers (who might think "Why should I weigh in, they already know the problems with the article?"). Try to fix the known problems first, then ask for a review. PR is a place to identify problems, not necessarily fix them, so "I know this has comma splice issues, and the references need to be converted to cite templates ..." is less likely to get a response than "I have tried to fix the comma splices, but could someone check for those, and could someone convert one of the refs to {{cite web}} so I have an example to follow for the rest ..."
What factors help make a Peer review more effective?
- Concrete examples of problems help – if an article needs references, I point out a paragraph or section that has none. One problem here is that people sometimes fix only the cited examples and ignore the other problems.
- The most effective thing is dialog – peer review should be a conversation about improving an article. Unfortunately, when PRs are archived with no replies to the reviewer's comments, it discourages reviewers. It may look like the review was done in vain, but often the suggestions have been used to improve the article, just not noted in the PR.
- Requesters who list any special concerns are helpful too. I look for a certain set of things initially, and might miss other concerns if they are not in the request.
How does your participation at Peer review help you with your own contributions to Wiki articles?
- The most obvious is an appreciation of the whole review process – not just Peer review, but also the Good article, Featured article and Featured list candidates. It helps to get more eyes looking at an article and suggesting things to improve or fix. At PR, there is no deadline – GAN and FAC are always work under pressure, but a PR can be more lesiurely. I used to worry time spent on PR was taking away from my other contributions, but I don't see it that way anymore. I am much better at seeing problems on a quick read, so if I don't look at an article I am working on for several days, I find things to fix, clarify or improve when I come back to it. The big picture helps – I could never write more than a stub on many of the topics I review, but many articles I have reviewed are now GA and FA, and I played a small part in improving them. I run across really interesting articles and editors doing PR. I will see new editing and template tricks or clever ideas on how to structure an article. When I need a peer review myself, I can ask a lot of editors to return the favor.
- My plea to anyone who has read this is to get involved in Peer review. Sign up to review in your areas of interest at Peer review/Volunteers. Pick a day each month to review something that has not yet gotten a response at the PR backlog or new requests. If the readership of the Signpost is in the thousands, and everyone did that at PR (or GAN or FAC or FLC), it would help a lot with all the backlogs. I want to close by thanking all the editors who already do peer reviews for their hard work and time, especially those who review items from the backlog.
See also
Discuss this story
I've been working on the old version, since David cut and pasted here rather than moving. Give me a minute to merge the old and new. SandyGeorgia (Talk) 03:50, 12 September 2008 (UTC)[reply]
Possible images from FAs I have nominated
Here are two each of a creek, a park and a bridge - pick what you like (the Cogan House CB is a twofer - also get FA Larrys Creek in the picture) Ruhrfisch ><>°° 16:17, 12 September 2008 (UTC)[reply]
More?
I originally thought this would be a "Focus on Peer Review" article for the Signpost. I am OK with it being an interview with me, but should it say somewhere (quoting Wikipedia talk:Featured content dispatch workshop) something like "User:Ruhrfisch who makes sure every peer review gets some input". I am not trying to toot my own horn, but it seems if that is the reason I was interviewed in the first place, perhaps it should be in the piece.
I also took out the "How does PR help writing FAs?" question, but if you think it is needed, I could answer some sort of "How does doing PR help you with your own contributions?" or something similar. Ruhrfisch ><>°° 04:19, 13 September 2008 (UTC)[reply]
While I do a lot of PRs, there are many other editors who also help a lot with the backlog - should they be thanked in some way? The Rambling Man did as many as I did or more for the first several months and still helps out some, Giggy, Finetooth, Dabomb87 - I do worry about forgetting someone if I thank specific editors though. Ruhrfisch ><>°° 15:10, 17 September 2008 (UTC)[reply]
Although the article was eventually deleted and I was accused of being a sockpuppet lol. Nothing has changed, i assure you. People are being driven away by this sort of stuff every day. 86.44.27.254 (talk) 15:29, 21 September 2008 (UTC)[reply]