Wikipedia talk:Wikipedia Signpost/2017-02-27/Special report

← Back to Special report

Discuss this story

Peer review is something I need to do more of. As a comment, what would be really good would be setting up some kind of "expert peer review" (maybe as part of review for an FA submission?) in which actual, credentialed experts are invited in to give a public assessment of articles. I've heard many times from experts that they grumble about how Wikipedia is bad - not just factual mistakes, that an article overemphasises the wrong things, or spends too much time talking about a theory most experts on the topic think is on the way out, or the hot new controversial idea that isn't very plausible. (See, for example, Emily Gould's comments on her article.)

It would be nice if we could actually invite people who know about a topic to publicly say this stuff and use it as a guideline for improving an article. I often find that experts on a topic don't automatically make the best writers about it - they may be too embedded in a point of view to give due weight to different sides of a debate - but having this kind of opinion on the talk page would be nice. Finding actual qualified experts like academics - or publicly known people like journalists, political consultants or lawyers - willing to donate their time to this kind of thing is the kind of thing the WMF sounds like it would be well-placed to negotiate (it would probably be a more efficient use of their time than actually editing the article directly and figuring out how the formatting works). How about it, guys? Blythwood (talk) 03:53, 28 February 2017 (UTC)

This has been tried a number of times, in various areas, and has produced some good results, but is very difficult to sustain. I've done it with both medical and museum experts, and WikiProject Medicine has taken an interest in the question for a long time. Part of the problem is that proper experts usually need to be given an article that is already at a pretty decent standard to get good detailed reviews. If it is full of issues they tend just to say so briefly, perhaps understandably; they are used to peer-reviewing work that is thought by the authors, professionals in the field, to be ready for publication. They are busy people and WP reviews tend to stay at the bottom of the in-tray unless they have a particular motivation to push them through. Also, their real top-level expertise may be limited to very specific areas within their subject and they can be reluctant to pronounce outside that, as they don't keep up as fully with the research. We have a number of regular editors closer to the pre/post doctoral level, and in practice they tend to give better sustained results. Johnbod (talk) 15:37, 2 March 2017 (UTC)

It's interesting to note that not much actually changed to the Emily Gould article, despite her comments.. 47.188.47.96 (talk) 16:40, 7 March 2017 (UTC)

Many thanks to Tom for his work improving Peer Review and for this article. For several years we had an agreement that no peer review request went unreviewed (which initially came about from how bad it felt to manually archive PR requests that were still unanswered after weeks or months at PR), and many editors chipped in to help with the task, including several already mentioned. The four who I recall doing the most reviews were Finetooth, Brianboulton, The Rambling Man and myself (apologies to all those I did not mention by name). The problem has always been getting enough reviewers.

Reviewers at PR have theoretically always been required to look at at least three things: they need to be able to 1) make suggestions on how well the article is written, 2) give advice on how well the article meets the Manual of Style requirements, as well as keeping an eye on what was needed to pass GAN or FAC or FLC, and 3) say how well the article covers the subject matter. The first two were relatively easy for experienced reviewers, while subject matter is often the most difficult part of a review (though even there some articles have obvious omissions, or do not do a good enough job making the topic clear to an interested layperson).

In an effort to meet the subject area requirement, for a while reviewers listed themselves by subject areas in which they felt they had some ability to review articles. One problem with that was a mismatch between articles at PR and the reviewers available (so there might be a large number of video game or pop music or sport articles per reviewer in those areas, while there were reviewers in more arcane areas (covered bridges, say) that rarely had articles nominated at PR. Another problem was that the reviewers would leave WP or lose interest in reviewing, which frustrated editors who would ask them for a PR and get no repsonse. Another problem was that many times an article would be nominated at PR in good faith by an editor thinking it was ready for FAC, when it had major issues with the writing and/or MOS that needed to be addressed first.

If there is an expert reviewer system, then it might make sense to have a "director" or "editor in chief" who assigns articles - this person would presumably be able to filter out the poorly written and non-MOS compliant articles, and would know who the expert reviewers were who were available and what their recent PR workloads had been. I am still not sure how expert reviewers would be recruited, but it is always good to give it a try and see what happens. Good luck, Ruhrfisch _><>°^° 16:36, 16 March 2017 (UTC)

PS I did not realize my edit summaries to Geometry guy were poetry. It's always good to / see ones' self through the eyes of / another - surprise!

Did you mean for this to be in Category:X1?

@LT910001: in an early edit, you put [[Category:foo]] in an example. A bot changed it from foo to X1. Did you mean to put this page into that category, or did you mean to type [[:Category:foo]], which would have rendered as Category:foo without adding the page to any category? davidwr/(talk)/(contribs) 22:40, 18 March 2017 (UTC)

Thanks davidwr for pointing this out. This was originally a quote... Fixed. --Tom (LT) (talk) 01:29, 19 March 2017 (UTC)