Wikipedia:Research recruitment

This is an essay.

It contains the advice or opinions of one or more Wikipedia contributors. This page is not an encyclopedia article, nor is it one of Wikipedia's policies or guidelines, as it has not been thoroughly vetted by the community. Some essays represent widespread norms; others only represent minority viewpoints.

This page in a nutshell: Researchers must first describe their research project and recruitment strategy (as explained below) and obtain consensus for their activities before mass messaging users.

This page documents the process that researchers should follow before asking Wikipedia contributors to participate in research studies such as surveys, interviews and experiments. Researchers must first describe their research project and proposed recruitment strategy (as explained below) and obtain consensus from the Wikipedia community for their activities before mass messaging users. This process exists because researchers have a great demand for research participants which the Wikipedia community is unable to meet, and so researchers are requested design their studies to be thoughtful and minimally invasive when they recruit the time of volunteers here.

Scholarly research of Wikipedia is useful for understanding the encyclopedia's content, readers, editors, history, current state, and future. These results also yield important knowledge applicable to other open content communities. In addition to driving scholarly knowledge of such systems, this work can also give results that improve Wikipedia itself. Much valuable research cannot be done without Wikipedia editors who volunteer to participate in studies by being interviewed, taking surveys or testing experimental interfaces. This policy describes an interface between academic researchers and Wikipedia editors that does the following:

allows the Wikimedia community to control recruitment campaigns from researchers looking for study participants
affords legitimacy to researchers whose plans are successfully vetted by the Wikipedia community

Who needs to go through this process?

[edit]

Anyone who wants to recruit a group of Wikipedians for a research study must go through the following process.

For the avoidance of doubt, if you want enough Wikipedians to respond that you are posting invites on mailing lists and/or enough talkpages that you might be considered to be spamming people then you need to go through this process first.

For Researchers

[edit]

Create a project page

To start the process of eliciting approval for your subject recruitment request, create a page describing your project and subject recruitment needs/plans.

To start from a template:

Replace StudyName with a title describing your research study in the box below and click the button.
Answer the questions that appear in the text box and save it. You'll be able to come back to make edits.

Hold an RFC

Once you've thoroughly described your project on the project page, you will need to hold a discussion to obtain consensus from the community to move forward.

Place the {{rfc}} template on your project page.
- This will make it clear to visitors that you'd like to hold a conversation about the project.
Canvass relevant WikiProjects and/or discussion forums (e.g. the Village pumps) for participants by inviting them to participate in your RFC.
- If you need help determining which Projects and forums might be relevant, post a topic on Wikipedia_talk:WikiProject Research asking for help.
Participate in the discussion.
- Wikipedians may bring criticisms, suggestions and support for your work.

Note that a populated discussion between editors will be necessary for consensus to be determined. It is your responsibility to elicit participation. If you need help gathering participants, try asking the members of Wikipedia:WikiProject_Research for help.

Obtain consensus

Once the discussion has completed and no more concerns are being raised, an admin will determine consensus and close the discussion for you.

Only once you have obtained consensus for your planned recruitment and study may you proceed with your proposed recruitment strategy.

Consensus is not the same as unanimity, but if you encounter objections your best option is to try and resolve those objections.

Replace the {{rfc}} template with {{recruiting_participants}} until your subject recruitment has been completed.

Recommendations

[edit]

Minimize sample size to minimize disruption. Studies that affect a larger number of editors will need wider consensus to move forward.

Consider vulnerable populations Members of the Wikimedia community become more stressed as they are targeted for intensive research more frequently. In general, community members who are highly active or members of highly researched minority populations are more often sought by researchers more often. Consider estimates of user demographics when designing a survey and see the estimates of members of frequently requested populations.

Don't repeat what has already been done. Lots of people have studied the Wikipedia community and some questions have become over familiar. Wikipedians are more likely to agree to a new survey that is likely to find out something new than one which merely replicates surveys that have previously been run. See wikipapers for an index of wiki research and meta:Editor_Survey for a description of surveys run by the Wikimedia Foundation.

Give back to the community. Your research results will likely have value to the community. Editors may be more likely to approve of your work if you can deliver your results in a freely licensed format for their consumption. Consider open licensing your manuscript, publishing your datasets, open sourcing your code and writing a summary for easy consumption.

Keep individual responses confidential Though the community expects aggregated results to be available, the norm is for individual results to be confidential. Please be clear as to where you are drawing the line between confidentiality and openness.

Keep your cool. Discussions can sometimes get heated over misunderstandings. Consider using the documented strategies for consensus building if the discussion ceases to be productive. Ask for help at Wikipedia:WikiProject_Research or meta:Research:Committee.

No breaching experiments. Do not violate the community's policies or norms. Damage or disruption, for any reason, is not tolerated and is a very fast way to generate hostilities between the Wikipedia community and the research community.

Operate transparently. Wikipedians tend to appreciate openness. The wiki keeps a public record of the history of all pages and conversations. You will aid Wikipedians in tracking your activities by ensuring that they happen on-wiki. Make sure you are logged in when participating in discussions and sign your comments with four tildes (e.g. ~~~~).

Be responsive, but patient. Reviews take time and you can't rush consensus. However, you can make sure that questions, suggestions and concerns posted to your project's talk page are addressed quickly.

For Reviewers

[edit]

All interested editors should act as reviewers. There are no special titles or rights necessary. Proposal reviews should be closed when there is consensus that the project is adequately documented, not likely to cause a disruption and likely to be successful at producing valuable research results.

Recommendations

[edit]

Use the checklist. ..and add to it.

Take a constructive role. Researchers may be missing an opportunity that you see. Offer your suggestions.

Assume good faith. Researchers may not understand the norms and terminology of Wikipedia, but it was their interest in understanding that brought them here in the first place.

If you don't understand... that is the researcher's fault, but you'll have to help them know what needs explaining.

Information for Wikipedians

[edit]

This section gives a brief overview of who researchers are and why they are interested in studying Wikipedia and its editors.

Who are researchers?

[edit]

There are a wide variety of backgrounds from which people approach studying Wikipedia.

Academics: the students, professors, and staff of colleges and universities
Industry researchers: the staff of private companies
Independents: individuals without an affiliation

They are here to perform scientific analysis of Wikipedia and its users. They often intend to publish the results of their work in academic publications.

What do they do?

[edit]

In the past, research in Wikipedia has built an understanding of how Wikipedia works,^[1] how editors interact with each other,^[2] what work is discarded and why,^[3] how admins are chosen,^[4]^[5] and how to detect vandalism.^[6]^[7] This research serves to increase understanding in how Wikipedia works and to improve its functioning. Researchers approach understanding Wikipedia in a few different ways.

Field experiment: Determines the limitations and strengths of Wikipedia's functionality or tests new functionality for editing, collaborating, navigating, etc., by developing and distributing modifications to Wikipedia's functionality. Field trials will usually need to recruit users under this policy.
Surveys and interviews: Learns various aspects of editors and editing (e.g. demographics, motivations, activities) using pre-written forms or back-and-forth conversations. Requests for participation can be either general (i.e. random), or targeted to specific editors. Requests for participation require sending unsolicited messages, so surveys and interviews frequently require recruitment under this policy.
Participant observation: Gains a close familiarity with the editing community by joining Wikipedia and doing the same work that Wikipedians do on a regular basis. They often casually converse with editors as a part of the collaborative editing process. Participant observers who do not send unsolicited requests do not usually fall under the SRAG's guidelines. However, they are recommended to disclose themselves as researchers and consult with the SRAG before starting their research project. In addition, participant observers who decide to interview or survey users need to recruit users under this policy.
Offline analysis: Analyzes database snapshots of publicly available information to examine the history of encyclopedia construction. Offline analyses will seldom need to recruit users under this policy.

Why are they here?

[edit]

Wikipedia is an interesting medium for scientific research. It is one of the most visited websites on the internet, serving as an information resource to millions of users every day.^[8] Many find it remarkable that an encyclopedia in which articles can be edited by anyone anonymously, and in which damage can only be repaired after it occurs, has quality comparable to traditional encyclopedias.^[9] They want to understand how the social dynamics of Wikipedia works. Further, Wikipedia is one of the few examples of millions of people working together with a single goal. The Wikimedia Foundation also supports the work of researchers by maintaining public mailing list (e.g. wiki-research-l and rcom-l), releasing public datasets, hiring them on fellowship and releasing periodic database snapshots for analysis.

Resources

[edit]

meta:Research - The research portal for all Wikimedia projects.
Wikipedia:WikiProject Research - A group of Wikipedians interested in scholarly research of Wikipedia. Many are interested in participating in and closing discussions about recruitment proposals.
Wikipedia:Research - Essay on scholarly research of Wikipedia
Wikipedian checklist - A checklist of concerns for research proposals. Use this to help you review research proposals.
Wikipedia:Ethically researching Wikipedia

Wikipedia:Academic studies of Wikipedia - A list of academic studies of Wikipedia.
Wikipapers - A semantic mediawiki instance for collecting and organizing academic papers about wikis and wiki-like systems.

References

[edit]

^ Kittur, Aniket; et al. (2007). Power of the few vs. wisdom of the crowd: Wikipedia and the rise of the bourgeoisie (PDF). alt.CHI at Conference on Human Factors in Computing Systems. Retrieved 2009-12-29.
^ Kittur, Aniket; et al. (2007). "He Says, She Says: Conflict and Coordination in Wikipedia" (PDF). Proceedings of the SIGCHI conference on Human factors in computing systems. Conference on Human Factors in Computing Systems. Vol. 1. ACM Press. pp. 453–462. doi:10.1145/1240624.1240698. ISBN 9781595935939. Retrieved 2009-12-29.
^ Halfaker, Aaron; et al. (2009). "A jury of your peers: Quality experience and Ownership in Wikipedia". Proceedings of the 5th International Symposium on Wikis and Open Collaboration. International Symposium on Wikis. ACM Press. doi:10.1145/1641309.1641332. ISBN 9781605587301. Retrieved 2009-12-29.
^ Burke, Moira; et al. (2008). "Taking Up the Mop: Identifying Future Wikipedia Administrators" (PDF). CHI '08 extended abstracts on Human factors in computing systems. Conference on Human Factors in Computing Systems. ACM Press. pp. 3441–3446. doi:10.1145/1358628.1358871. ISBN 9781605580128. Retrieved 2009-12-29.
^ Panciera, Katherine; et al. (2009). "Wikipedians Are Born, Not Made". Proceedings of the ACM 2009 international conference on Supporting group work. Conference on Supporting Group Work. ACM Press. pp. 51–60. doi:10.1145/1531674.1531682. ISBN 9781605585000. Retrieved 2009-12-29.
^ Viégas, Fernanda; et al. (2004). "Studying Cooperation and Conflict between Authors with history flow Visualizations" (PDF). Proceedings of the SIGCHI conference on Human factors in computing systems. Conference on Human Factors in Computing Systems. ACM Press. pp. 575–582. doi:10.1145/985692.985765. ISBN 1581137028. Retrieved 2009-12-29.
^ Priedhorsky, Reid; et al. (2007). "Creating, Destroying, and Restoring Value in Wikipedia". Proceedings of the 2007 international ACM conference on Supporting group work. Conference on Supporting Group Work. ACM Press. pp. 259–268. doi:10.1145/1316624.1316663. ISBN 9781595938459. Retrieved 2009-12-29.
^ "694 Million People Currently Use the Internet Worldwide According To comScore Networks". comScore. 2006-05-04. Retrieved 2007-12-16. Wikipedia has emerged as a site that continues to increase in popularity, both globally and in the U.S.
^ Giles, Jim (December 2005). "Internet encyclopedias go head to head". Nature. 438 (7070): 900–901. Bibcode:2005Natur.438..900G. doi:10.1038/438900a. PMID 16355180.