Talk:Crowdsourcing/Archive 1

Archive 1

Archive 2

Defining the outsourcing/crowdsourcing boundaries

Isn't the recruitment of all the public's ideas targeted to specific individuals or companies considered the crowd part of crowdsourcing? Look at www.ideasxchange.com.

OK, I've edited Microengagement's post a couple of times because it is NOT crowdsourcing. Linking a client with a few hired experts is the opposite of crowdsourcing. If you do any reading on the topic, the theory of crowds is that a bunch of 'generalists' are always smarter than a few 'experts' ... any thoughts on what do to with this?? I agree with Liface that 2.0 buzzwords drive me crazy, but I hate it even more when oldschool companies slap newschool buzzwords on their business models. Shazz Aug 11/06

Where does the line stand between outsourcing and crowdsourcing - RentACoder.com could be considered both? It almost seems that generally crowdsourcing needs many small inputs that build up to a whole, while outsourcing is for larger units of work? Benjaminhill 22:17, 1 June 2006 (UTC)

Hopefully the line stands so that all these ridiculous WEB 2.0 BUZZWORDS go away. Podcasting, crowdsourcing, who cares? It's just a giant nerd circlejerk. --Liface 02:25, 24 June 2006 (UTC)

Outsourcing is a contractual arrangement between a central agent and one or a few contractors known ex ante. Crowdsourcing is a quasi- or implicit contractual arrangement between one or many agents and many contractors who are only known ex post. I hope that satisfies. Unlike podcasting, I think crowdsourcing is a legitimate concept: Wiki and linux both come to mind as examples. Podcasting used to be called "digital recordings", and have not changed. Ehusman 01:07, 16 July 2006 (UTC)

Open Source Projects that are really distributed

MySQL certainly does not represent distributed OSS development, it's rather centralistic due to their dual-licencing model. — Preceding unsigned comment added by 130.149.23.190 (talk) 13:21, 20 September 2006 (UTC)

This is rather odd...

Somebody purchased a Google ad linking to this article. Try a crowdsourcing search to see. More discussion here: http://www.micropersuasion.com/2006/09/wikipedia_adver.html - EurekaLott 20:48, 20 September 2006 (UTC)

Many of the more commercial external links have been removed. Feel free to discuss them or add some back in if necessary. Many of them seemed to be of the "this is an example of crowdsourcing" type. I think it might be better to just use internal links for these, even if it's a redlink, and then describe why the company might be a good example of a crowdsourcing company. Certainly some of the companies are referenced in reliable sources [1]. --Interiot 04:41, 21 September 2006 (UTC)

I just removed all ext links except Jeff Howe's blog. These will need a careful eye kept on them. - David Gerard 10:36, 21 September 2006 (UTC)

I've traced the editor and company who placed the content on WP (and likely arranged for the adverts) and they have been (a) blocked, and (b) the article about themselves they had also added has been removed. WP is not a place for companies involved in a particular business to promote themselves nor a place for a list of such businesses. --AlisonW 11:39, 21 September 2006 (UTC)

Cambrian House, I take it. I'm not sure speedy deletion was appropriate there, does the company pass Wikipedia:Notability (companies and corporations) standards? If so we should just clean it up instead. Bryan 16:02, 21 September 2006 (UTC)

Large parts of it were pretty clearly COI/advert, so it would have had to have been mostly rewritten anyway. Considering that it launched 3 months ago, and has no mention in CNET/Wired/ZDNet/Google News (except for a small mention here), I'm not sure how it could meet WP:CORP. --Interiot 16:32, 21 September 2006 (UTC)

Advertising

As noted above, it has come to the attention of Wikimedia that someone appears to have purchased links from Google (and possibly elsewhere) leading to this article. Neither the Wikimedia Foundation not the Chapters arranged for any of this advertising, nor were they aware in advance that such advertising was being arranged by a third party. It is likely that an individual or company who had previously managed to place links on this page arranged for this advertising as some method of their gaining credibility and additional advertising. Wikipedia policies do not encourage links to commercial websites and our usual practice is to delete them where found, as has been done with this article. --Alison Wheeler, Chair Wikimedia UK. 11:20, 21 September 2006 (UTC)

---

Could Second Life be an example of crowdsourcing as well? After all, there is no way Second Life's creators, Linden Lab, would be able to create the whole content of this million-user virtual world on its own, even if they had a thousand 3D designers working for 10 years...

--Gwyneth Llewelyn 22:25, 17 October 2006 (UTC)

---

Crud

While the concept as a whole sounds somewhat valid, it feels as though it was conceived solely as a buzzword for articles and books and new, hip management styles to be pushed to execs. A google search still has the adword pointing to the wikipedia page, even though it's the first hit, and it's first page only has a businessweek article, a technorati cluster link and a handful of blogs set up JUST to discuss crowdsourcing.

Deletion would seem like a half decent idea. --hif 11:36, 4 November 2006 (UTC+5)

In English, it basically means "pay a bunch of suckers little or nothing to do your work for you". It's not a new concept...but now it's on THE INTERNET! *groan* --Takeel 14:58, 13 November 2006 (UTC)

Or getting individual users to build you an encyclopedia for little or no money, and gettign away with it by calling it a Wiki or something 65.94.190.146 03:02, 4 December 2006 (UTC)

Wikipedia isn't for profit, ya dig? --71.62.243.176 04:20, 5 January 2007 (UTC)

Crowdsourcing = a big company getting free intellectual property from which obscene amounts of money are made.

Bigdumbdinosaur (talk) 23:28, 19 February 2009 (UTC)

NPOV, Spam and Sources

I'm troubled by these warning indicators. I coined this term and have thus far declined to edit or comment on this page in any way. But this article has become so addled with these warnings that I felt compelled to step in ask for some assistance. Let me address each warning in order:

1) NPOV: The ideas in this section, such as collective customer commitment and mass customization, are founded in research conducted by MIT professors Frank Piller and Eric von Hippel, respectively. They are, imo, relatively small slices of the crowdsourcing pie, and I probably wouldn't have written this section, but the ideas are clearly related to Crowdsourcing. Further, this seems like a fairly neutral, flat description of the concepts, certainly not over-promotional. I'd suggest this warning should be removed.

2) Spam: This section is indeed a spam magnet, but generally some right-thinking editor comes along and nixes the offending addition. I removed "Steve Jackson Games" from the list of examples tonight. I've been tracking crowdsourcing assiduously for almost a year and I've never heard of the company. In addition, from what the entry said it doesn't sound like crowdsourcing to me (customer feedback is not crowdsourcing). However, the other 13 examples would be considered to follow a crowdsourcing model, a fact well-documented in popular literature (Cambrian House, for instance, has been written about extensively in the press as well as on many respected blogs. Again, the descriptions don't seem over-promotional.

3) Sources: I'm not in a position to contest this because I'm not sure what it means (and yes, I did go to the relevant explainer page). Are secondary sources like my Wired article insufficient? The one I did for Time Magazine? More to the point, other journalists have covered it from publications like the Boston Globe, the Canadian Globe & Mail, CNET, BNET, Business Week, Business 2.0, et. al. Right now Nexis shows 175 stories mentioning crowdsourcing. I'm happy to add the best, most thorough and balanced of these pieces to the "external references" section, but I'd like to make sure that's what's needed.

I'm concerned that some of this heightened scrutiny is rooted in a distrust and dislike for the emergence of the term--Everyone loves to hate a buzzword. No argument here. But love it or hate it, crowdsourcing seems to have crossed a tipping point in this regard. There are currently 2.6 million Google hits for the term; it's the subject of at least three doctoral dissertations that I know of; and Wired.com and NYU have just launched a major research [project][2] into crowdsourcing methodologies.

All this said, I do think a healthy skepticism is missing from the Crowdsourcing page. I'm no blind advocate of the trend. I think it will liberate people's potential creative and intellectual talents, and I think it will promote meritocracy over nepotism by creating a level playing field. I also think it will lead to exploitation in the hands of unethical companies. It will also lead to serious economic disruption, with people losing jobs and entire industries undergoing painful upheaval. All this is missing from the page, and I'd feel uncomfortable adding it myself. Hopefully one of crowdsourcing's critics (and there are more than a few) will write a thoughtful critique of the model.

I have taken the liberty to replace the definition with my own. Quite a bit of consensus has developed around this definition, and it's the one we're employing in Assignment Zero, the NYU/Wired research project I mentioned above. Other than this I'd rather not inflict my will on this page, leaving that up to the crowd that makes Wikipedia such a magical resource. JeffPHowe 10:23, 16 March 2007 (UTC)

Thanks again for taking the time to opine on this, and doubly so for keeping at arms length from the content. Your knowledge on this term will be very useful in building a well rounded article on the subject. Firstly, the indicators that you have termed warning indicators are often misunderstood to mean that, and its not hard to understand why: they are gross. That said, they are best understood if you think of them as editing/reviewing markers on an unpublished article. Basicly, this article isnt up to scratch yet, and contributors have decided to add the tags to ensure the reader doesnt think this article is representative of Wikipedia's best work.

Ok, onto each tag...in order of importance...

{{Unreferenced}}: This meanly means that the article doesnt provide sufficient reliable sources; yes, secondary sources are fine. If you can add references to five good quality articles amoungst the 175 on LexisNexis that mention crowdsourcing, that would be sufficient to remove this tag. When selecting the "best", give preference to ones where the article predominately about crowdsourcing, from reputable sources, and articles that can be accessed online without subscription (see google news archive. Also, some consideration should be given to demonstrating the term is in regular and continual use (currently the reader could come away from the Wikipedia article thinking the term was a flash in the pan during June/July 2006).

{{POV-section}}: This indicates the article isnt balanced; specifically the section "Advantages" was created without any indication that there was disadvantages. To not have a tag here would let our readers assume that outsourcing is a brilliant basis for a business plan, as there are no disadvantages. In your comments above you outline a number of macroeconomic disadvantages of crowdsourcing, which I think can be summed up as "TANSTAAFL". Unless the enthusiasts participating are gaining economic value from their investment of time, a company that is using crowdsourcing is a net-loss to the economy, and the market or courts usually dispenses with such companies at the first opportunity. I cant think of a good example of this (hopefully you can), but I expect we can find cases where the community has lashed out at the company with little provocation, simply because the community feels that they have as much ownership of the resource that has been created, and they see indicators of bad stewardship. This tag will be removed when the article covers the good, the bad and the ugly. John Vandenberg 02:35, 17 March 2007 (UTC)

{{cleanup-spam}}: As you point out, this section is being kept under control. The tag was added a few days ago [3] by User:Interiot because this section is attracting SPAM at a rapid rate. To address this, we need to work out why people are adding entries, and try to discourage it. The simplest fix is to remove any mention of companies that do not have an article, as that usually means they are Not Notable. At the moment, the article mentions NowPublic; if you think that company is notable, we can write a brief article for it. If not, we should remove it otherwise it attracts more entries of that ilk. Another way to keep this list trimmed is to enforce a decision that each entry must be accompanied with a secondary source to justify that it is an appropriate example. This approach would require that we find a source for each of the current examples. Another approach is to convert the list into prose that links the key examples of crowdsourcing. Lists in general attract unwanted additions, but prose attracts clever inappropriate additions that are more difficult to spot.

Hopefully one of crowdsourcing's critics ... will write a thoughtful critique of the model.

It isnt strictly necessary that the disadvantages be spelled out by a secondary source, as the the criticisms can be founded in economics and humanities, and thus attributed without the sources needing to mention "crowdsourcing" specifically. However, it would be a tautology for a contributor to Wikipedia to be sufficiently critical of crowdsourcing to do the leg work required to join those dots. :-) I've optimistically added a blank "Disadvantages" section to the article in the hope someone feels the need to demonstrate that they are not myopic. John Vandenberg 02:35, 17 March 2007 (UTC)

AS a first reader of this entry it seems rather naive in terms of placing the concept with general theories of innovation and the role of users in innovation processes. —Preceding unsigned comment added by 129.215.101.110 (talk • contribs) 16:49, 23 March 2007

Open Source vs. Crowdsource

I've declined to edit the "overview" section although I feel it is less an overview than a view on the distinction between crowdsourcing and open source projects, a fairly contained issue. Neither is it factually accurate: it is not true that "In this new model of collaboration the results of the global efforts returns only in the organization which leads the project." Crowdsourcing merely implies a form of labor procurement; it does not imply who benefits from the labor. Certainly the results of the crowd's efforts often benefit those members who contributed the labor, in both financial rewards and personal satisfaction. The global commons often benefits as well. The US Patent Office recently initiated a form of crowdsourcing, in which an open call for experts to review pending patents is being employed. The benefits from such a program would be widespread. The unfortunate association between crowdsourcing and corporate greed is a misapprehension. Like outsourcing, crowdsourcing is merely an approach to economic production, and can serve self-interest and the interest of the commons, sometimes at the same time. However, this is admittedly my view (even if it is backed up by considerable research) and as such I'll keep my argument to the talk page. JeffPHowe 11:15, 16 March 2007 (UTC)

My immediate reaction to reading this section was that open source is the most established form of crowd sourcing, but with the legal protection built into some licenses to prevent abuse (the truly free/open licenses). Open content is in the same boat as far as I can see. Even though open source does not initially start as a business model, it oft is a key factor in business models by other parties. e.g. IBM is now betting a decent part of their business on laissez-faire style development which they employ people to participate in, as they consider it a valid strategy to compete with Microsoft and Sun in the market place.(this is spelled out in the IBM court filings below) As a result, I can see why Wikipedians have focused on how crowdsourcing relates to open source.

The last sentence is rather odd; the article is talking about procurement, in which case there should be no expectation that anyone else other the company that pays will benefit. Any other benefits are bonuses. The problem is ensuring the 'employee' is adequately remunerated. I've been playing around with MTurk for the last two weeks as it sparked my interest in this subject. I'm actually wanting to create some HITs, so I thought I would attempt a few myself to see how it all works. So far, I've been paid $0.21c for about three hours effort. While I am feeling pretty happy about those microcents, I've come away thinking that its quite disgusting to see crowdsourcers are not willing to pay respectable amounts for difficult tasks. I've got a nasty streak of unionism in me, so the per HIT loose arrangements that are binding the employer and employee are already a bit of an issue; coupled with the extremely low rates of pay, it looks more like pure exploitation. As far as I can see, MTurk HITs can only be created by a US employer; I'm guessing this is because there are no minimum wages or other applicable regulation in the US. It would certainly create a stir here in Australia; we have started making provisions for these types of contractual employments (WorkChoices), but we have retained minimum wages as well (I haven't had to grapple with these new laws yet so I'm not sure how these are both true). Our unions like to stir up a lot of hot air about any scenario when a employee is taken advantage of. John Vandenberg 09:50, 20 March 2007 (UTC)

Open source as Crowdsourcing

As a wade in the open waters a bit, I've thought of a few cases that are worth mulling over.

Minor issues

• The Mozilla Foundation preventing third parties from distributing products using trademarks that are owned by the foundation. This prevents Debian from altering the source of Firefox in minor ways and calling the result "Firefox". They needed to call it Iceweasel.
• JasperReports "closed" an open source project. This was a very amicable and legal process, as there were few contributors at the time. However the result is this piece of software is very close to crowdsourcing -- JasperSoft is uniquely able to commercially benefit from improvements provided by others.
• MySQL and Qt have similar arrangements, where one company is able to make money licensing the code-base to others under a different license.[4] Its worth noting here that this was important enough that Trolltech has provided a special license that allows unrestricted use of Qt with KDE in order to pacify the peanut gallery.

Not real sure about these two

• Apple Safari does it page layout using a renderer called WebCore which is a fork of KHTML. There has often been "problems" cited that Apple is not "giving back" to the KHTML codebase. The most recent example was that was when Safari was the first browser to pass the Acid2 test, everyone expected that the fixes required to achieve this would be provided to KHTML.[5] IIRC Apple did in fact make the code changes available even before the controversy began, but in a supposedly unpalatable manner. It was amicably sorted out, and I'm guessing that Apple is a bit more cautious about this as a result.
• Apple Mac OS X is built on an open source OS called Darwin and there was doubts about whether it would be possible to rebuild Mac OS X from source.[6] (this is a similar problem to Tivoization below, but in this case the majority of the code was written by Apple so its other peoples work that is being misappropriated as it is people expectations that are being dashed) I've not followed this one so Im not sure what the outcome was.

Major problems

• Tivoization is a using a difficult to plug loophole in GPLv2 to restrict users from having the well defined benefits of the GPLv2. A lot of linux kernel hackers dont really mind, but a decent percent of free software adherents consider this illegal as it is against the spirit of the GPLv2. GPLv3 is attempting to resolve this, but the linux kernel cant adopt GPLv3 easily, as agreement from all kernel contributors would be required.
• The SCO Group. They tried to impose a fee on Linux users worldwide because they claim to own "Unix", yet at the same time they distributed multiple operating systems that used open source. Many open source projects stopped maintaining their projects for Open Server and Caldera, and some even used legalese to prevent their package from being included in these operating systems (e.g. nmap). If you are interested in this, I recommend reading IBM's latest filings. Specifically 979 - 981 which spell out the way that this company has broken the GPL and has been grasping at straws for the duration of the proceedings. The IBM lawyers toss in some wit to aid you through the reading.
  • This is a good example of the "crowd" mobilising against the sourcer :-) groklaw is evidence of the lengths that the open source community will go to in order to bury this company.

John Vandenberg 09:50, 20 March 2007 (UTC)

Spam Disclaimer

Given that the Spam disclaimer only refers to external links, it seems to be an inappropriate header since the links are all internal. If the list of crowdsourced companies is growing too long, perhaps all but the 3 most prominent should be eliminated? --Antelan ^talk 03:29, 23 March 2007 (UTC)

That section was sporting all sorts of links not so long ago. We have just move to a "must have a Wiki article before being considered worthwhile" type approach to keeping the list under control. As you point out, that then makes the tag a bit pointless. John Vandenberg 13:54, 23 March 2007 (UTC)

Cafepress

In the interest of further thinning out the long list of crowdsourced businesses, I've taken a look at Cafepress, and its inclusion seems somewhat off-topic for this article. Cafepress is analogous to a book publisher, and does not seem to match the definition of a crowdsourced business as stated in the opening of the article. I plan to remove the link one week from now unless there is disagreement. Antelan ^talk 17:35, 23 March 2007 (UTC)

I think it was included because it has a similar "feel" to what MTurk is doing. MTurk a marketplace for crowdsourcing, while Cafepress and Zazzle are market places for other innovation. CafePress was added here in November, and Zazzle was added in October. John Vandenberg 22:50, 23 March 2007 (UTC)

Thanks for your reply. The edit history is useful in understanding why someone might have added a link to Cafepress. However, given that you said that "Cafepress and Zazzle are marketplaces for other innovation," and given my previous analogy between Cafepress and a book publisher, I still don't feel that Cafepress's presence in this article is justified. Antelan ^talk 22:58, 23 March 2007 (UTC)

I agree that it is an odd entry. Do you want to ask the contributors of those two diffs whether they would like to comment ? John Vandenberg 02:23, 24 March 2007 (UTC)

Good call. One was added by an anonymous contributor, but I managed to find the other contributor and leave a message on his/her talk page two days ago. No response yet, but I'm in no rush. Antelan ^talk 17:19, 25 March 2007 (UTC)

Pending deletion of "Marketplaces" and "types of crowdsourced work"

Given that encyclopedia articles are not generally lists of services, I plan on deleting the "marketplaces" and "types of crowdsourced work" sections entirely in the next few days. If there is opposition, please state here. My intention is to clean up this article and make it legitimate in its own right, instead of allowing it to continue as little more than a disambiguation page for crowdsourced projects. Antelan ^talk 05:49, 29 March 2007 (UTC)

I'm writing an article on crowdsourcing under deadline this morning, and I found these examples to be helpful. But whatever you think. TimidGuy 15:51, 5 April 2007 (UTC)

I am not suggesting this course of action because I think the links are unhelpful; rather, I think the links do not contribute to this encyclopedic entry. To help a visitor understand what crowdsourcing is, it would be more appropriate to select three fairly orthogonal examples from that list and flesh them out. To preserve this list in a fashion more apt for Wikipedia, you could create a category 'Crowdsourcing' and each of the pages on this list could be tagged as such. Antelan ^talk 20:52, 9 April 2007 (UTC)

Good point. Thanks. TimidGuy 17:20, 17 April 2007 (UTC)

"Public Insight Journalism, A project at American Public Media to cover the news by tapping the collective and specific intelligence of the public. Gets the newsroom beyond the usual sources, uncovers unexpected expertise, stories and new angles."

That's pretty much advertising-speak... --John Lunney 23:59, 18 April 2007 (UTC)

Restored deleted marketplaces

I've temporarily restored the marketplaces as they were brought here in order to remove them from MTurk. I expect these market places need to be further rationalised and possibly merged into Marketplace, however I suspect that Mycroft and HumanGrid are not the marketplaces that we want to delete, as they are academically notable. John Vandenberg 19:57, 3 May 2007 (UTC)

This is not meant to be a list of crowdsourced projects. Imagine if protein had to list every known protein, or even every major one. Just because something is academically notable does not mean it should be linked in lists from every possibly related article. As an aside, if HumanGrid is notable, it should have its own Wikipedia article. --Antelan ^talk 23:21, 3 May 2007 (UTC)

This is a work in progress. Be constructive. At present, the list is valuable because the article isn't fleshed out with lovely prose. Examples with sources assist the reader understand, and also serve as a scratch pad for content that other Wikipedians can use to expand the article. Also as I said above, a lot of this content has been moved to here from MTurk as it clearly belongs here more than it belongs there. (and yes, I'll get started on the HumanGrid article as I know more about that project than Mycroft). John Vandenberg 00:16, 4 May 2007 (UTC)

I respect that you want to clean MTurk, but I do not think that this article's main page is the place to keep snippets that may be useful later. For now, I will leave the list here on the talk page. I think there needs to be some discussion about the role of this article. This is to be an encyclopedic article about crowdsourcing. As such, it is to be about the concept itself, why it exists, how it has been defined, etc. The role of this article is not to list off crowdsourced projects, even in prose form, although describing a small, representative set would be appropriate, especially with citations.

To see why I think this article's quality is slipping dangerously, consider this: An article on software would not enumerate as many software companies as it could find. Even an encyclopedia article on software companies would not enumerate as many software companies as it could find. Our massive list of potentially crowdsourced companies and websites is unnecessary and does not contribute to the goals of an encyclopedia. The far better solution would be to link to an outside reference that keeps track of what it considers to be crowdsourced projects.

As it stands, we're coming dangerously close to WP:NOR, since none of those items on the list have references demonstrating that they are crowdsourced. This compounds the fact that we're operating under a weak, disjunctive definition of crowdsourcing. I am moving the material to the talk page simply because I believe that doing so increases the quality of this article. As an aside, linking to a project's page in a statement is distinguishable from writing a sourced statement about that project. Antelan ^talk 20:33, 4 May 2007 (UTC)

Again, you have removed two sourced entries from "Types of crowdsourced work" in your zealousness to remove the Marketplaces section. Both sources clearly define them as crowdsourcing. Including a list of examples is normal practise. is it ideal? no, but its useful when a subject is in development by amateurs using their free time. Your comparison with an article on software is, quite ridiculous, stupid as that is a generic well established field, and the article is mature. John Vandenberg 09:15, 6 May 2007 (UTC)

The argumentum ad hominem personal assault was unnecessary. However, you did concede, by comparison, that the mature version of this article should not contain these lists. See also WP:TRIV. Antelan ^talk 15:51, 6 May 2007 (UTC)

==Marketplaces==

Further information: Marketplace § Internet Markets

Amazon Mechanical Turk co-ordinates the use of human intelligence to perform tasks which computers are unable to do.

Google Answers was a fee-based "answer brokering" search and research service offered by Google from April 2002 until late December 2006.

The Collaborative Human Interpreter by Philipp Lenssen also suggested using distributed human intelligence to help computer programs perform tasks that computers cannot do well.

Mycroft from the Berkeley School of Information (project site) applies a similar strategy of segmenting work into small tasks and capitalizing on those things that humans are good at but computers are not. Unlike MTurk, Mycroft is a distributed system that gathers knowledge through small, self-contained banners that appear on sites all over the web. In addition, Mycroft does not pay workers. Instead, it relies on a variety of non-monetary incentives.

In Europe, privately funded HumanGrid GmbH, is implementing such a crowdsourcing platform as well. HumanGrid is designed to be a very generic platform, able to map complex workflows and to include several quality management principles.

Both Rentacoder, Guru, IT-Globalized, and IPSwap allow people or businesses to request the development of computer programs; software developers can bid and get the contract to write the program.

Subvert and Profit allows advertisers interested in Undercover marketing to purchase actions on social bookmarking services such as Digg, and pays users of these services to perform the actions.

Changes to Definition and History

I made some slight changes to both sections. I wrote the original Wired article coining the term, and have conducted extensive research into the trend over the ensuing 14 months, including tracking it on my blog, crowdsourcing.com. I'm attempting to promulgate a semi-official definition that will throw some rough parameters around the usage of the term. I recently helped lead a six-month investigation into crowdsourcing (named Assignment Zero; referenced within the entry) and one of the major problems we ran into was differing, even contrasting understandings of the word.

Also, I added my name and Wired's into the history section. This doesn't seem self-promotional, only factually accurate. Crowdsourcing, the term, did not arise mysteriously out of common usage, as many now established words have. It was a highly conscious attempt to find a word that threw a lasso around a bunch of seemingly unrelated developments and show what they had in common. After fifteen years writing major magazine feature stories, I can tell you such hubris is rarely rewarded. This time it was. Wired's role, and my role, in inventing the term seem like a pretty elemental part of the idea's history.

Finally, I want to thank the various contributors, and John in particular, for all the excellent work cleaning up this page and putting some needed rigor to it. I'm proud to be associated with this page.JeffPHowe 18:56, 29 July 2007 (UTC)

---

Please sign all posts using the four tildes. Mrslippery 09:30, 29 July 2007 (UTC)

---

Hi Jeff, the reason I removed your name is because it seemed non notable to include it into the history section. Wired Magazine is notable enough to have an article in Wikipedia. If the same isn't the case for your name, it is sufficient enough to say that the term originated from Wired Magazine and that readers probably wouldn't care to know who at wired actually coined it. We could go as far as to say how you thought of the word, etc but we have to draw the line somewhere and it would make sense to draw it at WP's guideline for notability. Keep in mind that the notability of the act of coining the term hasn't even been asserted. i.e. What notable source mentions that Wired was the first to coin the term? Why is this worth mentioning? etc. The original research box is due to the definition not having a source and that much of the article in unverifiable. i.e. when you read the first sentence of the article, can you ask "Who said this?" and know the answer? same for the second, third sentence etc. Please take this opinion as constructive and know that I'm open to opinions and edits by others. What does everyone else think about the history section? Pdelongchamp 21:13, 30 July 2007 (UTC)

---

Thanks for the clarification, and I do take the opinion as constructive, though I'm still a little confused. My role and that of Wired's in coining the term is well-established. I've been quoted in numerous publications both scholarly and popular, have appeared on national television and at conferences discussing crowdsourcing. Nearly all of them mention my role in developing the term. I've hardly made a tally, but could probably come up with at least ten and probably 20 other references that note my and Wired's role in the development of the ideas around crowdsourcing. In terms of the definition, doesn't the combination of my blog and other writings constitute a verifiable source? I coined the term and have made a full-time job of investigating and exploring the application of the model over the course of the last 18 months. Why would my definition raise an "unverifiable" flag, but some other random contributors not? Fwiw, this definition has been parsed over the course of many conversations with the people who are pioneering crowdsourcing (in fields as diverse as journalism, graphic design, software development and journalism). I'm widely accepted as an expert on the subject and have sold a book on crowdsourcing that will be published in eight countries including the US, UK and Japan. I say all this not to establish that I'm notable (which I agree is arguable; outside of crowdsourcing and a seminal examination of the file-sharing underground, I have a thin claim on any kind of wide renown), nor to brag, but only to attempt to make the case that I, and my blog, crowdsourcing.com, are trusted sources. Again, thanks for the thoughtful note and help. I'm not a frequent contributor to Wikipedia and am anxious to follow etiquette and guidelines.JeffPHowe 02:39, 1 August 2007 (UTC)

I see that in The Long Tail article Chris Anderson of Wired is credited with having coined that term. If Mr Howe did devise the term crowdsourcing then it would probably be correct to note that in the article. Mrslippery 09:34, 1 August 2007 (UTC)

Jeff, if reliable sources mention you along with Wired as having coined the term, I would invite you to select the best source and reintroduce your name as well as the source back into the article. As for verifiability of the definition, I believe that tag has been there for quite a while and shouldn't be taken personally. It simply means that the definition isn't sourced. (i.e. there are no footnotes to tell us where the text in the definition came from.) In Wikipedia, even if you're an expert, (i mean you in a general sense) you can't just add your own thoughts or opinion to an article. Everything you add has to come from a reliable source. i.e. Even if I were a medical researcher and I knew that smoking caused cancer, if I wanted to add that to the Cigarettes article, I would have to quote a study. Or if I had invented WiFi and I wanted to write a definition for Wifi, i would still need to say exactly where the definition was coming from. and it would need to come from a reliable source. (not just from my own opinion, whoever I may be) This all has to do with verifiability. No one should trust anything that is written on Wikipedia. If you read a sentence and can't tell where the information is coming from, (i.e. it isn't sourced) you should consider it useless unreliable information. That's kind of the idea behind it. If you do have reliable sources to add to the information in the article, it would be great if you could add it. Pdelongchamp 14:30, 1 August 2007 (UTC)

MrSlippery/Pdelongchamp: Thanks to you both. This makes sense and has the added benefit of giving me greater insight into what makes Wikipedia tick. Essential, as my book on crowdsourcing naturally covers Wikipedia, a crowdsourced reference work. I'll be back with a few references in next few days.JeffPHowe 16:33, 3 August 2007 (UTC)

All: I've reinserted my name in the history section, with a citation referencing an article in Fortune Small Business from March, 2007. There are two other sources I could have used: The New York Times: http://www.nytimes.com/2007/03/19/business/media/19carr.html?ex=1186286400&en=1182f5593f0daded&ei=5070 and Business Week:http://www.businessweek.com/innovate/content/jan2007/id20070118_768179.htm?chan=search. Hope this satisfies.JeffPHowe 20:18, 6 August 2007 (UTC)

DARPA Grand Challenge?

From the examples on the page, the DARPA Grand Challenge is certainly an example of crowdsourcing. Was it not included because it would make the list too long or was it just not put in? Thylacine222 14:06, 6 August 2007 (UTC)

"Some people argue that the definition of Crowdsourcing is bound to online methods only." Clearly not so. See the historical examples of crowdsourcing. >>That's reasoning in circles. It just doesn't make sense to call everything with a prize attached to it crowdsourcing. It is a trend NOW because technology makes it possible to cooperate with large groups of people. I thus suggest "technology-driven" for consensus.

"Others (like Clay Shirky) say Crowdsourcing has to involve payment for it to be called crowdsourcing." Clearly not so. See the preparation of the OED and Wikipedia. Mrslippery 09:54, 25 August 2007 (UTC)

Wikipedia not cited as an example?

Has there been an explicit rejection of the notion that Wikipedia itself is a prominent example of crowdsourcing? As I read WP:WAWI, this article would seem to fit the last bullet: "Articles where Wikipedia is illustrative of the subject." --Sfmammamia (talk) 02:38, 13 February 2008 (UTC)

No comments here in 5 days, so I've made what I hope is a balanced attempt at reference to Wikipedia in relation to crowdsourcing. --Sfmammamia (talk) 03:31, 19 February 2008 (UTC)

Request for inclusion on this page

I would like to humbly request that a neutral user consider the ongoing Superdelegate Transparency Project for inclusion on this page. We have recently been featured in the New York Times, Wired, The San Francisco Chronicle, and Bill Scherr's Liberal Oasis. Wired magazine specifically cited us as an example of crowdsourcing. We have partnered with Huffington Post's Off The Bus crew giving us over 300 volunteers pulling together information for the project.

I'm not entirely sure if this elevates it to the proper level of notoriety or not, so i would just ask that the question be considered. Thank you.

Link

--Markometer (talk) 02:48, 22 February 2008 (UTC)

prizes, by themselves, are not crowdsourcing

I've made some changes to the "history" related bits. Whomever wrote them seems to have a totally non-discerning definition of crowdsourcing which means just announcing award money for some sort of task. This is not crowdsourcing. Crowdsourcing is about utilizing the collective resources of many individuals (usually via technology of some sort) to achieve some cumulative, large task. Longitude prize, for example, is NOT crowdsourcing—it's just a reward that is out there to find an answer to a problem, and yes, many people tried to find the answer, but only ONE person did, and the work of everyone else did NOT contribute to the final result. It's quite a different approach than a thousand people agreeing pour over satellite imagery—even though maybe only one person finds the thing people are looking for, the act of looking at the imagery itself is considered a contribution to the final result (it is as important to know what areas do NOT contain anything as it is to know which ones DO, in the end, and all of the work was being centrally coordinated to avoid duplication of effort).

I don't think the distinction I am proposing should be novel or controversial. What distinguishes crowdsourcing from everything else is the coordination of the crowd involved. Not just that many people are involved. Even then I think that the technology is probably essential to this—thinking about it just in terms of lots of people working towards some common goal leads one to conclude the Manhattan Project was crowdsourcing, which is obviously a ridiculous use of the term (the term loses any real meaning). I think saying that the OED solicitation of scraps of definitions from people in the 19th century is close enough to crowdsourcing to warrant mention, though even that I would only label as being "similar, sort of" to crowdsourcing (the people submitting definitions are not really very coordinated). --140.247.240.135 (talk) 18:06, 2 May 2008 (UTC)

Google Image Labeler

Does Google Image Labeler qualify as an example? -- Kq-hit (talk) 12:45, 11 November 2008 (UTC)

References editing

Not sure how to edit References, but it seems to not be open at the moment. If anyone has access, reference 17 is a dead link and that article is now located here: http://www.openpathdesign.com/lost_in_the_crowd.html

Thanks —Preceding unsigned comment added by Mcd001 (talk • contribs) 20:33, 18 November 2008 (UTC)

I'm not clear how the "Fourneyron's Turbine" is a historical example of crowd sourcing. It might be, but the linked-to article doesn't say anything about it. Mcswell (talk) 16:27, 20 July 2009 (UTC)

ref

Jeff Howe has written his book as of 2008: CROWDSOURCING. Thorough, lots of examples, explores the components of "crowdsourcing."

David —Preceding unsigned comment added by Smitti79 (talk • contribs) 23:47, 5 February 2009 (UTC)

The Image

What purpose does the image serve? As far as I can tell it could be just as effectively (if not more effectively) expressed as a numbered list, with the advantages of a lack of JPEG artifacts, no need to recreate the image in a non-lossy format, and inclusion in text search. Stannered (talk) 21:44, 25 April 2009 (UTC)

Vegemite fiasco

in 2009, the Australian food product vegemite attempted to source a new brand image, cited at Vegemite#Advertising_and_branding. I have heard this described as Crowdsourcing..but am not sure ...Feroshki (talk) 01:27, 26 October 2009 (UTC)which isnot

Definition seems different from the book

Hi, the definition of crowdsouring on wikipedia seems far narrower than the way Jeff Howe has outlined it in his book. As I understand it the word crowdsoursing is intended as an umbrella term for a variety of techniques aimed at harnessing voluntary contribution from everyday people. “When someone correct a misspelling on Wikipedia, uploads a video to YouTube, or suggests an edit to an author (-) who has posted their book online, that’s all crowdsourcing. As far as concepts go, crowdsourcing pitches a pretty big tent” (Jeff Howe, Crowdsourcing p177, 2008).

I would assume that Howe should be the definitive voice on the subject given that he coined the word and wrote a book about it? - Mike NZ —Preceding unsigned comment added by 130.123.192.23 (talk) 03:41, 13 January 2010 (UTC)

From what I understand of crowdsourcing via one of my college classes is that crowdsourcing is related more to business and advertisement. It requires the participation of many people to reach a common goal; but "crowdsourcing" often has to do with how businesses get their audiences to participate in buying their products. Crowdsourcing can also be implemented in an artistic form, where an artists piece of art relies on the help of others to make the piece. —Preceding unsigned comment added by Dracothejuggler (talk • contribs) 03:00, 6 May 2010 (UTC)

The definition of crowdsourcing term is still an evolving one and do not forget that it is currently considered a nomenclature and it yet has to cristalised. Numerous authors, especially Daren Brabham tried to separate crowdsourcing strongly from the open source topic. This separation however is arguably a little bit artificial at best. Having read widely on the topic within academic literature my current opinion on crowdsourcing is that it is more a term of convenience used in a business/organisation facing a problem, where some constrained problem is presented by a given entity context. However elements of Open Source have been defined as such in earlier works and by a number of very clear case studies (I can cite sources if need be). Anyway, it would seem that Crowdsourcing is more an instance of Open Source, or open source being an instant of crowdsourcing, depening on the particular problem at hand and your point of view. —Preceding unsigned comment added by 158.125.103.37 (talk) 18:24, 25 August 2010 (UTC)

Disadvantages

I'm going to throw down a few sentences to get people started. It would be a lie to pretend I don't have some vested interest in the promulgation of this term, so it puts me contributions in a specious light. I hope that such full disclosure of bias, and a warm embrace of balance will help offset that. I encourage any and all to add to my bit, or elide it in favor of their own. I have, for the record, removed the comment "get your ideas ripped off by big business!" as it smacks more of vandalism than a substantive addition.

Not quite sure what you are trying to get at. I suppose some disadvantages of crowdsourcing is that as you mentioned, some of the participats could have their "ideas ripped off". With some projects, there are many participants where it would be difficult to give credit to all of them. A disadvantage could be that people who submitted an amazing idea might not get the credit or even get paid for their contribution while the business makes the money. However, participants act in more of a voluntary sense and by participating in the crowdsourcing project they may not expect to be paid; unless the business announces some type of benefit for the volunteers. Dracothejuggler (talk) 03:07, 6 May 2010 (UTC)

Individuals getting more power

Perhaps a paragraph can be added about the individual getting more powerful trough crowdsourcing, and internet-related activities (eg harvesting power of botnet, ...). Can be used for environmental purposes, according to John Arquila. Gathering info from protected sites may also be useful for building gyroscopes, and robotic systems according to Shawn Carpenter (latter useful for certain environmental systems). See Cybercrime documentary (http://www.gsnmagazine.com/cms/market-segments/critical-infrastructure/458.html ) —Preceding unsigned comment added by 81.245.186.75 (talk) 12:10, 3 February 2009 (UTC)

It depends on what you mean by "power". Participants can get more power in a variety of ways. One perspective of power can be that the participant gets paid for their contributions; like how some participants of Mechanical Olympics were paid for getting first place, or how some businesses pay people to take surveys about their product/service. Power can also be in the form of publicity and getting your name out there, where you might become more socially popular (often more in a virtual sense) because they participated in a crowdsourced project (and say they won first place in mechanical olympics). Another way they can gain "power" is possibly by seeing how their contribution effected the project in a whole, bringing up a sense of accomplishment.Dracothejuggler (talk) 03:14, 6 May 2010 (UTC)

"Appeal" section

I added the "appeal" section, because there did not seem to be any explanation on the page which explains why people participate in crowdsourcing. Echalhou (talk) 21:34, 29 April 2010 (UTC)

Methods of web-based crowdsourcing/examples

While there are examples throughout the document, it would be nice to include something about the current technologies used, their implementation, and their adoption rates. I feel this is important in the Web-based section, even is this is just links to other articles, as some of these articles already exist. —Preceding unsigned comment added by Jjblalock (talk • contribs) 00:06, 24 November 2010 (UTC)

Early examples

As far as I know, The Internet Movie Database was the first example of a collaborative database on the Internet, and as such I think it merits mentioning Kjetilho (talk) 21:38, 30 January 2011 (UTC)

More information on web-based crowdsourcing?

More examples of web-based crowdsourcing might be illuminating as to the sort of solutions that can be generated on such websites. For example, the article "Science by the Masses" (requires subscriber access) describes how crowdsourcing websites can allow a variety of people from different backgrounds who enjoy solving problems to find real-world problems to solve. The article shows how crowdsourcing can be particularly effective for solving problems that have not been solved in a field: people of a different academic or industrial background will often have a different perspective on a problem, allowing them to solve it when the experts in the field of the problem cannot. Web-based crowd-sourcing can allow problems to reach a variety of backgrounds and solve long-outstanding problems. Articles like this one could reveal the powerful effects of crowdsourcing. Sean (talk) 03:25, 31 January 2012 (UTC)

Purely speculation

The line: Grover relates that "maybe just asking a question is too simple. Maybe there has to be more complexity."

doesn't really seem to add anything valuable to the article, and implies that something has to be complex to be useful. The context is relating to a site that ended up the way it did for a variety of more plausible reasons than being 'too simple' an idea. Many other popular sites have flourished using just that concept. The fact that both statements start with "maybe" should be a clue that this is just pure speculation. It's not a very good quote, doesn't add any relevant information, and in fact might confuse readers trying to understand the concept of crowd sourcing. —Preceding unsigned comment added by 98.190.179.59 (talk) 14:03, 8 February 2011 (UTC)

Crowdsourcing Description, Origins, Examples

There are a number of changes that would increase the clarity and value of this article. The first would be to reframe the initial definition of Crowdsourcing in order to better align it with the ways in which terms are typically defined and describe on this site. For example, we could change it to something like "Crowdsourcing is a general term used to describe the process by which work is distributed amongst contributors." As it stands, the introductory section is rather hard to understand if you're not already familiar with the domain.

Next, I'd like to suggest changes to the History section. Although web-based crowdsourcing is a relatively recent phenomena, the general concept has been around for many years. It therefore seems prudent to start with a discussion about how different types of activities have been crowd sourced in the past and then talk about how the Internet has shaped the way in which these types of tasks are accomplished.

Finally, the recent examples section is rather sparse. There are many other examples we could include - InnoCentive, the Darpa Red Balloon Challenge, and Duolingo, that would help describe the range of activities that utilize crowdsourcing to accomplish tasks. RebeccaGulotta (talk) 08:40, 31 January 2012 (UTC)

More clarity and examples

I agree with Rebecca that the introduction of this article seems quite sudden and without an adequate amount of description. I also would argue that Jeff Howe should be better identified. As the article currently stands, the reader has no easy means to discover who Jeff Howe is or why he should be considered an authority figure on this subject matter. A citation of a source with this information would make it much more credible.

In the appeal section, it would also be beneficial to cite some of the relevant research which has been conducted surrounding social theory and how it may apply to the phenomenon of crowdsourcing. Although crowdsourcing is relatively new, there is nevertheless a wide body of research which could be referenced in this section to help readers gain a better understanding of why crowdsourcing has become so valuable. Sybek (talk) 05:59, 01 February 2012 (UTC)

Jeff P. Howe is the author of the book Crowdsourcing. — Preceding unsigned comment added by 72.251.57.97 (talk) 15:44, 25 February 2012 (UTC)

Merge from Ubiquitous human computing

It seems to me that Ubiquitous human computing is just another more fancy name for the same phenomenon. Thoughts? --_{Piotr Konieczny aka Prokonsul Piotrus| talk to me} 16:14, 12 January 2012 (UTC)

Looks like we've got a couple more neologisms at digital sweatshop and microwork; would support merging all of them into this article. --McGeddon (talk) 13:26, 10 February 2012 (UTC)

Ubiquitous human computing and Crowdsourcing are very different. Please read the paper by Dr. Jonathan Zittrain for clarification Saqib (talk) 05:01, 12 March 2012 (UTC)

Merging limits ideas and imagination! Lewis B. Sckolnick, Coyote Data Security — Preceding unsigned comment added by 72.251.57.17 (talk) 20:19, 23 February 2012 (UTC)

Prizesourcing

I guess, we should make 2 different terms: crowdsourcing - Wiki-like and, I guess, Yahoo Answers-like gamified models would be there too and prizesourcing - competitions rewarded with money and real-life prizes. Those are too different to have same name.— Preceding unsigned comment added by 62.221.56.166 (talk • contribs) 16:19, 22 February 2012‎

That reward systems vary across crowdsourcing systems doesn't seem a useful fault-line to split the article along, especially not if "prizesourcing" is a neologism that the world hasn't felt the need to adopt. --McGeddon (talk) 11:06, 12 March 2012 (UTC)

reference needed

This claim, which sounds logical, needs a reference.

"Some have questioned the ethical validity of providing no compensation or small amounts of compensation to members of the crowd that contribute to tasks."

I hope I put this in the right place. — Preceding unsigned comment added by 50.100.185.215 (talk) 19:31, 21 March 2012 (UTC)

An overall phenomenon on collective force

I've started a section on the Internet Talk page bringing up an overall phenomenon that's similar to crowdsourcing, but not exactly on this process. Maybe you guys or anyone could comment on it and discuss the phenomenon? - M0rphzone (talk) 06:22, 10 April 2012 (UTC)

Improving introduction

I want to rewrite the introduction part; it is very short and maybe a little bit more explanation would help. Also I've seen some books like a guide to open innovation and crowdsourcing and getting results from crowds that may be useful as new references. Nebeleh2 (talk) 03:48, 19 October 2012 (UTC)

Removed 'crowdfunding' from the lead

I wasn't able to find any reputable sources that used 'crowdfunding' as a synonym for 'crowdsourcing.' All the references I've seen limit crowdsourcing to the sourcing of labor, analogous to outsourcing. Below is the material I removed. I didn't remove the section in the article on crowdfunding, however.

Crowdsourcing is also another name for crowdfunding which has been introduced into law by the JOBS Act. Right now the SEC is continuing to develop and tweak the specifics of how the actual fundraising process will work. They will have until Jan. 2013 to make the final adjustments to the rules under the Act. Currently, online platforms will be able to raise up to 1 million dollars while sidestepping many of the strapping regulations which typically hold smaller companies at bay. By easing the regulations and the investor requirements, it will open up a lot of potential capital to smaller companies. Prior to the JOBS Act, most investors were only qualified if they were "accredited." Everybody is standing by to see what the final tweaks will be by the SEC, and a lot of online platforms are waiting in the wings to launch their funding programs. Robertekraut (talk) 19:36, 13 May 2012 (UTC)

Here’s a reference that draws a distinction between crowdsourcing and crowdfunding. 27th Annual Intellectual Property Law Conference Crowdfunding/Crowdsourcing Inventions and the IP Challenges See pages 3 and 4.--Nowa (talk) 20:43, 13 May 2012 (UTC)

Definitions of crowdsourcing

The proliferation of competing crowdsourcing definitions is bothersome. Jeff Howe, a journalist, coined the term "crowdsourcing" in the June 2006 issue of 'Wired' magazine, a popular press publication. Since then, it seems everyone has their own definition of this term. Bloggers, marketing consultants, and journalists in other reputable publications (Forbes, Businessweek, etc.) have all adjusted this definition to meet their needs, usually in a self-serving way (in the case of the marketing consultants, who appoint themselves experts on crowdsourcing) or without having read the emerging scientific literature on crowdsourcing (in the case of bloggers and journalists).

There are now 19,000 scientific articles on crowdsourcing, according to Google Scholar. There is a ton of peer-reviewed, academic literature on crowdsourcing out there and many people who have engaged in serious empirical studies to better understand this phenomenon. Recently, Estelles-Arolas and Gonzalez did an exhaustive survey of the scientific and popular literature on crowdsourcing. They found no less than 40 different definitions for crowdsourcing floating around. They synthesized these findings into a complete, authoritative definition.

In light of this, I think it is time to stop the vandalism of this page by marketing consultants and people who work for crowdsourcing start-up companies who want to place their own unsupported, unverified definitions of crowdsourcing on this page. It is also time to stop including links to every new start-up company that claims it deals in crowdsourcing. If and when these companies make a demonstrable, measurable, verified impact in the crowdsourcing landscape, then it will be time to include them in this encyclopedia entry (i.e., Threadless, iStockphoto, InnoCentive...these companies all exist on this page because they appear dozens of times in the scholarly literature on crowdsourcing). Per Wikipedia's standards, peer-reviewed academic literature trumps articles in the popular press, and both of these forms trump the occasional blogger or consultant claiming they know the "right" answer on crowdsourcing. To make a mark in this enormous scholarly discourse on crowdsourcing, it is necessary to actually engage it an a neutral-point-of-view way, with verifiable sources backing up claims. --Dbrabham (talk) 19:40, 3 January 2013 (UTC)

Given some particularly influential popular definitions of crowdsourcing that have circulated, it is also good to include a few of these most influential ones (e.g., van Ess'). I have updated the page to reflect this. Dbrabham (talk) 18:19, 15 January 2013 (UTC)

Verifiable Sources About Crowdsourcing

As a Wikipedia page, this needs the same standards of verifiability as an encyclopedia would. Please see Wikipedia:Verifiability for more information about this. Generally, peer-reviewed scholarly sources are the best kind of source, and there are more than 19,000 scholarly articles using the word "crowdsourcing" according to Google Scholar. With this wealth of scholarly knowledge, there is little need to incorporate other non-verified sources.

The next-best sources would be popular press articles (magazines, newspapers, non-scholarly books) from reputable experts in the field. Often, these reputable experts writing in the popular press have also written scholarly work on the subject or are, in the case of Jeff Howe, who coined this term "crowdsourcing," journalists who wrote important, foundational articles and books on the subject.

Slide decks, blogs, and other non-refereed works written by marketing people, people who happen to be involved in a start-up company that claims to do crowdsourcing, or by students who have blogged about this topic for their classes are not relevant for this Wikipedia page and should be left off. Consultants and business owners who continue to inject their own vanity links on this site should be seen as spammers trying to promote their interests at the expense of fair scholarly discourse.--Dbrabham (talk) 12:44, 11 January 2013 (UTC)

Merger proposal

Is anybody against the merger proposals? I dream of horses (T) @ 01:22, 12 March 2012 (UTC)

Well, somebody said "Merging limits ideas and imagination!" slightly further up this page. So long as the final article mentions the terms that we're merging, I don't see a problem, though - it's better to have one article that explains all similar terms in context, than three that partly repeat one another. --McGeddon (talk) 10:58, 12 March 2012 (UTC)

And oh, someone else said that "ubiquitous human computing and crowdsourcing are very different" and told us to go and read a paper for a clarification of that unexplained statement. This may just mean that the current two-sentence article on ubiquitous human computing is simply inaccurate. --McGeddon (talk) 11:02, 12 March 2012 (UTC)

Oppose - This article is already too long and too complex. I would like one article providing an overview, and separate articles going into details. Wikfr (talk) 15:28, 13 March 2012 (UTC)

It's ridiculous that there's no mass peer review article. What about making crowdsourcing, digital sweatshop, microwork all redirects to mass peer review? Or, alternatively, microwork as the main article (shortest term, least specific) with the others as redirects to sections within? — Preceding unsigned comment added by Corrector623 (talk • contribs) 15:50, 29 March 2012 (UTC)

Oppose - All these different terms refer to slightly or completely different things. That's why they use distinct terms (or were coined), and not the same term. - M0rphzone (talk) 03:55, 24 May 2012 (UTC)

re:Microwork . CrowdSourcing is a major topic, and its many subsidiary ideas will grow beyond the size of this page over the next couple of years. While Software_as_a_service (like Microwork) is a subsidiary of the Cloud Computing sector - it deserves to be its own wiki page in the same way that Microwork deserves to be its own page. A few companies have tried to use the page for advertising so it has continually been stripped down. FlorianMettetal (talk) 08:03, 23 October 2012 (UTC)

re:Digitial_sweatshop . IMO this is a derogatory term for Microwork. Why not merge it as a subheading to Microwork and fill out the Microwork page with the rest of the good content from the Digital Sweatshop page. FlorianMettetal (talk) 08:04, 23 October 2012 (UTC)

I don't think that it's a derogatory term for all kinds of microwork, but a specific exploitative business model built on microwork. Given that microwork is the genus here, and digital sweatshops are a species, I agree with your proposal to merge the two under microwork.Jooster b (talk) 19:16, 31 January 2013 (UTC)

I would agree with merging Digital sweatshop to microwork. No need for two articles on basically the same topic. However, microwork is probably sufficiently developed to differentiate itself as a spin-out from crowdsourcing (e.g. microwork as a type of crowdsourcing)--Obi-Wan Kenobi (talk) 21:10, 12 April 2013 (UTC)

Harvard TB example in the "wisdom of the crowds" section is NOT an example of wisdom of the crowds

The second sentence of the following quote is wrong: "Harvard Tuberculosis Lab teamed with CrowdFlower to help identify drug resistant TB cells in mouse cortex slides. Employing wisdom of the crowd allowed the project to be completed efficiently by being able to expand beyond their limited employee base."

The task that was crowdsourced was the "menial and time-consuming job of examining and manually labeling each image" (quote is from the cited source on the WIkipedia page). It is not an example where the desired outcome is a prediction and where the prediction of the 'crowd' is more accurate/better than the predictions of 'experts' (which is the meaning of "wisdom of the crowd"). The crowdsourced workers were not 'better' or 'more accurate' or whatever at this straightforward menial task than the Harvard graduate students that otherwise would have had to do the work. There were just more workers when it was crowdsourced and they were far cheaper, so the benefit was just that the work was done faster and cheaper. In short, there is nothing "wisdom of the crowds" about this. It is just an example of crowdsourcing.

Therefore, I will delete the example.

Relatedly, what makes the Day in the Life documentary an example of the "wisdom of the crowds"? That example should be deleted too, unless someone can edit it to explain what, if anything, is "wisdom of the crowds" about it, as opposed to just being an example of crowdsourcing. The only actual example of "wisdom of the crowds" is the third example of the hedge fund.

Dear Wikipedian That Did Not Sign The Above Message,

Thank you for your contribution and catching the invalid examples in the stated section of the article. I just wanted to let you know that you did not have to rant about it here. If you see a content mistake, fix it and move on. There was no need to post about it here solely to justify your edit. Also, remember to sign any posts that you make on talk pages with four tildes (~). Keep up the good working with editing and content checking, just don't crowd article talk pages with your edit summaries. Metsfreak2121 (talk) 02:14, 23 February 2013 (UTC)

CrowdMed

http://www.newscientist.com/article/dn23392-crowd-diagnosis-could-spot-rare-diseases-doctors-miss.html --Espoo (talk) 21:07, 25 April 2013 (UTC)

Wikipedia

Is Wikipedia an example of crowd-sourcing or not? In either case, I think it should be mentioned, since at least prima facie it seems to be. Kdammers (talk) 12:41, 19 March 2013 (UTC)

I am new to Wikipedia - I agree that Wikipedia should be mentioned as crowd-sourcing, because thousands of years later when wikipedia will be changed and transformed into something else when someone will trace back the history no mention of Wikipedia just because wikipedia was chivalrous and non-selfish means a hole in encyclopedia information. — Preceding unsigned comment added by Arunkw (talk • contribs) 09:53, 16 May 2013 (UTC)

Wikipedia is frequently lumped together with crowdsourcing in both the scholarly and popular press, but I have argued many times over in peer-reviewed scholarly literature (including my recent book with MIT Press) that it is not. Wikipedia is certainly an interesting participatory culture phenomenon, but it falls into the "commons-based peer production" label that legal scholar Yochai Benkler talks about, not under the umbrella of crowdsourcing. This distinguishing point, as I have argued, is based on the degree of control the sponsoring organization has over the day-to-day production of content...at Wikipedia, there is no one directing the daily work of Wikipedians. Wikipedia merely provides the sandbox and its users self-organize to create the content (ditto open source software production). Plus, Wikipedia's own Jimmy Wales hates the term crowdsourcing and has been interviewed many times saying so. For conceptual hygiene's sake, let's keep the terms separate. Dbrabham (talk) 21:09, 19 May 2013 (UTC)

I'm new to wikipedia but if there are as you say, scholarly press that believe wikipedia is an example of crowdsourcing and if there is also scholarly press that believe it is not crowdsourcing, shouldn't there be a section documenting the debate? StartlingCanary2 16:49 11 September 2014 (UTC) (I hope I did the UTC right)

On recent, erroneous editions to the article.

Hi. This is Jeff Howe. I coined the term crowdsourcing in a Wired article in 2006. In 2008 Crown/Random House published [book] on the subject. The book has sold tens of thousands of copies and been translated into 12 languages. Throughout all this, I have been widely (and correctly) credited as the coiner of the term crowdsourcing. A version of the origin for the term can be found in this New York Times column by the (sadly deceased) columnist William Safire.

Every once in a while I check the page, and today I see that Steve Jurvetson, a Venture Capitalist I believe, is claiming credit to having "coined" the term. There is no merit to this claim, though it is fair to say he was one of the first people to *use* the term in public, though he used it in a very different sense. I actually called him shortly before my article came out, because we at Wired were naturally quite concerned about whether or not anyone would use the term before us. At the time Steve believed he had in fact heard the term used by Chris Anderson, Wired's editor-in-chief, before the article came out, then proceeded to use it (wrongly) in an offhand post on Flickr.

I'm choosing *not* to edit the page, as I'm clearly an interested party. But I thought someone should hold Jurvetson, or his claimants, to the same burden of proof to which I've always been held. Whenever I've tried to make the claim to having coined the term, I've had to provide popular and/or academic articles backing up that claim. I can do so again, though again, the examples are numerous at this point.

What the page should *really* say is that the word originated in a conversation between myself and my editor, Wired's Mark Robinson. For a few years, it indeed did credit us both, but then someone nixed Mark's name. I make it a point not to edit this page, as it would be a conflict of interest. But I did want to make the above points known to the Wikipedians in charge.

Thank you ... — Preceding unsigned comment added by Jeffreyphowe (talk • contribs) 17:05, 19 February 2013 (UTC)

Yeah I agree with Jeff Howe on this, though I have been known to fall into this trap and waver a bit. Jeff Howe (AND Mark Robinson) coined the term. Jurvetson was an early user of the term but not its creator. The NY Times article jives with what I know about this term's origin and I trust the NY Times and Wired's research department. I've been studying this term since Howe's article came out and I've never heard anyone verify that Jurvetson was really a *creator* of the term, only an early user of it. And no scholarly literature exists that credits Jurvetson either. I'll make the fixes. Dbrabham (talk) 21:13, 19 May 2013 (UTC)

Crowd Searching Needs More Explaining

Crowd searching is one of the types of crowd sourcing, but it is not well defined here. Also, before talking about Micro works, it has already said, "...as crowd searching, which differs from Micro work in that ...". There should be at lease a link to Micro work.

LouisHam (talk) 16:37, 16 October 2013 (UTC)

Crowdsourcing Improvement

After reviewing, I noticed that the article does not directly follow the criteria to be considered a good article. I posted the link for good article criteria below this statement. http://en.wikipedia.org/wiki/Wikipedia:Good_article_criteria The article does not meet the third requirement. A section of the article under "Modern Methods" does not fully discuss explicit crowdsourcing. The article talks about explicit crowdsourcing and says: "With explicit crowdsourcing, users can evaluate particular items like books or webpages, or share by posting products or items. Users can also build artifacts by providing information and editing other people's work." After that short quote the article does not discuss explicit crowdsourcing anymore. This causes an inconsistency because the article has a separate paragraph specifically for implicit crowdsourcing. I plan to fix this inconsistency by adding an article specifically about explicit crowdsourcing. This is surprising because wikipedia is a form of explicit crowdsourcing. In fact, we are all engaging in social computing and explicit crowdsourcing by participating in this talk page! The article also needs examples of explicit crowdsourcing websites.

Xavierdillahunt (talk) 03:00, 17 October 2013 (UTC)

Correctness of statements and proper references

Regarding paragraph: "It is usually expected from a crowdsourced project to be unbiased by incorporating a large population of participants with a diverse background. However, most of the crowdsourcing works are done by people who are paid or directly benefit from the outcome (e.g. most of open source projects working on Linux). In many other cases, the end product is the outcome of a single person's endeavour, who creates the majority of the product, while the crowd only participates in minor details.[76]"

Looking up reference 76, I only find a news article with merely the personal opinion of the writer -- no further references therein whatsoever to support the claims made in the section listed above. As a critical (scientific) reader, I must therefore deem the paragraph non-factual, without evidence; they do not belong in this form in an encyclopedia. For instance, the statement that Linux is based on projects should be detailed; above all: how much a part is financially independent, and how much is not? If financial dependence is marginal, the above paragraph appears not worth mentioning; if is, then it is.

It is clear that the problem outline is a valid concern. But without further details and examples based on actual research, the statements definitely appear made up. In short, the statements are not based on factual research. — Preceding unsigned comment added by 134.76.222.49 (talk) 22:08, 23 December 2013 (UTC)

More examples of non-internet predecessors

We've added two more useful examples of non-internet uses of crowdsourcing. One is the use of crowdsourcing to gather information about the Leonid meteor showers. The other is the gathering of information about the number of birds in certain places on christmas day which provides useful data to ornithologists. - StartlingCanary2 A.stass21 Studently Danie949 15:53 UTC 17 September 2014 — Preceding undated comment added 15:53, 17 September 2014 (UTC)

Additional information about authors + additional citation

Added information about Daren C. Brabham, Henk van Ess, Enrique Estellés-Arolas, and Fernando González Ladrón-de-Guevara. Also fixed some [citation needed] tags with references. — Preceding unsigned comment added by Cadunne (talk • contribs) 15:58, 17 September 2014 (UTC)

Reorganizing the "Modern methods" and "Predecessors" sections

User stuen007 changed "Predecessors" to "Historical Examples." This brings the section more in line with the definition provided earlier in the article, which allows for examples of crowdsourcing that occur offline. Brndn.js (talk) 15:41, 17 September 2014 (UTC)

In addition, we moved "Crowdsourcing Typology" to the top-level "Modern methods" section, as this content describes a method of looking at how crowdsourcing is broken down into categories. In addition, we made "Types of Crowdsourcing" a top-level heading, and changed its name to "Examples," to better describe its content. This way, the "Historical Examples" and "Examples of Crowdsourcing" sections are parallel. Brndn.js (talk) 15:41, 17 September 2014 (UTC)

"Wisdom of the Crowd" heading moved to See Also section. Solar Nebula (talk) 16:01, 17 September 2014 (UTC)

Separate Page that might need to be incorporated into the main page

https://en.wikipedia.org/wiki/Crowdsourcing_as_Human-Machine_Translation This topic seems an odd choice to have it's own page, especially due to its title. I do not know if it would be better off under the Crowdsourcing page or the Machine translation page. I will add this same post to the Machine Translation page. StartlingCanary2 (talk) 16:02, 17 September 2014 (UTC)

Updating Ethics section to include Citizen Science ethics

Within the citizen science community there are some agreed upon ethics, such as crediting crowdworkers and only using them when they are necessary, that aren't in the article. Me and my classmates have started on a section, pasted below. Someone else can feel free to continue.

[Citizen Science]] projects, strongly tied to academia, have to operate under a more strict ethical code than private institutions utilizing crowdsourcing. For example, they have to make sure that participants are properly credited in papers for which they provided the data and analysis. They also need to make sure participants are doing work that is both necessary and unable to be done by a machine as to not waste their time (CITE: http://www.citizensciencealliance.org/proposals.html).

Darch Paper- https://ay14.moodle.umn.edu/pluginfile.php/741572/mod_resource/content/1/darch2014.pdf — Preceding unsigned comment added by Cadunne (talk • contribs) 17:00, 10 December 2014 (UTC)

Theoretical Perspectives - Crowd Capital

I removed the section once again. Even if it was properly written in a manner that provides context and indicates why it is noteworthy, it requires some independent secondary sources to establish the context and noteworthiness. --Ronz (talk) 16:18, 19 July 2015 (UTC)

Removed it again. I'd like to think here is some way we can make it into something encyclopedic, but that will require independent secondary sources. The content as is reminds me of something from SCIgen. --Ronz (talk) 15:53, 21 July 2015 (UTC)

External links modified

Hello fellow Wikipedians,

I have just added archive links to 2 external links on Crowdsourcing. Please take a moment to review my edit. If necessary, add {{cbignore}} after the link to keep me from modifying it. Alternatively, you can add {{nobots|deny=InternetArchiveBot}} to keep me off the page altogether. I made the following changes:

• Added archive https://web.archive.org/20141129054631/http://www.tiki-toki.com/timeline/entry/323158/Crowdsourcing-Back-Up-Timeline-Early-Stories/ to http://www.tiki-toki.com/timeline/entry/323158/Crowdsourcing-Back-Up-Timeline-Early-Stories/
• Added archive https://web.archive.org/20140824051327/http://www.audubon.org/thank-you-0 to http://www.audubon.org/thank-you-0

When you have finished reviewing my changes, please set the checked parameter below to true to let others know.

This message was posted before February 2018. After February 2018, "External links modified" talk page sections are no longer generated or monitored by InternetArchiveBot. No special action is required regarding these talk page notices, other than regular verification using the archive tool instructions below. Editors have permission to delete these "External links modified" talk page sections if they want to de-clutter talk pages, but see the RfC before doing mass systematic removals. This message is updated dynamically through the template {{source check}} (last update: 5 June 2024).

• If you have discovered URLs which were erroneously considered dead by the bot, you can report them with this tool.
• If you found an error with any archives or the URLs themselves, you can fix them with this tool.

Cheers.—^{cyberbot II}_{Talk to my owner:Online} 03:39, 16 January 2016 (UTC)

Possible merge?

I recently declined a no context speedy on the article Crowdsolving, as the article had enough context to tell what it was. However I don't know that it's really able to pass notability guidelines on its own, as the term isn't widely used at this point in time and when it is used, it's usually in relation to the overall idea of crowdsourcing. I'm not starting an official merge discussion, but if someone decides that this would be a good idea, I'm not opposed to this turning into one. Tokyogirl79 (｡◕‿◕｡) 05:04, 17 February 2016 (UTC)

External links modified

Hello fellow Wikipedians,

I have just modified 2 external links on Crowdsourcing. Please take a moment to review my edit. If you have any questions, or need the bot to ignore the links, or the page altogether, please visit this simple FaQ for additional information. I made the following changes:

• Corrected formatting/usage for http://www.fta.dot.gov./planning/programs/planning_environment_8711.html
• Corrected formatting/usage for http://www.ics.uci.edu/~jwross/pubs/RossEtAl-WhoAreTheCrowdworkers-altCHI2010.pdf

When you have finished reviewing my changes, please set the checked parameter below to true or failed to let others know (documentation at {{Sourcecheck}}).

This message was posted before February 2018. After February 2018, "External links modified" talk page sections are no longer generated or monitored by InternetArchiveBot. No special action is required regarding these talk page notices, other than regular verification using the archive tool instructions below. Editors have permission to delete these "External links modified" talk page sections if they want to de-clutter talk pages, but see the RfC before doing mass systematic removals. This message is updated dynamically through the template {{source check}} (last update: 5 June 2024).

• If you have discovered URLs which were erroneously considered dead by the bot, you can report them with this tool.
• If you found an error with any archives or the URLs themselves, you can fix them with this tool.

Cheers.—^{cyberbot II}_{Talk to my owner:Online} 09:00, 1 April 2016 (UTC)

Addition To Modern Methods

Hello fellow Wikipedians,

Firstly, I'm new to Wikipedia, so please forgive any newbie behavior. Secondly, I want to cite my employer's blog, where we publish research and best practices from our crowdsourcing business. I have read Wikipedia:Blogs_as_sources and I believe that our company is a source of authority in crowdsourcing and that I am an expert. But I want to call that out from the beginning.

I would like to add a paragraph to the bottom of the Crowdsourcing#Modern_methods section. It could also go into Crowdsourcing#Crowdvoting but it is related to modern methods, not examples. My addition:

Crowdsourcing often allows participants to rank each other's contributions, e.g. in answer to the question "What is one thing we can do to make Acme a great company?".  One common method for ranking is "like" counting, where the contribution with the most likes ranks first.  This method is simple and easy to understand, but it privileges early contributions, which have more time to accumulate likes.  In recent years several crowdsourcing companies have begun to use pairwise comparisons, backed by ranking algorithms such as Elo. Ranking algorithms do not penalize late contributions. They also produce results faster. Elo, for example, is at least 10 times faster than manual stack ranking. REF  One drawback, however, is that ranking algorithms are more difficult to understand than like counting.

Many thanks, TDQuigley (talk) 17:50, 26 May 2017 (UTC)

The main concern I have with the content are the advert mentions and referencing source. Comments "backed by ranking algorithms such as Elo" can be simplified to "backed by ranking algorithms"; and "Elo, for example ..." could be more generically stated in terms of ranking algorithm performance improvement over manual methods. Lastly, the only source provided is a primary source; per WP:RS, we should instead use an industry white paper, journal, news article, or similar third-party reliable sources. --- Barek (talk • contribs) - 23:26, 30 May 2017 (UTC)

I can take out the specific reference to Elo, and just mention "ranking algorithms." In other words, your edit is fine. On the reference, I have not been able to find another source that specifically measures the time savings of using ranking algorithms. That, I think, is the essential point. While our blog is not an industry white paper, journal, or news article, it's purpose is to publicly share data and methods from our work. That page, in particular, does not fall into the category "Individual web pages that primarily exist to sell products or services."; per Wikipedia:Identifying_reliable_sources#Vendor_and_e-commerce_sources. So, taking your edit and keeping the reference, we'd have this:

Crowdsourcing often allows participants to rank each other's contributions, e.g. in answer to the question "What is one thing we can do to make Acme a great company?"  One common method for ranking is "like" counting, where the contribution with the most likes ranks first.  This method is simple and easy to understand, but it privileges early contributions, which have more time to accumulate likes.  In recent years several crowdsourcing companies have begun to use pairwise comparisons, backed by ranking algorithms. Ranking algorithms do not penalize late contributions. They also produce results faster.  Ranking algorithms have proven to be at least 10 times faster than manual stack ranking.  One drawback, however, is that ranking algorithms are more difficult to understand than like counting.

TDQuigley (talk) 15:16, 31 May 2017 (UTC)

Sorry for the delayed reply (travelling with limited online time, not enough to focus on the above wording until now). The modified version reads with a more neutral point of view. If that's the only available ref, then we will need to use it until a source that better meets Wikipedia's guidance of using a 3rd party reliable source becomes available. Before I left to travel, I had also requested on the talk page of two related Wikiprojects that some of their they comment here - but either members of those projects missed the request, are busy elsewhere, and simply have no opinion on this topic. Feel free to add the revised wording to the article; or, if you prefer, I can add it and convert the ref to using a standard {{cite web}} template. --- Barek (talk • contribs) - 22:34, 4 June 2017 (UTC)

Thanks for your time. I used {{cite web}} and added the paragraph. TDQuigley (talk) 18:33, 5 June 2017 (UTC)

Crowdsourcing Creative Work

The subsection "Crowdsourcing creative work" starts with "Main article: Crowdsourcing creative work" but that just redirects back to (the top of) this page. --Antistone (talk) 18:41, 6 June 2017 (UTC)

External links modified

Hello fellow Wikipedians,

I have just modified one external link on Crowdsourcing. Please take a moment to review my edit. If you have any questions, or need the bot to ignore the links, or the page altogether, please visit this simple FaQ for additional information. I made the following changes:

• Added archive https://web.archive.org/web/20141202072036/http://www.unv.org/fileadmin/docdb/pdf/2011/SWVR/English/SWVR2011_full.pdf to http://www.unv.org/fileadmin/docdb/pdf/2011/SWVR/English/SWVR2011_full.pdf

When you have finished reviewing my changes, you may follow the instructions on the template below to fix any issues with the URLs.

This message was posted before February 2018. After February 2018, "External links modified" talk page sections are no longer generated or monitored by InternetArchiveBot. No special action is required regarding these talk page notices, other than regular verification using the archive tool instructions below. Editors have permission to delete these "External links modified" talk page sections if they want to de-clutter talk pages, but see the RfC before doing mass systematic removals. This message is updated dynamically through the template {{source check}} (last update: 5 June 2024).

• If you have discovered URLs which were erroneously considered dead by the bot, you can report them with this tool.
• If you found an error with any archives or the URLs themselves, you can fix them with this tool.

Cheers.—InternetArchiveBot (Report bug) 22:56, 14 August 2017 (UTC)

External links modified

Hello fellow Wikipedians,

I have just modified 2 external links on Crowdsourcing. Please take a moment to review my edit. If you have any questions, or need the bot to ignore the links, or the page altogether, please visit this simple FaQ for additional information. I made the following changes:

• Added archive https://web.archive.org/web/20151122025307/http://i3wue.de/staff/matthias.hirth/author_version/papers/conf_410_author_version.pdf to http://i3wue.de/staff/matthias.hirth/author_version/papers/conf_410_author_version.pdf
• Added {{dead link}} tag to http://delivery.acm.org/10.1145/1930000/1924442/p86-doan.pdf?ip=71.182.229.123&acc=OPEN&CFID=87199630&CFTOKEN=25539817&__acm__=1330408287_b904f7bffdd69c8ad85d83c619e9a3c1
• Added archive https://web.archive.org/web/20120227173340/http://schader.bwl.uni-mannheim.de/fileadmin/files/publikationen/Kaufmann_Schulze_Veit_2011_-_More_than_fun_and_money_Worker_motivation_in_Crowdsourcing_-_A_Study_on_Mechanical_Turk_AMCIS_2011.pdf to http://schader.bwl.uni-mannheim.de/fileadmin/files/publikationen/Kaufmann_Schulze_Veit_2011_-_More_than_fun_and_money_Worker_motivation_in_Crowdsourcing_-_A_Study_on_Mechanical_Turk_AMCIS_2011.pdf

When you have finished reviewing my changes, you may follow the instructions on the template below to fix any issues with the URLs.

This message was posted before February 2018. After February 2018, "External links modified" talk page sections are no longer generated or monitored by InternetArchiveBot. No special action is required regarding these talk page notices, other than regular verification using the archive tool instructions below. Editors have permission to delete these "External links modified" talk page sections if they want to de-clutter talk pages, but see the RfC before doing mass systematic removals. This message is updated dynamically through the template {{source check}} (last update: 5 June 2024).

• If you have discovered URLs which were erroneously considered dead by the bot, you can report them with this tool.
• If you found an error with any archives or the URLs themselves, you can fix them with this tool.

Cheers.—InternetArchiveBot (Report bug) 02:35, 19 September 2017 (UTC)