User talk:Syced/Wikipedia Reference Search

This page is part of WikiProject Reliability, a collaborative effort to improve the reliability of Wikipedia articles. If you would like to participate, please visit the project page, where you can join the discussion and see a list of open tasks.ReliabilityWikipedia:WikiProject ReliabilityTemplate:WikiProject ReliabilityReliability articles

More websites

Lists of more reference websites: [1] [2] [3] To be continued! Nicolas1981 (talk) 12:00, 16 February 2009 (UTC)[reply]

Code to generate the list

Download the annotations as "annotations.xml", and use those files:

annotations.xsl

<?xml version="1.0" encoding="ISO-8859-1"?>
<?xml-stylesheet type="text/xsl" href="annotations.xsl"?>

<annotations-shortcut/>

annotations-launcher.xml

<?xml version="1.0" encoding="ISO-8859-1"?>
<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform">

<xsl:template match="/">
	<xsl:apply-templates select="annotations-shortcut"/>
	<xsl:apply-templates select="Annotations/Annotation"/>
</xsl:template>

<xsl:template match="annotations-shortcut">
	<html>
	<p>
	<!-- Change this to where you downloaded the Google annotations -->
	<xsl:apply-templates select="document('/home/nico/Desktop/annotations.xml')"/>
	</p>
	</html>
</xsl:template>

<xsl:template match="Annotations/Annotation">
	<xsl:value-of select="@about"/>
	<br/>
</xsl:template>

</xsl:stylesheet>

— Preceding unsigned comment added by Syced (talk • contribs) 08:44, 18 February 2009 (UTC)[reply]

Rename the project

"Wikipedia Reference Search" can be interpreted of searching for a "wikipedia reference", which it is not about. I am currently thinking about a new name, any idea is welcome. After 5 minutes of reflexion, my suggestion is "SearchRef", with no initials form and no expanded form. A Google search shows that SearchRef is not associated to any big project. Please tell me whether this name is OK, and feel free to suggest more names. Thanks! Nicolas1981 (talk) 11:42, 19 February 2009 (UTC)[reply]

Wiki-editable websites list

It would be really cool if the list could be edited right here on the Wiki, instead of having to use Google's web interface. Google has done a tool that can scrape all "href" attributes in an HTML page, that's a good start but I don't think we can use it as is, because it would include links to Wikipedia itself. A small PHP script would probably be enough, I will look into that when I have time. In other news, WRS has been serving around 15 queries per day since January :-) Nicolas1981 (talk) 10:53, 24 April 2009 (UTC)[reply]

Blacklist???

I attempted to put a link to this search engine on my userpage. To my surprise Wikipedia rejected the link as a blacklisted site! What's that about? --MelanieN (talk) 16:44, 20 November 2013 (UTC)[reply]

airliners.net

I think airliners.net should be removed from the list. Or at the very least /aviation-forums/ hits should not show up. --82.136.210.153 (talk) 07:22, 25 April 2015 (UTC)[reply]

Thanks

Thanks for WRS, it is quite useful. --82.136.210.153 (talk) 07:38, 25 April 2015 (UTC)[reply]

Not policy

This appears to return hits from a list of sources that get added to this user page. This user page is not a policy page. Thus, the first sentence is false. WRS does not return sources that policy pages identify as reliable. Or am I missing something? NewsAndEventsGuy (talk) 02:01, 31 March 2019 (UTC)[reply]

Thanks for letting me know! After a lot of discussion with the community at the section you linked to, and after changes in the project, the matters now seems to be settled :-) Syced (talk) 13:37, 6 April 2019 (UTC)[reply]

Yep, thanks for being gracefully willing to accept critical feedback NewsAndEventsGuy (talk) 14:37, 6 April 2019 (UTC)[reply]

Suggested additions

If you want another website to be added, append it here. Don't hesitate to add many websites, it's cheap!

Hi. Can you please add: formula1.com, autosport.com, itv-f1.com? More will come from WP:F1. Thanks Cdhaptomos talk–contribs 15:52, 17 February 2009 (UTC)[reply]

Done Nicolas1981 (talk) 07:55, 18 February 2009 (UTC)[reply]

Also: grandprix.com, fia.com. Thanks Cdhaptomos talk–contribs 21:23, 17 February 2009 (UTC)[reply]

Done Nicolas1981 (talk) 07:59, 18 February 2009 (UTC)[reply]

Also: wwe.com, tnawrestling.com, slam.canoe.ca, f4wonline.com, wrestleview.com. D.M.N. (talk) 21:39, 17 February 2009 (UTC)[reply]

Done Nicolas1981 (talk) 07:59, 18 February 2009 (UTC)[reply]

Also www.consumerreports.org Smallman12q (talk) 21:45, 17 February 2009 (UTC)[reply]

X -- Consumer Reports has a known bias within its reporting regarding American versus foreign vehicles (read up on it), and did not even test Toyota-made vehicles it recommended. I cite the following Nicolas1981 (talk) 23:38, 15 March 2009 (UTC)Google result, which includes a link to CR's 2007 statement: CR used to automatically wave through Toyota vehicles as recommended without testing. -- Guroadrunner (talk) 07:22, 18 February 2009 (UTC)[reply]

Done Included as an experiment, I let both of you discuss the topic and reach a consensus (I am neutral) Nicolas1981 (talk) 07:59, 18 February 2009 (UTC)[reply]

Guroadrunner: I read CR's statement as something quite different: not that they didn't *test* Toyota vehicles, but rather that, because Toyota vehicles have historically been so reliable, they were assuming that new and redesigned vehicles would also be. When that turned out (after consumers had owned and driven the new models for some time, then CR changed their approach. (Also, when you're trying to prove a point, citing the best two or three sources is much, much better than providing a google search.) And if our criterion for sources is *perfection*, then I don't know of any source that qualifies. -- John Broughton (♫♫) 15:27, 19 February 2009 (UTC)[reply]

I have a problem including CR: they capitulated when the Bose corporation threatened to sue after a less than flattering review. Also, their reviews are for the layman, they rarely get into the technical aspects of anything that they review. – ukexpat (talk) 17:17, 19 February 2009 (UTC)[reply]

I suggest the following. -- John Broughton (♫♫) 00:20, 18 February 2009 (UTC)[reply]

philly.com (Philadelphia Inquirer)
sfgate.com (San Francisco Chronicle)
oregonlive.com (Portland Oregonian newspaper)
cleveland.com (Cleveland Plain Dealer)
wsj.com (Wall Street Journal). Most content is available only to subscribers, but they do have some free content

Done Nicolas1981 (talk) 08:07, 18 February 2009 (UTC)[reply]

I also don't understand why you don't just include the entire .gov domain, rather than listing a bunch of separate parts of it. Is hhs.gov less reliable than ftc.gov, for example? Or a state government website less reliable than a federal one? Similarly, I don't understand why you have chosen to list some universities (USC, U. of Virginia, Tufts, etc.) and not others (JHU, UCB, UCSF, etc.) - why not just list the entire .edu domain? Universities often host student pages - see, for example, http://www-scf.usc.edu/ , so obviously editors need to make some distinctions among the pages of the .edu sites you already list - the same distinction that could be made for all edu pages. -- John Broughton (♫♫) 00:20, 18 February 2009 (UTC)[reply]

Done I added *.edu and *.gov to the list. but it does not seem to work... maybe it needs websites names and not just a TLD. The initial list comes from statistics on which websites are most linked from Wikipedia. Nicolas1981 (talk) 08:07, 18 February 2009 (UTC)[reply]

I would like 3 checkboxes to blanket-add anything returned by Google Scholar, Google Books, and Google News. Likewise, if there are other broad categories that are easy to implement and turn on/off with a check-box, do so. Broad categories should be used if by far the majority of sites/books/whatever in the category are reliable, with a user warning that not all results are in fact reliable. davidwr/(talk)/(contribs)/(e-mail) 01:51, 18 February 2009 (UTC)[reply]

X -- I disagree with Google News as a reliable starting point for sourcing, and I can speak from personal experience: I work in the media and for my local area Google News includes results from a non-reliable blog as a news source, possibly more. I also understand Google News lists material from Associated Content, which is a user-created regurgitation of news with little to no firsthand reporting. Recommend not going with this advice on Google News. -- Guroadrunner (talk) 07:22, 18 February 2009 (UTC)[reply]

Done Added Scholar Books News as an experiment, I let both of you discuss the topic and reach a consensus (I am neutral). Nicolas1981 (talk) 08:13, 18 February 2009 (UTC)[reply]

I also agree that Google News is overbroad; I've definitely seen blogs in the results that are in no way acceptable as a reliable source. -- John Broughton (♫♫) 15:27, 19 February 2009 (UTC)[reply]

Davidwr, "broad categories" are a good idea, and technically feasible. Can you handle the hard work of forming those categories? Thank you! Nicolas1981 (talk) 08:13, 18 February 2009 (UTC)[reply]

The "broad categories" should be made at the request of the community. As each is its own checkbox, and users are free to check them or not, this can be open-ended. In the long run, if user-logins are ever enabled, user-definable categories and "preset checkbox settings" could be added as well. I may want Google Scholar and News plus my 50 favorite web sites, another user may want his favorite 25 web sites but nothing else beyond the canned list of reliable sources that everyone gets. davidwr/(talk)/(contribs)/(e-mail) 16:07, 18 February 2009 (UTC)[reply]

could you add 8w.forix.com (forix motorsport site)? Thanks. D.M.N. (talk) 15:45, 28 February 2009 (UTC)[reply]

Done Nicolas1981 (talk) 16:09, 3 March 2009 (UTC)[reply]

motorsport.com, f1-live.com, mclaren.com, planet-f1.com, f1technical.net, gpupdate.net, brawngp.com, crash.net too please. Cdhaptomos talk–contribs 16:29, 6 March 2009 (UTC)[reply]

Apart from mclaren.com and brawngp.com, none of the others as of yet satisfy WP:RS. D.M.N. (talk) 18:56, 6 March 2009 (UTC)[reply]

Why are we using them as sources inarticles then? Cdhaptomos talk–contribs 19:42, 7 March 2009 (UTC)[reply]

Done Added all of these websites as an experiment, I let both of you discuss the topic and reach a consensus (I am neutral). Nicolas1981 (talk) 23:38, 15 March 2009 (UTC)[reply]

I don't know really, but IMO it doesn't pass RS. D.M.N. (talk) 08:22, 16 March 2009 (UTC)[reply]

http://mentalfloss.com/ is also a print magazine with articles on a wide variety of topics. Supernerd11 _{Firemind ^_^ Pokedex} 17:22, 10 February 2015 (UTC)[reply]

It would be great if two other editors could confirm whether mentalfloss is indeed considered a good reference website. Thanks! :-) Syced (talk) 13:18, 31 March 2019 (UTC)[reply]

I suggest http://archives.chicagotribune.com. Eddie Blick (talk) 01:38, 26 April 2017 (UTC)[reply]

Done Thanks! Syced (talk) 13:18, 31 March 2019 (UTC)[reply]

Relationship with WP:RSP?

@Syced: I just came across this tool while working on some changes to {{Find sources}}. It's really interesting, but I'm curious about the list's relationship to WP:RSP, which seems to be a more-widely vetted list of which sources Wikipedians find reliable. I see some URLs currently on the list, such as foxnews.com and forbes.com, that are yellow-listed at RSP and that I'm not sure should be included if the point of this is to include only websites that editors can be confident are reliable. {{u|Sdkb}} ^talk 20:22, 30 September 2021 (UTC)[reply]

I actually did not know about RSP (or forgot about it), I am happy to accept your pull request that uses that list instead, thanks! Instructions are at https://github.com/nicolas-raoul/Wikipedia-Reliable-Sources#how-to-contribute Cheers! Syced (talk) 20:15, 3 October 2021 (UTC)[reply]

@Sdkb and @Syced, I created a Reliable Sources Engine with only WP:RSP sources (before someone at TeaHouse pointed me to this really cool project). Let me know if you have thoughts - wondering if there's room for a second, more focused search engine (say, for political topics and/or for new editors)? Superb Owl (talk) 23:32, 4 June 2024 (UTC)[reply]

Your idea of reusing the RSP sources list is great! Of course there is room for two, especially if you sync from RSP regularly. Cheers! 🙂 Syced (talk) 13:50, 12 June 2024 (UTC)[reply]

Of course there is room for two I strongly question this. Maintaining two lists is basically twice the work, which is a huge maintenance cost to bear. It's a fork. Sdkb ^talk 13:55, 12 June 2024 (UTC)[reply]

My list has only 100 sources on the non-paywalled sources version and 140 on the full version. I actually haven't forked any of the sources from here, but may try and set up a GitHub system like yours if you'd recommend it. I have come around and want to merge projects Superb Owl (talk) 16:27, 12 June 2024 (UTC)[reply]

Wanted to second @Sdkb on Forbes and Foxnews not being reliable, and also flag Kreml.ru, .world.guns.ru/, *.tretyakovgallery.ru/*, *.rsl.ru/*, .mfab.hu/*, .fotomuzeum.hu/*, sstm.org.cn, (and any sites that are not vetted with country codes originating in autocracies without academic freedom or free speech). The Kremlin in particular seems like a poor fit when this resource is being described as "a Google search that only searches sites vetted by Wikipedians" on various templates. Update: it looks like the commit worked this time Superb Owl (talk) 20:02, 12 June 2024 (UTC)[reply]

Starting another list of sources that are not generally reliable:
- spectator.co.uk
- spectator.us
- news.yahoo.com has mostly reprints of articles, many from unreliable sources (haven't figured out how to only include original reporting, so I excluded it)
- info.gov.hk (for the same reasons above re: autocracies)
It's a lot to go through but I can keep combing through it if it's helpful Superb Owl (talk) 02:41, 15 June 2024 (UTC)[reply]

We don't need another list. We need to be consolidating our lists, and given its prominence WP:RSP is the clear place to centralize. Sdkb ^talk 12:50, 15 June 2024 (UTC)[reply]

updated: Let's do it. I'll submit more pull requests and merge the two lists Superb Owl (talk) 16:48, 15 June 2024 (UTC)[reply]

@Syced, are you able to easily download and upload the lists to update the search engine? Just in my brief experience I don't know if that workflow still exists... Superb Owl (talk) 14:39, 21 June 2024 (UTC)[reply]

Thanks both for the very constructive work! After all I tend to agree than one list is less work to maintain, and if I receive GitHub pull requests when changes are needed then I guess things will continue smoothly for a few more dozens of years :-) I merged the GitHub pull request and will update the search soon (the format has changed so I need to convert it). By the way, some statistics: the search is getting around 30 queries per day, more than half being searches for people names. Syced (talk) 17:42, 26 June 2024 (UTC)[reply]

Awesome! how do you see these statistics on the number (and type) of queries per day? Superb Owl (talk) 18:39, 26 June 2024 (UTC)[reply]

There is a simple Statistics section within the GSE admin user interface. :-) Syced (talk) 10:09, 27 June 2024 (UTC)[reply]

Very cool! thanks! Will be really helpful for consolidating other search engines... Superb Owl (talk) 17:45, 1 July 2024 (UTC)[reply]