Wikipedia:Bots/Requests for approval/Commons fair use upload bot 2

The following discussion is an archived debate. Please do not modify it. To request review of this BRFA, please start a new section at WT:BRFA. The result of the discussion was

Approved.

Commons fair use upload bot

Operator: Dcoetzee (talk · contribs · SUL · edit count · logs · page moves · block log · rights log · ANI search)

Time filed: 01:56, Monday August 27, 2012 (UTC)

Automatic, Supervised, or Manual: Unsupervised and automatic, runs on Toolserver (however, its actions are triggered by administrators on Commons)

Programming language(s): Python, mwclient

Source code available: Yes, from https://github.com/wikigit/Commons-fair-use-upload-bot

Function overview: Cross-wiki bot. If an image currently in use at En is going to be deleted on Commons, a Commons administrator may choose to tag it with {{Fair use delete}}, and the image is automatically reuploaded here with a {{di-no fair use rationale}} tag (see example). Any articles where the image is in use receive talk page notices, reminding them to complete a fair-use rationale if they wish to continue using the image (example).

Links to relevant discussions (where appropriate): This is a request to renew authorization after the expiration of the bot's 7-day trial in April 2011 (see Wikipedia:Bots/Requests for approval/Commons fair use upload bot). The bot ran for about 15 months after that without authorization (I forgot to renew - my fault), and was recently blocked. See recent ANI discussion, which was archived prematurely and so inconclusive. I've added to the bot's user page to clarify its precise function and link to requests for approval as a result of that discussion.

Edit period(s): Continuous

Estimated number of pages affected: Roughly 2-5 per day

Exclusion compliant (Yes/No): Yes

Already has a bot flag (Yes/No): No, not needed

Function details: Most details of operation already described above. Full page and file history is included for license compliance. The bot runs once an hour. The bot also sometimes uploads images that are PD in the US but not in their source country, which are permitted here but not on Commons. Such works are tagged with the {{PD-US-1923-abroad-delete}} tag on Commons, and then transferred here with {{PD-US-1923-abroad}} tags.

I should note that, due to a bug in User:CommonsDelinker, images will still be removed from articles when they are deleted from Commons even if they have been reuploaded here using Commons fair use upload bot. I just looked up the old thread in which I contacted the bot manager and they said "I cannot fix that. Patches for CommonsDelinker are appreciated. The code is in pywikipediabot's svn.". So there is a possibility they might accept a patch for this.

Commons fair use upload bot now reverts edits by CommonsDelinker that are affected by the redirect bug, including the cases affecting Commons fair use upload bot's re-uploaded files. It would still be preferable to fix CommonsDelinker if we can find a way to do this, but so far I haven't been able to get in touch with anyone who can help me do that.

Discussion

As one that commented before, I think the assurances (particularly tagging for rationale) is appropriate. I wonder if we also need to tag {{Di-no license}} if the reasoning that the image can't be used at Commons is unclear. --MASEM (t) 02:15, 27 August 2012 (UTC)[reply]

I think that would probably depend on the image. Usually the author/source/license information is accurate, but sometimes (when the claimed license is invalidated during the deletion request on Commons), the false license tag should really be removed before tagging it with Fair use delete. In such a case Di-no license seems like a good idea, and it may be possible for the bot to detect that no license tag is present. Dcoetzee 02:22, 27 August 2012 (UTC)[reply]

Correction on the timeline: The bot was approved for trial in April 2011, and performed the trial then. Then there was some discussion about an alternative implementation, and then the operator did not respond to requests for a status update, and the request was closed as expired on 2 July 2011. Then the bot started operating again on 15 July 2011, and apparently no one who noticed it running noticed that it was never approved until recently. Anomie ⚔ 02:35, 27 August 2012 (UTC)[reply]

Oops thank you for the update, my mistake. I don't remember what happened before as it was a very long time ago, but I think I just wasn't checking on the request page and was under the misapprehension that it was already approved. The bot seems to work fine in its current condition and I'm not planning an alternative implementation any longer. Dcoetzee 03:03, 27 August 2012 (UTC)[reply]

Question about a different issue: I once reported that there was a problem with transfers preserving template codes in image description pages that have incompatible meanings on Commons and on en-wp, especially the {{copyvio}} template. Has this been fixed? Fut.Perf. ☼ 06:55, 27 August 2012 (UTC)[reply]

Yes, these are removed in the current version. Not all such templates have yet been discovered but I can add them easily at a moment's notice. Dcoetzee 21:46, 27 August 2012 (UTC)[reply]

How about {{PD-release}}? That's a rather scary template. The Commons template says that the file has been released to the public domain by the uploader. The Wikipedia template also says that the file has been released to the public domain, but does not tell by whom it was released to the public domain. --Stefan2 (talk) 12:37, 5 September 2012 (UTC)[reply]

That one seems pretty rare and I think it can be handled manually after re-upload for now. Dcoetzee 09:24, 10 September 2012 (UTC)[reply]

I don't think you should be using regex to parse templates, for reasons vaguely outlined here. A Python package for template parsing exists at http://pypi.python.org/pypi/mwparserfromhell/ . →Σ σ ς. 07:29, 27 August 2012 (UTC)[reply]

I agree - I was not aware of this package and it looks excellent and I'll use it, although for the moment I think the regex parsing is sufficient. Dcoetzee 21:47, 27 August 2012 (UTC)[reply]

Is there a JavaScript port for this template parser? -- Rillke (talk) 17:48, 26 September 2012 (UTC)[reply]

There is another common "free in USA only" tag, {{FoP-USonly|France}}, for photos of recent French buildings. Replace "France" with "Belgium", "Russia" or some other country name for buildings in other countries. Wouldn't it be a good idea to add support for that tag too? I see FOP-related deletion requests on Commons all of the time. --Stefan2 (talk) 12:37, 5 September 2012 (UTC)[reply]

I think that would be another good one to add. Another one is for works that meet the threshold of originality in their source country but not the US. However, I need to work on the support for multiple kinds of reupload tags, which is currently ad hoc, so this would be a feature to add later. Dcoetzee 09:24, 10 September 2012 (UTC)[reply]

Can we please expedite this request? It is a necessary function, and having it blocked at this end is problematic for the wider functioning of xwiki issues, and will more likely result in the loss of viable images. The functionality is required, it has the base requirements, user is trusted and competent, so that leaves the requirement around the bot edits. — billinghurst sDrewth 10:59, 10 September 2012 (UTC)[reply]

Approved for trial (7 days). Please provide a link to the relevant contributions and/or diffs when the trial is complete. MBisanz ^talk 00:39, 11 September 2012 (UTC)[reply]

I've asked User:J Milburn to unblock the bot so that it can function during its trial, at User_talk:J_Milburn#Please_unblock_User:Commons_fair_use_upload_bot. Dcoetzee 13:19, 11 September 2012 (UTC)[reply]

{{OperatorAssistanceNeeded|D}} Bot's been unblocked. Any update? MBisanz ^talk 16:04, 19 September 2012 (UTC)[reply]

7-day trial since time of unblocking is now complete, so the bot is now disabled on English Wikipedia. Unfortunately during that time period nobody chose to use it. If I received another trial I could encourage people on Commons to use it during the trial period and find some images to use it on myself. Alternatively we could look at its historical contributions prior to the trial. Dcoetzee 20:58, 19 September 2012 (UTC)[reply]

Approved for extended trial. Please provide a link to the relevant contributions and/or diffs when the trial is complete. to give you more flexibility. MBisanz ^talk 13:22, 20 September 2012 (UTC)[reply]

Okay thanks! I've re-enabled it. I'll go through some DRs soon and probably run across some candidates for fair use upload. Dcoetzee 23:20, 22 September 2012 (UTC)[reply]

You have the most bureaucratic bot policy I've ever seen (and I wonder why people are still wanting to write bots for en.wp). If you don't integrate {{Fair use delete}} into Common's tools, this will be never successful. I suggest commons:MediaWiki talk:Gadget-QuickDelete.js or even commons:MediaWiki talk:Gadget-DelReqHandler.js (but this script lost its maintainers, though it is highly used). -- Rillke (talk) 17:36, 26 September 2012 (UTC)[reply]

{{OperatorAssistanceNeeded|D}} Any updates? MBisanz ^talk 18:42, 29 October 2012 (UTC)[reply]

The bot has been used several times over the last month and is apparently working normally, although CommonsDelinker is, frustratingly, still removing the images after they are locally uploaded, frequently causing the images to be deleted as orphaned. I have no control over CommonsDelinker, and the owner is uncooperative, but I have modified my bot to automatically revert its incorrect edits - this will be tested the next time it's used. I agree with Rillke that integrating it into Commons' tools would be helpful and is a good idea that I hadn't considered. Dcoetzee 20:12, 29 October 2012 (UTC)[reply]

Will it only revert CommonsDelinker once? I'm not at all sure that approving bots to revert other bots is a precedent we want to set. — madman 00:54, 1 November 2012 (UTC)[reply]

{{OperatorAssistanceNeeded|D}} Any updates? MBisanz ^talk 00:56, 31 December 2012 (UTC)[reply]

I think that this bot is working properly and that the bug is a problem with a different bot. --Stefan2 (talk) 12:06, 31 December 2012 (UTC)[reply]

So, does that mean we can approve this bot? MBisanz ^talk 00:20, 6 January 2013 (UTC)[reply]

{{botApproved}} MBisanz ^talk 00:19, 15 January 2013 (UTC)[reply]

A user has requested the attention of the operator. Once the operator has seen this message and replied, please deactivate this tag. I have reverted the approval for now. I could not find anywhere where the bot operator said it was working fine, and not conflicting with another bot. If CommonsDelinker is screwing up, then that should be fixed before this bot is approved. Otherwise the bot is importing orphaned fair-use images, and that's not of any use. Finally, I would like to see a comment by Dcoetzee to confirm everything is OK. That would make sure we're all on the same page so there are no screwups like last time. Maxim_(talk) 00:40, 15 January 2013 (UTC)[reply]

Sorry for not updating. The bot is working normally and transferring files as expected. As expected, some files have had FURs filled out after being transferred and remain, while others were deleted after 7 days because no one took an interest in preserving them. The bot is continuing to revert problematic edits by CommonsDelinker almost immediately, so files will not be orphaned, and most of the reverts I reviewed are correct (with the exception of one rare case involving a DYK). They won't get in a revert loop, because CommonsDelinker never repeats an edit (unless a file is restored and re-deleted). I realise bot-to-bot battle is undesirable, but most of CommonsDelinkers' edits are correct, so I would prefer not to block it, and I can't get in touch with the operator to actually get the redirect bug fixed - they're completely nonresponsive - although I'm sure the fix would be very simple. Dcoetzee 00:49, 15 January 2013 (UTC)[reply]

Approved. Looks good. I didn't mean to be a bureaucrat about this, but I was just confused; a comment such as "per IRC" from MBisanz next to the Approved template would have been sufficient. :p Maxim_(talk) 00:57, 15 January 2013 (UTC)[reply]

No problem, transparency is good anyway! Dcoetzee 00:59, 15 January 2013 (UTC)[reply]

The above discussion is preserved as an archive of the debate. Please do not modify it. To request review of this BRFA, please start a new section at WT:BRFA.