Wikipedia talk:Date formattings/script/MOSNUM dates

This is the "master" version of my MOSNUM script where all new ideas would go. These changes would go into production version only after additional testing. The workflow proposed, to allow more users to collaborate and to facilitate bug fixing, is something as follows:

$User comes up with an idea for an improvement of the script.
He applies his changes on User:$User/test/mosnum.js (a version of the script in his own userspace). Some testing is done to ensure there's no significant breakage.
$User applies the changes on Wikipedia:Date formattings/script/MOSNUM dates. Any significant changes are discussed at Wikipedia talk:Date formattings/script/MOSNUM dates.
Interested users (beta-testers) can import the changes from User:Ohconfucius/master/mosnum to their own userspace and test them. The results of the testing are posted to the talk page of the master script.
Ohconfucius imports the changes from the master script to his own userspace and tests them throughout. Lots of testing may not be needed if beta testers indicate that a lot of testing has already been done and everything works well.
Ohconfucius introduces the changes to the production version.

:Useful links

Utility scripts

Bug reports

Testsuite

test1 ( all to dmy, all to mdy)

Some questions about changes by Ohconfucius before 12 May

This changeset contains changes to Ohconfucius' test script that I didn't understand fully. Let's have some review and then incorporate them to the main script if everything is OK :)

Line 741

- regex(/\[\[(AD|BC|CE|BCE)([\s_]?)(\d{1,4})\]\]/gi, '$3$2$1');
+ regex(/\[\[(AD|BC|CE|BCE)([\s_]?)(\d{1,4})\]\]/gi, '$1$2$3');

Why don't we reorder dates into consistent format like 123 AD after unlinking? The change removes this feature. 1exec1 (talk) 21:11, 3 June 2012 (UTC)[reply]

I suspected that there would be a greater possibility of false positives by that approach, especially when the years are not four digits, so I opted to do it straight out of the links. --Ohconfucius ^¡digame! 02:44, 4 June 2012 (UTC)[reply]

Agree. Though I think it's worth to remove this rule altogether, because it would only match false-positives. I've already removed all links that point to the year pages. 1exec1 (talk) 16:06, 4 June 2012 (UTC)[reply]

Removed

Line 762

- ohc_regex(/\[\[@day[\s_](?:of[\s_])?@month\|([^\]]{1,30})\]\]/gi, "$1");
+ ohc_regex(/\[\[@day[\s_](?:st|nd|rd|th|)[\s_](?:of[\s_])?@month\|([^\]]{1,30})\]\]/gi, "$1");

Wouldn't @th? be sufficient instead of (?:st|nd|rd|th|)? 1exec1 (talk) 21:11, 3 June 2012 (UTC)[reply]

Possibly. I'll put that in. --Ohconfucius ^¡digame! 02:44, 4 June 2012 (UTC)[reply]

Done

Line 762

-    ohc_regex(/\[\[@Day\]\]/gi, "@Day");
+ // ohc_regex(/\[\[@Day\]\]/gi, "@Day");

Is there's something bad with unlinking days? 1exec1 (talk) 21:11, 3 June 2012 (UTC)[reply]

Er, not as such, but I didn't see the utility in doing lone 2-digit numbers only on the date range (ie 1-31). All numbers up to at least 2020 should be unlinked. I already have the following in the script:
regex(/\[\[([12]\d{3}|\d{1,3})\]\]/gi, '$1');
--Ohconfucius ^¡digame! 02:47, 4 June 2012 (UTC)[reply]

Ok, then it's probably worth to remove it. 1exec1 (talk) 16:11, 4 June 2012 (UTC)[reply]

Removed

Lines 875 and 881

+  ohc_regex(/(\()@YYYY[-–]@MM[-–]@DD(\))/gi, '$1@Day @Month @YYYY$2');
...
+  ohc_regex(/(\()@YYYY[-–]@MM[-–]@DD(\))/gi, '$1@Month @Day, @YYYY$2');

What's the reason for these additions? Was it that multiple dates within the same citation weren't converted? If it is so, then this change isn't necessary, as I've added proper fix for this problem. 1exec1 (talk) 21:11, 3 June 2012 (UTC)[reply]

Yes, palliative to correct ISO dates within simple parentheses that were not converted. It's a very specific string that is also very common, so I added that line. --Ohconfucius ^¡digame! 02:49, 4 June 2012 (UTC)[reply]

But this regex would convert ISO dates not only in citations, but everywhere. Is that by intention? If not, the regexes within ohc_ISO_to_dmy_in_references() should handle all ISO dates within references already. Are there any cases when they don't work? 1exec1 (talk) 16:23, 4 June 2012 (UTC)[reply]

It's a catch-all, useful in very specific cases. I've come across the occasional (military history) article where ISO dates are used in the body. Having this button gives convenience in converting these. --Ohconfucius ^¡digame! 09:30, 6 June 2012 (UTC)[reply]

Line 898

-    ohc_regex(/\{\{date\|(@yyyy-@zm-@zd)(?:\|dmy|)\}\}/gi, "$1"); //Template:date converts to dmy by default
+ // ohc_regex(/\{\{date\|(@yyyy-@zm-@zd)(?:\|dmy|)\}\}/gi, "$1"); //Template:date converts to dmy by default

What was the problem with the date template? Are there cases where the template doesn't result in dmy yet it is matched by the regex? 1exec1 (talk) 16:37, 4 June 2012 (UTC)[reply]

Its unnecessary, thus it was commented out. {{date}} renders dates to dmy by default, but I think I already have code that removes it earlier in the script. --Ohconfucius ^¡digame! 14:38, 5 June 2012 (UTC)[reply]

Agreed. 1exec1 (talk) 16:55, 5 June 2012 (UTC)[reply]

Removed

Lines 898 and 909

+ regex(/([^\d\w\/\-%,])@YYYY-@MM-@DD(<\s?\/ref.*?>)/g, '$1@Day @Month @Year$2');
...
+ regex(/([^\d\w\/\-%,])@YYYY-@MM-@DD(<\s?\/ref.*?>)/g, '$1@Month @Day, @Year$2');

(Note to myself): These aren't necessary; it's better to update regexes in ohc_ISO_to_dmy_in_references(). 1exec1 (talk) 16:37, 4 June 2012 (UTC)[reply]

Line 955

-  ohc_regex(/([^\d][^\w\d])@Month((?:\s@day?,?){1,6}),?(\/|\s?(?:[-–]|&ndash;)\s?|(?:[ _]|&nbsp;)(?:and|&|to|or)(?:[ _]|&nbsp;)+?)@Day,?\s(?:of\s)?(\d{3,4}[^\w\d][^\d])/gi, "$1$2$3@Day @LMonth $4");
+  ohc_regex(/([^\d][^\w\d])@Month((?:\s@day?,?){1,6}),?(\/|\s?(?:[-–]|&ndash;)\s?|(?:[ _]|&nbsp;)(?:and|&|to|or)(?:[ _]|&nbsp;)+?)@Day,?\s(?:of\s)?([12]\d{3}[^\w\d][^\d])/gi, "$1$2$3@Day @LMonth $4");

Are there any problems with 3-digit years? Also, it's probably better to use @yyyy instead of [12]\d{3} as we're in ohc_regex anyway. 1exec1 (talk) 16:54, 4 June 2012 (UTC)[reply]

I often find constructions like "Between June 6 and 7, 5000 soldiers attacked", so the [12] reduces the incidence of false positives. --Ohconfucius ^¡digame! 14:51, 5 June 2012 (UTC)[reply]

Agreed. It's certainly worth to limit the number of false positives. 1exec1 (talk) 16:37, 5 June 2012 (UTC)[reply]

Done

Lines 955 and 998

- ohc_regex(/([^\d][^\w\d])@Day\s@Month\s@YYYY(?=[^\w\d][^\d]|\b)/gi, "$1@Day @LMonth @Year");
+ ohc_regex(/([^\d][^\w\d])@Day\s@Month\s@YYYY(?=[^\w\d][^\d]|\b)/gi, "$1@DD @LMonth @Year");
- ohc_regex(/([^\d][^\w\d])@Day\s@Month(?=[^\w\d][^\d]|\b)/gi, "$1@Day @LMonth");
+ ohc_regex(/([^\d][^\w\d])@Day\s@Month(?=[^\w\d][^\d]|\b)/gi, "$1@DD @LMonth");
...
- ohc_regex(/([^\d][^\w\d])@Month\s@Day,?\s@YYYY(?=[^\w\d][^\d]|\b)/gi, "$1@LMonth @Day, @YYYY");
+ ohc_regex(/([^\d][^\w\d])@Month\s@Day,?\s@YYYY(?=[^\w\d][^\d]|\b)/gi, "$1@LMonth @SD, @YYYY");
- ohc_regex(/([^\d][^\w\d])@Month\s@Day(?=[^\w\d][^\d]|\b)/gi, "$1@LMonth @Day");
+ ohc_regex(/([^\d][^\w\d])@Month\s@Day(?=[^\w\d][^\d]|\b)/gi, "$1@LMonth @SD");

This first one must be a bug: @DD is day with leading zero. As for the second one, we already use mostly @Day for days without leading zeroes, so it makes sense to leave this as is for consistency. 1exec1 (talk) 16:54, 4 June 2012 (UTC)[reply]

Agreed. --Ohconfucius ^¡digame! 14:51, 5 June 2012 (UTC)[reply]

Done

Line 1073

- regex(/([\|\{]\s*(?:file|image\d?|image location\d?|img|pic|title|quote|journal|url|work|doi)\s*=)([^\|\}]*)([\|\}])/gi, protect_function);
+ regex(/((?:file|image\d?|image location\d?|img|pic|title|quote|journal|url|work|doi)\s*=)([^\|\}]*)([\|\}])/gi, protect_function);

I think it's worth to keep the check of preceding { or | symbols as it somewhat ensures that we're in a template. Were there any cases where this broke anything? 1exec1 (talk) 16:54, 4 June 2012 (UTC)[reply]

It seemed redundant because the above are all used as parameters inside the myriad of templates that exist in WP. If you think it's safer, then OK to restore. --Ohconfucius ^¡digame! 14:46, 5 June 2012 (UTC)[reply]

The current code already matches either of template name or parameter name (both | and { characters are matched). Thus preserving the check doesn't reduce the usefullness of the regex. 1exec1 (talk) 16:41, 5 June 2012 (UTC)[reply]

One problem with reinserting the ([\|\{]\s* regex string is that parameters will be evaluated by the regex in relation to adjacent parameters. From here, the parameter adjacent to a protected one will not be acted upon because the closing pipe mark will have been already grabbed thus leaving no opening pipe mark for the one that follows viz:

|url=http://pandora.nla.gov.au/pan/23790/20091105-0000/issue1026.pdf |title=The ARIA Report: Week Commencing 26th October 2009|work=[[Pandora Archive]]|issue=1026|format=pdf|accessdate=November 21, 2009}}

becomes

|url=⍌17⍍|title=The ARIA Report: Week Commencing 26th October 2009|work=⍌18⍍|issue=1026|format=pdf|accessdate=November 21, 2009

We've already established that lookaheads do not work in this context; the contents of |title=, which ought to be protected, aren't. Do you have any suggestions to fix this? --Ohconfucius ^¡digame! 09:19, 6 June 2012 (UTC)[reply]

Now I see. What if we run the same regex second time? It should protect the remaining titles. 1exec1 (talk) 12:57, 6 June 2012 (UTC)[reply]

OK, that can work. --Ohconfucius ^¡digame! 15:04, 6 June 2012 (UTC)[reply]

...

I'll add more later. 1exec1 (talk) 21:11, 3 June 2012 (UTC)[reply]

customization points

The customize* functions are there to be overridden by the user in his own .js file. It doesn't make sense to put anything there, we should just make another function. 1exec1 (talk) 16:59, 4 June 2012 (UTC)[reply]

OK. Didn't realise what it was for. I needed an extra code loop and it was convenient. But these are my own customisations to a degree, and I don't expect anyone else to use them. I forgot to remove the customisations in the production script. These are used by me to 'correct' Reflinks (which inserts a lot of yyyy-mm-dd dates). --Ohconfucius ^¡digame! 14:45, 5 June 2012 (UTC)[reply]

Testsuite

I think it's worth to combine the known tests into a common testsuite so that this knowledge would become more "public". I propose the following:

There's a common page (or several pages if the testsuite becomes very large) at, say, Wikipedia:Date formattings/tests/MOSNUM dates
There's several other pages containing the expected output of the script. For example, say, there's Wikipedia:Date formattings/tests/MOSNUM dates1/all_to_dmy with the text that would result by applying all to dmy to the Wikipedia:Date formattings/tests/MOSNUM dates page. The same for every operation.
Testing workflow: $User opens Wikipedia:Date formattings/tests/MOSNUM dates and hits the relevant script button in the toolbox. After the script completes, $User does not save the page. Instead, he copies the resulting text, opens relevant page with the expected output (say, Wikipedia:Date formattings/tests/MOSNUM dates1/all_to_dmy) and pastes the text into the edit box. $User does not save the page. Instead, he pushes the show changes button and verifies there have been no changes.

Any opinions? 1exec1 (talk) 17:26, 4 June 2012 (UTC)[reply]

I agree we need a test suite with methodology. I've been using borrowed test files up to now, but never worked out against what to evaluate the output, so what you propose makes perfect sense. --Ohconfucius ^¡digame! 09:57, 6 June 2012 (UTC)[reply]

We probably need somewhere to file bug reports too. --Ohconfucius ^¡digame! 09:58, 6 June 2012 (UTC)[reply]

Yes, this is missing. I think it's worth to change the URL scheme then, to have test and bug pages as subpages of the script page. Also, upstream and master became redundant after the scripts have been moved to Wikipedia namespace - the location says by itself that the scripts here are the master/upstream versions. What do you think about Wikipedia:Date formattings/scripts/MOSNUM dates, Wikipedia:Date formattings/scripts/MOSNUM utils for the script, Wikipedia:Date formattings/script/MOSNUM dates/test1 (and similar) for tests and Wikipedia:Date formattings/script/MOSNUM dates/bugs for bug reports? 1exec1 (talk) 13:07, 6 June 2012 (UTC)[reply]

No quibbles there. --Ohconfucius ^¡digame! 15:06, 6 June 2012 (UTC)[reply]

Could you post a list of still not fixed bugs here ? The issues that were raised on my talk page seem to be fixed. 1exec1 (talk) 15:46, 12 June 2012 (UTC)[reply]

Conflict with 2017wikitexteditor

I thought that you should know that I found (through trial and error) and this conflicts with the 2017 Wikitext editor.
SSSB (talk) 09:43, 27 January 2021 (UTC)[reply]