User talk:Mathglot/sandbox/Templates/Alberta templates/Municipalities of Alberta

	Canada portal This page is within the scope of WikiProject Canada, a collaborative effort to improve the coverage of Canada on Wikipedia. If you would like to participate, please visit the project page, where you can join the discussion and see a list of open tasks.CanadaWikipedia:WikiProject CanadaTemplate:WikiProject CanadaCanada-related articles
NA	This page does not require a rating on Wikipedia's content assessment scale.
	This page is supported by WikiProject Alberta.
	This page is supported by WikiProject Canadian communities.

Convert table to template with display parameters

Legacy discussion

This discussion was originally started at Wikipedia:Help desk, and added here for context. Click 'show' to view.

This discussion was originally started at Wikipedia:Help desk. It is pasted here for context to continue discussion on this template talk page.

The following discussion is closed. Please do not modify it. Subsequent comments should be made on the appropriate discussion page. No further edits should be made to this discussion.

Hello Help deskers! I would like to convert a wikitable to a template. See {{Municipalities of Alberta}} for a truncated table of municipalities in Alberta that I intend to evolve into a working template with display parameters. Of the eight rows, there are two instances of four different municipal status types. I would like parameters for each of the four municipal status types.

{{Municipalities of Alberta|cities=yes}} displays rows for municipalities that have city status and the "Total cities" sortbottom row
{{Municipalities of Alberta|towns=yes}} displays rows for municipalities that have town status and the "Total towns" sortbottom row
{{Municipalities of Alberta|villages=yes}} displays rows for municipalities that have village status and the "Total villages" sortbottom row
{{Municipalities of Alberta|summervillages=yes}} displays rows for municipalities that have summer village status and the "Total summer villages" sortbottom row

Can anyone coach me on how to do this please? I am a relatively quick learner in a lead by example situation. In other words, please edit {{Municipalities of Alberta}} and I will analyze and learn from the diffs. Cheers, Hwy43 (talk) 00:47, 8 January 2022 (UTC)[reply]

Hi Hwy43. I tried to mark off a single row of the table with an <onlyinclude> section, but it didn't work. I have done something similar, but not for individual rows in a table, which may need a different solution. If I get some time, I will take a copy of your template a do some fiddling, because I would like to see if a solution is possible. --Verbarson ^talk_edits 12:13, 8 January 2022 (UTC)[reply]

Hello Verbarson. Thanks for taking a stab. I look forward to any input in future. I will approach the Wikipedia:WikiProject Templates as well. Cheers, Hwy43 (talk) 02:02, 9 January 2022 (UTC)[reply]

@Hwy43:, partially converted; summary rows currently non-parametrized (all summaries always included). See example below.

Show `{{Draft:Municipalities of Alberta\|cities=yes\|towns=yes}}`
Draft:Municipalities of Alberta

Is that pretty much it? ~~There's an introduced bug that messed up the values in the change percentage column. Haven't looked at that, maybe you can spot the problem and fix it.~~ Also, it would be beneficial to change the conditionals to use the template {{yesno}} which allows more values than just yes (including y, true, 1, etc.) and so is more forgiving. Hope this helps, Mathglot (talk) 08:51, 9 January 2022 (UTC)[reply]

Fixed the bug; summary rows still need parametrizing by the same method; try it! Mathglot (talk) 09:20, 9 January 2022 (UTC)[reply]

As a practical matter, it looks like your small table is just a tester, and you plan to make a complete, A-Z table with lots of cities, right? Since the syntax of each row is not complex but extremely tedious, copy/pasting or typing errors are highly likely, and one misplaced vertical bar and the whole table will come out junk, and be extremely hard to debug. Even if you get it right, finally, it will be unmaintainable as the values change over time, no one will dare touch it, so it will slowly go out of date and become useless. So instead, create a subtemplate that generates one row with the proper syntax, e.g., {{muni AL row|Airdrie|C|January 1, 1985|61581|43271|84.57}} to generate row #2 with the proper syntax, and then the main table is just a collection of dozens of these row templates, one per urbanization, with the proper parser conditional around it (or pass the params through to the row, and have it return empty when appropriate), and then the whole thing will be both easier to create, and far easier to maintain. Good luck! Mathglot (talk) 09:42, 9 January 2022 (UTC)[reply]

Hi Mathglot. Thanks for the help! Dropping an interim note for now to acknowledge and advise that I will be studying what was done. I will return to ask some questions and more fulsomely respond. One spoiler alert though. I will want to further evolve the template, with optional parameterized columns as well. Cheers, Hwy43 (talk) 20:20, 9 January 2022 (UTC)[reply]

Hwy43, you're welcome. The current version should be considered only a way station on the road to a parametrized row template, without which the table-level template is ultimately doomed. For how to approach parametrizable columns, see for example this table template and this row template, but I can't stress enough how important it is to deal with the rows, first. A good exercise for you would be to alter the Alberta template so that only the proper summary rows are visible, depending on the parameters.

Also, I'm thinking that the general-interest portion of this discussion at the Help desk has about run its course, and as this discussion gets more technical maybe we should carry on at Template talk:Municipalities of Alberta, where it's more likely to be found and get feedback from interested readers. If you have no objection, I'll initiate one there (or you can just respond to this there instead of here) and we can link the conversations with {{Moved discussion}} so one can follow from one page to the other. Are you okay with that? Mathglot (talk) 22:16, 9 January 2022 (UTC)[reply]

Mathglot, I'd be thrilled with that! I have created the talk page and will start a thread using {{Moved discussion}}. Cheers, Hwy43 (talk) 22:36, 9 January 2022 (UTC)[reply]

The discussion above is closed. Please do not modify it. Subsequent comments should be made on the appropriate discussion page. No further edits should be made to this discussion.

Goals

So my goal is to have a single table-based template covering all municipalities in Alberta with parameterized rows based on municipal status type and parameterized columns based on the scope of the articles intended to call the template. The benefit of this will be only having to edit quinquennial federal census results in one location for all ~340 municipalities rather than in multiple locations in redundantly built tables. The onlyinclude/noinclude approach has been utilized to date to minimize the amount of locations but hasn't eradicated redundancy.

The following articles/sections are those with existing tables:

Parameterized rows will be based on the following municipal statuses:

City
Town
Village
Summer village
Specialized municipality
Municipal district
Improvement district
Special area
Métis settlement

Common to all instances of the called template will be the following columns:

Name
Population (20xy)
Population (20xz)
Change (%)
Land area (km²)
Population density (per km²)

Parameterized columns will be the following, which will be placed between column 1 ("Name") and column 9 ("Population (20xy)"):

Status
Region
Census division
Neighbouring municipality
Incorporated date
Council size
Municipal census population (and year)

Hwy43 (talk) 23:33, 9 January 2022 (UTC) (sig replicated here by Mathglot (talk) from next sig after the header below)[reply]

Development approach

Once the code for the parameterized rows and columns is understood, I will add the remaining ~330 rows of municipalities by concatenation to build the necessary code in Excel.

At the moment, the steps as I see moving forward are as follows:

Widen the table to accommodate columns 3 through 8.
Solve the sortbottom rows issue.
Add the code to parameterize columns 2 through 8.
Add the remaining ~330 rows.

Surely there is more. I will start on step 1 while awaiting any comments. Cheers, Hwy43 (talk) 23:33, 9 January 2022 (UTC)[reply]

You've been busy! I see you've already added the six new columns, bravo. Your development plan sounds reasonable, but imho it requires a new step, "Create table row template", so that the table template becomes long, but simple to read and modify, namely, 330 transclusions of the row template. Whether that step happens before or after current step 2 (sortbottom rows) is immaterial. I'll look into creation of some kind of row template for you, unless you wave me off and prefer to do it yourself; I can give you tips on how to approach it, in that case.

As a practical matter, you'll have a max of, what—sixteen columns? The table is currently already quite wide; you might want to do some purely cosmetic editing, narrowing the column widths wherever you can. Some ideas: change the date column to default to format yyyy-mm-dd (and abbreviate the column header as necessary; maybe Incorp{{br}}date); abbreviate column headers for census division, council size and so on; if it looks too cryptic when short, add a tooltip, or a legend below the table explaining the meaning of the column headers; abbreviate col headers for all pop. columns; maybe even reducing left- and right-padding. Mathglot (talk) 00:37, 10 January 2022 (UTC)[reply]

I am amenable to considering a row template. I would greatly appreciate you creating it as offered. I suggest the title of the template be {{Template:Muni AB row}} where AB = the province's postal abbreviation (saving AL for Alabama).

We are at thirteen columns but the maximum needed at any of the benefitting list articles is ten. I have abbreviated some column headers and applied tooltips to explain the abbreviations. Regarding the date format, let me think about that. It might be a last resort if confirmed necessary.

Would any of this changes thus far, or implementation of the last resort, compromise the WP:FL status of List of municipalities in Alberta?

Note that once this is implemented across Alberta lists, I will be leading the charge on similar efforts for the remaining provinces and territories of Canada. Cheers, Hwy43 (talk) 01:55, 10 January 2022 (UTC)[reply]

In the meantime, I have tried parameterizing four of the sortbottom rows. I have the template called five different ways at User:Hwy43/sandbox#Template:Municipalities of Alberta tests. Note how all continue to be displayed in all versions. Was this the same barrier you were bumping into yesterday? Cheers, Hwy43 (talk) 02:19, 10 January 2022 (UTC)[reply]

I had anticipated a number of things you mentioned, including the expansion to other provinces. Giving myself a WP:TROUT for momentarily forgetting the 'AB' postal abbreviation.

As far as FL status, I'm not the best person to ask about that, but to the extent that it enhances the ability to present information to viewers, and reduces the possibility for a dozen or fifty different pop. tables getting out of sync with each other, because all of the data is being kept in one place, I would think it would count as a net positive toward Featured status. It's hard to predict how FL evaluators would assess the impact of the template. To the extent that the template row will isolate all the complexity inside it and change only very slowly over time by template writers, and the separation of all the actual city data will be in the updated Municipality table with almost no other code present, making it easily understandable and maintainable (even easier than a wikitable, but a good amount), I would think that would also count as a positive. But, as I say, I don't know how they do their evaluations.

I haven't looked at how you've managed the bottom row stuff yet, but I'll have a chance tomorrow, probably. I wasn't bumping into a barrier, there's no particular difficulty to it, I just figured that once you saw the paradigm for doing conditional rows in the table body, you'd be able to use the same idea and apply it to the footer material, so I assume that's what you did.

A final word concerning the 300+ row conversion: we should probably have another subsection about that when the time comes, but just fyi, it might be possible to develop a regular expression to convert the data you find online into our template row format, so you don't have to do a lot of repetitive typing. Or is that what you meant by using Excel? We can talk about that later, though. Thanks, Mathglot (talk) 04:41, 10 January 2022 (UTC)[reply]

My revised diff above will give you a snapshot of what I did. You are correct. I applied the paradigm I had observed but am coming up short. I feel I am close though.

It appears we are speaking the same thing. I use Excel's concatenate function to build a regular expression. All rows in the tables across the various list articles in the Goals section above were built with ease due to this function. I have even built full paragraphs on content and infoboxes using the same. Hwy43 (talk) 05:41, 10 January 2022 (UTC)[reply]

Let me know if you get stuck with the summary rows. Ah, good to know you used regexes to create the table. Mathglot (talk) 19:29, 10 January 2022 (UTC)[reply]

Mathglot, I am stuck with the summary rows. I feel like I am so close. Regexes are the way to go. It would be so painfully slow to build and avoid human errors without it. Hwy43 (talk) 03:17, 11 January 2022 (UTC)[reply]

@Hwy43: I'll look at that, not sure if I'll get to it today. If you decide to move on it before I do, please make changes in the sandbox; you can test either using Special:ExpandTemplates or using the testcases page. Mathglot (talk) 04:40, 11 January 2022 (UTC)[reply]

@Hwy43:, summaries fixed in the sandbox version. See the test cases, or try out {{Draft:Municipalities of Alberta/sandbox|cities=yes}} for example. If all the test cases are doing the right thing, we can copy the sandbox back to the main template. Mathglot (talk) 18:43, 11 January 2022 (UTC)[reply]

@Mathglot:, appreciated. Three minor things noticed in test cases:

In cities=yes, "Total cities" row, final column, the value in the cell is top-justified;
In towns=yes, "Total towns" row, final column, the value in the cell is top-justified; and
In summervillages=yes, "Betula Beach" row, final column, the value in the cell is top-justified.

I recall fixing something similar in an earlier version of the template. It is like there is a hard return or two in the last cell forcing the cell value up to appear as though it is top-justified. Here is the edit I believe that fixed this last time. I will try to replicate. Hwy43 (talk) 03:36, 12 January 2022 (UTC)[reply]

This fixed #3! Hwy43 (talk) 03:43, 12 January 2022 (UTC)[reply]

But I have been unable to fix #1 and #2 at Draft:Muni AB footer. :( Hwy43 (talk) 04:07, 12 January 2022 (UTC)[reply]

One other thought: I know you considered anchors earlier, at least to identify the "A" cities, "B", and so on. I'm a fan of direct row-addressability in tables, and you can use the same approach you did to anchor every row, the most natural anchor being the municipality name. But maybe you don't need that. Mathglot (talk) 04:51, 10 January 2022 (UTC)[reply]

I need to think about that. I removed the anchors when I re-sorted the rows as I realized that numerous anchors would be lost depending on which parameters are not enabled. Hwy43 (talk) 05:41, 10 January 2022 (UTC)[reply]

Re "numerous anchors would be lost", not sure I understand (but maybe I don't need to). Naturally, with conditional rows, some locations would not be in the table, such as towns when only cities were selected, so that if someone targeted #Athabasca (town) when only cities were showing, that would just go to the top of the page. Is that what you meant by "missing"?

There is another approach to conditional tables which I didn't mention because it's generally inappropriate, which would give a different effect if a user tried to target #Athabasca on the cities-only table. That is to *always* generate the entire table code, but control the display of unselected rows via css. That is, when you are showing only cities, the row for Athabasca (town) would still appear in the wikicode, along with its anchor, but would not be rendered on the article page by the browser, due to the presence of a css property of display:none. I'm not sure if wikitables support this property, maybe in the style it could be added. If so, and you added the css where needed, then the tables would always be full-length in the wikicode of any page where the template was transcluded (increasing load time of the page slightly) but the same length as now in the browser window (no effect on scroll time). It's an unorthodox approach, but it would give you an interesting effect, which is that if a user chose to link #Athabasca on a page that displayed only "cities", it would go to the next row in the table *after* Athabasca that was a city, presumably "Beaumont". I'm pretty sure you won't want to do this, but I thought I'd better mention it, just in case. Mathglot (talk) 19:29, 10 January 2022 (UTC)[reply]

I am not seeing a need to anchor to any municipalities within the template. I was originally thinking alphabetical anchors such as {{anchor|A}} to enable navigation from {{Compact ToC||q=|u=|x=|z=|name=no}} (see at List of municipalities in Alberta#List of urban municipalities) but if we render cities only then we won't get a working A-anchor to Airdrie because the anchor is assigned to a village (Acme). Similarly the B-anchor won't go to Beaumont because it is assigned to a town (Banff). Regardless, should a need for anchors emerge, let's resist the css route as suggested. Hwy43 (talk) 03:52, 11 January 2022 (UTC)[reply]

Your decision sounds fine to me. Should you change your mind, it's still possible to have an 'A' anchor, by not assigning it to a particular municipality row, but to an extra table row that has zero vertical height and is always there. Mathglot (talk) 06:14, 11 January 2022 (UTC)[reply]

Interesting. Let's park this for future consideration. Hwy43 (talk) 04:08, 12 January 2022 (UTC)[reply]

A bit of housekeeping I forgot: I agree with your suggested name of {{Muni AB row}}. At some point during development, the current Draft:cpmr should be renamed; maybe first to Draft:Muni AB row, to make sure all the transclusions are fixed appropriately before going live with it, and then to Template:Muni AB row once everything looks good. Cheers, Mathglot (talk) 22:53, 10 January 2022 (UTC)[reply]

Agree, but with it evident that I can use concatenation to build the wikicode with ease and also minimize human error, is the need for a row template now waning? I realize it may then be harder for others to edit, but this is a template that should need minimal editing between every five-year census release. Perhaps there would be protection applied to the template once live to prevent novice, unregistered and non-autoconfirmed users from editing. Hwy43 (talk) 03:55, 11 January 2022 (UTC)[reply]

Responded at #Row template below. Mathglot (talk) 06:22, 11 January 2022 (UTC)[reply]

I've created sandbox and testcases files for testing. The sandbox currently contains the fixes for the summary row that was a sticking point. Mathglot (talk) 10:26, 11 January 2022 (UTC)[reply]

Row template (8-column)

This older format is now deprecated, in favor of the #Thirteen-column format.

I've added this subsection to help organize the discussion; this is about the creation of the row template, currently at Draft:Cpmr. I had been looking at your previous version of the Municipalities table with eight columns when I created it, so it matches that format, not your current, wider one. The point is, to show how it simplifies the construction and maintenance of the main table.

Here's what transcluding the row template would look like, using that format and additional helper templates for the header and footer:

Example of generating the 8-col version of the table using the row template
Draft:Cpmh Draft:Cpmr Draft:Cpmr Draft:Cpmr Draft:Cpmr Draft:Cpmr Draft:Cpmr Draft:Cpmr Draft:Cpmf

Note that this is the first draft of the row template, so it doesn't have the conditional parser code in it yet that allows you to choose just cities or towns, and so on. So, there are two enhancements to be made to it:

expand the row to the wider table size
add conditional rendering based on the four existing parameters

You've already done #1 in the current version of the main table, so doing it in the row template should be easy for you. You've seen how the conditionals work in the main template, and I think you'll be able to mirror that here. Let me know if you need help with anything.

Note that this design will also make it possible to do the conditional columns later, which otherwise would have been impossibly complex, and resulted in inscrutable code. There will still be some complexity, but it will all be isolated in the row template, while regular editors can just change the values in the main template when some updated population or other figures and dates come out, which should be a relatively low-risk operation. Mathglot (talk) 04:05, 10 January 2022 (UTC)[reply]

Thanks. Looking very good. We may need to reduce some of the functionality associated with the "prov" and "loc" parameters in order to avoid redirects but more importantly to avoid dabs. Here are some examples (I am going out of province for some of these, thinking ahead to other provinces).

{{cpmr|[[Edmonton]]|City|January 1, 1985|61581|43271|84.57}} – to avoid redirect
{{cpmr|[[Langley, British Columbia (district municipality)|Langley]]|District municipality|January 1, 1985|61581|43271|84.57}} – to avoid dab (typical article name disambiguation format)
{{cpmr|[[Clermont, Abitibi-Témiscamingue, Quebec|Clermont]]|Township|January 1, 1985|61581|43271|84.57}} – to avoid dab (different article name disambiguation format)
{{cpmr|[[Plessisville, Quebec (parish)|Plessisville]]|Parish|January 1, 1985|61581|43271|84.57}} – to avoid title for a different municipality that is the primary topic
{{cpmr|[[Municipal District of Lesser Slave River No. 124]]|Municipal district|January 1, 1985|61581|43271|84.57}} – as certain municipality articles will not follow the typical "Name, Province" article naming format
{{cpmr|[[Lloydminster]] (part)|City|January 1, 1985|61581|43271|84.57}} – to apply a qualifier after the wikilink (in this case a single municipality is split by a municipal boundary)

Surely there are other unique examples. I also want the ability to wikilink entries in the "type" parameter and add ref tags such as {{#tag:ref|Sample note...|group=lower-alpha}} and <ref>{{cite web | ... }}</ref>. I will study all the list articles to see if any other functionality needs exist.

Note my attention to this is about to wane despite all this initial momentum. Back to the grind tomorrow after three weeks off. Hwy43 (talk) 06:37, 10 January 2022 (UTC)[reply]

Good point about redirects and links, hadn't thought about that. Assuming that the concatenation of TownName + ', ' + ProvinceName is correct most of the time with a bunch of exceptions, then what I'd do is leave it as is by default, but add a new, optional parameter which, if present, would override the default concat to generate the link target for the city. There is plenty of precedent for this in templates, see for example param |lt= in {{ill}}, or param |disp= in {{lnc}}. If you decide to go this route, a good name for it, imho, might be |muni-link= on analogy with the |author-link= param in {{citation}}. Ditto for the unlinked part of Lloydminster.

I added two optional params to deal with this, resulting in the following; is this what you'd expect to see?:

Tabular result for your examples above
Draft:Cpmh Draft:Cpmr Draft:Cpmr Draft:Cpmr Draft:Cpmr Draft:Cpmr Draft:Cpmr

Note: If the table above looks broken, it may be due to changes to the templates made later. Below is a "subst'ed" version of the same thing, that should freeze the result as it was at that time:

Same as above, but substed from rev. 1064915132

List of urban municipalities in various provinces (illustrative example only; province name not shown)
Name	Status	Incorporation date (current status)	2016 Census of Population
Name	Status	Incorporation date (current status)	Population (2016)	Population (2011)	Change (%)	Land area (km²)	Population density (per km²)
Clermont	Township	January 1, 1985	61,581	43,271	+42.3%	84.57	728.2/km²
Edmonton	City	January 1, 1985	61,581	43,271	+42.3%	84.57	728.2/km²
Langley	District municipality	January 1, 1985	61,581	43,271	+42.3%	84.57	728.2/km²
Lloydminster (part)	City	January 1, 1985	61,581	43,271	+42.3%	84.57	728.2/km²
Lesser Slave River (dist. #124)	Municipal district	January 1, 1985	61,581	43,271	+42.3%	84.57	728.2/km² Draft:Cpmr

Note that the above table is valid as of rev. 1064915132 of Draft:cpmr; when the draft template is changed, the above may change as well as it is transcluded, not substed; or it may break, depending on what is done.

You could, of course, do it the other way, requiring |loc= to include all necessary wikicode, including correct positioning of brackets; that would require every city in the table to have correct linkage. Whether that option is better than the positional param approach, only you (and other WikiProject Canada members) can decide. To some extent, it would depend how frequent the exceptions are, and how much you trust editors to fix the long tables and not mess it up with malformed links. A hybrid approach, would be to fold parameters |muni-link= and |suffix= into one, keeping just the first, and requiring all linkage markup to be included in it; e.g., |muni-link=[[Lloydminster, Alberta]] (part) if it's easier to understand that way, and has the advantage that |loc= needs no bracket markup. (You could go even further: permitting unbracketed 'loc' values in the normal case which would then be properly marked up by the template the way they are now, but also detecting '[[' in the first two characters of the value, and then handling those cases as given.)

If you find other special cases in the wild, I recommend following the same path, i.e., adding optional, named params to handle them. If you get some municipality that is completely wacky and breaks all the rules, remember that you don't *have* to use the template, it's there for convenience and clarity; if it would be clearer to the reader of the wikicode, you can always skip the template for that wacky row, and just include the required wikitable code for that one row instead. You don't have to turn yourself into a pretzel by adding sixteen optional params, just to force a wacky case to come out right via the template; that would be counterproductive. Haven't looked at type or ref. Mathglot (talk) 21:28, 10 January 2022 (UTC)[reply]

Well crap. My fulsome replies to the above paragraphs were replaced by a single '`'. I'll try to respond again here:

Re: new optional parameter... yes, let's do that and go with |comm-link= as this could eventually be used for unincorporated communities in addition to municipalities.

Re: my expectations to see... visually, five of six are to my expectations. For Lesser Slave River, I am expecting to see the entire article name (i.e., Municipal District of Lesser Slave River No. 124). Alternately I would shuffle it to read as Lesser Slave River No. 124, Municipal District of to maintain proper alpha sorting, or better yet Lesser Slave River No. 124, MD of to shorten and maintain proper alpha sorting. Non-visually, Lloydminster should avoid the Lloydminster, Alberta redirect and go straight to Lloydminster.

Re: |loc= including all necessary wikicode... I am okay with that if the new optional parameter above is not pursued. Ultimately, I agree with folding parameters {para|muni-link}} and |suffix= into simply one like |comm-link=[[Lloydminster]] (part).

Re: other special cases, there may be instances in which two or more wikilinked values are required in a single cell. I will investigate the various lists to see how frequent to determine if there is a case for additional optional parameters or if we can simply not use the row template for those. Good idea.

This should catch me up on this part of the thread. Hwy43 (talk) 04:44, 12 January 2022 (UTC)[reply]

Row template vs regex generation of main table template

Above, you asked about whether there is really a need for a row template:

is the need for a row template now waning?

In brief, I would say the need for a row template is increasing, and will increase a great deal more when the conditional parser code is added to control conditional columns. If you're having doubts about how to proceed, you needn't take my word for it; you could post a request at a WikiProject, or call on trusted template editors to have a look at this situation, and give you their opinion.

Note: Brief background for new editors arriving here: you have a pdf file with data from some 300 or more Alberta municipalities, and you wish to create a templatized Wikitable containing 300 rows somewhat like the sample table shown at the top of Draft:Municipalities of Alberta, with the possibility of having conditional display of certain rows and columns. A selection of rows, such as "all cities", or "all towns", and so on, may be displayed depending on the value of the "status" cell in the row (such as 'city', 'town', etc.), and whether it matches a user-provided parameter (e.g., |towns=yes): this would generate the table of towns, and will make the template usable in multiple different articles about Alberta. (See #Smoke_test section at testcases.) You originally defined eight table columns of information, depending on six cell values; this was upped to thirteen columns recently. This will involve six fixed columns, and seven dynamic columns which may be chosen, presumably, via additional template parameters. Eventually this will be generalized beyond Alberta to all Canadian provinces.

If I understand what you are proposing, you wish to evaluate whether it's possible or advisable to use a regular expression against the pdf file containing your raw data, to convert it into wikicode that would directly generate your "Municipalities of Alberta" template, without the need for a row-generating template.

Generally speaking, I think that wouldn't be the best approach, for several reasons. Even if it's possible to do what you say using a regex (and I consider myself a crackerjack regex user), I think the resulting template would be long, and impossibly complex. A single misplaced vertical bar or pair of curlies might be extraordinarily difficult to track down. You might respond, that you could just use the regex again on the original file to recreate the whole thing, and although I have my doubts whether truly template-ready code could be generated from your dataset without some tedious manual postprocessing, even if it could, that would leave you with a template that only you could modify.

Using the row-generator sub-template (currently at Draft:cpmr for the older, 8-column format), the Alberta template would consist of 300 lines, with one transclusion per city, plus a few lines at the top and bottom for the table envelope, and your summary rows at the bottom (currently in Draft:cpmh and Draft:cpmf). One transclusion of the 8-col row template is about 50-120 characters (Airdrie=55 chars, Plessisville=120); so a row in the main template would look like this in the wikicode for the 6-param, 8-column table:

{{cpmr|Airdrie|City|January 1, 1985|61581|43271|84.57}}

If the average is around 80 characters, the Alberta template would be 300 * 80 or 24,000 characters. (This calculation is based on the row-template that generates eight columns; since you increased the columns to 13, let's call it 300 * 120 or 36kb, and there would be five additional params in each line above.)

Now, if you don't use the row-template, the same, 8-col row would expand to 360 characters per row without conditionals, probably double that with, and times 300 if you use the regex gives you around 260kb of dense wikitable code full of parser conditionals. I think that would be close to unmaintainable, even if you only have to touch it every five years after each Canadian census.

Anyway, like I say, the template will end up being used by you and the Canada group so I don't really care how it ends up, but imho, you'd be wise to isolate all of the complexity into the row template, get it right once, and then transclude it 300 times, rather than try to build a giant, conditional table template from a regex applied to external data. An example of what the straightforward wikicode would look like for a six-row table using the row template can be found on the /doc page of Draft:cpmr in the Draft:cpmr#Examples section. Note that this version of the row-template generates the older, 8-column format, and does not yet contain code for dynamic row and column generation, so the row template code will expand significantly, and the transclusions will expand slightly to add new params (and probably using the named aliases will make it more understandable, as six positional params is about all I can handle without getting confused).

I'd certainly encourage you to solicit additional opinions on this, linking this section of the discussion. Thanks, Mathglot (talk) 08:45, 11 January 2022 (UTC)[reply]

You have successfully walked me off the ledge. I will just have to redesign some aspects of my Excel-based concatenation approach. Hwy43 (talk) 04:50, 12 January 2022 (UTC)[reply]

|}

Conditional columns

This is premature for where we are now with the rest of it, but just wanted to add this section as a place to stick any questions about it, and also to give you a heads-up to look at this template as an example of one that handles conditional columns. We can come back to this later, I don't advise doing anything now about it, until everything else is pretty stable, and we've tried it first with at least one full (hundred+) list of municipalities, and preferably also with one other province, at least with a short list. Mathglot (talk) 05:02, 10 January 2022 (UTC)[reply]

Thumbs up. Hwy43 (talk) 06:37, 10 January 2022 (UTC)[reply]

What with all the other things cropping up with the full table expansion in the sandbox, my current thinking is that since the conditional column feature is at least as complex or more so perhaps than the current issues, it might be worth putting this off to a phase two development after the initial release. If you try to get this into the initial release, it may push it off to who knows when. Unless lacking dynamic columns is a deal-breaker and would make the template useless to anybody, I think it would be best to concentrate on getting a useful, working version out there without this feature for the first launch. We can then circle back and attack this in phase two. Mathglot (talk) 09:33, 16 January 2022 (UTC)[reply]

Sounds good. Hwy43 (talk) 06:22, 17 January 2022 (UTC)[reply]

Development and testing, and live vs. draft

Are you familiar with the use of Template sandboxes, and the testcases page used for testing templates? I noticed because you had a run of 37 edits over a short period to the live template {{Municipalities of Alberta}}, some of them reverted, within a short period. This looks like normal development to me, but normally when a template is live, we don't make changes to the live template; rather, we copy the last known stable, working version to the sandbox subpage. Then, changes to the sandbox are tested using test cases you add to the testcases subpage. (I recommend using the 'create' links at the bottom of Template:Municipalities of Alberta, as it will start you off with some useful content.)

Practically speaking, since you're the creator of the template and it's not yet being transcluded by any article, it doesn't really matter, and in theory you could just carry on editing the live template as before. But I recommend using the sandbox + testcases pages anyway: for one thing, it will get you into the right habits, and for another, the page history of the template will be much cleaner and easier to read, if there are only working changes added to it, copied over from the sandbox after you've tested them. If you've never done this and need help, I can kickstart the process for you by creating a sandbox and basic testcases page. Certainly when it comes time to do the conditional columns, the sandbox and testcases pages should be used.

The other issue with live vs. draft, is that there are a large number of abandoned live templates in Template space, and there's a user group very recently created, that is scouring the Template namespace for unused templates, and recommending them for deletion. That won't happen, if your template is in Draft space, or is a user subpage. You mentioned above that the R&R is over for now, so your participation may trail off here; that makes it more likely it will get picked up for deletion. You could try adding {{in progress}} or {{under construction}}, but I'm not sure if that will help. Anyway, that's something else to consider also with respect to where you want to leave the template, if you decide to not work on it for a while. Ping me when you start up again, or if you have questions in the interim. Mathglot (talk) 22:06, 10 January 2022 (UTC)[reply]

Thanks. I have moved it to Draft space and have attempted to tag the redirect for deletion. Hwy43 (talk) 03:13, 11 January 2022 (UTC)[reply]

Great. I've created the sandbox and the testcases pages, and will look at the summary row issue there. Mathglot (talk) 04:35, 11 January 2022 (UTC)[reply]

I believe the summary row issue is resolved, at least for the 8-column format, although it would need to be updated to apply to your more recent table format. Mathglot (talk) 10:33, 11 January 2022 (UTC)[reply]

@Mathglot:, thanks. Largely resolved. Just some top-justified issues for two cells - one in the "Total cities" row and one in the "Total towns" row. See my replies midway through #Development approach above for further detail. Hwy43 (talk) 04:53, 12 January 2022 (UTC)[reply]

Thirteen-column format

@Hwy43:, For the new, 13-column format, I created new header and footer templates Draft:cpmh13 and Draft:cpmf13 that work with it, and a new row template Draft:cpmr13. I updated the sandbox, starting with a copy of your last revision (1064957210‎ of 02:27, 11 Jan.) ‎of the main template, with two changes: one change to use the new header/footer/row templates, and the other, to allow short parameter aliases ('t' in addition to 'towns', and also 'c', 'v', and 's') so you can use either the long form, or the one-letter abbreviation and it works the same either way. (I also alphabetized the rows by municipality name.) Haven't moved it over yet, in case you'd like to look at it; you can try this:

{{Draft:Municipalities of Alberta/sandbox|c=yes|s=yes}}

which generates the following:

Output of {{Municipalities of Alberta/sandbox|c=yes|s=yes}}

{{Draft:Municipalities of Alberta/sandbox|c=yes|s=yes}}

Example moved to /examples#Thirteen-column format due to performance consdierations

I'm thinking about whether the conditional row stuff belongs in the row template (in which case we'd have to pass the values of cities, towns, villages, and summervillages to it), or keep it as is. Arguing in favor of moving it, is isolating all the complexity in the row template, and keeping the main template, destined to have 300+ rows, as simple as possible. Mathglot (talk) 02:34, 15 January 2022 (UTC)[reply]

@Mathglot:, looks very slick! I recall you asking for the diff that fixed the first instance of top-justification from before the template was split into multiple templates. Here it is: [1]. Cheers, Hwy43 (talk) 06:20, 15 January 2022 (UTC)[reply]

@Hwy43: I think that problem may have gone away with recent changes for other things. The sandbox has been upgraded again; it now uses a fixed format "if" helper template to do the conditional row generation. This was done so the template table creation can be done by regex expansion without any programming intelligence, which wasn't true up till now because of the difference between the status type, and the parameter names. Can you point me to the data source for the census figures you've been using to create the table? I want to try to blow the sandbox up to 300 lines transcluding the new row template, and see what happens. There may be some special cases not currently handled by the code that works fine for the first ten rows that you have in the sandbox that we might need to fix up. We're getting close, I think. (Not counting dynamic columns; but I think we can release the table to Template space before that happens, once any glitches are fixed.) Mathglot (talk) 10:11, 15 January 2022 (UTC)[reply]

I found this table from Statistics Canada which has all the data from 7 of the 8 columns of your 8-column format, except for the city/town/village status,. I found the status info here, but not in a useful format. I don't see one page where you can find all the data for the 13-column format, including status, neighbour, and council number and size; is there one downloadable page that has all of this information or did you mash up several pages to create your 13-col format? Mathglot (talk) 19:40, 15 January 2022 (UTC)[reply]

@Mathglot:, here is the proper table from StatCan that includes municipal status. I can pull together the content for the other columns relatively quickly. How about I code all the lines and add them? Hwy43 (talk) 22:11, 15 January 2022 (UTC)[reply]

@Hwy43:, sounds fine to me. Can you add them to the Sandbox first, following the pattern there? One minor point: when people come to the Template page itself in the future, we don't want to make them scroll past a 300+ line table, before they get to the template instructions (/doc) part of the page, so I'd show only 10 or 20 table rows on the Template page, and all rows only when transcluding the template. I can do that once you're done, if you want. Looking forward to seeing this released! Mathglot (talk) 22:18, 15 January 2022 (UTC)[reply]

@Mathglot:, I have a lot of your comments to respond to. Let me respond paragraph-by-paragraph as it will be unwieldy otherwise. Though I will ping you with every reply, perhaps hold off until I am done. The edit summary of my final reply will indicate I am done. Re: this paragraph, agreed. Happy for you to do this prior to deployment. Hwy43 (talk) 07:33, 16 January 2022 (UTC)[reply]

I can see you've got the cities in there, good job! I think the default sort should be by municipality name (column one), don't you agree? If you're going to do the other muni types in stages, do you plan to alphabetize the list at the end? I can do it if it's a problem; lmk. Mathglot (talk) 03:21, 16 January 2022 (UTC)[reply]

@Mathglot:, thanks. Also agree. For the testing right now, however, I prefer by status first. Let's change default sort at the end. Hwy43 (talk) 07:37, 16 January 2022 (UTC)[reply]

I see you're all done, cool! Looks good, with some anomalies noted below, but they look pretty easy to handle in the row template. Great job! Oh, and no need to manually fix up any of the errors in the table, I need them in order to be able to test adjustments to the row template to handle them, and also, the table format is going to change anyway, as a result of the Conditional row thing (but that shouldn't affect you). Don't forget to read section #Conditional row generation below, also. Mathglot (talk) 07:29, 16 January 2022 (UTC)[reply]

@Mathglot:, thanks. I have a checklist of anomalies in my sandbox that I will add to the below. Hwy43 (talk) 07:39, 16 January 2022 (UTC)[reply]

Data sources

@Hwy43:, I'm confused about where the data is coming from. Above, you listed this table ("Population and Dwelling Count Highlight Tables, 2016 Census") and I see some of the data columns there, namely our columns 1 & 2 (name & status), and the last five columns (pop. 2016, 2011, change %, area, density) but not the others (region, CD, neighbour, inc date, council size, muni census, muni census year). Where are you getting the data in columns 3 - 8 from? Mathglot (talk) 18:24, 17 January 2022 (UTC)[reply]

@Mathglot: A variety of sources including StatCan, Alberta Municipal Affairs, etc., and sometimes the municipalities themselves (for municipal censuses starting in 2020). We are consolidating information from multiple sources into a single table. This further expanded StatCan table gives us CDs (notice nesting in the first column). This set of municipal profiles for Alberta's villages gives us neighbour, incorporation date, and council size. These (https://open.alberta.ca/publications/2368-7320 population lists] give us more recent population counts and census years. Hwy43 (talk) 01:13, 18 January 2022 (UTC)[reply]

The above are those sources that can be nested within column headers. However there are also in-row sources needed for things like 2016 census result amendments (affecting Bonnyville and Burnstick Lake for example) and post-2019 municipal census results (like for Morinville and Blackfalds). Hwy43 (talk) 01:49, 18 January 2022 (UTC)[reply]

The needs for in-row refs and notes can be observed by skimming the three columns of tables wikilinked in the #Goals section. Hwy43 (talk) 01:52, 18 January 2022 (UTC)[reply]

(edit conflict) Thanks. The timing of your message is pretty good; I just completed the initial steps to to switch the row template over to the new style, although it's not complete or fully tested yet. It works off a more streamlined sandbox2 file, and all four of the old, "foo-type=yes" params to control which municipalities are displayed are gone in this update, and replaced by the single param |disp=; you can see the description of it here. You can now use the |disp= param to select the rows you want, including urban or rural; full list of values at the link. I literally just finished this step, and the footer isn't updated yet; it always shows all rows currently. Mathglot (talk) 01:57, 18 January 2022 (UTC)[reply]

Mathglot Thanks for the latest updates. I have officially gone dark. Will be dark until Thursday evening. Once back in the saddle I will review the latest sandbox examples and reply fulsomely. In the meantime, I will continue to monitor your messages on this talk page, and maybe reply with very short and quick answers/comments, like the one I am about to issue about abbreviating “Council”. Stay tuned. Hwy43 (talk) 08:31, 18 January 2022 (UTC)[reply]

@Mathglot: the |disp= approach works. Would you like me to add references to the header template? Hwy43 (talk) 04:54, 20 January 2022 (UTC)[reply]

@Mathglot: here are the inline citations I have compiled for the column headers.

Inline citations code for columns in header template

Name: <ref name=municodes>{{AltaMC}}</ref>
Status: <ref name=municodes/>
Region: not applicable
CD: <ref name=2016StatCan/>
Rural neighbour (cities): <ref name=BaseMap>{{cite map | url=https://extranet.gov.ab.ca/srd/geodiscover/srd_pub/imageryBaseMapsEarthCover/ProvMapSeries/750K/750KMunicipalities.pdf | title=2021 Provincial Base Map: Municipalities | publisher=[[Alberta Environment and Parks]] | date=July 26, 2021 | accessdate=January 20, 2022}}</ref>
Rural neighbour (towns): <ref name=TownPro/>
Rural neighbour (villages): <ref name=VilPro/>
Rural neighbour (summer villages): <ref name=SumVilPro/>
Incorporated (cities): <ref name=CityPro>{{cite web | url=http://www.municipalaffairs.gov.ab.ca/cfml/MunicipalProfiles/basicReport/CITY.PDF | title=Municipal Profiles Summary Reports: Cities | publisher=[[Alberta Municipal Affairs]] | date=January 14, 2022 | accessdate=January 20, 2022}}</ref>
Incorporated (towns): <ref name=TownPro>{{cite web | url=http://www.municipalaffairs.gov.ab.ca/cfml/MunicipalProfiles/basicReport/TOWN.PDF | title=Municipal Profiles Summary Reports: Towns | publisher=[[Alberta Municipal Affairs]] | date=January 14, 2022 | accessdate=January 20, 2022}}</ref>
Incorporated (villages): <ref name=VilPro>{{cite web | url=http://www.municipalaffairs.gov.ab.ca/cfml/MunicipalProfiles/basicReport/VILG.PDF | title=Municipal Profiles Summary Reports: Villages | publisher=[[Alberta Municipal Affairs]] | date=January 14, 2022 | accessdate=January 20, 2022}}</ref>
Incorporated (summer villages): <ref name=SumVilPro>{{cite web | url=http://www.municipalaffairs.gov.ab.ca/cfml/MunicipalProfiles/basicReport/SVIL.PDF | title=Municipal Profiles Summary Reports: Summer Villages | publisher=[[Alberta Municipal Affairs]] | date=January 14, 2022 | accessdate=January 20, 2022}}</ref>
Cllrs (cities): <ref name=CityPro/>
Cllrs (towns): <ref name=TownPro/>
Cllrs (villages): <ref name=VilPro/>
Cllrs (summer villages): <ref name=SumVilPro/>
Muni. census pop.: <ref name=2019MAPL>{{cite web | url=https://open.alberta.ca/dataset/daab9fce-c2f6-49d1-a433-375b2b7aee24/resource/61cd908d-e2b9-4837-939b-533848d723b9/download/2019_mapl_web.pdf | title=2019 Municipal Affairs Population List | publisher=[[Alberta Municipal Affairs]] | isbn=978-1-4601-4623-1 | date=December 2019 | accessdate=January 20, 2022}}</ref>
2016 Census of Population: <ref name=2016StatCan>{{cite web | url=https://www12.statcan.gc.ca/census-recensement/2016/dp-pd/hlt-fst/pd-pl/Table.cfm?Lang=Eng&T=304&SR=1&S=87&O=A&RPP=20&PR=48 | title=Population and dwelling counts, for Canada, provinces and territories, census divisions and census subdivisions (municipalities), 2016 and 2011 censuses – 100% data (Alberta) | publisher=[[Statistics Canada]] | date=February 7, 2018 | accessdate=January 20, 2022}}</ref>

Note that for three columns there are unique citations depending on municipal status type. They will have to be coded so that only those display depending on the |disp= parameter used. Want me to add the above to the header template? If so, please code in the conditions associated with one of the 12 type-dependent citations. I will observe what you did in the diff and then apply the same approach to the other 11 citations as part of my learn-by-observation approach to template-building. Hwy43 (talk) 09:21, 20 January 2022 (UTC)[reply]

@Hwy43:, just noticed this, as I've been off on some other things for a bit. As the header and footer templates are one-offs, that is, they aren't transcluded 300 times by the main template, I'm not overly concerned whether the refs get included in-line, in the "normal" way, or using the LDR method I indicated with Arrowwood. I think either is okay, as the header and footer templates are fairly simply, and if get "ugly" and cluttered with refs, it's not so serious. Otoh, if you want to go with LDRs there as well, that will certainly work, and will make it easier to see what's going on in the header and footer.

Speaking of the footer: have you considered putting all the refs in the footer, instead of the header? There's no special reason they have to come first (i.e., top of column) and if you think about refs in articles, they come after what they verify. No reason we can't do the same thing here, and include them in the footer. I see two advantages: it keeps the header less obtrusive, and not as tall, and secondly, the footer already has conditional code for emitting certain rows depending on |disp=, so if you stick the refs in the right footer row, they will come out conditionally on disp. I think either way could work, and you can even try it both ways (even at the same time) and see which one you like better. Unlike sandbox2 and the row template, I don't mind if you mess around with the header and footer, to try stuff out. I'll probably get back to finishing up the changes to the neighbour column tomorrow. Mathglot (talk) 08:01, 25 January 2022 (UTC)[reply]

Mathglot I am back after an inactive period due to work. The LDR method is interesting. I would prefer to leave the refs inline within the header for now. In the footer is something that can be considered after the template is live. Speaking of, let me know if there is something I can do to help move things forward until some of the others things settle for you. Hwy43 (talk) 04:19, 30 January 2022 (UTC)[reply]

@Hwy43:, wb! Sure, no worries. That's pretty much it, I just need to find the time to fix two of the subtemplates for the neighbour cell and reintegrate them into the row template. Mathglot (talk) 04:23, 30 January 2022 (UTC)[reply]

Okay, the neighbour cell stuff is done now, I think, but hasn't been fully tested. See the neighbour section, below. Mathglot (talk) 09:37, 31 January 2022 (UTC)[reply]

Scrolling

We're getting ahead of ourselves again here, but at some point, you'll want to think about the issue of table length, and scrolling. A 300-line table is pretty annoying in the middle of an article. Fortunately, there are various solutions, including a fixed-length scroll window, in which you can scroll up and down to see a fixed number of rows, although not the whole table at once. (There's an 'expand' link that users can click to get the whole table.) See Help:Table#Scrolling for more on this. Mathglot (talk) 10:23, 15 January 2022 (UTC)[reply]

@Mathglot:, sounds good. Something to address near the end. Hwy43 (talk) 07:41, 16 January 2022 (UTC)[reply]

Naming

We are using Draft space for development, which is a good thing, but you might want to think about "permanent", or at least, initial names for things. Here is an initial proposal:

current	proposed name (template space)	possible redirects (template space)
Draft:Municipalities of Alberta	{{Municipalities of Alberta}}	{{Alberta municipalities}}
Draft:Cpmh13	{{Muni AB header}}	{{MuniABh}}
Draft:Cpmf13	{{Muni AB footer}}	{{MuniABf}}
Draft:Cpmr13	{{Muni AB row}}	{{MuniABr}}
Draft:Cpmrowif	{{Muni AB row if}}	{{MuniABrif}}

I put the name you suggested in row 4 column 2 ({{Muni AB row}}), and made up some similar names to fill out the table, along with some possible redirects, but you can choose whatever names make sense to you. Only the middle column needs to be decided before it goes live; the third column can wait, and you don't have to have redirects if you don't want them. Mathglot (talk) 04:04, 16 January 2022 (UTC)[reply]

@Mathglot:, as of right now, the above names and redirects look good. Hwy43 (talk) 07:43, 16 January 2022 (UTC)[reply]

Hwy43, Adding the following expanded table showing the templates and other pages required for release:

current	subtplt	proposed name (template space)	possible redirects (template space)	remarks
Draft:Municipalities of Alberta		{{Municipalities of Alberta}}	{{Alberta municipalities}} {{Alta munis}} {{AB munis}}	main template
	*/sandbox		(no move)	rev. 1065981462‎ or earlier is obsolete; sandbox2 may move here.
	*/sandbox2		(recreate post-launch)	current testing version; does not require release, as sandbox can be recreated post-release.
	*/testcases	{{Municipalities of Alberta/testcases}}		test cases for the main template
	*/doc	/doc		doc page for the main template
Draft talk:Municipalities of Alberta		Template talk:Municipalities of Alberta		Template talk page
Draft talk:Municipalities of Alberta	*/examples	*/examples		Examples off-loaded from the talk page for performance reasons
Draft:Comma flip2		*/comma flip	{{comma flip}}	For an inverted string with a comma, flip the order and pipe the original (w/o brackets). Possibly of general use as a string template, but start it as a subtemplate.
Draft:Cpm dab2		*/dab expand	{{MuniABdab}} {{CA dab}}	expands locations that would link to a disambig page if not adjusted; could easily be updated to handle all provinces
Draft:Cpmc13		*/cell	{{MuniABcell}} {{MAB cell}}	supplies the value for one cell of a row when special handling is needed
Draft:Cpmf13b		*/footer	{{MuniABf}} {{MAB f}}	generates footer rows at the bottom of the table
Draft:Cpmh13		*/header	{{MuniABh}} {{MAB h}}	generates the column header row at the top of the table
Draft:Cpm nb cell		*/rn cell	{{MuniABrn}} {{MAB rn}}	links up to 4 slash-separated rural neighbour items
Draft:Cpm nb item		*/rn item	{{MuniABrnitem}} {{MAB rn itm}}	adjusts one slash-separated rural neighbour item and adjusts for commas and abbreviations
Draft:Cpmr13b		*/row	{{MuniABrow}} {{MAB row}}	generates one row of the table
Draft:Cpmr13b	*/doc	*row/doc		doc page (Note: H:CEGWR err in doc)
Draft:Cpmrefs		*/refs	{{MuniABrefs}} {{MAB refs}}	contains list-defined references for the AB municipalities table
Draft:Cpmrowifb		*/row if	{{MuniABrowif}} {{MAB rowif}}	decides if a conditional row should be emitted for this municipality type
Draft:Cpmrowifb	*/doc	*/row if/doc		doc page
Draft:MD expand		*/abbrs	{{MuniABabbr}} {{MAB abbr}}	expand abbreviations such as "M.D."
Draft:Muni AB references				this is a development worksheet, not for release
Draft:Nth slash		{{str Nth slash}}	{{Nth slash}}	Return Nth expression split by slash, or 1st item. General use string handling routine, should reside directly in Template space.

The table also serves as a checklist during the release process, because all the red links should go blue if it's been properly done. Mathglot (talk) 00:22, 2 February 2022 (UTC)[reply]

Thanks. I have added a couple shortcuts to the main template for now. Will review and potentially add more later. Hwy43 (talk) 04:40, 2 February 2022 (UTC)[reply]

Updated to add checkbox column so I can use it to keep track of what's been moved and code-adjusted. Testing is tracked in testcases (t.b.d.), not here. Mathglot (talk) 07:14, 5 February 2022 (UTC)[reply]

Conditional row generation

Adopted; follow-up at #Selective row display – switchover to multi-value display param.

@Hwy43: There's a fundamental design decision that needs to be made very soon about how the conditional row generation is handled. Up till now, the row conditionality is based on your initial parameter suggestions of |cities=yes, |towns=yes, |villages=yes, and |summervillages=yes. This has advantages and disadvantages. On the plus side, you can have tables showing just "cities" and "towns", as in some of the examples. On the other hand, it makes the code lengthier and more complex in places due to the fact of having to manage four parameters for the four municipality types, to the point where we had to add a special template Draft:Cpmrowif to do the somewhat complex checking for whether the row should be generated or not. The current situation is not too too bad, as long as there are only four "types" (statuses), but it's not ideal due to having to have parser code in the main template.

When I was looking at StatCan, I noticed many other municipality types beside just those four, including two kinds of Indian reservation or settlement, various types of district, and some others. This could easily boost the number of "types" to eight or more, and the conditional statement for the row test would be possible, but quite long and complex; longer than the template to generate the row. In reviewing the #Goals subsection above, I noticed links which imply additional types: "specialized municipalities", "rural municipalities", "Metis settlements", "municipal districts", "Improvement districts", and "Special areas", some of which I remember seeing at the StatCan page.

The crucial question is: is the ability to have more than one "type" of municipality in the same table, like "towns" and "cities", important, or not? Does anyone really want a table that just shows "cities", "summer villages", "Indian reservations" and "Metis settlements" and not the rest? Probably not, I'm guessing. If we can limit the template to showing only one "type" per transclusion, i.e., only "cities", or only "towns" or only "summer villages" (or any of the other types) it will greatly simplify everything.

It seems like all the links in your #Goals section include tables with just one "type". If that is how this table-template is to be used, then instead of having the four template parameters |cities=yes, |towns=yes, |villages=yes, and |summervillages=yes, plus the possibility of having to add another eight or ten of them, we should just have one parameter to replace the existing four, called "type" (or "status" if you prefer, so that instead of |cities=yes, you'd have |type=city, and instead of |summer villages=yes, you'd have |type=summer villages. This would eliminate the existing, long conditional above each row in the Draft table, and permit a much cleaner design, where the new |type= param would be passed to the row template, which would itself decide whether to emit a row or emit nothing; as a result, the main template would have only multiple transclusions of the row template, and no parser code at all. The result would be a Main template of 300+ rows, plus one for the header, and one for the footer; very clean and simple and easy to understand. (As a fringe benefit, it would simplify your regex for generating the table, even if it's once per five years, others could more easily do it, or modify it if need be.) Having one "type" param, instead of the four or more, is also advantageous, because you don't have to know in advance what the particular municipality types of a province are; having just "type" means you can switch the template to Ontario, and it will automatically work, even if the municipality types there have different names than in Alberta.

I suggest we make this change. It will involve a change to the row template (to interpret the new parameter), to the main template (to drop the four existing parameters, add the new one to each row transclusion, and drop all the parser code, as wellas changes to the two doc pages involved. We could do it before you move on to adding towns, villages, and so on to the current sandbox, unless you prefer to finish adding them using your current regex. Please lmk which you prefer. It's important that you understand what is involved here, so if there's anything that's not clear, please let me know so we can clear it up. But we should figure this out and handle it one way or the other, ASAP. Mathglot (talk) 04:49, 16 January 2022 (UTC)[reply]

@Mathglot:, we are going to need nine "types" (statuses) for Alberta. To date we have only focused on the four that are urban in nature. Hwy43 (talk) 07:45, 16 January 2022 (UTC)[reply]

@Mathglot:, there are two links within #Goals that require more than one "status". First at List of municipalities in Alberta#List of urban municipalities, we will need |cities=yes (or |c=yes), |towns=yes (or |t=yes), |villages=yes (or |v=yes), and |summervillages=yes (or |sv=yes), which represent Alberta's four "urban municipality" statuses. Ideally, a |type=urban would trigger a yes for all four urban municipality statuses.

Second at List of municipalities in Alberta#List of rural municipalities, we will need |municipaldistricts=yes (or |md=yes), |improvementdistricts=yes (or |id=yes), and |specialareas=yes (or |sa=yes), which represent Alberta's three "rural municipality" statuses. Ideally, a |type=rural would trigger a yes for all three rural municipality statuses. Hwy43 (talk) 08:03, 16 January 2022 (UTC)[reply]

@Mathglot:, what are your thoughts on this in light of my previous replies above with new information? Hwy43 (talk) 08:08, 16 January 2022 (UTC)[reply]

@Hwy43:, yes, it could be done relatively easily with |type=urban, |type=rural; also, any other "multiples" determined later, could be added to the row template relatively painlessly, without touching the main template with the 300 rows. I will look at this tomorrow. the row template could also resolve all of the anomalies I identified at #Anomalies, bugs, etc.. So, the design of isolating all the complexity in the row template is turning out to be a big win. Mathglot (talk) 08:18, 16 January 2022 (UTC)[reply]

This approach was adopted; follow up at #Selective row display – switchover to multi-value display param. Mathglot (talk) 19:40, 5 February 2022 (UTC)[reply]

Anomalies, bugs, etc.

General

Here are my general observations and emerging needs.

In the header template, remove the default table title and replace with a |tabletitle= parameter so that we can customize the table title to suit the parameter(s) that are selected (e.g. if we only enable cities we should title the table as "List of cities in Alberta", not "List of municipalities in Alberta").
The "Province of Alberta" row in footer shouldn't sort.
In the "Region" column, pipe Northern Alberta to Northern, Central Alberta to Central, Southern Alberta to Southern, and Alberta's Rockies to Rockies.
Add note and ref tags such as {{#tag:ref|Sample note...|group=lower-alpha}} and <ref>{{cite web | ... }}</ref> for a certain amount or all cells within each row (to be confirmed).

This list will likely grow. Hwy43 (talk) 08:26, 16 January 2022 (UTC)[reply]

#1 table title – it is already in there, it's currently called |caption=. It works together with |prov=; see the doc at Draft:Cpmh13#Parameters, and that section of the doc needs to be copied over to the doc page of the main template as well.

#2 non-sortable footer rows – not looking at sorting just now; that's something relatively easy to fix whenever.

#3 Region piping – that makes sense; we can special handle that.

#4 notes/refs – Some tables, rather than stick refs or tags anywhere, stick then all in one, dedicated column at the end of the table (which can have the column header, "Notes/refs"). This is generally cleaner, less distracting, keeps the other cells narrow(er), and has other benefits. To prevent the table code itself from getting ugly and fat, I'd recommend looking into WP:List-defined references, with all refs defined below the main table with brief refnames, and just the named refs in the main table. But we can look again later, after there's more clarification about exactly what is needed. Mathglot (talk) 08:56, 16 January 2022 (UTC)[reply]

#4 @Mathglot: The majority of refs will be placed in the column headings, but there is always a need for municipality-specific notes and refs. I want to avoid a notes/refs column in which the vast majority of its cells are empty. The row template calls on {{Change}} to generate the values in columns 9-11. That template has |suf1= and |suf2= to allow for notes/refs for columns 9 and 10. I would like the row template to allow for the same parameters to feed {{Change}} when necessary. Hwy43 (talk) 06:36, 17 January 2022 (UTC)[reply]

@Hwy43: added |pop2-ref= and |pop1-ref= for this. See the doc at Draft:Cpmr13b#Named for details, and see row "Arrowwood" in Draft:Municipalities of Alberta/sandbox2 to see how to use it. This is not yet fully tested, but those examples appear to be working. As sandbox2 is my main development and test bed now, please avoid making changes to Sandbox2, until after it's merged back to Sandbox (probably tomorrow). You can always edit Sandbox2 to run your tests in Preview mode without saving; or if you need to test more extensively, please take a copy of Sandbox2. Thanks, Mathglot (talk) 21:47, 21 January 2022 (UTC)[reply]

@Mathglot: that does the trick. Well done. I recall there being the need additional refs in any given row depending on column. At some point, I will re-review what I wrote previously and double check all existing tables that will be replaced. Hwy43 (talk) 02:08, 22 January 2022 (UTC)[reply]

@Mathglot: I have done my re-review and determined which column cells need optional ref parameters in the row template. See #References and sourcing. Hwy43 (talk) 05:22, 30 January 2022 (UTC)[reply]

Neighbour red and orange links

At this point, with the sandbox containing the four muni types converted, numerous municipalities have red links in the "neighbour" column. This occurs for various reasons, and all could be handled by using an override parameter added by a human editor after the automatic generation of the table, or the existing |nb= parameter could handle it, with some intelligent parsing added to it, looking at commas, slashes, and disambig links. Even if the row template can handle such things, the override could always be chosen on top of it, or for unique situations not handled by it.

not a bug? – numerous cities have red links in the neighbour field. If this is because the "neighbour" is properly formatted and the article does not exist yet, then the red link is proper, and it's not a bug. Maybe a redirect or an article needs to be created to resolve it.
comma suffix – some towns have neighbors listed in inverted order with a comma, such as Bassano whose neighbour is listed as "Newell, County of". An en-wiki article, if it existed, would likely not be named like that. Similarly, Beaverlodge has "Grande Prairie No. 1, County of"; Taber's neighbour is "Taber, M.D. of", and so on. These seem to link properly if inverted, e.g., 'St. Paul No. 19, County of' ⟶ County of St. Paul No. 19, and if that works most/all of the time, the row template could be beefed up to handle that, or, the |nb= override could do it with additional post-processing by a human editor.
parenthetical muni name – some neighbours are red, because they have a parenthetical part containing the muni name, and the article exists on en-wiki, but without the parenthetical; e.g., Banff's neighbour is listed as "Improvement District No. 9 (Banff)", which is red, whereas Improvement District No. 9 exists. This type of problem should be fixed by just adding a redirect to it from the longer version with the unnecessary parenthetical.
multiple neighbours – some munis have red links, because more than one locality is listed, e.g., for "Calgary" we have "Foothills County / Rocky View County" which is red, because that is two locations, not one; it should be linked as two links: Foothills County / Rocky View County instead. Or Drumheller, which links three locations as one. All the ones I've seen use slash as a divider, and if it is systematically used that way, the row template could be smartened up to detect multiple locations separated by slash, and link them properly.
disambig page – some are not red links, but link incorrectly to a disambig page, such as Strathmore, whose neighbour is linked as Wheatland County, but that is a disambig page. It should link to Wheatland County, Alberta instead, and coded as [[Wheatland County, Alberta|Wheatland County]]. Or, Viking's neighbour, Beaver County, which should link to Beaver County, Alberta, coded as [[Beaver County, Alberta|Beaver County]]. Ditto Caroline (Clearwater County), Rockyford, Ryley, and Standard.

That's all I've seen for now, with cities, towns, villages, and summervillages now converted into the sandbox template. Mathglot (talk) 07:53, 16 January 2022 (UTC)[reply]

Here are my bugs relating to this topic. Some of these overlap with yours above.

In the "Name" column, need to add that |comm-link= parameter as mentioned above at #Row template to overcome redirects, dabs, suffixes, etc. (I created a Lloydminster (Part), Alberta redirect to Lloydminster in the meantime to avoid a redlink).
In the "Neighbouring municipality" column, need to add a |neigh-muni-link= parameter similar to |comm-link= to overcome redirects, etc. and enable piping to achieve proper alpha sorting (e.g. Grande Prairie No. 1, County of instead of County of Grande Prairie No. 1) and abbreviations (e.g. Bonnyville No. 87, MD of instead of Municipal District of Bonnyville No. 87).
Also in the "Neighbouring municipality" column, we could have up to four entries in which each should be separated by a <br>. Right now they are separated by commas on a single line.
In same column, there are at least three possible entries that require disambiguation – Beaver County, Alberta, Clearwater County, Alberta, and Wheatland County, Alberta.

I will stop here for now. Hwy43 (talk) 08:26, 16 January 2022 (UTC)[reply]

Most of these can be solved by intelligent parsing in the row template, and won't need new parameters, or human post-processing to fix links. Adding redirects is sometimes a good idea, as in the case where you did. In particular, we won't need |neigh-muni-link= as all of the cases I've seen in the long table (inverted comma case, slash-multiples case) can be resolved in the row template. The disambig cases probably will mostly have to be added by human post-processing: i.e., just retouch the |nb= value to add the proper target. If there are too many of them, we can look into other approaches. Mathglot (talk) 09:04, 16 January 2022 (UTC)[reply]

@Hwy43: I noticed some tweaks you made to the Sandbox since the table was loaded from the PDFs (here and here), but these are unnecessary now, for two reasons: the switchover to the new display param design, and once that's done, adjustments to handle the anomalies listed above. So, any changes you make now to the sandbox are likely to be overwritten at some point, so until the new stable version comes out, there's probably no point to changing the sandbox. That doesn't mean you absolutely shouldn't change it—if you want to test things, see what happens, go right ahead; just be aware that any change to the sandbox after rev. 1065979331‎ of 07:01, 16 January 2022 will likely get overwritten more or less without warning when the new design is ready, or during testing of it. Switchover development plan coming soon... Mathglot (talk) 01:59, 17 January 2022 (UTC)[reply]

@Mathglot: sounds good. The tweaks were made only to apply a consistent format across all anomalies in the same column. It was driving me nuts that I was inconsistent. Hwy43 (talk) 06:41, 17 January 2022 (UTC)[reply]

@Hwy43:, no worries. In fact, you can consider the sandbox your personal sandbox for now, as I might shift to sandbox2 for the switchover, so I can see working examples the old way, vs. the new way, in case we want to compare them. I should be back shortly with a sketch of the plan. After that, I'll be out for a couple of days, most likely. Mathglot (talk) 06:48, 17 January 2022 (UTC)[reply]

I did have a couple of questions: the slashes between multiple places in "neighbor", is that in the original StatsCan pdf, or is your Excel op or regex adding that? Same question for the "Foo, County of" (or M.D.) with the inverted order and the comma--StatCan, or your regex? Thanks, Mathglot (talk) 06:50, 17 January 2022 (UTC)[reply]

(edit conflict) No longer seeing any slashes in that column. I switched them over to “, ” and “ and ” but am happy to return to simplify with just slashes throughout. Anyway, it is not yet in my Excel regex but can be easily added into it. As for the inverted ordering, that is a fed into the regex using a separate column that I maintain. Hwy43 (talk) 07:24, 17 January 2022 (UTC)[reply]

The neighbour column involves more complex parsing than any of the others, so it's important to decide on a stable format and stick with it. I currently have several helper templates just to deal with that column: one for disambig pages, one for dealing with the "M.D." abbreviation, one for multiple neighbors separated by slashes, and one to bring them all together to generate the proper link(s) for the cell. If you need to improve or change the format, that's fine, but please let me know in advance so we can talk about your needs.

Currently in testing, is a version that handles up to four neighbours per cell separated by slashes (e,g., "N one / N two / N three / N four"; is four enough? It's easy to increase it, but I haven't seen a location that requires more than four so far). Each cell may have abbreviations for "Municipal District" (currently handling both "MD" and "M.D."—is this from the data tables and therefore required, or it your Excel generating two different ones and it could be narrowed down to just one?); and handling neighbour strings that have a maximum of one comma (e.g., 'Willow Creek No. 26, Municipal District of' is good, but 'Willow Creek No. 26, Alberta, Municipal District of' is not). Does this agree with the format you are using, and can we keep this format? If not, let's discuss further. Mathglot (talk) 22:07, 21 January 2022 (UTC)[reply]

@Mathglot: four neighbours per cell is the maximum. When we get to specialized municipalities, the three types of rural municipalities, and Metis settlements, we will have blank entries and the column will be hidden. As for municipal district abbreviations, let’s only go with "MD". I will fix my Excel file accordingly to remove "M.D." For the double commas, I can create undisambiguated redirect that remove ", Alberta" to make sure we don’t have two commas. All instances will be primary topic. This may not be the case when we create equivalents for other provinces in Canada. Hwy43 (talk) 02:20, 22 January 2022 (UTC)[reply]

@Hwy43: thanks, that all sounds good. Don't worry too much about the MD/M.D. thing for now, the template already deals with both (and it's easy), so no need to change the Excel for that at this point; but going forward, yes, it would be good to have a consistent rule about it, so going with "just 'MD' " will make it easier for others who may need to fiddle with the templates some day. As far as other provinces, don't worry about that at all, if they have different rules, that's fine, as long as each province is internally consistent with how they handle things. Each province might end up having have their own subtemplates to deal with the particularities of their situation, but it will be much easier to do that once Alberta is done and they can use it as a jumping off point. Thanks for the response, it all sounds good, so I can carry on now in confidence. Mathglot (talk) 03:09, 22 January 2022 (UTC)[reply]

@Hwy43: Update: there's a sticking point with the subtemplates of the row template in dealing with the neighbour field. It has nothing to do with your format choices for 'neighbour', which are all good as previously discussed; it's a technical issue having to do with parsing vertical bar in templates using Lua regular expressions (and the escape {{!}} does not work in that context). So, I'm going to have to redo a couple of them. It's just as well, because that will actually make things easier to follow, and maintain; but it does need doing, so that's what I'm up to in the next day or two. This is just fyi. In the meantime, if you find bugs unrelated to nb column, please add a subsection to #Anomalies, bugs, etc.. Thanks, Mathglot (talk) 00:54, 23 January 2022 (UTC)[reply]

@Hwy43: okay, rn (neighbour) now appears to be working:

{{Draft:Municipalities of Alberta/sandbox2|disp=u}}

Example moved to /examples#Neighbour due to performance consdierations

Please check the neighbour column for anything fishy looking. I'm interested in any other problems, too, but the only thing changed lately is |rn= handling. Please lmk if you find anything. Mathglot (talk) 07:50, 31 January 2022 (UTC)[reply]

P.S. The subtemplates currently handle either MD or M.D. in the neighbour column, so there shouldn't be any red links in that column in sandbox2 due to that. We can just leave it like this for phase I in the interest of getting this done, but we can simplify one of the subtemplates next time around, if your Excel only generates one of them. Actually, if we are going to generate just one, then maybe "M.D." is better than "MD", in order to keep the replacement code easier, otherwise the code will have to take care with towns with names like "Camden" (in some other province, maybe) and take care not to substitute the 'md' when it shouldn't. But again, please don't take any action on this now, let's just keep it on the list of stuff to do next time around. Mathglot (talk) 21:28, 31 January 2022 (UTC)[reply]

@Mathglot: everything seems to work. Prior to implementation of phase one, I will be consistently applying MD. Use of M.D. makes me shudder. Good pragmatism on "Camden". To avoid however, the trigger for the replacement code can be MD of instead of MD. Hwy43 (talk) 05:33, 1 February 2022 (UTC)[reply]

Missing pop. references

Bug: ~~The refs for pop1 and pop2 for Arrowwood are generating empty footnotes in the references section now. Not sure if this is related to the fix today for expansion of neighbour column or not.~~ Mathglot (talk) 09:29, 31 January 2022 (UTC)[reply]

Not a bug; test omitted the LDRs. Mathglot (talk) 10:52, 31 January 2022 (UTC)[reply]

Extra curlies emitted

Bug: currently emitting }}}}} }}</noinclude> at the end of the table. See /examples/bundled citations for an example. Mathglot (talk) 09:09, 4 February 2022 (UTC)[reply]

Haven't seen this lately, and not sure if I inadvertently fixed it while working on something else, or what. I'll leave this open for a while, and if it doesn't turn up again, we can close it. Mathglot (talk) 06:37, 5 February 2022 (UTC)[reply]

Province of Alberta row in footer

When sorting any of columns 2 through 7 ascending I see that the footer's "Province of Alberta" row sorts as the first row. This row should be docked to the footer and unsortable. Hwy43 (talk) 03:14, 5 February 2022 (UTC)[reply]

Thanks, I'll look into that. Mathglot (talk) 06:20, 5 February 2022 (UTC)[reply]

Done Mathglot (talk) 06:36, 5 February 2022 (UTC)[reply]

References and sourcing

Mathglot within the row template, there are cells for a maximum 13 columns. The following is a list of cells by column and whether they may need ref parameters to accommodate sources different than those refs published in the header (or footer).

Name – ref parameter needed and |suffix= can be used Done; unless you want to reserve that for text suffixes only in favour of a |loc-ref=
~~Status – ref parameter not required~~
~~Region – ref parameter not required~~
~~CD – ref parameter not required~~
Rural neighbour – |rn-ref= required
Incorporated – |inc-ref= required
~~Cllrs – ref parameter not required~~
Muni. census pop. – |mpop-ref= required
Pop. (2016) – |pop2-ref= required Done
Pop. (2011) – |pop1-ref= required Done
~~Change – ref parameter not required~~
Land area – |area-ref= required
Pop. density – |popd-ref= required; population density is generated using {{Pop density}}, so a ref parameter may first need to be added to that template

So in total, eight cells may need their own references of which there are already parameters for three of them. Hwy43 (talk) 05:22, 30 January 2022 (UTC)[reply]

@Hwy43: understood, thanks. Do you have some rough figures for approximately what percentage of each column would need refs? E.g., 10% of rn's, 50% of areas, and so on. Guesses are fine. Also, how will the number of refs per row break down, e.g., if we have 300 rows, will half of them have no refs, 75 of them one ref, 25 two refs, 10 three refs, and so on. Depending how this shakes out, i.e., if there are too many refs, it might be better to find another approach rather than one more parameter per column. Mathglot (talk) 05:31, 30 January 2022 (UTC)[reply]

@Mathglot: in the columns, no more than 10% for Name. No more than 5% for Incorporated and Muni. census pop. No more than 2% for Rural neighbour, Land area, and Pop. density. In rows, no more than 2% would would have two refs. One row would have up to five refs (thanks to Lloydminster being a border city split between two provinces). One row would have three refs. Hwy43 (talk) 06:05, 30 January 2022 (UTC)[reply]

Thanks, that helps to know that. Will take it up again tomorrow. Mathglot (talk) 06:16, 30 January 2022 (UTC)[reply]

@Hwy43: Do the references have systematic names and urls that the template could generate automatically using some pattern? E.g., if the reference url and title for the Red Deer and Spruce Grove muni censuses look like these examples:

Red Deer – statcan.ca/pop/Red_Deer.pdf, "Red Deer municipal census"
Spruce Grove – statcan.ca/pop/Spruce_Grove.pdf, "Spruce Grove municipal census"

and all (or most) of the other ones follow that pattern, then rather than have a parameter, let's just create them automatically, or with a shorter param value, such as |mpop-ref=yes. Ditto for the other fields, do the references follow some pattern we could use to generate them? Mathglot (talk) 08:03, 31 January 2022 (UTC)[reply]

@Mathglot: you asked about systematic names and urls for references. The answer is no. Nearly all of these references will be unique. Hwy43 (talk) 05:10, 1 February 2022 (UTC)[reply]

Thanks; understood. Mathglot (talk) 07:22, 1 February 2022 (UTC)[reply]

Gathering references

A separate issue (not to be confused with decisions about how or where in the table the references should be placed as discussed above) is the task of simply gathering them all. I've created Draft:Muni AB references as a plaintext list (not a template) where we can make a list of them. Please add references there, in whatever sort order makes sense. For references destined for use in the main table, I've created subsections so we can them subdivided by type (cities, towns, etc.). Within each type, let's use whatever order that would help make it easier to find them; most likely alphabetical by municipality, I would think (but perhaps if there are sources that turn up again and again for different municipalities, some other order might work better). Thanks. Mathglot (talk) 21:11, 31 January 2022 (UTC)[reply]

@Mathglot: so is Draft:Muni AB references dedicated only to in-row/table body refs? If inclusive of those in the header as well, then I don't fully understand. Within #Data sources above I have listed "Inline citations code for columns in header template". It is not intuitive to me yet how those have been translated to Draft:Muni AB references. At Draft:Muni AB references can I add a Header section before the Table body section in which I can list all refs by column title and further by status when necessary? I can then list the in-row refs within the subsections under the Table body section.

One thing we should be careful about is the 2021 census results are released on February 9. Many of the in-row references will change on that date, as well as some of the column header refs. Hwy43 (talk) 05:54, 1 February 2022 (UTC)[reply]

@Hwy43: The page Draft:Muni AB references isn’t well-defined, other than as a scratchpad or holding pen so if we want to go to one place to see all of the refs, that is the one page that will have them. In theory, we don’t even have to use it, I created it mostly as a scratchpad to manage complexity, so you’re welcome to add a header section or do whatever you think will help you manage referencing on the project in that page; that’s what it’s there for. The idea was, to use that page as a source for where the refs are handled programmatically. That is in template Draft:Cpmrefs, but that is brand new and still under development; it’s design may well change.

I replied to the other comments but only pinged here, so check around. If I missed one, please reping from there. Mathglot (talk) 07:38, 1 February 2022 (UTC)[reply]

@Mathglot: I just went nuts at Draft:Muni AB references. Through this exercise, I have learned the following:

Of the 256 rows of urban municipalities, 23 rows require one or more in-row/cell references;
Of the 23 rows, only one row (the Lloydminster row) requires more than one in-row/cell references (it requires five references, not the maximum four I mentioned earlier);
17 references are required in the "Name" column, which can be accommodated by the |suffix= parameter;
4 references are required in the "Pop. (2016)" column, which can be accommodated by the |pop2-ref= parameter;
1 reference is required in the "Pop. (2011)" column, which can be accommodated by the |pop1-ref= parameter;
4 references are required in the "Muni. census pop." column, which does not yet have an associated |mpop-ref= parameter; and
1 reference is required in the "Land area" column, which does not yet have an associated |area-ref= parameter.

Your call if you want to reconsider two of the requested additional in-row/cell refs and pull together |mpop-ref= and |area-ref= parameters before we officially release the first version. During the third release, when we add 87 rows for specialized/rural municipalities and Metis settlements, we may or may not discover a need for any further in-row/cell refs. Hwy43 (talk) 04:35, 2 February 2022 (UTC)[reply]

@Hwy43: Wow, great! But please don't go nuts, it will leave me stranded without a collaborator, lol... Thanks for your efforts on this. With the numbers and data you provided, I think there may be a much better way to handle references, than to overload the row with all sorts of parameters just for refs that get used fairly rarely, but will clutter up the page when they do (especially when there is more than one of them). Are you familiar with bundled citations? Here's an example of it for some imaginary Airdrie sourcing^[1]: note that you get one bracketed footnote number inline, which leads to a single footnote in the references section that lists several different sources, for whatever columns that need them. Bulleting is often used in bundled citations, because it makes a nice, clean listing that is easy to read.

This is fully approved as a sourcing method by the Manual of Style, and may work very well for this application. There is no need for every cell to have its own bracketed note to fulfill WP:Verifiability, and it would let us get rid of all of the "ref" parameters, except for one, in which we would bundle whatever citations were needed. Easier to code, keeps the columns a bit narrower, and easier (and less cumbersome) for the reader viewing the table, when it's not littered with bracketed numbers all over the table, and keeps the references section itself down to a reasonable number of total footnote numbers, without missing a single needed citation. (For an over-the-top example of bundled citations, see note [a] in the first sentence at Eric Zemmour; this example is almost comical, but it does illustrate the fact there's really no limit how many citations you can include with just one numbered (or lettered) footnote bundle.) It also means we don't have to keep creating new parameters, every time some new column that never needed a source before, now needs one for some municipality. Even browsing the reference section directly will be far easier. Win-win, I think.

I'll set up an example in sandbox2 tomorrow so you can better see what it would actually look like. Looks like Lloydminster would make a good guinea pig, unless you'd prefer I pick a different one. There's one final advantage I just thought of: you could create the ref param that goes in the table row programmatically with your Excel, because for the Lloydminster row, it would just be |ref={{R|Lloydminster}}. You'd still have to create the citation itself separately for the LDR section, but you've already got that now, or close to it, from your going-nuts episode. Mathglot (talk) 07:54, 2 February 2022 (UTC)[reply]

@Mathglot: so would the CITEBUNDLE approach only be used for the 23 rows that have one or more in-row/cell ref tags? Are you suggesting the same for the header as well? Lloydminster is definitely the best guinea pig for the in-row/cell test. If you want to run a test for the header for inspection as well, please do. Admittedly applying such to the header row is tougher for my mind to reconcile at the moment. One important caution about the in-row/cell ref tags though before you proceed. Nearly every ref tag is a note instead of a citation (though citations are within the notes themselves). So most CITEBUNDLEs will have to land within the {{Notelist}} template while only a handful will land within the {{Reflist}} template. Hwy43 (talk) 01:56, 3 February 2022 (UTC)[reply]

@Hwy43: bundled citations work equally well for both. It’s probably not worth doing for the header, but will really help for the main table. More later. Mathglot (talk)|

@Hwy43: ref stuff is in a transitional state, with bundled refs partly working for two cities (Lloydminster, Wetaskiwin), one town (Athabasca), one SV (Kapasiwin), and two villages (Bittern Lake, Ryley). I say 'partly', because most of those have named references and those aren't connected yet. You can see an example at Draft talk:Municipalities of Alberta/examples#Bundled citations (note: that page gets slower with each new example; might need to break it up into multiple example poages). This requires a new (not yet documented) row parameter |r= which takes the municipality name (minus embedded blanks) as its value; so, for example, |r=Athabasca or |r=BitternLake; you can see some examples in /sandbox2. (This also deprecates all the other "ref"-type parameters.) Next step is to connect them to the header or other refs, and then to fill out the rest of them. Mathglot (talk) 09:44, 3 February 2022 (UTC)[reply]

I think the examples page hit some threshold maximum, because I sometimes see about a dozen "Template: broken ref" warnings at the end of "Bundled citations" example, and it doesn't expand the table at all. If you see that when you try to view it, then edit the section and hit Preview; that will expand only that one section, and will work; then cancel out. I'll have to split the examples page up, it's getting too big, I think. Mathglot (talk) 09:49, 3 February 2022 (UTC)[reply]

@Mathglot: I have reviewed the citebundled notes at Draft talk:Municipalities of Alberta/examples/bundled citations#Example: urban municipalities with notes. Looks good. I'm not seeing the references at the end of each note yet. Perhaps those are what you mean by "named references [that] aren't connected yet." Regardless, I see how the |r= at Draft:Municipalities of Alberta/sandbox2 produces the citebundle ~~but I am not seeing where the content within each note is stored~~ and how it calls from Draft:Cpmrefs. Hwy43 (talk) 02:41, 5 February 2022 (UTC)[reply]

Yes, that's what I meant, getting the explanatory notes to either embed or link to references, named or not, and display it all properly. This is discussed at WP:Nesting footnotes. One combination of that even has a Phab ticket open about it, but can't find it right now. Mathglot (talk) 19:57, 5 February 2022 (UTC)[reply]

@Hwy43: Above, you mentioned that "One row would have three refs"; is that still the case? I don't see a locality with three refs at Draft:Muni AB references. Mathglot (talk) 19:57, 5 February 2022 (UTC)[reply]

@Mathglot: no longer the case. When collecting all refs on the scratchpad I was able to eliminate numerous in-row/cell refs thanks to updated column header refs. Hwy43 (talk) 04:33, 6 February 2022 (UTC)[reply]

Thanks! Mathglot (talk) 19:06, 8 February 2022 (UTC)[reply]

Sample refs

^
Sources for Airdrie:
- 2011 pop.: "Canada national census of 2011". 2016. Retrieved 1 February 2022.
- 2016 pop.: "Alberta provincial census of 2016". 2016. Retrieved 1 February 2022.
- 2019 pop.: "2019 Municipal Affairs Population List" (PDF). Alberta Municipal Affairs. December 2019. ISBN 978-1-4601-4623-1. Retrieved January 20, 2022.
- area: "Candian land municipal area database". 2020. Retrieved 1 February 2022.

Placement of notes and references

@Hwy43: I followed the link from your comment at #Incorporating the template post-launch and viewed the current table at List of cities in Alberta#List. I notice that the notes are displayed right after the table, and this makes a lot of sense to me. The way the template is designed, in particular the subtemplate Draft:cpmrefs, they can be placed anywhere, by transcluding that subtemplate where you want them. Up to now, I had presumed that this would be in the "References" section at the bottom of whatever article the table was to appear in; this requires the user to add one transclusion to the article for the table (e.g., in section #List at that article) and one more transclusion, to include "cpmrefs" in the #References section at the bottom of the article.

If we know in advance that the notes will always go right after the table in whatever article it appears in, then we can simplify things for the user so they only have to make one transclusion, namely the table itself. If that's the case, we can just add {{Draft:cpmrefs}} at the end of Template:Municipalities of Alberta, right after the transclusion of the subtemplate that generates the footer rows. If we're not sure, or if the notes might sometimes appear with the table, and sometimes in the #References section at the bottom of the page, then we should leave it as it is.

Either way, as a technical matter we should define a subgroup for the notes, so that if we resolve them at the end of the table, we want to show only the notes from the table, and not every note that happened to be in the article above the table. Currently the article your linked uses group "AB" so that the notes after the table are numbered "AB 1", "AB 2" and so on, and that's fine if you want to keep that style, or just the letter "N" (e.g., 'N 1', 'N 2', etc.). (In future phases, we could default to the two-letter province code if desired.) Mathglot (talk) 04:35, 6 February 2022 (UTC)[reply]

@Mathglot: there will be instances where we need notes to appear at the bottom of the article with the references as some articles have notes in prose to accompany those notes in the table. Hwy43 (talk) 05:21, 6 February 2022 (UTC)[reply]

Understood; so we won't merge in the refs subtemplate into the main template, but keep it separate, as it is now, so the editor transcluding it in an article can decide on the placement of the notes. Thanks, Mathglot (talk) 05:28, 6 February 2022 (UTC)[reply]

@Hwy43: I should have added, we'll keep them separately defined, i.e., in the refs subtemplate, as it is now, so the editor transcluding it in an article can decide on the placement of the notes. In order to accommodate this, I had to "detour" for the last couple of days to come up with a scheme to do this, which was just an extension to the functionality of the existing "cpmrefs" subtemplate so it could handle end-of-article footnotes (i.e., the "regular" way) or end-of-table footnotes (the way it's done at List of cities in Alberta#List). This resulted in a robust new version (currently at Draft:Cpmrefs2, but needs to be merged back to "cpmrefs") which allows the user to decide on placement of the footnotes. A corollary of this, is that I had to create user-assignable group names, so whether it's 'AB' or something else, is something the user can decide. This will require a concomitant change to the row template, to generate the expected named reference, instead of just automatically generating the a-b-c... sequence that comes out of using {{efn}} (that is, a reference with group name lower-alpha). I'll get to that today or tomorrow, and then I'll have an example for you showing the AB-series of footnotes right below the table (or whatever label you want to use). That still leaves us with the task of adding references within the notes, which are still hidden. Mathglot (talk) 01:58, 8 February 2022 (UTC)[reply]

I do like the two-letter provincial abbreviation suffix approach. Such approach is used for many at List of cities in Canada. Hwy43 (talk) 05:29, 6 February 2022 (UTC)[reply]

@Hwy43:, I noticed that it is, and there's no problem with that other than width, and I do think it makes sense at the article List of cities in Canada. But from the list of around 20 articles you linked here, all of them are about Alberta, and having the bracketed note say 'AB 7' doesn't really tell you anything you don't already know; after all, the "regular" footnotes at the bottom of the page are just meaningless numbers, and nothing is lost from their being so. Having the flexibility to choose the "reference group-name" (which is what the 'AB' prefix before the digits is called generically, in Wikipedia-speak), whether it be 'AB' or something else, would be a plus. As it happens, this was a freebie that came out of my "detour" over the last couple of days, to resolve the referencing issue I alluded to at #Incorporating the template post-launch. But it's your call, and all I really need to know, is what the default is, if the user doesn't pick a group name. (And it's better if they don't, because it's simpler.)

One final question: in all the places where the template will be used, about what percentage of them will the footnotes be displayed right after the table, and what percentage at the end of the article? If it's mostly at the end of the table, then we can simplify things for the user, by defaulting to that, so if they didn't do anything, they would automatically come out right after the table. We'd add a simple override param to the main template, like |refs-in-appendix=yes (okay, that's a crappy name, can't think of one right now) and if they pick "yes", then the refs come out mixed in with the others in the appendix, instead of at the end of the table. That's a more complex case so I'm hoping that the majority of the time, the notes are supposed to stick with the table; that will make it easy to invoke the main template and the references will be generated at the same time in the same place. Mathglot (talk) 01:58, 8 February 2022 (UTC)[reply]

@Mathglot: agree on abandoning the postal abbreviation approach for reasons stated. Alternatives might simply be 'TN' for 'table/tempalte notes' or 'LN' for list notes.

Right now, of the ~20 links at #Goals, only two of them have the notes following the table (so 10%). Habitually it seems my preference has been to embed the notes at the bottom of the article, but I am not adverse to considering more widespread implementation following the table. A slightly less crappy name might be |bottom-notes=yes.

I think I am now caught up to everything you've added to this subsection of the discussion. Now onward to elsewhere you've been chatty! Hwy43 (talk) 06:49, 3 March 2022 (UTC)[reply]

Nested LDR reference issue

I can see now why I was having issues earlier that caused me to postpone adding the embedded references into the notes and leave them hidden for now. It's an actual bug in the Wikimedia software where "list-defined references do not work correctly when references are nested" which is exactly what we are trying to do (see Phab:T22707), and is decribed at Wikipedia:Nesting footnotes#5. List-defined references. I'm looking for a workaround. A couple of possibilities spring to mind:

put them in separate refs in series (i.e., instead of {{refn|Airdrie: pop increased that year.<ref>Statcan 2016</ref>}} we would put them in series ((i.e., {{refn|Airdrie: pop increased that year.}}{{refn|Statcan 2016}}); this would create two superscript items in the header or row instead of one (e.g.[AB 1][AB 2] instead of just [AB 1]) with the possibility of reusing the second one if it is named.
write the embedded note out in plain text without the <ref> tags: {{refn|Airdrie: pop increased that year.''source:'' Statcan 2016}}; it could still use a full {{citation}} template, but it would appear in its entirety in the first reference instead of linking to a second reference; this would show up as only one superscript item in the header or row. Otoh, the embedded citation couldn't be separately named, so if another ref in the header or body needed that same, "embedded" reference, it would have to be spelled out again in another, identical {{citation}} template, which is okay, but annoying (and if one of them changed, we'd have to remember to change the other).
find another solution.

This is a bit of a setback, but we'll find a solution. Items 1 and 2 each have their pro's and con's, neither is ideal but may be acceptable. I'm hoping for a better solution, and I am looking into that now. My approach is to use a direct CITEREF-style piped wikilink to link to a previously defined list-defined reference by adding a |ref= id to it, and then referring to that ref using a regular piped wikilink via its CITEREF name; the basics of this are described here, if you're curious. I think this could work, but I have to create and run some tests to see what happens. If it works as I hope, it would appear seamlessly on the page like one nested LDR footnote and solve the problem. (That would be useful as a workaround solution for anyone who runs into this mediawiki bug elsewhere, and so probably would become a generalized template that implements the workaround.) Mathglot (talk) 19:06, 8 February 2022 (UTC)[reply]

@Mathglot: ah, interesting. So the citation ref embedded within a note ref is the problem. In my travels, I am certain I have embedded two or more citation refs within a single note ref, if that complicates things further. As List of municipalities in Alberta is a featured list, items 1 and/or 2 may appear sloppy and therefore compromise its status. It may also not have an impact whatsoever. I could always inquire with the FL director or delegates. Have you made any headway on the direct CITEREF-style piped wikilink approach? Cheers, Hwy43 (talk) 07:13, 3 March 2022 (UTC)[reply]

@Hwy43:, tbh, haven't thought about it since then. I knew you were busy with the Feb elections for a bit, and I needed a break to think about other things, so the timing was good for a break. I already had several possible solutions in mind before the break, so I just need to rejigger my mind back to this topic, and look into which of the ideas I had makes the most sense going forward. Give me a few days, and I'll be back onto it. Mathglot (talk) 08:38, 3 March 2022 (UTC)[reply]

Selective row display – switchover to multi-value display param

This section is about deprecating the four params currently used to control selective row display in the template, and switching over to one multi-valued param instead (provisionally named, |disp=).

This is

Resolved; click show for details.

The goal is to make the table template easier to generate in the first instance, and easier to modify and maintain subsequently for the inevitable tweaks necessary after an initial automatic operation to load the table, by isolating coding complexity such as conditional row display or handling article linkage and other cell display issues in the row template. Going forward, this design will also permit much easier integration of other functionality, such as #Conditional columns.

Background

Up till now, the main Draft table and the sandbox templates are capable of selective row display using params |cities=yes to show cities (i.e., rows with |type=City), |towns=yes to show Towns, |villages=yes to show Villages, and |summervillages=yes to show Summer villages. However, as previously described, this is both unwieldy because of the requirement for parser conditionals in the main table above each data row, as well as being unscalable. In fact we need to be able to selectively test for (at least) five more location types besides those four, as well as at least two combo-types ('urban', and 'rural'); so if it's unwieldy now, it would only get much worse under the current design.

In addition, the multiple display params currently more or less force the current Municipality table template to contain parser conditionals to control conditional row display, which is not ideal for various reasons.

Design overview

The solution is to change the design in two ways:

drop the four existing, binary-valued params (|cities=, |towns=, |villages=, |summervillages=) in favor of one multi-valued param, |disp= which takes the values: 'city', 'town', 'village', 'summervillage', (five more), 'urban', and 'rural'.
move the parser conditional and transclusion of the subtemplate cpmrowif into the row template

This will leave the main Table template consisting of 300+ transclusions of the row template, one per municipality, plus one header template at the top, and one footer template at the bottom.

Note: sidebar about Html tables vs. wikitable markup: the more this complexity is moved into the row template, the less need there is to maintain the current use of wikitable markup. That is, if in the future it would be advantageous to use Html table elements instead of wikitable markup, that would be easy to do. There is no need for that now, and no change to this is currently contemplated.

Implementation

The switchover to the new design involves the following:

Use the "automatically generated" Sandbox rev 1065979331 as of 07:01, 16 January 2022 as a starting point; copy it to sandbox2. I.e., do not include subsequent manual fixes to tweak slashes and commas to ands, ors, or inverted wording.
Create new row template doc Draft:Cpmr13b/doc as a copy of Draft:Cpmr13/doc, and update it for new functionality.
Create new row template Draft:Cpmr13b (or Draft:Cmpr13/sandbox based on the old, and add conditional row display
- start with copy of existing row template rev. 1065740852
- add conditional display based on new |disp= param with nine values, plus urban and rural
- conditional-row checking moves into the row template [/sandbox]
Conditional row checking
- create new row condition checker cpmrowifb as a copy of Draft:Cpmrowif
- new doc page (or inline doc)
Sandbox2 updates
- strip all row-checking (remove cpmrif above each row)
check header and footer templates to make sure no changes required (they should be okay)
Start new Testcases
Continue narrowing columns
- Pop density – 'km2' units not needed in every cell, already in header
- Change percent – '%' units not needed in every cell
Add special handling functionality to row template for some cells
- Region pipes: already has 'Metro' abbr.; needs pipes for Northern, Southern, Central, and AB's Rockies, etc.
- Neighbour special handling for slash (multiple links) (e.g. Calgary), inverted order with comma (e.g. Barrhead), etc.
- Abbrev. 'Council' and add tooltip

ran out of time; this is subject to update. Mathglot (talk) 07:20, 17 January 2022 (UTC)[reply]

10. Template naming – per #Naming and other factors. Mathglot (talk) 21:31, 19 January 2022 (UTC)[reply]
11. Test case page – standard testcases page needed Mathglot (talk) 21:31, 19 January 2022 (UTC)[reply]

@Mathglot: re: abbreviation of Council, ~~apparently Cou would be acceptable, according to this. Alternately,~~ a review of council agendas and minutes reveals “Cllr” is an abbreviation for “councillor” so we could go with Cllrs. Good night! Hwy43 (talk) 08:36, 18 January 2022 (UTC)[reply]

This is on hold for the moment, until the #Data sources issue is clarified, as it may affect the row template changes. Mathglot (talk) 18:28, 17 January 2022 (UTC)[reply]

Okay, most changes through step 5 are done. Here's an example with just cities, and a caption:

List of cities in Alberta

{{Draft:Municipalities of Alberta/sandbox2|caption=List of cities in Alberta|disp=cities}}

Example moved to /examples#Implementation due to performance consdierations

Test case page(s) need(s) to be added, and I added some more steps. This is getting close. There's going to be a bunch of housekeeping to make it releasable: applying correct names, moving to Template namespace, adjusting template code and doc pages to use the new names (and namespace); etc. Mathglot (talk) 05:47, 18 January 2022 (UTC)[reply]

@Mathglot: I have a reprieve from work. Deadline pushed five days so I have some newfound time to spend on this. Just made a couple bold edits to the templates in progress. Let me know if I break anything. Good opportunity for me to see if I am getting the hang of things. I want to change |nb= to |rn= to match my column title change of "Neighbouring municipality" to "Rural neighbour" but I didn’t want to create too much reversion work if I am getting carried away! Hwy43 (talk) 05:05, 20 January 2022 (UTC)[reply]

@Hwy43:, The ones I've seen are:

Draft:Cpmh13 rev. 1066790524 – change hdr to 'rural neighbor'; narrower col hdrs
Draft:Cpmr13b rev. 1066794237 – muni-link ⟶ comm-link
Draft:Cpmr13b/doc rev. 1066794417 – same as above

They all look fine! We should set a goal to release the first version before you have to go back; five days should be enough. Nbd in case there are unforeseen issues or if we prefer to go out and smell the roses, but it's nice to have a goal. Mathglot (talk) 06:15, 20 January 2022 (UTC)[reply]

Resolved. Deprecation of the four params in favor of |disp=) is now complete. Collapsing this section to save vertical space. Mathglot (talk) 22:15, 21 January 2022 (UTC)[reply]

Display by region, CD, CMA, etc.

Mathglot, I just had a wild idea. We have articles for all CMAs/regions and CDs in Alberta. It would be nice to have display parameters for the five regions and 19 CDs at some point in the future. Hwy43 (talk) 06:31, 6 February 2022 (UTC)[reply]

Since we already have param |disp=, and a single subtemplate which decides whether to display a row or not based on |disp= and type, this ought to fall out very naturally and easily from the existing code. Mathglot (talk) 06:37, 6 February 2022 (UTC)[reply]

Sounds good. Just thought of additional. We have the following in Alberta:

six geography regions (data in third column);
19 CDs (data in fourth column);
four CMAs (need to code within template without creating a new column);
two metro region boards (need to code within template without creating a new column);
seven land-use framework (LUF) regions (need to code within template without creating a new column).

I will try to rein in my ambitions to only above sets of additional geographies. Hwy43 (talk) 06:49, 6 February 2022 (UTC)[reply]

Column order and row order

Discussion about sorting by location or type/status is

Resolved; click show for details.

@Hwy43:, had a thought about the order of the columns, in particular, columns one and two which currently contain the municipality (in col. 1) and the "status" or type ('City', 'Town', etc.). I understand why the municipality name is in col. 1, as the table is a "list of municipalities", but it's odd for a naive viewer to look at the table in "all locations" mode (or in "urban" mode, which currently amounts to the same thing as "urban" as we appear not to have an rural locations in the table yet) with column one *appearing* to be in alphabetical order as you let your eyes scan down the page (on my laptop, I can see Airdrie through Edmonton with the header hidden) and I don't learn that my assumption is wrong until I get to Athabasca, the first town, right after Witaskiwin, and realize that the table is actually not alphabetized by its first column, as I might have expected, at least when viewing more than one type of municipality.

I see two possible solutions to this (maybe there are others), and I wonder what you think:

swap columns one and two, so first you have all the cities ('C'), then the summer villages ('S'), then towns ('T') then villages ('V'). Within each status type, it would be alphabetical by location, as it is now, and the expected, sort-by-column-one standard now applies.
re-sort all the table rows so it actually *is* alphabetical by location name

With the new row template isolating the parser code and the main table (in sandbox2) nothing but a list of municipal data (plus header and footer), it's now a trivial operation to sort the table by location, so I did that in sandbox2. Here's what that would look like, for |disp=urban:

List of urban municipalities in Alberta

How does that look to you, with respect to what you think a user would expect to see, when they are displayed an "urban" list? I think it's what I would expect; sort-by-location. Do you have a preference, either for the old way or one of the two other options above? Thanks, Mathglot (talk) 00:35, 19 January 2022 (UTC)[reply]

@Mathglot: I believe/hope I previously indicated that prior to deployment we were going to change the default sort to alphabetical by municipality name. The current sort by status type then alpha by name is only temporary. Hwy43 (talk) 09:01, 19 January 2022 (UTC)[reply]

@Hwy43:, no doubt you did, there's enough stuff going on that I forgot. At least as far as sandbox2 is concerned, it's now already sorted by location name as in the example above (i.e., when you expand the green collapse bar), so if that's what you were planning all along, I assume that means you're good with how it looks there? If so, I'll carry on with the next steps based on the name-sorted version currently in sandbox2. Mathglot (talk) 21:39, 19 January 2022 (UTC)[reply]

Good with A→Z by "Name"! Hwy43 (talk) 04:20, 20 January 2022 (UTC)[reply]

This is Resolved. Mathglot (talk) 03:32, 22 January 2022 (UTC)[reply]

Performance

Are there web pages where you might transclude the template more than once? I ran some tests, and one copy of the |disp=u page took 6 1/2 seconds to transclude:

Performance for {{Municipalities of Alberta/sandbox2|disp=urban}}

Performance report for a simple test page containing {{Municipalities of Alberta/sandbox2|disp=urban}} (6.5 seconds):

Transclusion expansion time report (%,ms,calls,template)
100.00% 6587.540      1 -total
 92.24% 6076.154      1 Draft:Municipalities_of_Alberta/sandbox2
 90.15% 5938.845    256 Draft:Cpmr13b
 53.47% 3522.532    768 Draft:Cpmc13
 51.59% 3398.302    256 Draft:Cpm_nb_cell
 43.75% 2881.800    726 Template:Replace
 33.08% 2179.204    266 Draft:Cpm_nb_item
 26.82% 1766.465    363 Draft:Cpm_dab2
 25.58% 1685.116    363 Draft:MD_expand
 21.66% 1427.063   1387 Template:Str_count

In fact, you might notice ~~this Talk page taking longer to render~~, as there are several tests above; it might make sense to move some of those tests out to subpages, so that this Talk page loads rapidly again, as it used to). Examples moved; performance issue is resolved.

But displaying cities only (|disp=cities) took less than a second in one trial:

Performance for {{Municipalities of Alberta/sandbox2|disp=cities}}

Performance report a simple test page containing {{Municipalities of Alberta/sandbox2|disp=cities}} (less than one second):

NewPP limit report
Parsed by mw1321
Cached time: 20220131085654
Cache expiry: 1814400
Reduced expiry: false
Complications: [vary‐revision‐sha1]
CPU time usage: 0.934 seconds
Real time usage: 1.009 seconds
Preprocessor visited node count: 14271/1000000
Post‐expand include size: 113989/2097152 bytes
Template argument size: 18940/2097152 bytes
Highest expansion depth: 28/100
Expensive parser function count: 0/500
Unstrip recursion depth: 0/20
Unstrip post‐expand size: 10463/5000000 bytes
Lua time usage: 0.388/10.000 seconds
Lua memory usage: 3392578/52428800 bytes
Number of Wikibase entities loaded: 0/400
-->
<!--
Transclusion expansion time report (%,ms,calls,template)
100.00%  929.015      1 -total
 79.06%  734.512      1 Draft:Municipalities_of_Alberta/sandbox2
 66.73%  619.915    256 Draft:Cpmr13b
 28.05%  260.609     57 Draft:Cpmc13
 26.65%  247.575     19 Draft:Cpm_nb_cell
 17.56%  163.177      4 Template:Cite_web
 16.00%  148.620     23 Draft:Cpm_nb_item
 15.86%  147.345      1 Template:AltaMC
 15.49%  143.891     46 Template:Replace
 11.25%  104.533     99 Template:Str_count

so it kind of depends how you plan to use the template on article pages. Mathglot (talk) 09:07, 31 January 2022 (UTC)[reply]

I've moved three examples that perviously generate a full table on this talk page, to the subpage /examples. This should make this Talk page load fast again. Mathglot (talk) 19:13, 31 January 2022 (UTC)[reply]

@Mathglot: that explains why my sandbox was crawling to load! For phase one, I intend to display:

cities only at List of cities in Alberta#List
towns only at List of towns in Alberta#List
villages only at List of villages in Alberta#List
summer villages only at List of summer villages in Alberta#List
four instances at List of communities in Alberta - one instance for the Cities section, one instance for the Towns section, one instance for the Villages section, and one instance for the Summer villages section.

So yes, one article in which the template will be transcluded four times.

I also intend to display all four municipality types in a single template at List of municipalities in Alberta#List of urban municipalities, but I won't until phase two as I don't want that instance to have all 13 columns.

Then there will be a phase three to display three other instances of the template at List of municipalities in Alberta. Hwy43 (talk) 06:07, 1 February 2022 (UTC)[reply]

Yes, that’s why. Good that the one with all four won’t come out till Phase 2, that will give us time to look at any performance considerations. Mathglot (talk) 07:15, 1 February 2022 (UTC)[reply]

Just leaving this here, so I don't forget later: based on some things I read at WP:PEIS, nested transclusions increase the count, but I'm not sure if that figures in performance. It wouldn't be too difficult to reduce nesting by combining all the three or four subtemplates currently used to create the rn cell, into one template. Another alternative involves redoing the row conditionals, so that instead of having one conditional transclusion per row (which allows mixed types such as "urban" to be displayed in municipality alphabetic order) we could have one conditional per type (currently column two). This would save around 300 transclusions of cmprowifb, at the cost of displaying the table alphabetized by type (all cities first, then all towns, and so on) instead of by municipality. Once displayed, the user could click the column header to get to alphabetic by municipality. Not clear how much this would save, since cmprowifb is a simple template that doesn't transclude any other templates, but it should save something. To the extent that you mostly want to display "cities only", "towns only", and so on as you described above, this may be a more logical approach anyway. Mathglot (talk) 03:07, 5 February 2022 (UTC)[reply]

Okay. I will defer to you. Hwy43 (talk) 03:16, 5 February 2022 (UTC)[reply]

None of this is for Phase I anyway, just wanted to list things while I think of them. Mathglot (talk) 06:11, 5 February 2022 (UTC)[reply]

Another method is subst'ing some or all of the subtemplates, and maybe the main template, too. The more subtemplates that are subst'ed, the less work the mediawiki engine has to do when the page is loaded, and the faster it is. Mathglot (talk) 05:42, 6 February 2022 (UTC)[reply]

Phased release

We should start talking about the release of the first version. One issue is testing: there is a (rump) testcases file, which needs a lot of expansion. The release itself will require a lot housekeeping which is somewhat tedious but needs to be done, mostly relating to all the helper templates needed to implement the table. Besides the main template, currently at {{Draft:Municipalities of Alberta}}, there are ten helper templates, three stand-alone /doc files, two sandboxes, and one testcases file; they all need to be moved to the right place as subtemplates of the main one, and have Template space redirects for them so they can be referred to more easily. This means a lot of code changes to drop all the "Draft:" prefixes all over the templates, and replace the old names with the new ones (or redirects) in the code; this is somewhat error-prone so I anticipate some breakage *during* the release process, but once everything is in place, we run the testcases until everything looks good.

I don't think we should go further with references in the first release, at least, not the new, per-row or per-cell references. If you feel you need them in order to satisfy WP:Verifiability, then let's just make a list of all of the references for the Alberta table (a task which has to be done at some point anyway) and add them all as LDRs (I can help), and create one new, 12-column span footer row, and include all of the refs there for now. If you want to use the two pop. refs that already exist, let's discuss if it makes sense to do that for the first release or not, depending how complex it is and how many there are. Part of the reason for this, is just the sheer amount of effort required for all of this, and I need to get back to some other projects that are languishing. If we try to get too much into the first release, it might get pushed off, and that wouldn't be good. I feel we can wrap this up in a week and get something out there if we draw a line in the sand now, and start planning for it. Once the table is out there, then we can start planning for the next phase, and you can decide how to prioritize next steps, whether you prefer in-cell refs, conditional columns, or whatever else you have in mind. Shall we try to shoot for a 2/7 release? Think about it, and let me know. Mathglot (talk) 09:07, 31 January 2022 (UTC)[reply]

@Mathglot: let's use just the two pop. refs for now. The per-row/cell refs can wait until a subsequent release. Can even forego the new, 12-column span footer row for now since I won't be implementing phase one at List of municipalities in Alberta (a featured list) yet, in which WP:V could be most scrutinized. So a February 7 target sounds good, though I may not apply it until the dust settles from the February 9 release of the 2021 census results; so late that day or the following day or more. Hwy43 (talk) 06:39, 1 February 2022 (UTC)[reply]

All sounds good. If you’re expecting a new census on Feb 9, maybe we should shoot for Feb 16 just in case, and if things go unexpectedly well, just release it earlier than that. Mathglot (talk) 07:10, 1 February 2022 (UTC)[reply]

Sounds good. Hwy43 (talk) 01:07, 2 February 2022 (UTC)[reply]

One thing I could start doing if there's a lull, is to start the subtemplate moves (along with the code changes to point to the new names). The subtemplates are pretty stable now and don't need the main template to be finished to do it. There's a fair bit of tedium involved, and there will probably be some misfires or glitches along the way, until everything is in the right place and named properly, with the redirects in place. The actual "release" will involve placing the main template there, and if all the tedium was executed properly, it should just work. I wouldn't say that this template is very complex (the row template is probably the kernel of whatever level of complexity it has), but it does have a lot of moving parts so there's always the possibility of a major hiccup for some unforeseen circumstance that would stall the release or require a rollback, so don't be too surprised or concerned if that happens; we'll deal with it as it comes. But for now, things seem to be progressing smoothly, and I don't foresee problems at this point. Mathglot (talk) 08:20, 2 February 2022 (UTC)[reply]

@Mathglot: I am okay with this but have one question first. I would like to properly regenerate the 256 instances of the rows template to replace the "M.D. of" instances with "MD of", replace Foothills No. 31, MD of with Foothills County (to reflect a municipal name change a couple years back), and fix inconsistent rural neighbour wikilink approaches used in the cities rows compared to the other rows (e.g. for the Grande Prairie row should have Grande Prairie No. 1, County of instead of County of Grande Prairie No. 1 so that when the column is sorted alphabetically is sorts by "Grande" and not "County". Hwy43 (talk) 02:06, 3 February 2022 (UTC)[reply]

@Hwy43: it's actually easier to parse and handle "M.D." than "MD", because of the possibility that some town like "Camden" (or any town with ---md---) might show up as a municipality or country in some province. But as the row template already handles both, it's not worth regenerating the table just for that. If there's just a handful of other changes, like that of "Foothills" and "Grande Prairie", you can change that by hand also. If you prefer to regenerate it, do you want to try to create sandbox3 from the tsv result you get, or do you want me to do that? In principle, from the tsv to the sandbox just is another regex conversion, but with recent changes for the bundled citations thing, I'd have to at least update the documentation at the row template to reflect that, and that hasn't been done yet. The line you're trying to generate for Lloydminster for example, is this:

{{Draft:cpmr13b|Lloydminster|City|Central Alberta|10|County of Vermilion River|January 1, 1958|7|19740|2015|19645|18032|24.04|t=c|suffix=(part)|r=Lloydminster|d={{{disp|}}}}}

The order of the fields matters up through 24.04; after that, they may be in any order. Let me know your preferences. Mathglot (talk) 18:39, 3 February 2022 (UTC) updated by Mathglot (talk) 01:45, 4 February 2022 (UTC)[reply]

@Mathglot: please use "MD of" as the trigger instead of "MD" as the better way to avoid any municipality with "---md---". "M.D." is not preferred for the same reason it is not preferred to use "N.A.T.O.", "T.V.", and "U.S.B." (see MOS:POINTS). Hwy43 (talk) 02:46, 4 February 2022 (UTC)[reply]

@Hwy43: not sure if you read the code of the subtemplate, but in fact that is what it is doing now. We can keep it that way, and drop the checking for "M.D." at some point. Mathglot (talk) 02:50, 4 February 2022 (UTC)[reply]

@Mathglot: ah. I am not following along as closely as I can be. Thanks. I have yet to reply to you on a couple other things. Will do so tomorrow. Hwy43 (talk) 08:44, 4 February 2022 (UTC)[reply]

@Hwy43: no worries, this is a volunteer project, no obligations here. Upgrades today on the references subtopic have brought us close to the final look, as it now generates a complete table, with conditional rows on demand, and with all header and row references present (all the ones I'm aware of that you added to the ref scratchpad). You can see that output at /examples/bundled citations, which is based on the upgraded /sandbox2 and Draft:cpmrefs. (There's a bug that emits a bunch of '}}}' at the end of the table, which I don't expect will be too problematic.) The main missing bit now, is that municipality footnotes display only the text portion of the note, and not any embedded citations, which are commented out for now (but present in the code as hidden text). That's the last step that I'm aware of that still needs doing, before the Feb. 9 data set upgrade. But the current example looks pretty close to how it will look when it goes live. Performance is still an issue, and will probably have to be addressed in a subsequent phase. Mathglot (talk) 09:04, 4 February 2022 (UTC)[reply]

@Mathglot: understood. It appears I am catching up and following along now. Hwy43 (talk) 02:51, 5 February 2022 (UTC)[reply]

Regenerating the template vs. fixing the issues

@Hwy43: above, you mentioned regenerating the template above in order to fix some errors. If there's only a few, it's probably not worth it; it's very easy to change the sandbox (i.e., the future template). You noted the following issues:

"MD" vs "M.D." issue – already working; we can simplify next time, but messing with this now would be "if it ain't broke, don't fix it" problem, imho;
Grande Prairie – an easy fix; Done (see example)
'Foothills No. 31, MD of' to Foothills County – this looks like an easy fix, there are four of them, in: Black Diamond, High River, Okotoks, and Turner Valley. Do you want those four fixed?

If that's all there is, I don't see any point in recreating the template just for that. Are there others? You could make a note to yourself for Phase 2 to fix your regex so it generates it correctly next time around, but I don't see any point to recreating it now just for that, do you? There's also a risk to recreating it this close to release of the first phase, in case there are unexpected problems, because the current version is close to where we want it to be. Mathglot (talk) 02:01, 5 February 2022 (UTC)[reply]

Mathglot I have fixed the four Foothills instances and Cold Lake's rural neighbour that was yet to be fixed. I also did the MD thing anyway. This exercise was easy. I just copied all code and pasted into Word, then did a find/replace exercise, and then copied and pasted the code back in. Such a quick exercise. If there were unexpected problems, feel free to revert. Hwy43 (talk) 03:06, 5 February 2022 (UTC)[reply]

@Hwy43: Good job, it looks fine. Wikipedia has a regex editor, you can find it in your left sidebar under 'TemplateScript' header, but like you, I don't use it either, I have an offline text editor I use which has strong regex features. Mathglot (talk) 03:31, 5 February 2022 (UTC)[reply]

I've made some simple width adjustments, to rn, to pop. density, and also moved the sort widget to its own header row, with the result that the whole table is narrower and looks much better now, and almost fits in standard width without horizontal scrolling. If we abbreviated months, so that instead of "September 19, 1911" we used "Sep 19, 1911" (an acceptable format per MOS:DATEFORMAT in tables or where brevity is required), I think you could have 13 columns with no horizontal scrolling. Mathglot (talk) 05:51, 5 February 2022 (UTC)[reply]

@Mathglot: I am supportive of the "Sep 19, 1911" approach. Note that when we copy and adapt this over for use in other provinces, sometimes we only have an incorporation year instead of the exact date. If one row had a precise date of "Sep 19, 1911" and all others we had year only, could such be accommodated and properly sortable? That is, the row with "Sep 19, 1911" follows all other rows with the year of 1911 before then displaying all rows with the year 1912. Hwy43 (talk) 06:24, 6 February 2022 (UTC)[reply]

@Hwy43: Depending on how refs stuff goes, we'll see if we can squeeze shortened dates into Phase I; it would be nice, as the table would fit almost without any horizontal scroll, I think. As far as year-only date, no problem, but it might take some looking into; but since this is for other provinces, by definition that would be a later phase, so won't affect us now. Mathglot (talk) 06:31, 6 February 2022 (UTC)[reply]

Indeed. Hwy43 (talk) 06:35, 6 February 2022 (UTC)[reply]

@Hwy43: D'oh, I hit on a very easy way to do it, i.e., brute force (after all, their are only 12 months, and my editor can mass-change them) so it's done, and that has shrunk the width so the whole table fits on the screen and the horizontal scrollbar has disappeared, so that's a big win in my book. I do have one question about the column header for the date column: why the parenthetical "current status"? If that's some sort of explanation about what the "incorporated" date actually means, can we just move the term "current status" into the footnote for that column? If you took out "current status", and changed "incorporated" to "incorp<br/>date", the column might shrink a bit more, as "incorporated" is now the widest element in the column. See /sandbox2. Mathglot (talk) 07:03, 6 February 2022 (UTC)[reply]

@Mathglot: see the three incorporation dates in the infobox for Chestermere. We are presenting the date in which it incorporated under its current municipal status (in this case, when it became a city). I support you trying to instead tackle this as a note. Hwy43 (talk) 07:27, 6 February 2022 (UTC)[reply]

@Hwy43:

Done. See Chestermere at /examples/bundled citations. (Again, citation is there, but hidden for now.) Mathglot (talk) 08:05, 6 February 2022 (UTC)[reply]

(edit conflict)@Mathglot: to elaborate on what I mean re: "current status", I am thinking a change to the column header as follows would do the trick. {{abbr|Incorp|Incorporated}}<br/><ref name=foo/>{{#tag:ref|A municipality may have two or more incorporation dates for different municipal statuses throughout its history. If a municipality first incorporates as a village, then as a town, and most recently as a city (i.e., its current municipal status), then this column presents the municipality's incorporation date under its current municipal status.|group=lower-alpha}} Make sense? Hwy43 (talk) 08:17, 6 February 2022 (UTC)[reply]

@Hwy43: yes, it makes sense and would be helpful, and we should do that. I'm in the middle of trying to adjust the refs and hitting a snag, so I may not get to this right away. Remind me if I forget. Mathglot (talk) 08:23, 6 February 2022 (UTC)[reply]

@Hwy43: Updated "Incorp" column header with the new note per your suggestion, and the whole column looks cleaner now. No new example needed, as the existing /examples/bundled citations automatically picks up the updated header and demonstrates the new appearance. Mathglot (talk) 01:02, 7 February 2022 (UTC)[reply]

@Mathglot: also, and I think this is on your list, the %-sign can be removed from each row in body and in footer as it is already in parentheses in the header. In fact, the column header can be changed to {{abbr|Chg|Change}}<br/>(%). Hwy43 (talk) 07:48, 6 February 2022 (UTC)[reply]

Such might require a change to {{Change}}, which I assume is being used to generate values in this and its two preceding columns. Hwy43 (talk) 07:52, 6 February 2022 (UTC)[reply]

@Hwy43: Good idea. Doable, either as a template change (which I won't tackle now, because of a highly visible template that would require lots of regression testing even for a simple change) or by adding a replacement wrapper to remove the '%' character (which would cause a slight increase in load time). Imho, neither is worth doing now, the template change is worth doing later. Mathglot (talk) 08:05, 6 February 2022 (UTC)[reply]

Incorporating the template post-launch

We should incorporate the template into at least one article right after the template is launched. The reason for this is that over the years, a lot of templates have accumulated in template space that are not used and have become a lot of clutter. A group has recently been established to find these unused templates and delete them (and that's a good thing), but we just need to make sure that at the point we want to launch the template, you already know which article you want to include it in so we can add it right after launch, so that the template doesn't show up on the list of unused templates and get nominated for deletion. Thanks, Mathglot (talk) 19:35, 5 February 2022 (UTC)[reply]

@Mathglot: if you think it is viable to launch as early as tomorrow, no opposition to doing so at the above and at List of cities in Canada#List and List of communities in Alberta#Cities. Hwy43 (talk) 05:32, 6 February 2022 (UTC)[reply]

@Hwy43: tomorrow seems early, unless you can live without the references in the notes, and even then is iffy, because I'm kind of expecting glitches in the #Naming and moving process. We could always try it, and then just roll back if it doesn't work right. Currently, Template:Municipalities of Alberta is a redirect, so "launch" basically means copying Draft:Municipalities of Alberta on top of the redirect. This should then work out of Template space, assuming that all of the subtemplates have been moved into the right places, and all the code with "draft" has been changed to the new name as appropriate in each template. You can check the checkmarks in the table at #Naming to see about the moves so far. There's a tester file here using the live subtemplates (less the refs subtemplate), so they appear to be working in situ, so, so far, so good. What's left now, is upgrading the refs subtemplate for embedded citations, and group name. Mathglot (talk) 06:06, 6 February 2022 (UTC)[reply]

@Mathglot: good point about references in notes. I am ready when you are, although Wednesday and Thursday I will be obsessively focussed on editing Alberta articles with the new census results. So you’ll have my attention if you want to launch before Wednesday or after Thursday. Cheers, Hwy43 (talk) 06:10, 6 February 2022 (UTC)[reply]

@Hwy43: got it, thanks! Mathglot (talk) 06:18, 6 February 2022 (UTC)[reply]

@Hwy43:, there's another issue which arose related to your linking List of cities in Alberta#List above. I should've looked at this earlier, because the fact that the refs are right under the table is important information which I wasn't aware of before, and has resulted in a detour for the last couple of days to accommodate it. (I should go into that in more detail at #References and sourcing.) In any case, that's now mostly resolved, leaving the next step still as it was, namely the "references in notes" issue still to be resolved. As far as this section is concerned, that means release will be no earlier than the weekend, and probably next week is more realistic. Mathglot (talk) 00:55, 8 February 2022 (UTC)[reply]

Thanks for the reminder; time to get back to this soon... Mathglot (talk) 06:26, 3 March 2022 (UTC)[reply]

Ha! That wasn't a reminder to get back to this. It was a hint to you that I haven't forgotten about you and this collaboration. Pardon the three weeks of momentum-killing silence. Work/life/wiki balance is out of sorts. Could be another week and a bit before I return to a fraction of the previous attentiveness but I am going to try to process every message you sent since February 5 and respond where necessary. Cheers, Hwy43 (talk) 06:34, 3 March 2022 (UTC)[reply]

Not at all; I needed the break. I think we collaborate very well, each with their own strengths and weakenesses, and the whole is greater than the sum of the parts. A week (or whatever) would be great, that will let me transition gradually back into this. (Can be longer, RL is priority, this is secondary.) Don't worry about responding to every message, unless you want to. I barely remember what I wrote, at this point, and might have to spend some time processing my own messages

. Mathglot (talk) 08:46, 3 March 2022 (UTC)[reply]

Hi. I have noticed numerous one-character edits on a variety of pages associated with this idling effort with the edit summary of "bump". Are you looking to reinstate the collaboration? Hwy43 (talk) 19:24, 10 July 2022 (UTC)[reply]

[Airdrie-1] Sources for Airdrie:
2011 pop.: "Canada national census of 2011". 2016. Retrieved 1 February 2022.

2016 pop.: "Alberta provincial census of 2016". 2016. Retrieved 1 February 2022.

2019 pop.: "2019 Municipal Affairs Population List" (PDF). Alberta Municipal Affairs. December 2019. ISBN 978-1-4601-4623-1. Retrieved January 20, 2022.

area: "Candian land municipal area database". 2020. Retrieved 1 February 2022.

[2] 2011 pop.: "Canada national census of 2011". 2016. Retrieved 1 February 2022.

[3] 2016 pop.: "Alberta provincial census of 2016". 2016. Retrieved 1 February 2022.

[4] 2019 pop.: "2019 Municipal Affairs Population List" (PDF). Alberta Municipal Affairs. December 2019. ISBN 978-1-4601-4623-1. Retrieved January 20, 2022.

[5] rea: "Candian land municipal area database". 2020. Retrieved 1 February 2022.

[1]