Module talk:Transcluder

Edit request in re nofollow

I added a |nofollow= to disable following redirects. See Module:Transcluder/sandbox. This has two main uses:

1. Sometimes redirect pages have interesting information on them, that is tedious to copy over whenever it changes. E.g., User:Psihedelisto.
2. When making many redirects to the same page, that will all have exactly the same wikitext, it's annoying to have to keep typing the {{R shell}}, when you can just, {{subst:#invoke:Transcluder|main|User:Psihedelisto|nofollow=y}}.

Pinging Sophivorus as this is their module. Psiĥedelisto (talk • contribs) ^{please always ping!} 02:01, 19 July 2020 (UTC)

@Psiĥedelisto: Thanks for the contribution! I did a few tweaks and just made it live. Notice that the option is called "noFollow" rather than "nofollow" for coherence with the other options. Let me know of any issues or further requests, cheers! Sophivorus (talk) 13:46, 19 July 2020 (UTC)

only didn't work on more than 1 element for me

Here, called there. — 𝐆𝐮𝐚𝐫𝐚𝐩𝐢𝐫𝐚𝐧𝐠𝐚 (talk) 22:11, 24 May 2021 (UTC)

@Guarapiranga Hi yea, I actually removed that feature some time ago but forgot to remove it from the documentation. I just did, thanks. By the way, you can simulate the functionality by doing multiple calls to the module, cheers! Sophivorus (talk) 00:58, 25 May 2021 (UTC) 👍 ‍

Could this module be lighter and faster with mw.string instead of mw.ustring?

I'm trying my hand at it in the sandbox, but still gotta debug. — 𝐆𝐮𝐚𝐫𝐚𝐩𝐢𝐫𝐚𝐧𝐠𝐚 (talk) 21:16, 30 May 2021 (UTC)

@Guarapiranga Honestly I used mw.ustring only because it seemed safer, without really understanding the difference with mw.string. If you think you understand the difference and that it won't matter, and all the test cases look good, I'll be happy to incorporate your changes, cheers! Sophivorus (talk) 00:45, 31 May 2021 (UTC)

My sandbox is nowhere near ready—I was hoping someone more experienced could jump in and find my mistakes. It was from Johnuniq I read that ustring is a lot slower.^[1] The reason is that Scribunto implements mw.ustring by using Lua to replace the regex library which is a breathtaking accomplishment but which makes it slower than using plain strings. ^[2] Mr._Stradivarius also says that using mw.ustring has the drawback of having to cross back and forth between Lua and PHP all the time, which reduces the performance by quite a bit.^[3] — 𝐆𝐮𝐚𝐫𝐚𝐩𝐢𝐫𝐚𝐧𝐠𝐚 (talk) 02:21, 31 May 2021 (UTC)

I'm not interested in modules that attempt to add another layer of wikitext parsing to the complex PHP of MediaWiki. I can imagine why something like this module would be wanted for special noticeboards but using it in articles is certain to fail periodically and is a maintenance problem. Johnuniq (talk) 02:52, 31 May 2021 (UTC)

This module is used in Module:Excerpt, which I gather was built for creating topic portals, and I use it at WP:User scripts/List/sandbox, Johnuniq. I don't think it's used in articles. — 𝐆𝐮𝐚𝐫𝐚𝐩𝐢𝐫𝐚𝐧𝐠𝐚 (talk) 03:50, 31 May 2021 (UTC)

Try "what links here" in the article namespace. Johnuniq (talk) 04:06, 31 May 2021 (UTC)

@Guarapiranga @Johnuniq This module was primarily designed to be used by Module:Excerpt which implements Template:Excerpt, a template used in 2000+ articles in the English Wikipedia (including COVID-19 pandemic and other high profile articles) and 1500+ more in the Spanish Wikipedia (including heavy use in high profile articles like Science, Philosophy, country articles and many more). Cheers! Sophivorus (talk) 13:05, 31 May 2021 (UTC)

I stand corrected 😊 — 𝐆𝐮𝐚𝐫𝐚𝐩𝐢𝐫𝐚𝐧𝐠𝐚 (talk) 13:07, 31 May 2021 (UTC)

For an example of problems, see Electoral results for the district of Parramatta (which is currently showing a million "time allocated for running scripts has expired" errors). That is solved by removing all the mw.ustring stuff but the bigger question is why articles are using this guaranteed-to-fail method of padding articles. Johnuniq (talk) 05:26, 18 July 2021 (UTC)

@Johnuniq Hi! Thanks for the info, switching to mw.string has just sharply increased its priority, I'll try to implement it asap. Regarding your bigger question, I'm not sure I understand it or why you say "guaranteed-to-fail", since this issue only seems to affect one or at most a few articles, and seems solvable. Cheers! Sophivorus (talk) 12:32, 18 July 2021 (UTC)

I did a massive edit to replace many mw.ustring with plain string. Unfortunately, the module is trying to do too much work and Electoral results for the district of Parramatta is still broken. I left 30 mw.ustring and replacing them would fix that article. I left them because some functions may need mw.ustring (depending on how the functions are called), and some I lost interest in because I had already done a lot of changes. I'll explain about "guaranteed-to-fail" some other time because I don't want to distract from seeing if this module can be further edited, or whether the failing article needs a drastic edit (yes it does need a drastic edit because it's just silly to repeat so much information, but that's not a problem for discussion here). Johnuniq (talk) 03:57, 19 July 2021 (UTC)

There are now 25 articles in Category:Pages with script errors because my editing this module means some pages have been purged. Johnuniq (talk) 05:03, 19 July 2021 (UTC)

@Johnuniq Hi, thanks for the work done so far! I copied your changes to the central sandbox and some of the test cases seem to be failing now. This may mean broken or buggy excerpts on the English Wikipedia. Do you know what may be the cause? If not, I'll try to figure it out and fix it this week, but I may have to revert to the previous stable version until I do. Many thanks again!! Sophivorus (talk) 12:42, 19 July 2021 (UTC)

@Sophivorus: I edited mw:Module:Transcluder to fix a couple of problems and copied that module to Module:Transcluder. That means "non" is accepted as "no" which frankly is not a problem but if it were, it can be removed later. Having the two modules the same is simpler now. mw:Module:Transcluder/testcases is showing one problem, namely testReferences:

Expected = acfgk

Actual = acfgk<ref name="l" />

I think the test text for that is the following (from mw:Module:Transcluder/test#References).

a<ref>b</ref>c<ref name="d">e</ref>f<ref name="d" />g<ref name="h" group="i">j</ref>k<ref name="l" />

I cannot see why the code would ever remove <ref name="l" />. It only removes <ref name="d" /> because there is another ref with name="d". Any ideas? Johnuniq (talk) 09:43, 20 July 2021 (UTC)

After considerable trouble, I have worked out why the current module (with my changes) fails testReferences while the previous mw:Module:Transcluder revision from 12:40, 9 January 2021 passes. It's because this edit removed the following code from getReferences:

if flags and not truthy(flags) then
    text = mw.ustring.gsub(text, '<%s*[Rr][Ee][Ff][^>/]*>.-<%s*/%s*[Rr][Ee][Ff]%s*>', '')
    text = mw.ustring.gsub(text, '<%s*[Rr][Ee][Ff][^>/]*/%s*>', '')
    return references, text
end

Johnuniq (talk) 03:56, 21 July 2021 (UTC)

@Johnuniq Thanks so much for figuring it out! I just restored the code and now all the test cases pass. I also added a comment explaining why I put that code there in the first place. It's kind of hacky but it's the best I could come up with so far. Same with 'non': it's there to make it work on the French Wikipedia, but it's also kind of hacky so I added a short comment to try to clarify it. Electoral results for the district of Parramatta still displays timeout errors but I think it displays way less than before. The page is also kind of extreme so I think the blame can't be put much on the Transcluder module or the excerpt concept. In fact, this reminds me that some time ago I tried to construct a table of COVID stats with data pulled out entirely from Wikidata (using Module:Wd) but encountered timeout errors that prevented me from accomplishing it. I remember I thought: why should volunteers have to work more in order to save computers from working? Would you agree? Do you think that maybe the max execution time should be upped a bit? Thanks again for the hours put into this! Sophivorus (talk) 13:46, 23 July 2021 (UTC)

I am a pragmatist and believe we should work with what we've got. While asking for more execution time might be reasonable, the fact is that previewing an edit to any of the many articles which excessively use {{excerpt}} is painfully slow. More execution time would only make those previews/saves take longer. The fundamental problem is that duplicating content is a broken idea. If it really would be desirable to do that, MediaWiki has a built-in mechanism that is extremely fast, namely transclusion. Rather than trying to write a module which deals with all the weird and unpredictable ways wikitext might be written, those wanting to mirror excerpts around the place should use transclusion instead. That would not break and would be very fast. Johnuniq (talk) 01:49, 24 July 2021 (UTC)

I had another go at making the module work at that failing article. Some debugging showed that Module:Excerpt is also trying to do too much work and it appears to contribute quite a lot to the 10 seconds execution limit. The modules are trying to replace the MediaWiki parser and that is not going to work reliably. Johnuniq (talk) 10:25, 24 July 2021 (UTC)

@Johnuniq Thanks again for the improvements!! Problem with plain transclusions is that they transclude the entire page, so you have to go use <includeonly> tags which complicate the wikitext and make it more difficult for some users to understand, adopt and use correctly. In the first version of excerpts, I was using the #lsth parser function that at least can transclude just the intro of a page or a specific section. Problem was that it also transcludes undesired elements like infoboxes, notices, navigation templates with undesired automatic categorization, and many others. Name conflicts with references were also rampant (especially because the visual editor names all references the same), so ugly <noinclude> tags and other complications were still necessary. Long story short, modules like Transcluder and Excerpt remove all these difficulties and result in easy, beautiful excerpts at the cost, granted, of a performance loss. But luckily, your efforts are making me see that so many things can be improved on that front that maybe that drawback can be greatly reduced! Thanks again!!! Sophivorus (talk) 12:34, 24 July 2021 (UTC)

function getNamespaces is misbehaving

Unfortunately, function getNamespaces is not performing as intended the function

local function getNamespaces(name)
	local namespaces = mw.site.namespaces[name].aliases
	table.insert(namespaces, mw.site.namespaces[name].name)
	table.insert(namespaces, mw.site.namespaces[name].canonicalName)
	return namespaces
end

Although 'namespaces' is a local variable, the table.insert is actually adding to the global values

A test

function p.td()
	tab = getNamespaces('Category')
	mw.log(#tab)
	tab = getNamespaces('Category')
	mw.log(#tab)
	tab = getNamespaces('Category')
	mw.log(#tab)
end

returns 2, 4, 6

I think mw.clone() would help eg

local namespaces = mw.clone(mw.site.namespaces[name].aliases)

Desb42 (talk) 10:06, 24 November 2021 (UTC)

Interesting bug! Your solution is correct, thanks. The original defines namespaces as a pointer to the aliases table in mw.site.namespaces and unexpectedly it allows that table to be modified (presumably it's only a temporary table so the modifications have no effect). The function should be a bit smarter and not add duplicates—in this example both .name and .canonicalName are "Category" and getNamespaces returns a table including both. I'll have a look later to see if a fix would be warranted (that is, how is getNamespaces used?). I checked for changes since I last looked at the module and there is another problem: local min, max = string.match is no longer adequate now that en dash and em dash have been included in the regex. One fix would be to use ustring.match. It might be better (I haven't thought about it yet) to instead use gsub to replace en/em dashes with a hyphen before doing the match. Johnuniq (talk) 02:59, 25 November 2021 (UTC)

I started looking at function parseFlags but have abandoned it for the moment because other stuff should be cleaned. The regex '^(%d+)%s*[-–—]s*(%d+)$' has a typo that would make it almost never work—the second s should be %s. I can't (quickly) work out what to do about the returned flags table—for a range like 3-5 the table would have numeric keys 3, 4, 5 but for a plain number such as 3 the table would have a string key '3'. Finally, the function should use tonumber first rather than looking for a range then setting min, max to the same value if the text is a number rather than a range. Johnuniq (talk) 04:26, 25 November 2021 (UTC)

@Sophivorus: You might like to investigate these issues. Johnuniq (talk) 05:52, 27 November 2021 (UTC)

@Desb42 @Johnuniq Thanks for the report and the fixes! I just fixed the namespace issue and the %s typo and replaced string.match for mw.ustring.match in parseFlags. Johnuniq, I couldn't quite follow your reasoning about the problem with the returned flags table, but feel free to experiment at the central https://www.mediawiki.org/wiki/Module:Transcluder (since the module is used increasingly on several Wikipedias) and testing your changes at https://www.mediawiki.org/wiki/Module:Transcluder/testcases Cheers! Sophivorus (talk) 12:20, 27 November 2021 (UTC)

Those working on the module need to decide whether input 3-5 should result in keys that are numbers while input 3 gives a key that is a string. It doesn't make sense for them to be different although quirks in the code could possibly make it work. To illustrate the difference:

t = {}
t[1] = 'My key is a number'
t['1'] = 'My key is a string'
for i, v in ipairs(t) do ... end  -- processes t[1] only

Johnuniq (talk) 23:13, 27 November 2021 (UTC)

@Johnuniq Thanks, I think I understand now and I agree. Do you think this change should suffice? Sophivorus (talk) 21:56, 29 November 2021 (UTC)

@Sophivorus: Sorry, I was wrong (or, I cannot now work out why I thought the code was mixing numbers and strings). It is true that mw.ustring.match will return strings given something like '3-5', but in for i = min, max do flags[i] = true end, i can only be a number and min, max will be automatically converted to numbers if possible. I think your code would work but I edited mw:Module:Transcluder to show what I recommend. There is very little practical difference but I think checking for input "3" before checking for "3-5" is cleaner. The line

min = tonumber(range:match('^%s*(%d+)%s*$'))

could simply be

min = tonumber(range)

if you don't need to worry about input like 1.2e1 (which tonumber regards as 12) or broken input like 1.2 (which would give a non-integer key). Johnuniq (talk) 06:17, 1 December 2021 (UTC)

some (hopefully) performance improvements

I have just created Module:Sandbox/Desb42/Transcluder to test out a number of ideas

1. change getNamespaces to create a list of unique namespace names (as this produces a list of regexes to be processed one by one)
2. change how template black listing is processed (check the first letter of the template name and only process those blacklist names that start with that letter). See function matchBlacklist and the reorganised xtemplates (currently within this Module)
3. the function removeString could be called many times in getTemplates (this 'text' is potentially rebuilt many times). Instead of using gmatch to work through 'text', take a note of how far the search has progessed (using the '()' regex instruction), and copy each snippet up to this to build one text string at the end of the loop

Desb42 (talk) 16:27, 27 November 2021 (UTC)

@Desb42 Awesome!! Every performance improvement is welcome since some pages include many excerpts and we have encountered script timeouts before. Furthermore, I think excerpts will slowly become more and more common so performance improvements will become more and more of a deal. I looked at your code but I need to study it better (I'll do so this week, promise). Feel free to experiment as much as you want and then deploy your changes at https://www.mediawiki.org/wiki/Module:Transcluder so that we can test them against https://www.mediawiki.org/wiki/Module:Transcluder/testcases before deploying on the wikis. Lastly, please continue trying to make your code and comments as readable as possible so that future developers can understand it easier and are motivated to contribute! Kind regards, Sophivorus (talk) 11:47, 30 November 2021 (UTC)

MediaWiki update

I noticed the MediaWiki version is slightly out of data with the English WP version:[4] fgnievinski (talk) 03:01, 29 January 2022 (UTC)

bad argument #2 to 'sub' (number expected, got nil)

On Platform Engineering Team/SOPs/Intake Process the last section is failing with this error when attempting to transclude "intake" from another page. I traced this to the second sub call on line 443, which is string.sub(text, suffix). Perhaps someone here will know what causes this and how to repair the module and/or the page using it (If the latter, perhaps we can handle the issue and render a more helpful error). Krinkle (talk) 03:26, 5 February 2022 (UTC)

@Krinkle: As you note, there is no check of the result of the string.match at line 438 and it fails if the pattern is not found. At any rate, in your application someone has gone to trouble of using a labeled section transclusion although they included an extra heading which may or may not have been wanted. At mw:Platform Engineering Team/SOPs/Intake Process#Platform Operations and Dumps the {{excerpt}} should be replaced with the following which has a much lower overhead.

{{#section:Platform Engineering Team/Platform Operations and Dumps|intake}}

Johnuniq (talk) 06:06, 5 February 2022 (UTC)

@Krinkle Hi! Thanks for the bug report, I'll try to fix it but in the meantime, I fixed your excerpt by adding "subsections=yes" since the content of the <section> tags was only subsections and excerpts by default strip out subsections. Cheers! Sophivorus (talk) 12:49, 5 February 2022 (UTC)

Parameters — exact match

Hello, over at wikispecies, Felis catus (this version) is using this module to display a vernacular name where on enwiki "Featured article" or equivalent would display. However, if one chooses French (fr) as one's language from the language selector, the Vernacular name is displaying the entries both for fr=Chat domestique and frr=Kaat; how do you ensure only (fr=) Chat domestique is displayed?

<indicator name="vernacular pagename">[ {{{style|<span style="border-bottom:1px dotted">}}}{{#invoke:Transcluder|main|{{BASEPAGENAME}}|only=parameters|parameters={{Uselang}}}}</span> ]</indicator>

Thank you, Maculosae tegmine lyncis (talk) 09:30, 15 March 2022 (UTC)

@Maculosae tegmine lyncis Hi, sorry for the late reply. I haven't fully tested it, but I think something like the following will work: {{#invoke:Transcluder|main|{{BASEPAGENAME}}|only=parameters|parameters=^{{Uselang}}$}}. That is, use a regular expression with ^ and $ to indicate the start and end of the parameter. Cheers! Sophivorus (talk) 15:20, 2 December 2022 (UTC)

Template-protected edit request on 19 May 2022 - Regex casing

This is a change to some of the regexes. Ive noticed that for some reason, the onlyinclude tag is case sensitive. For example, transclusing the content on this revision of my page would return the content 13</onlyincludE> (Yes, even the commented onlyinclude took effect. Not gonna ask). Do note however that includeonly and noinclude are not case sensitive. (Try transcluding this revision of my page and the results are perfectly normal). Implemented on the sandbox. Aidan9382 _(talk) 10:39, 19 May 2022 (UTC)

 Done. P.I. Ellsworth - ed. ^put'r there 13:06, 19 May 2022 (UTC)

Respecting the order in getParameters

Hey! I had recently come across a situation in which I needed the parameters of a template parsed, but with the order tracked due to later processing reasons, so I had to add a specialised local version of this module's getParameters which did so. I was wondering, do you think it would be worth it to include the order as one of the returns? For backwards compatability reasons, it'd need to be the 3rd value returned as to not ruin the function of current templates that dont need or want the order, but I think it might be worth having this as an option. An example of how this works is in the sandbox. Aidan9382 _(talk) 15:25, 7 October 2022 (UTC)

@Aidan9382 I just deployed your changes, thanks! Sophivorus (talk) 15:11, 2 December 2022 (UTC)

overuse of ucfirst/lcfirst

the local function matchFlag

local function matchFlag(value, flags)
	if not value then return false end
	value = tostring(value)
	local lang = mw.language.getContentLanguage()
	for flag in pairs(flags) do
		if value == tostring(flag)
		or lang:lcfirst(value) == flag
		or lang:ucfirst(value) == flag
		or ( not tonumber(flag) and mw.ustring.match(value, flag) ) then
			return true
		end
	end

end

could benefit from having invariants removed from the for loop eg

local function matchFlag(value, flags)
	if not value then return false end
	value = tostring(value)
	local lang = mw.language.getContentLanguage()
	local lcvalue = lang:lcfirst(value)
	local ucvalue = lang:ucfirst(value)
	for flag in pairs(flags) do
		if value == tostring(flag)
		or lcvalue == flag
		or ucvalue == flag
		or ( not tonumber(flag) and mw.ustring.match(value, flag) ) then
			return true
		end
	end
end

This is a function that is used frequently within this module. Desb42 (talk) 14:01, 2 December 2022 (UTC)

@Desb42 I just deployed your changes, thanks! Sophivorus (talk) 15:11, 2 December 2022 (UTC)

Another ucfirst/lcfirst change

The following function can be improved (I believe)

local function removeSelfLinks(text)
	local lang = mw.language.getContentLanguage()
	local page = escapeString(mw.title.getCurrentTitle().prefixedText)
	text = text
		:gsub('%[%[(' .. lang:ucfirst(page) .. ')%]%]', '%1')
		:gsub('%[%[(' .. lang:lcfirst(page) .. ')%]%]', '%1')
		:gsub('%[%[' .. lang:ucfirst(page) .. '|([^]]+)%]%]', '%1')
		:gsub('%[%[' .. lang:lcfirst(page) .. '|([^]]+)%]%]', '%1')
	return text
end

by moving the uc/lcfirst to

local function removeSelfLinks(text)
	local lang = mw.language.getContentLanguage()
	local page = escapeString(mw.title.getCurrentTitle().prefixedText)
	local ucpage = lang:ucfirst(page)
	local lcpage = lang:lcfirst(page)
	text = text
		:gsub('%[%[(' .. ucpage .. ')%]%]', '%1')
		:gsub('%[%[(' .. lcpage .. ')%]%]', '%1')
		:gsub('%[%[' .. ucpage .. '|([^]]+)%]%]', '%1')
		:gsub('%[%[' .. lcpage .. '|([^]]+)%]%]', '%1')
	return text
end

A slightly more controversial change is to check if the text contains '[[' at all

local function removeSelfLinks(text)
	if text:find('%[%[') then -- any links at all?
		local lang = mw.language.getContentLanguage()
		local page = escapeString(mw.title.getCurrentTitle().prefixedText)
		local ucpage = lang:ucfirst(page)
		local lcpage = lang:lcfirst(page)
		text = text
			:gsub('%[%[(' .. ucpage .. ')%]%]', '%1')
			:gsub('%[%[(' .. lcpage .. ')%]%]', '%1')
			:gsub('%[%[' .. ucpage .. '|([^]]+)%]%]', '%1')
			:gsub('%[%[' .. lcpage .. '|([^]]+)%]%]', '%1')
	end
	return text
end

Desb42 (talk) 16:43, 2 December 2022 (UTC)

@Desb42 Deployed, thanks again! Sophivorus (talk) 13:42, 23 January 2023 (UTC)

Confusing escape logic in getParameters

Hey! I've recently been trying to diagnose an issue within a separate module that relies on Transcluder, and I've pinned the issue down to the getParameters function. I'm specifically looking at link processing in a template here, so unsure if this effects subtemplate processing too. The main issue here appears to be escapeString( mw.ustring.gsub(link, '%%', '%%%%') ) on this line which was added in this revision. This appears to be over-escaping the string (turning something like "[[:File:20200324 Global average temperature - NASA-GISS HadCrut NOAA Japan BerkeleyE.svg#Pairwise correlation|pairwise correlations exceeding 98%]]" into "%[%[:File:20200324 Global average temperature %- NASA%-GISS HadCrut NOAA Japan BerkeleyE%.svg#Pairwise correlation|pairwise correlations exceeding 98%%%%%]%]", even though escaping "%" should really lead to "%%", not "%%%%", and this causes the code to treat the wikilink's pipe like a parameter pipe as it's never escaped properly. @Sophivorus: was this intentional and fixed some other bug, or is this safe to just plainly revert? (And if the latter, could you do so?) Thanks. Aidan9382 _(talk) 13:52, 18 January 2023 (UTC)

Note: After some testing, it seems this is required or else it does some weird capturing logic, but this does need some fixing per the above bit. Once I've found a proper solution I'll post an edit request. Aidan9382 _(talk) 08:39, 23 January 2023 (UTC)

Edit request

Changes are on the sandbox. This implements 2 fixes for getParameters()

1. Fixes the way % is handled when hiding pipes in wikilinks (previously, if a % was present, the link would fail to gsub, leading to pipes in the link not being escaped and instead being interpreted like the start of a new parameter), which fixes the issue described above
2. Escapes = in subtemplates and links while processing parameters to avoid accidentally treating them as defining parameters (e.g. thinking [[Page|String=with equals]] means "[[Page|String" = "with equals]]" instead of 1="[[Page|String=with equals]]")

I've tested the fix using Template:Transclude lead excerpt/testcases to ensure nothing broke and my sandbox to ensure the fix actually fixes anything. These are testable further using the Debug Console while editing and the code on said sandbox (look inside the collapsable box). Thanks. Aidan9382 _(talk) 10:11, 23 January 2023 (UTC)

 Completed. Editor Aidan9382, thank you sincerely for your edits! Have you considered getting the TE bit? P.I. Ellsworth , ed. ^put'r there 12:49, 23 January 2023 (UTC)

@Aidan9382 Thanks for the bug report and the fixes! Sorry for not replying on time, I guess I'm not as active as before. If I recall correctly, the double escaping %%%% you removed used to fix this issue with URLs that contain % characters, but after your fix, the issue doesn't seem to have reappeared, so we're good. BTW, if you decide to request the template-editor permission, you have my support, cheers! Sophivorus (talk) 12:52, 23 January 2023 (UTC)

To editor Sophivorus: No problems, and glad to hear it hasn't caused any other issues. I imagine URLs should would work fine, as its only escaped so it can be gsubbed safely (meaning it basically immediately unescapes), but if it turns out something does crop up, I'll be sure to refine it further (though hopefully that won't happen).

To editor Paine Ellsworth: Thanks for implementing this. I have, though I'm just a little short on the guidelines. Once I'm past that though, I probably will, especially if Module:Excerpt/portals starts picking up more transclusions (its close to the 5k TE protection, and it seems I can't get away from it just yet). Aidan9382 _(talk) 13:00, 23 January 2023 (UTC)

Didn't realize – it seems like you've been around for longer. Learning a lot from you. Happy New Year and Best of Everything to You and Yours! P.I. Ellsworth , ed. ^put'r there 13:10, 23 January 2023 (UTC)

Numerical keys in getParameters

Hey! Currently, when getParameters receives a numerical key, the type it returns can vary. For example, if a template went like {{ABC|X|Y}}, getParameters would return {[1]="X", [2]="Y"}, but if the keys were explicitly defined like {{ABC|1=X|2=Y}}, you would instead get {["1"]="X", ["2"]="Y"}. Should this be standardised to always be a string/number, and which? Aidan9382 _(talk) 10:41, 21 February 2023 (UTC)

@Aidan9382 Agree! I guess I'd favor numbers rather than strings, per being cleaner? However, if you decide to implement it yourself, feel free to choose whichever you like. You may want to know that the master module has several testcases that you can make use of and extend. As to me, I've taken note of the issue and will implement it eventually, someday, if you don't do it before. I usually work by "streaks", so it may take me some time but eventually I'll do a catch up with this and several other ideas I got in mind. Cheers! Sophivorus (talk) 14:39, 21 February 2023 (UTC)

@Sophivorus: if you want it numerical, I've theorised it on the sandbox (note that it specifically matches integers >=1, not all numbers like -1.5). I'm wondering if it's worth having an __index metatable that would automatically try to find the numerical index if given a string, purely to make working on the parameters with template inputs to be more convenient since they always give strings (wouldn't be overly complicated to implement), but I'm undecided - though I think keeping it numerical either way is preferable, since that allows for ipairs to be used, which is probably helpful. Thoughts? Aidan9382 _(talk) 17:02, 21 February 2023 (UTC)

On second thought, I think just forcing numerical is fine. It would definitely be more natural to reference numerical parameters using a number, and so forcing strings is probably the wrong move (and likely no good for backwards compatability too). The metatable is probably overkill - on the rare occasion someone needs to work with template inputs in such a specific way that this sort of key type consistency is required, that can be handled pretty easily on that module's end (like here). Aidan9382 _(talk) 15:22, 1 March 2023 (UTC)

Avoid interpreting no-value keys as numerical index values in getParameters

Somehow keep finding more bugs to do with getParameters. Currently, if getParameters is presented with a situation like {{Test|A=|B= |C=}}, it'll return {[1]="A", [2]="C" ["B"]=""}. The behaviour of B goes as expected, but A and C are interpreted as numerical indexes. This is because of the check seeing if its an empty string after a concatenation not considering that the given value could've just been empty. I've changed it in the sandbox to properly account for this.

(When implementing the changes, avoid implementing the changes on lines 289-291 on the sandbox, as that bit is to do with the discussion above and not this bugfix). Aidan9382 _(talk) 12:58, 23 February 2023 (UTC)

Nevermind, I can implement it myself now.  Done, I suppose. Aidan9382 _(talk) 08:53, 24 February 2023 (UTC)

To editor Aidan9382: the TE bit couldn't have happened to a better editor! Gonna miss your TERs Congrats! P.I. Ellsworth , ed. ^{put'er there} 20:05, 24 February 2023 (UTC)

Appreciated! Wasn't even expecting to get it this soon, but it ended up coming to me. Hopefully gonna get around to handling some edit requests myself at some point once I've got a good idea on how to handle em. Aidan9382 _(talk) 21:02, 24 February 2023 (UTC)

Square brackets retained despite links=no parameter.

Moved from Module talk:Excerpt § Square brackets retained despite links=no parameter.

I recently used {{excerpt|Frenemies (podcast)|Label 1=Frenemies|links=no|hat=no}} This may well an error of implementation on my part but it returned the following paragraph with open square brackets at the start of the first (parent page) wikilinked term.

Frenemies is a podcast hosted by Trisha Paytas and Ethan Klein on the H3 Podcast. Running from September 15, 2020 to June 8, 2021, the podcast focused on discussing personal experiences, pop culture, internet drama and mental health. It consisted of 39 episodes and two accompanying vlogs. Paytas and Klein's friendship was occasionally tumultuous, leading to episodes that were incomplete due to a verbal argument. The podcast came to an end during its 39th episode, after a dispute with Paytas and Klein about podcast revenue and production ownership. Despite this, Frenemies gained recognition in the podcasting realm and found a place in numerous Top 50 lists, frequently pulling in millions of viewers per episode.

References
References

I manually pasted the content into the article, but I thought I'd bring it up in-case it is a bug. Pabsoluterince (talk) 12:45, 2 August 2023 (UTC)

@Pabsoluterince You might want to post this over at Module talk:Transcluder, as that's the module that actually handles retrieving the text and removing the wikilinks. --Ahecht (TALK
PAGE) 13:47, 2 August 2023 (UTC)

At Module:Transcluder#L-584, [^|] should probably become [^%]|], so it doesn't mistake [[Link1]] and [[Link2|Pipetext]] for a single wikilink to article called "Link1]] and [[Link2". I can't verify this easily, because previewing a module change without template editor privilege requires a lot of tedious copying to titles we can edit. On reflection, doesn't that whole block require a lot of % escapes for magic characters such as [ and ]? Certes (talk) 13:48, 2 August 2023 (UTC)

I've added the missing %] to Module:Transcluder/sandbox, which seems to fix the problem. I've also created an older version which adds further missing % signs to escape other magic characters in character sets. Looking at Template:Excerpt/testcases with either version shows lots of invisible differences. I think, but am not sure, that these are all adding &editintro=Template:Excerpt/editintro to the [edit] links on one side only and differing ^[numbers] on the citations. Certes (talk) 14:41, 2 August 2023 (UTC)

The differences in testcases were already present. A simple copy-paste without formatting shows the same differences [sic] before and after my change. Certes (talk) 14:48, 2 August 2023 (UTC)

@Pabsoluterince Bugfix deployed @Certes Thanks as always. As to your older version with further bugfixes, I'll take a closer look asap, but I believe that sometimes escaping with % is not necessary when the regex structure can only mean that the character must be escaped, as in [^]], which doesn't need to be [^%]], cheers! Sophivorus (talk) 15:27, 2 August 2023 (UTC)

This module isn't documented elsewhere

This is a very useful module, but it isn't mentioned in Help:Transclusion or Help:Labeled section transclusion. Should these pages be updated to describe it?

Both of them mention {{Excerpt}}, which is the easiest way to access Module:Transcluder's functions unless you're writing another Lua module. Those help pages are more about how to use transclusion than internal implementation details, so I think that's the right level to link at. Perhaps the links could be more prominent. Certes (talk) 10:58, 1 September 2023 (UTC)