User:Want/Is ifexist expensive function?
 | This is an essay. It expresses the opinions and ideas of some mediawiki.org users, but may not have wide support. Feel free to update this page as needed, or use the discussion page to propose major changes. |
I recall, MediaWiki.org originally was intended to serve as a source of usable templates and help pages for administrators of independent MediaWiki instances. Today exists here 12,860 templates, but most of them practically impossible usable.
Anyone who has tried it knows that it's not easy. You need to manually copy the template, find the dependencies, copy the missing templates and modules, and finally fix the code according to the situation. Here most templates uses modules of the Extension:Scribunto which complicated it. – Why?
MediaWiki.org does not use the advanced ParserFunctions, loops, user variables, etc. provided by native PHP extensions because someone in the past thought they were a risk. I understand it. The complexly template in the pure wikicode are not very clear (brackets, brackets, brackets,..) and a small error in the code of the frequentaly uses template, as {{Ll}} for example, can cause the entire website to be unavailable. From this perspective, it's posed a risk.
Scribunto (development start in 2012), allows you to write scripts directly into wiki pages (modules) and processed this code through an independent server-side interpreter. But isn't it a ticking time bomb in terms of MediaWiki development?
The advantage of native PHP extensions is that they are not dependent on other software. If the performance of PHP improves, their performance will automatically improved too. In 2019, at the Wikimedia Hackathon in Prague, I learned that the deployment of a new version of PHP (from 5.x to 7.x), which according to tests was supposed to bring a significant improvement in performance, had practically no effect. – Why?
Problem of the Main page Mediawiki.org
[edit]I recently stirred up the murky waters here by trying to solve a problem that unnecessarily discourages beginners who don't speak English well. The language bar on the main page does not switch languages subpages as usual, but do links to the translatable template. Moreover, it does not apply to all translations. I've been trying to figure out where this is coming from to possibly fix it, but I haven't been able to find it.
But that's not the only problem of this language bar. If you have a small display (e.g. on a mobile phone) it's not visible because it is at the very bottom, outside the screen. So I used the {{Language}} template, which although switches languages correctly, but also doesn't refer to all the languages into which the content of the main page is translated and started working on a new {{Mt}} template that would she solved it.
My change sparked the discussion (see Topic:Y8l01lc8uein5fng), that showed that is there a long-term unresolved problem. And I was surprised at how expensive templates MediaWiki.org uses.
The following table compares the performance of my Template:ll and local template {{Ll}}. I'm not using caching and yet the values are very similar. However, my template has a much wider use, because its output can be used as the input of another template. Real examples of use are described in documentation, which is of course translatable.
| Limit | Template:Main page | MediaWiki | Woodcraft | Hlavní strana | |
|---|---|---|---|---|---|
| Templates | {{Ll}}, {{Localized link}}, {{Pagelang}}, {{TNTN}}, {{Translatable}}, {{Translatable/en}}, {{Translatable template name}} Module:Template translation and message with title | {{Ll}}, {{Localized link}}, {{Pagelang}}, {{Translatable}}, {{Translatable/en}} and Module:Template translation | Template:ll, Template:pagelang | Template:ll, Template:pagelang | |
| CPU time usage | 0.028 seconds | 0.035 seconds | 0,023 sekund | 0,043 sekund | |
| Real time usage | 0.041 seconds | 0.051 seconds | 0,049 sekund | 0,053 sekund | |
| Preprocessor visited node count | 1,000,000 | 57 | 57 | 58 | 58 |
| Post-expand include size | 2,097,152 bytes | 253 bytes | 253 bytes | 304 | 414 |
| Template argument size | 2,097,152 bytes | 93 bytes | 93 bytes | 27 bajtů | 42 bajtů |
| Highest expansion depth | 100 | 11 | 11 | 9 | 9 |
| Expensive parser function count | 500 | 1 | 1 | 2 | 2 |
| Unstrip recursion depth | 20 | 0 | 0 | 0 | 0 |
| Unstrip post-expand size | 5,000,000 bytes | 0 bytes | 0 bytes | 0 bajtů | 0 bajtů |
| Number of Wikibase entities loaded | 400 | 0 | 0 | unused | unused |
| Lua time usage | 10.000 seconds | 0.003 seconds | 0.004 seconds | unused | unused |
| Lua memory usage | 52,428,800 bytes | 662,697 bytes | 662,697 bytes | unused | unused |
| ExtLoops | 2600 | unused | unused | 0 | 0 |
Languagebar
[edit]I've written a lot of templates for my multilingual wiki, but most of them can't be used on MediaWiki.org because used functions unavailable there. But the {{Mt}} template can be compared because was wroted for using on MediaWiki.org.
First tested on my wiki. After placing it on MediaWiki.org, I noticed that when applied to all language codes, page languagebar exceeds the Expensive parser function count, which limited to 500. So I was looking for another way to solve it and found Manual:Checking for page existence.
A redesigned template is embedded here as {{Mu}}. It uses the #ifeq function instead of the expensive #ifexists function. And since it does the same thing and uses the same parameters, it was possible to do following test pages:
You can do yourself test, if you do any change in code and use only the preview.
Test existence by #ifeq is wrong!
[edit]When I tried the {{Mu}} template, I was surprised that the processing of the User:Want/languagebar page code crashed after 15 iterations, because limit the post-expand include size was full. This is really proof that using an expensive function #ifexist is a more reasonable choice than check existence of the page by #ifeq function, which does not count towards the number of expensive parser functions!
| Limit | {{Mu}} | |
|---|---|---|
| Status | stopped after 'ang' (15 items done on my wiki done 156 items) | |
| CPU time usage | 25.294 seconds | |
| Real time usage | 27.051 seconds | |
| Preprocessor visited node count | 1,000,000 | 584,178 |
| Post-expand include size | 2,097,152 bytes | 2,097,152 bytes |
| Template argument size | 2,097,152 bytes | 1,752,121 bytes |
| Highest expansion depth | 100 | 35 |
| Expensive parser function count | 500 | 14 |
| Unstrip recursion depth | 20 | 0 |
| Unstrip post-expand size | 5,000,000 bytes | 30,660 bytes |
| Number of Wikibase entities loaded | 400 | 0 |
| Lua time usage | 10.000 seconds | 7.247 seconds |
| Lua memory usage | 52,428,800 bytes | 1,394,903 bytes |
Here is output Lua Profile returned after processing crash:
| recursiveClone | <mwInit.lua:45> | 2820 ms | 38.7% |
| MediaWiki\Extension\Scribunto\Engines\LuaSandbox\LuaSandboxCallback::getExpandedArgument | 1300 ms | 17.9% | |
| MediaWiki\Extension\Scribunto\Engines\LuaSandbox\LuaSandboxCallback::newTitle | 980 ms | 13.5% | |
| ? | 500 ms | 6.9% | |
| type | 380 ms | 5.2% | |
| (for generator) | 360 ms | 4.9% | |
| <mwInit.lua:45> | 160 ms | 2.2% | |
| MediaWiki\Extension\Scribunto\Engines\LuaSandbox\LuaSandboxCallback::expandTemplate | 120 ms | 1.6% | |
| newFrame | <mw.lua:153> | 120 ms | 1.6% |
| tostring | 80 ms | 1.1% | |
| [others] | 460 ms | 6.3% |
Is the function #ifexist expensive?
[edit]So I did one more test. For one item only.
| Limit | #ifexist (User:Want/Mt af) | #ifeq (User:Want/Mu af) | |
|---|---|---|---|
| CPU time usage | 0.010 seconds | 0.194 seconds | |
| Real time usage | 0.014 seconds | 0.248 seconds | |
| Preprocessor visited node count | 1,000,000 | 42 | 2,629 |
| Post-expand include size | 2,097,152 bytes | 220 bytes | 109,357 bytes |
| Template argument size | 2,097,152 bytes | 85 bytes | 5,838 bytes |
| Highest expansion depth | 100 | 10 | 31 |
| Expensive parser function count | 500 | 2 | 12 |
| Unstrip recursion depth | 20 | 0 | 0 |
| Unstrip post-expand size | 5,000,000 bytes | 0 bytes | 0 bytes |
| Number of Wikibase entities loaded | 400 | 0 | 0 |
| Lua time usage | 10.000 seconds | unused | 0.080 seconds |
| Lua memory usage | 52,428,800 bytes | unused | 1,303,599 bytes |
| Lua Profile | unused | unused |
Because the User:Want/languagebar page has same effect as the {{Language}} template. I can do next test. Difference of it, that the User:Want/languagebar page works with all 452 language codes. However the {{Language}} handles only 72 of them. That was the reason to do two tests: First for full set of language codes and second for set language codes as used by the {{Language}} template.
| Limit | {{Language}} | {{Mt}} | |
|---|---|---|---|
| Note | only 72 items done | all 452 languages (same set language codes as {{Language}}) | |
| CPU time usage | 0.695 seconds | 0.903 seconds (0.141 seconds) | |
| Real time usage | 0.960 seconds | 1.225 seconds (0.224 seconds) | |
| Preprocessor visited node count | 1,000,000 | 10,126 | 29,261 (6,651) |
| Post-expand include size | 2,097,152 bytes | 164,062 bytes | 106,155 bytes (23,426 bytes) |
| Template argument size | 2,097,152 bytes | 43,298 bytes | 60,846 bytes (11,396 bytes) |
| Highest expansion depth | 100 | 18 | 13 (13) |
| Expensive parser function count | 500 | 243 | 548 (134) |
| Unstrip recursion depth | 20 | 0 | 0 (0) |
| Unstrip post-expand size | 5,000,000 bytes | 7,685 bytes | 0 bytes (0 bytes) |
| Number of Wikibase entities loaded | 400 | 0 | 0 (0) |
| Lua time usage | 10.000 seconds | 0.410 seconds | unused |
| Lua memory usage | 52,428,800 bytes | 1,692,425 bytes | unused |
You can see for yourself that my template, which does not use the Lua interpreter, is 5x faster and much less memory intensive. The only parameter that limits its use for all language codes used by MediaWiki is more lower the limit for expensive parser functions. My wiki use 2000 for it. Here is set only 500. It is not possible to process all language codes without exceeding of it!
Template as a tool
[edit]My own wiki use templates very extensively. Not only for dynamic content of my Main page, but also as an auxiliary tool.
For example, Template:transstat generate calculations for paid translators. For links to specific places in the content of a site, I widely use navigation templates, because my version of the Template:ll supports anchors across all languages of content.
I use specialized templates that allow the page content to be translated into other languages, although the page itself is not translatable because I need the links to have only one destination at any given time. See for example 3AA3 – There's a lot waiting to be translated, but I'm working for the future. Even a beginner can translate a text. But only an experienced user can write documentation on how to use the features of a new template or extension.
I continually write own translatable help pages, for functionalities what the MediaWiki.org not documented. Info about technical changes wiki give the page Current events. There you can find all the information about updates and installed extensions from November 2013, when the wiki was launched, until now. You can try my widget, which generate link and send page to google translate.
Unfortunately, nothing from it can be used on MediaWiki.org because not installed the necessary extensions. But anyone who maintains own wiki can use it, if installed needed extensions and register and log in to my wiki – that's the only way he can get access to the code, which he can copy. Anonymous visitors do not have access to it.
- Note: The dependency of the template on the extension can be recognized even without logging out of the categorization.
Singlepages
[edit]I use expensive features quite heavily on my wiki. And you can test for yourself how fast the pages load. The most expensive page my wiki is "The Book of Woodcraft, 1922 (book)", a Singlepage for a book that has 624 pages. Due to these pages, I have increased some limits (see char *)
| Limit | The Book of Woodcraft, 1922 (book) | |
|---|---|---|
| CPU time usage | 3.943 seconds | |
| Real time usage | 4.707 seconds | |
| Preprocessor visited node count | 1,000,000 | 21,415 |
| Post-expand include size* | 6,266,880 bytes | 5,020,421 bytes |
| Template argument size* | 6,266,880 bytes | 17,259 bytes |
| Highest expansion depth | 100 | 13 |
| Expensive parser function count* | 2000 | 0 |
| Unstrip recursion depth | 20 | 1 |
| Unstrip post-expand size | 5,000,000 bytes | 2,608 bytes |
| ExtLoops | 2600 | 638 |
My experiences with MediaWiki
[edit]My first contact with Wikipedia website was begin of August 2004. I work only as user into 2008.
Since 2008 summer I am the administrator of the wiki https://wiki.control.fel.cvut.cz, which is still running today and has been gradually updated to the current version LTS 1.39.x. In the same year 2008, I created the first version of the Extension:AccessControl and started own experimenting with templates. That wiki has a total of 151 templates, of which 56 templates are unused. They were created for testing purposes when I was learning how to write templates. Most of the 52 open non-existent templates are a relic from June 2010 when I tried to import the first templates from wikibooks.
This is proven by calling the 'Template:Hiddenh3' in the template documentation Template:Ref/doc, which the wiki has, but you can find it at wikibooks. In total, this wiki contains 7431 pages (of which 2750 are content) and 3507 files. About 10 of the 206 registered users have been continuously creating content during that time. And thanks to the change history, we could effectively intervene in the event of a patent dispute, as we could prove that a patent application was filed up to 10 years after it was published on our wiki work on the subject.
User, sysop, translator, root, developer and maintainer
[edit]In 2013 I started my own wiki https://www.thewoodcraft.org
Content of it is multilingual from the very beginning and intensive dynamically generated. The content template that generates the content of the main page was created in April 2015. It's a sophisticate of codes, sometime inspired by templates from MediaWiki.org, as the ll template for example, that generates links to multilingual pages. Unlike the {{Ll}} template, support anchors and also responds to the currently selected interface language - if there is a translation of the article title in the relevant language, it will use it.
That wiki has a total of 654 templates. Half of it consists of translations of the specialized template concept, used in translations. 204 templates is by system MediaWiki marked as unused. In fact, it is the already mentioned translations that are called indirectly, through the concept template or the templates to insert code. Those templates can also be used in another instance of MediaWiki, it's pure wikicode, but some of them depend on extensions that MediaWiki.org doesn't support - Scribunto offers the same functionality as them. But is really effective? I don't know.
In total, this wiki contains 38650 pages (of which 7358 are content) and 1872 files. About a third of the 63 registered users continuously participated in the creation of content during that time.
Why I don't use Scribunto on wikis?
[edit]I don't use Scribunto because I see no reason why the wiki should run next interpreter of programming language on the server side. Widgets are something else. They work with javascript, which is interpreted on the user's side by the browser.
I have an advantage over other developers because I don't have to ask anyone if I want to try or test something.
The downside for me is updating the MediaWiki core. I maintain the code myself and the user base is too small to rely on the help of other developers. That's why I stick to code from the official git repositories as much as possible and that's why I'm also the maintainer of the AccessControl and EImage extensions.
My life is complicated only by changes between releases MW, that force changes in the code of the extensions used. I don't have time to continually keep track of partial changes and correct minor errors in the PHP code or in the code that Scribunto interprets.
The project I maintain is not as extensive as the WMF projects. I am mainly creater of content, not programmer or sysop. Every user of my wiki has absolutely freedom of creation. He must only respect others users and changes of their content do only after consultation with hims.
I do templates maximally effective, because the efficient code garantee a good speed of environment. And the host's performance is not unlimited too.
I wrote a lot of manuals to my wiki. And I also contribute to the translations of those on MediaWiki.org - it saves me work.
My work on MW
[edit]Through the page Project:translation/cs, I try to pass on my experience to other translators in order to make the journey as easy as possible for beginners and those who want to run their own multilingual wiki.
I am pleased that, thanks to the Special:ActiveLanguages/cs group of Czech translators, although it is not very numerous, Czech is in fourth place in the ranking of translated content here.