Talk:Local development quickstart

macOS and PHP

Since macOS Monterey, PHP is no longer preinstalled. So perhaps that should change to reflect an install from Homebrew or Macports or something else if that is preferred. TTaylor (WMF) (talk) 13:50, 31 October 2023 (UTC)

Reference: php.net installation on macOS TTaylor (WMF) (talk) 13:51, 31 October 2023 (UTC)

Thanks! I'll fix that. APaskulin (WMF) (talk) 16:26, 1 November 2023 (UTC)

I'd really like to add apcu for Mac, but that requires several additional steps. Perhaps I should add that to Manual:Running MediaWiki on macOS. —TheDJ (Not WMF) (talk • contribs) 10:17, 23 November 2023 (UTC)

I do think that it's too many extra steps for this guide, but I think adding it to Manual:Running MediaWiki on macOS is a great idea. APaskulin (WMF) (talk) 21:10, 4 December 2023 (UTC)

Serving

There's a composer script called serve for running php -S localhost:4000, if that's easier. It doesn't make it possible to change the port, is probably the main drawback. Sam Wilson 03:26, 22 November 2023 (UTC)

Good point! I had considered this but felt that it obscured the function of the command unnecessarily and that using the full command provided the option to change the port if the user needed to. But maybe it would be worth including it as an option since it is easier to type. APaskulin (WMF) (talk) 18:54, 22 November 2023 (UTC)

@APaskulin (WMF): Yeah, I do like the fact that it makes it clear that it's a pretty simple command, that's a good point. Sam Wilson 06:36, 23 November 2023 (UTC)

I'd also suggest going with composer serve, for the same reason that we also wrap the contents of the install.php invocation. Also, we might want to update the composer servecommand to include includes/dev/router.php. KHarlan (WMF) (talk) 07:22, 24 November 2023 (UTC)

@APaskulin (WMF) @Krinkle comopser serve now sets the log directory, prints logs to the console, and sets the number of CLI workers. So IMHO we should update the docs to use composer serve. KHarlan (WMF) (talk) 12:56, 16 January 2024 (UTC)

Thanks, @KHarlan (WMF)! Switching to composer serve sounds good to me. Feel free to make the updates, otherwise I can make them next week. APaskulin (WMF) (talk) 02:02, 20 January 2024 (UTC)

Node / npm

You really need node/npm, even grunt for some of the development work. Do we want to add that here as well ? Shouldn't be too many additional packages right ?

If not, I suggest we add a section with what this does NOT do (npm, performance optimisation, php debugging, MariaDB, thumbnailing etc) and then from that section, we can link to further instructions ? —TheDJ (Not WMF) (talk • contribs) 10:19, 23 November 2023 (UTC)

Or we point people directly at fresh.. but that brings a few very MW specific things into the picture again, which I guess we don't really want here ? —TheDJ (Not WMF) (talk • contribs) 10:39, 23 November 2023 (UTC)

That's a good point, but considering the extra steps, I'd rather link out to docs that cover this setup in the Next steps section. For now, we're aiming for parity with MediaWiki Docker, which doesn't include this extra setup. We currently have a link to Manual:How to debug, which covers some of what you mentioned. Are there any other pages that you think would be useful for contributors new to MediaWiki? APaskulin (WMF) (talk) 21:23, 4 December 2023 (UTC)

Use shallow clone

I suggest using --clone-depth=1 for cloning MW and Vector. That takes 13 seconds or so instead of several minutes. After the initial clone, you could make the MW repo "unshallow" (a full clone). For example, this command would use a shallow clone, and then put the multi-minute process of obtaining a full clone in the background:

git clone https://gerrit.wikimedia.org/r/mediawiki/core.git --depth=1 mediawiki && git clone --depth=1 https://gerrit.wikimedia.org/r/mediawiki/skins/Vector.git mediawiki/skins/Vector && cd mediawiki && echo "Fetching full MediaWiki code in background; you may proceed" && git fetch --unshallow > /dev/null 2>&1 & KHarlan (WMF) (talk) 07:10, 24 November 2023 (UTC)

On further reflection, the git fetch --unshallow > /dev/null 2>&1 & is maybe too confusing and magical. What if we use the --clone-depth=1 argument and then in "Next steps" we offer to the user that they should run git fetch --unshallow in the core and Vector directories? KHarlan (WMF) (talk) 08:43, 29 November 2023 (UTC)

I love this idea! But unfortunately, I think it's too confusing to be worth the few minutes it saves. APaskulin (WMF) (talk) 21:32, 4 December 2023 (UTC)

I thought also about this option to accelerate the download (and reduce the advertised time "Install MW in 7 minutes"), but it makes the command line longer (probably already a bit long for some newbies) and it would be a bit difficult to explain in next steps, particularly given there are two repositories.

Another option would be to add a quick note "if the download is too long, you can use this command line: git clone --depth=1 …" but it would complexify the quickstart (btw I love the current state with only 4 commands excluding dependencies). ~ Seb35 [^_^] 11:43, 14 August 2024 (UTC)

The shallow/unshallow part of the git command could be behind a composer command, so the user doesn't really need to be aware of it. Writing from somewhere with very slow download speeds: I do think it's valuable to make the initial clone and setup as fast as possible. You could make the "unshallow" part automatically initiated as part of running "composer serve". KHarlan (WMF) (talk) 08:27, 21 August 2024 (UTC)

Multiple workers for PHP server

We might want to suggest to the user that they set e.g. PHP_CLI_SERVER_WORKERS=4 per the documentation in https://www.php.net/manual/en/features.commandline.webserver.php. So the command would be PHP_CLI_SERVER_WORKERS=4 composer serve Not sure if it will make a big different in practice. KHarlan (WMF) (talk) 07:24, 24 November 2023 (UTC)

This is a good point. I wasn't able to reproduce any slowness or stalling with the single-threaded process when I tested using Visual Editor locally. Have you observed any slowness or stalling? I think it's a good idea to add PHP_CLI_SERVER_WORKERS=4 within the composer serve command, but I'd like to verify that we're seeing issues because of this limitation. APaskulin (WMF) (talk) 21:34, 4 December 2023 (UTC)

I don't see any appreciable difference with multiple CLI server workers (roughly tested by loading Main_Page as a logged out user with HTTP cache disabled and looking at DOMContentLoaded / load times, with and without the multiple workers flag). So, I think we can leave out this flag. cc @Krinkle in case you have a different opinion. KHarlan (WMF) (talk) 13:12, 12 December 2023 (UTC)

@KHarlan (WMF) The next thread on this page talks about the "blocked requests" caveat, which is what raising the default worker count would address. If we're comfortable with the default, and are confident enough that this does not warrant a note of any kind, we could leave it that way. However, for those using the composer serve command, it'd cost nothing to add it regardless.

In my own testing, I do see a clear difference in timeline, but the overall load times are indeed quite close, especially if you're used to a Docker or Vagrant VM.

• Page load Before, After.
• VisualEditor load Before, After.

Note how "After" isn't just a bit faster, the latency numbers actually reflect the time for that logical request to generate a respond from MediaWiki, instead of accumulating the queue time for the previous request, which is not obvious to most devs I imagine.

Broadly:

• More production-like. It makes waterfall charts more realistic, more comparable between environments, and between local and production.
• Less confusing. When analyzing performance on a single-threaded server, concurrent requests will appear to be slower when really they are just waiting. You and I know this, but it's a needless barrier that people have to be "in the know" about, before which they will make incorrect measurements and conclusions. They might even conclude that "composer" is slower than some other environment, when it's actually baremetal and close to the fastest it can be.
• A tiny bit faster. Every millisecond counts :)
• Allows benchmarking out of the box. I'm working with Derick at the moment on benchmarking the impact of a change. In doing so, I'm suggesing to use the composer setup instead of Docker, as it has far less variance and more consistent response times, however, as written he'd have to set ab -c1 (to avoid contaminating data with stalled requests) or to dirty the working state by setting custom workers. If we instead set PHP_CLI_SERVER_WORKERS=8 by default, then you can actually run a reasonably large benchmark in a matter of seconds rather than minutes. It's one thing to be able to notice "hey this sets workers to 8" and use that, then to have it not set anything, and having to know that that means there is no concurrency. The former is more transparent and obvious imho. I'd almost recommend setting WORKERS=1, except that if we do, we might as well set it to 8.

These are all minor points, but given it's so easy to add, I'd say why not? Krinkle (talk) 16:00, 19 December 2023 (UTC)

Sounds good to me. (Went to make a patch and then found https://gerrit.wikimedia.org/r/c/mediawiki/core/+/984225, thanks @Krinkle!) KHarlan (WMF) (talk) 07:20, 20 December 2023 (UTC)

Warn about blocked requests?

Maybe this is out of scope for the quick start document, but wanted to flag that the built-in server manual warns:

The web server runs only one single-threaded process, so PHP applications will stall if a request is blocked.

In the past, I saw this cause problems for extensions like VisualEditor, which made internal API requests as part of the process for saving an edit. That doesn't seem to be an issue anymore, but we might want a "Troubleshooting" section which explains that if the page stops loading, one possible cause is the single-threaded process. KHarlan (WMF) (talk) 07:58, 24 November 2023 (UTC)

This is related to my comment in response to your previous comment, but I think adding it to a troubleshooting section is a great idea. APaskulin (WMF) (talk) 21:41, 4 December 2023 (UTC)

Load DevelopmentSettings.php from the outset

Without that, it's really hard for a novice to figure out why errors are occurring, or to get more detailed feedback from someone else. I'd suggest a one-liner as part of composer mw-install:sqlite that adds this to LocalSettings.php. KHarlan (WMF) (talk) 08:00, 24 November 2023 (UTC)

Patch proposed in https://gerrit.wikimedia.org/r/c/mediawiki/core/+/977136 KHarlan (WMF) (talk) 08:44, 29 November 2023 (UTC)

Great idea! Thanks for the patch! APaskulin (WMF) (talk) 21:41, 4 December 2023 (UTC)

Help user set up git for pushing patches

At the end of this guide, the user will have cloned core and Vector with https, but in order to push a patch for review, they'll need to switch to e.g. ssh://USERNAME@gerrit.wikimedia.org:29418/mediawiki/core in their .git/config. It might be worth calling that out specifically (Gerrit/Tutorial#Set up Git) in next steps. KHarlan (WMF) (talk) 08:48, 29 November 2023 (UTC)

Great point! I'll add a note about this. APaskulin (WMF) (talk) 21:45, 4 December 2023 (UTC)

@php or php

I was looking at the composer magic commands (because of @putenv), and it made me wonder if we should switch composer to use @php instead of directly calling php. It seems that is how laravel is doing it https://github.com/laravel/laravel/blob/07fdfbc8d8d88440634bd1c42075653dbb5df402/composer.json#L38 —TheDJ (Not WMF) (talk • contribs) 09:21, 20 December 2023 (UTC)

Let's go with @php per the Composer docs. KHarlan (WMF) (talk) 09:30, 22 December 2023 (UTC)

job runner

The job runner rate is rather low by default, which can get in the way. Maybe we can provide some sort of default setup for this as well ? I was looking if maybe we could run parallel commands, to also have a job runner when running serve, but that doesn't seem to be very easy to do with composer. And if we need to list it as a separate composer command we might as well list it as a direct instruction or link to it. Anyone else have ideas ? —TheDJ (Not WMF) (talk • contribs) 09:32, 20 December 2023 (UTC)

I’m afraid asking the newbie developer to run a second command would be too complicated. I aggree the default value for $wgJobRunRate = 1 is quite low and depending of type of non-executed jobs it may produce unexpected behaviours (I have in mind when CirrusSearch is initialized – I know it is an advanced extension – there are no search results until the jobs are executed).

An intermediate solution would be to increase $wgJobRunRate to run more jobs through the Web interface. ~ Seb35 [^_^] 13:58, 14 August 2024 (UTC)

Docker

Maybe a Docker based solution can be useful for some, guess some parts of it can be improved by images that wmf also provide to enhance the experience, here is my recipe at least, maybe would be useful for myself in future at least,

git clone https://gerrit.wikimedia.org/r/mediawiki/core.git mediawiki && cd mediawiki
git clone https://gerrit.wikimedia.org/r/mediawiki/skins/Vector.git skins/Vector

# Create a container named mediawiki from debian image and use bash just to keep it alive
docker run -it -d -p 4000:4000 --expose 4000 --volume $PWD:/app -w /app --name mediawiki debian bash

# Update
docker exec -it mediawiki apt update

# Install required things
docker exec -it mediawiki apt install -y php php-intl php-mbstring php-xml php-apcu php-curl php-sqlite3 composer

# update
docker exec -it mediawiki composer update

# sqlite
docker exec -it mediawiki composer mw-install:sqlite

# run
docker exec -it mediawiki php -S 0.0.0.0:4000

ebrahim^talk 21:14, 19 June 2024 (UTC)

Unless you can’t/don’t want to use Docker Compose for some reason, there’s an official solution: MediaWiki-Docker. —Tacsipacsi (talk) 08:16, 20 June 2024 (UTC)

Tacsipacsi: Fantastic, I'm using this now, thanks for letting me know about my blind spot 😊 −Ebrahim^talk 12:08, 27 June 2024 (UTC)

I created Category:Development environments as an initial attempt to improve the discoverability of various systems that currently exist. I'm not sure if these things rise to the level of importance of working on nav templates to connect them all as well or not at this point, but that's something folks could think about. -- BDavis (WMF) (talk) 15:57, 27 June 2024 (UTC)

At the end instruct about the generated files

I just tried this quickstart and it’s really easy, contragulations to those who created it!

At the end, is it worth to instruct the developer what are the generated and important files? I think about:

• the LocalSettings.php: where to configure MW,
• cache/my_wiki*.sqlite: the database files, may be useful if the developer want to reinit the DB
• logs: for the debug

Also, I don’t know if this page is the right page, or if it would be better in a follow-up page, but at the end it is may be instructed to modify $wgSitename or $wgLanguageCode to see the visible effect of some parameters. ~ Seb35 [^_^] 14:22, 14 August 2024 (UTC)