Continuous integration/Tutorials/Adding a new release pipeline for MediaWiki

This is an advanced tutorial, mostly aimed at members of the Wikimedia Release Engineering Team, to add a new pipeline to Wikimedia CI for MediaWiki core, extensions, skins, and a few other repos which are branched to the REL1_xx and wmf/1.XX.X-wmf.xx release and production branches.

In the example, we are using the work done for creating the REL1_34 pipeline (i.e., the release branch for MediaWiki 1.34).

Prerequisites

A Wikimedia Gerrit account to be able to submit changes for review
The branch for which you are adding CI does not need to exist yet, but this shouldn't be done

Editing Zuul configuration

The configuration is held in integration/config repository, in the file zuul/layout.yaml.

The file is made of four sections:

Pipelines: define event filtering and result handling behavior
jobs: finely adjust the behavior on a per job basis
project-templates: define common jobs triggering
projects: define, for a Gerrit repository, the pipelines to uses and the hierarchy of jobs being triggered.

We will need to adjust each of these:

First, define the new pipelines, one for "test" and one for "gate and submit", and define the triggers for them. The test one should go after the previous version's test pipeline:

  - !!merge : *test_pipeline
    name: test-1.34
    description: >
      Variation of the "test" pipeline reserved for MediaWiki 1.34
    trigger:
      gerrit:
        - !!merge : *test-event-patchset-created
          branch: ^REL1_34$
        - !!merge : *test-event-comment-recheck
          branch: ^REL1_34$
        - !!merge : *test-event-plus-one
          branch: ^REL1_34$

… and similarly, the gate-and-submit one should go after the previous version's gate-and-submit pipeline:

  - !!merge : *gate_and_submit_pipeline
    name: gate-and-submit-1.34
    trigger:
      gerrit:
        - !!merge : *gate-and-submit-comment-added
          branch: ^REL1_34

Now you need to determine what the test jobs for this version of MediaWiki should be, so you can ensure that they are reflected in your pipelines. For 1.34, the initial target was identical to 1.33, as the supported PHP versions didn't change until later in the development cycle (HHVM, and PHP 7.0 and 7.1 were all dropped from that branch before release). In general, the new branch will follow whatever master has, as new requirements are added to master first.
Next, we need to alter each of the project-templates which have MediaWiki-like branches. As of June 2020, these are:
- extension-quibble
- extension-quibble-composer
- extension-quibble-noselenium
- extension-quibble-composer-noselenium
- skin-quibble
- skin-quibble-composer
- node10-rundoc-docker-for-MW-branches
- mwgate-npm
- mwgate-rake
- mwgate-tox-docker
- extension-broken
There are also commented out references to it in extension-phan and skin-phan which you should adjust, to avoid later confusion.

Take care to ensure that these align with the PHP and other testing environment requirements you have determined above. This the original patch for extension-quibble following the determination of supported versions, copying the 1.33 values:
```
    test-1.34:
      - mwgate-node10-docker
      - quibble-composer-mysql-php72-docker
    gate-and-submit-1.34:
      - mwgate-node10-docker
      - quibble-composer-mysql-hhvm-docker
      - quibble-composer-mysql-php70-docker
      - quibble-composer-mysql-php71-docker
      - quibble-composer-mysql-php72-docker
      - quibble-composer-mysql-php73-docker
```
… and this is the eventual version as of June 2020, supporting only PHP 7.2 and 7.3 (note that the selection of quibble jobs changed as well):
```
    test-1_34:
      - mwgate-node10-docker
      - quibble-composer-mysql-php72-noselenium-docker
      - quibble-composer-selenium-docker
    gate-and-submit-1_34:
      - mwgate-node10-docker
      - quibble-composer-mysql-php72-noselenium-docker
      - quibble-composer-mysql-php73-noselenium-docker
      - quibble-composer-selenium-docker
```

Next, you need to define the use of the new pipelines for the direct jobs for MediaWiki itself. Again, take care to ensure that these align with the PHP and other testing environment requirements you have determined above.

    test-1.34:
      - mediawiki-quibble-composer-mysql-php72-docker
      # FIXME use a normal composer job?
      - mediawiki-quibble-composertest-php72-docker
      - mediawiki-core-jsduck-docker
	
      - mediawiki-core-php72-phan-docker

    gate-and-submit-1.34:
      - mediawiki-quibble-composer-mysql-hhvm-docker
      - mediawiki-quibble-composer-mysql-php70-docker
      - mediawiki-quibble-composer-mysql-php71-docker
      - mediawiki-quibble-composer-mysql-php72-docker
      - mediawiki-quibble-composer-mysql-php73-docker
      # FIXME use a normal composer job?
      - mediawiki-quibble-composertest-php72-docker
      - mediawiki-quibble-composer-sqlite-php72-docker
      - mediawiki-quibble-composer-postgres-php72-docker
      - mediawiki-core-jsduck-docker

      - mediawiki-core-php72-phan-docker

The last step in layout.yaml is to look for special repos that also get MediaWiki-like branches but don't use any of the templates; as of June 2020, there's just one, VisualEditor/VisualEditor, which you can configure like so:
```
    test-1_34:
      - visualeditor-node10-browser-docker
    gate-and-submit-1_34:
      - visualeditor-node10-browser-docker
```
Finally, you need to tell the CI tests themselves about the new pipeline, in tests/test_zuul_scheduler.py, by extending MEDIAWIKI_VERSIONS like so:
```
    'Release 1.34': {
        'branch': 'REL1_34',
        'pipeline-suffix': '1.34',
    },
```

Save your changes to the files, and send it for review. Your changes will be diffed with the previous configuration, and if you are listed in the CI allow list, your configuration will be automatically validated.

To have it deployed, see the Zuul page; this requires shell access on the continuous server running it.

Note that there may be some JJB changes needed, depending on the CI configuration at the time.

Example patches:

Adding 1.34
Removing 1.33 (similar process but in reverse, and split for ease of reverting):
- Remove job and job-templates uses of the pipeline
- Remove the pipelines themselves