Selenium/Ruby/mwext-mw-selenium Jenkins job

For this example, you will need internet access. mwext-mw-selenium Jenkins job runs every time you push a patch to Gerrit (for repositories that have the job configured to run), but you can also start it manually.

Advantages

• You do not have to install anything on your machine.
• You are probably already familiar with Gerrit and Jenkins.
• While the tests are running, you can continue using your machine as usual, because everything is happening in Jenkins.
• Debugging failed tests will be easier, since you will see the browser, logs, screenshots and video of the test run.
• It will be reasonable fast.
  • MobileFrontend Selenium test run takes about 5 minutes on my machine.
  • It takes about 5 minutes for mwext-mw-selenium Jenkins job to run for MobileFrontend.

Disadvantages

• You will need internet connection while running the tests.
• You might not be familiar with Gerrit or Jenkins, so you will have to learn how to use yet another tool.

Check if mwext-mw-selenium is configured to run for a repository

• In layout.yaml look for:
• extension-selenium template

- name: mediawiki/extensions/MobileFrontend
  template:
    - name: extension-selenium

• Or mwext-mw-selenium job

- name: mediawiki/extensions/Flow
  experimental:
    - mwext-mw-selenium

Run the Selenium tests for jobs with extension-selenium template

• Upload a patch to Gerrit for a repository.
• Example: gerrit:309594 for MobileFrontend
• mwext-mw-selenium job runs automatically
• when all Jenkins jobs configured for the repository are finished, jenkins-bot will leave a comment in Gerrit with links
• click mwext-mw-selenium link
• optionally, click Full Log link at the top of the page

00:00:00.000 Started by user anonymous
00:00:00.033 Building remotely on integration-slave-trusty-1011 (phpflavor-hhvm contintLabsSlave UbuntuTrusty phpflavor-php55) in workspace /mnt/jenkins-workspace/workspace/mwext-mw-selenium
...
00:00:43.937 + vendor/bin/bundle exec cucumber --color --tags @integration --tags '~@skip' --format pretty --format junit --out /mnt/jenkins-workspace/workspace/mwext-mw-selenium/log/junit
...
00:05:06.737 20 scenarios (1 undefined, 19 passed)
00:05:06.737 118 steps (3 skipped, 1 undefined, 114 passed)
00:05:06.737 4m19.672s
...
00:05:07.097 Finished: SUCCESS

• To run mwext-mw-selenium job again without pushing another change to Gerrit, reply to a commit with recheck

Run the Selenium tests for jobs with mwext-mw-selenium job

If the mwext-mw-selenium job is in test or postmerge pipeline, the job will run automatically after a patch is pushed to Gerrit, the same as for extensions that have extension-selenium template.

If the mwext-mw-selenium job is in experimental pipeline (like it is for Flow):

• Upload a patch to Gerrit for a repository.
• Example: gerrit:310408 for Flow
• Reply to the commit with check experimental
• mwext-mw-selenium will run
• when all Jenkins jobs configured for the repository are finished, jenkins-bot will leave a comment in Gerrit with links
• click mwext-mw-selenium link
• optionally, click Full Log link at the top of the page

00:00:00.000 Started by user anonymous
00:00:00.011 Building remotely on integration-slave-trusty-1017 (phpflavor-hhvm contintLabsSlave UbuntuTrusty phpflavor-php55) in workspace /mnt/jenkins-workspace/workspace/mwext-mw-selenium
...
00:00:20.287 + vendor/bin/bundle exec cucumber --color --tags @integration --tags '~@skip' --format pretty --format junit --out /mnt/jenkins-workspace/workspace/mwext-mw-selenium/log/junit
00:00:22.282 0 scenarios
00:00:22.282 0 steps
00:00:22.282 0m0.000s
...
00:00:22.448 Finished: SUCCESS