Developer account/Subversion

This article was a guide to applying for and using MediaWiki commit access with Subversion. A person with commit access has write access to the repository and can submit their own patches or patches from others. This has changed since our migration from Subversion to Git on March 21 2012; please instead apply for a developer account to use Git/Gerrit/Cloud Services/Toolforge.

Requesting commit access

Prerequisites

Before applying for commit access, you should typically have:

A demonstration that your request is made in good faith. For example, someone may be employing you to work on MediaWiki, or you may be known to the Wikimedia volunteer community by way of past work.
Plans to contribute to MediaWiki or extensions in a substantial way.
Programming skills appropriate for the type of work you propose to do. Our community will help new participants, especially with MediaWiki-specific skills, but we don't have time to train programmers from scratch.
An account on this wiki (www.mediawiki.org) with an authenticated e-mail address set in your Special:Preferences.

Request format

Commit access requests should be sent to commit-access-requests@wikimedia.org.

Your request should contain the following information:

Requested user name

Your user name must match the following regular expression /^[0-9a-z_\-\.]{1,17}$/. That is, it may only contain from 1 to 17 numbers and lower case ASCII letters, as well as the . (full stop), - (hyphen-minus) and _ (low line) characters.^[1] Additionally, the user name should not conflict with the existing user names at http://svn.wikimedia.org/users.php.

Your www.mediawiki.org user name

You must be registered and have e-mail features enabled to receive code review feedback.

What you expect to work on

Which extensions, core modules, etc.

A link to your public key (public key openssh format)

Your SSH key will be used to authenticate you when you commit changes.
For security reasons, you should temporarily host your public key on your personal webserver and supply the link to it. Less preferably, post it to a subpage of your mediawiki.org user page, using your mediawiki.org user name. The key should be in the openssh format (not SSH2 format), see example :

ssh-rsa AAAB3NzaC1yc2EAAAAB3FWqgypbL ... Lgfel80W3Bm9sbb+5JcsHV comment...

in plain text, and not embedded in HTML. ssh-keygen (for Unix-like operating systems) and PuTTYgen (for Windows-family operating systems)^[2]^[3] are examples of programs that can generate a key for you.

Warning:

Do not send a key as an e-mail attachment, as unsigned e-mail is not a secure means of delivering your SSH key. Dropbox, Megaupload, Rapidshare, and the like are also unacceptable.

Past work on MediaWiki: Patches, extensions, bug reports, etc. Please provide links or attachments to specific patches, changesets, or codebases that you wrote; links are preferred.
Past programming experience: Other open source projects, etc.

It may take up to three weeks for your request to be processed. If you've been waiting more than two weeks, feel free to ping Sumana Harihareswara.

Getting started and set up

Once you have an account, you will need to follow some steps before you can commit anything.

Read Subversion to get an overview of the Subversion source control system.
Read Subversion/auto-props and do exactly what it says to set up auto-props.
Create a USERINFO file.
Checkout using svn+ssh:// instead of http:// , or it simply won't work (you may get a 403 Forbidden unexpected return value).
If you are prompted for a password during checkout and your SSH key doesn't need a passphrase, this may be an indication that your account has not been set up correctly. If you're using Windows, check that Pageant is running with your private key stored.
Check out two copies of MediaWiki: one instance for development and testing, and another instance that you will apply patches to and commit from. This structure makes it easier to commit clean patches. To do this, use:
svn co svn+ssh://username@svn.wikimedia.org/svnroot/mediawiki/trunk/phase3 wikimedia-dev

svn co svn+ssh://username@svn.wikimedia.org/svnroot/mediawiki/trunk/phase3 wikimedia-commit
Have a coder link your SVN account to your wiki account, so that you receive emails when somebody comments on your commits. If you're feeling bold, you could instead ask a coder to make you a coder; you can then link your account yourself at [[Special:Code/MediaWiki/author/<your name>]].

Updating/replacing your SSH key

If you have a labsconsole user account and it's linked to our LDAP system, you can manage your own keys at https://labsconsole.wikimedia.org/wiki/Special:NovaKey . Otherwise, please:

Go to this subpage of your personal user page on this wiki.
Click "create". Paste in the text of your new key, then click "save page".
You will be redirected back to the page, and it will now have your key on it. Copy the URL to your clipboard.

Send an email to commit-access-requests@wikimedia.org of the form:

Hi,

I'm a MediaWiki developer. Could you please update my ssh key for the user name insert username here? My SSH key is at paste URL here.

Thanks,
your name

Guidelines for applying patches

Remember that trunk is supposed to be live code -- keeping things up to date and ready to run at all times is our biggest method of quality control. Code must work at all times, except of course when it's broken by accident.
Always update the RELEASE-NOTES.
Consider possible security implications (e.g. see this security note for extension authors).
Do not backport any new features to the stable trunk without prior permission. In fact, best to just leave the stable trunk alone. Leave all backporting of fixes to the release manager, and save yourself grief. Things should only be backported if they are a security fix, or are vital for the software to run. Also, new features don't go into maintenance releases of old versions.
Don't touch the release tags at all. Ever!
Try not to break things, or people may get cranky. For example, if you are changing the parser, you can run parserTests before and after to help determine if your change introduced any new breakages, like so:

cd tests
php parserTests.php --quiet --quick --color=no

... and compare the failures before with the failures after, to check that there were no regressions.

Test all patches thoroughly before applying them to SVN HEAD.
Try not to make any big changes right before the tree is branched into a stable release (this currently happens about once every 3 months).
Try to avoid introducing new preferences, if possible, by supplying sane defaults. Remember that new preferences should very rarely be added.
Try to follow MediaWiki's Coding conventions where possible. Read them every so often because they might be updated.
MediaWiki caches parser output and revision diffs. Take into account that Wikimedia and other sites use HTTP caching (with Squid or Varnish) in front of the webserver. Try not to introduce code that makes caching difficult.
I18n (internationalisation) and new message texts must be added (or changed) on translatewiki.net, not in the SVN. The changes will be committed to Wikimedia SVN by the translatewiki.net developers in due time.
Before committing someone else's software to Wikimedia SVN, it is best to discuss with them first. Ideally, invite the original developer to apply for commit access and wait for that request to be fulfilled so they can commit it themselves.

To commit to SVN

Once you have done all the above steps, you are ready to make your first commit. You should join MediaWiki developers' IRC channel (irc.freenode.net) #mediawiki. This way you people can give you direct feedback about your commits and get to know you and you them.

Always make sure to have your local copy as close as possible to the head version (update), check the files you have changed, added, deleted or not committed (status), and always compare your local copy including your changes with the latest version in the repository (difference local to HEAD). Commit only, if everything is as expected:

cd my_workingcopyfolder
svn up
svn status
svn diff -rHEAD > my_workingcopyfolder-to-head.diff
svn commit -m "my message"

To prepare a commit, run svn diff List of files you want to commit. If you don't give filenames, svn picks up all files recursively from your current directory. Review the diff once more for any unrelated changes and bugs in the code. When you are ready run svn ci Same list of files. Your default text editor will pop up and ask for a commit summary. The following tips help to make good commit summaries:

Say what you did and why it's a good thing or needed.
The first line should contain concise summary of the commit.
In the following lines you can go into details of what and why. For bug fixes, try to give clear instructions to reproduce the bug, so reviewers can confirm that the commit does fix the problem.
Reference related bugs with bug #### and revisions with r##### for each bug and revision. You need to use that exact syntax so that those will become links automatically.
Once you have written the whole summary, read it once to catch any errors.

If you want to write the summary in steps, you can use an ordinary file and the --file for the svn ci command.

Then you should get output back like this:

Sending        RELEASE-NOTES
Sending        includes/User.php
Transmitting file data ..
Committed revision 16794.

Then you can close any affected bugs (if applicable) in bugzilla, using a line like "Fixed in r#####."

Then hang around on #mediawiki for at least a few minutes, so that people can find you in case you broke anything. You might also get feedback on Code review, so be sure that your user account is linked and that you allow email notifications.

Some useful resources or commands

To see what changed in a particular revision number, go to http://svn.wikimedia.org/viewvc/mediawiki?view=rev&revision=16789 and replace the revision number in the URL as appropriate.
The "svn status" command will show you any files that differ between your local version and the central version.
The "svn diff includes/file_you_changed.php" command will show what exactly has been changed in a certain file, relative to the last checked out version.
If you want to see what changed between two revisions from the command line:

svn diff --revision 17109:17114

Show differences between your local copy and the current code in the repository

svn diff -rHEAD

You should probably subscribe to the wikitech mailing list.
If you want to see what's changed recently, you can use the CVS mailing list.
Sometimes there is a small bit of lag between commits, and getting notified via CVS email. There is no lag however with SVN, so to see what's changed recently, or even not-so-recently, you can use the "svn log" command. For example this will show all changes between the specified revision numbers, and details of which files were changed:

svn log --revision 17109:17114 --verbose

You can always ask questions on the #mediawiki IRC channel. Also if the CIA-7 (or 60, or 57, or whatever the number may be) bot is working, it will print out notifications of SVN commits to the #mediawiki channel as they happen.
If you commit a change, and it turns out to be completely broken, then you can revert it by doing something like this (undoing changes explained in detail).

You can add the merge option --dry-run to preview the changes as it does not apply changes to the working copy.

svn up
svn merge -r REVISION_NUM_YOU_WANT_TO_UNDO:REVISION_NUM_TO_REVERT_TO .
svn status
svn diff
svn commit --message="Self-revert accidental breakage" includes/file-you-changed.php includes/another-file-you-changed.php

For reverting a single version, you can use the changeset notation (take note of the - sign before the revision number to apply the reverse difference)

svn up
svn merge -c -REVISION_YOU_WANT_TO_UNDO .
svn status
svn diff
svn commit --message="Self-revert accidental breakage" includes/file-you-changed.php includes/another-file-you-changed.php

To view a specific bug in bugzilla, go to http://bugzilla.wikimedia.org/show_bug.cgi?id=#### (replace #### with the bug number in the URL as appropriate).
When closing a bug in bugzilla, include a comment with something like "Fixed in r####" in it - it autolinks the revision, and it helps us to keep track of changes related to bugs.
When committing a change which fixes a bug, always include the bug number in the commit summary.

Commit Summaries

With each commit to the MediaWiki SVN repository, please include an informative commit summary.

Try to ensure that your first line of your commit summary is as informative as possible. The code review tool only displays the first line of the commit summary in the overview. Try to be concise in this first line, and then elaborate in further detail below.

In the case of controversial, or non obvious (why you are doing them/why this is a benefit) changes, please link to any relevant discussions (if appropriate), like bug comments, Bugzilla bugs, mailing list posts. Else, if these are not the case, explain why you are doing this commit (an essay isn't required, but "Fix bug" isn't enough information).

If you're merging from trunk to a branch, "MFT r####" is the customary commit message.

Link to relevant bugs and revisions

If you are doing work towards (or to fix a bug), please use the bug #### syntax to link it (example). If this commit follows on, reverts, or is related to other commits, please reference them using the r#### syntax (example).

Syntax like

bug 12345
r12345
follow up patch to r12345

in a commit message or code comment automatically creates a link to the specific bug or revision, or set a follow-up indication at the specific revision it follows up. When used in a commit message specifically, this syntax also creates links between revisions (to note follow-up revisions, for example).

Going live

Code in Subversion won't immediately be live on Wikimedia wikis like Wikipedia. This is to avoid immediate breakage of the sites and to allow for code review to take place.

When a system administrator (typically Brion Vibber or Tim Starling in this case) has reviewed code changes, they will perform a standard svn update on the production cluster. At this point, changes are live on http://test.wikipedia.org, and last-minute checking can be done to ensure nothing has been broken under Wikimedia's idiosyncratic configuration.

If everything appears okay, then changes will be synchronised to application servers. This is also known as "scapping", after the "scap" script which does this.

To find out who last modified a particular line of code

Run a command like this (using the includes/Parser.php file as an example) :

svn blame includes/Parser.php | less

... and then search for the line by typing "/" + your search term (e.g. the function name). It may take about 20 seconds for the svn blame output to be generated.

This will show which revision a line was modified in, and by whom, like so:

  4452      timwi     $argc = count($args);
  9860 kateturner
  4452      timwi     for ( $i = 0; $i < $argc-1; $i++ ) {
  4904     hashar          if ( substr_count ( $args[$i], '[[' ) != substr_count ( $args[$i], ']]' ) ) {
  4904     hashar                $args[$i] .= '|'.$args[$i+1];

... you can then look up that particular revision number using the SVN web interface, to see what the commit log message was to get an explanation of why a change was made. If necessary, you can then ask the user; there is a list of SVN committers in the SVN server with contact information.

Setting up a non-standard port

Tip: Only needed if you use a non-standard port for SSH normally, otherwise ignore this. You can force SVN to use the right port number by adding this to ~/.subversion/config :

[tunnels]
ssh = $SVN_SSH ssh -p 22

Branching and merging

You may wish to do ongoing experimental work in a branch, so that the development trunk remains stable until your change is ready to merge. Branches are also used for the quarterly releases, so that additional bug fix releases can be made easily.

See the Subversion/branching guide for more...

Notes

↑ While valid Unix user names can contain many other characters other than /^[0-9a-z_\-\.]{1,17}$/, we should avoid the use of metacharacters that can cause us trouble in some contexts if we forget to escape any meta-characters.
↑ OpenSSH under Linux cannot read ppk files generated by PuTTY under Windows. When you got commit access granted and want to use the ssh-add command to add your private key as identity, and when you see "Bad passphrase, try again for /path/my.key:" then you may need to reload your ppk key using PuTTYgen and export it as OpenSSH key:
PuTTYgen → Conversions → Export OpenSSH key (source)
↑ "Dealing with Private Keys in Other Formats" - article about the different private key formats of PuTTY, OpenSSH, and ssh.com and how to convert between them