User:DWalden (WMF)/A11y playbook draft1

Introduction

Imagine a world in which every single human being can freely share in the sum of all knowledge. About 15% of the world's population lives with some form of disability. Therefore, the Quality and Test Engineering team, along with the development teams, must ensure people with disabilities are able to interact with Wikimedia products. That is to say, that they can perceive, understand, navigate, and contribute.

The WMF Design Style Guide and Principles

The Wikimedia Design Style Guide ensures a consistent look and behavior for our products. Our interfaces are our brand. These principles and guidelines aim to help designers and developers with their Wikimedia projects, as part of the Foundation or in a volunteer capacity. The style guide features unique focus areas like accessibility, internationalization, and localization.

Also, according to Wikimedia's Accessibility Statement, the Foundation aims to support WCAG 2.1 AA guidelines. Make sure you read them carefully and incorporate them to your daily testing procedures.

Possible accessibility testing techniques

There are different strategies to testing accessibility. In this section we’ll outline 5 approaches.

Coverage-based
1. Coverage based on items in WCAG21
2. Coverage based on types of disability
Risk-based
1. Which functionality is likely to cause users the most problems?
2. For an/each area, identify what kinds of accessibility issues it will most likely have
3. For an/each WCAG21 item, identify which areas are mostly likely to have that particular issue
Activity-based
1. Follow a script or checklist depending on the activity or product you are testing. (See play 8)
  - e.g. https://www.w3.org/WAI/test-evaluate/preliminary/
  - e.g. Testing checklist for Abstract Wikipedia
2. Run accessibility checking tools (e.g. aXe, WAVE)
3. Perform exploratory testing (possibly using assistive technologies, e.g. screen readers)
4. Test as particular personas
  - e.g. https://www.w3.org/WAI/people-use-web/user-stories/ and https://www.w3.org/WAI/people-use-web/abilities-barriers/
Evaluation-based
1. Automated accessibility checks (e.g. aXe, perhaps run in CI)
2. Comparison with WCAG21 guidelines (quickref) and techniques
People-based
1. Testing by end-users
2. Testing by accessibility experts

Examples

VisualEditor watchlist functionality

When saving an edit with VisualEditor, the user has an option to add it to their watchlist for a particular amount of time. It had been mentioned that the UI controls which allow a user to do this might have accessibility problems. I wanted to do some accessibility testing following the playbook above. I give a report below, identifying the techniques I used.

I only had a limited amount of time to test this. Therefore, I took a focused approach.

First, I used Firefox’s default accessibility checker (available via devtools). This found two issues related to the watchlist elements, but I consider them to be false positives (the failure was “Clickable elements must be focusable and should have interactive semantics”, but I believe we are following all the recommendations mentioned there). This is using the technique 4.1 mentioned above.

Then, I used the aXe Firefox extension. This found no accessibility issues with the watchlist elements. Again, following technique 4.1.

I then took a risk-based approach to identify the accessibility issues the functionality is most likely to have (technique 2.2). I thought being able to access the UI controls using only a keyboard and having those controls read correctly by a screen reader were most likely.

For keyboard only use, I tested interacting using only a keyboard (technique 3.4, taking on the persona of a user who cannot use a mouse). I found I could do everything with a keyboard I could do with a mouse. The tabbing order was logical. I was able to achieve my task.

For screen readers, I checked that we were following the correct WCAG21 guidelines (technique 4.2). I assumed that guideline 1.1 (Text Alternatives) was the correct guideline we should be following to make it possible for screen readers to read our UI. I identified that the checkbox (“Watch this page”) is using technique H44 and using a <label> element to identify it. The dropdown is following technique ARIA9, using elements with the attribute “aria-labelledby”. These are marked by WCAG as techniques “sufficient” to ensure we have successfully met guideline 1.1.

To make sure, I could have installed a screen reader and checked that it read the UI controls correctly, but I did not deem it necessary.

Colour contrast of IPInfo tool

See phab:T309828#8206171. Use of technique 4.2 and WCAG21 technique G18.

Could also have used technique 3.2 as accessibility checking tools like aXe can find content with insufficient colour contrast.

Documentation and useful links

Note: Here could be the links to QA documentation, e.g. test planning, test instructions (test scripts, API calls, queries, step-by-step instructions, test cases/test results reports etc.)