Accessibility testing
What the standard actually requires, which parts a tool can check, and why the same work makes your locators more stable.
Accessibility has an unusual property among quality attributes: improving it makes your test suite better as a side effect. A control with a proper role and an accessible name can be found by getByRole; a styled div with a click handler cannot be found by anything except its styling, which is the least stable thing about it.
The scope is set by WCAG, the W3C's Web Content Accessibility Guidelines. Most legal regimes reference level AA. It is a large document, but the shape is simple: content must be perceivable, operable, understandable and robust, and each of those breaks down into testable criteria.
The number that matters for a test strategy is how much of this automates, and it is worth being precise because the marketing around it is not. Deque, who maintain axe, publish 57% — and immediately qualify it: that is 57% of issues by volume across their audit data, while the share of WCAG success criteria a machine can decide at all is 20 to 30%. Both numbers are true and they answer different questions.
What automates is the mechanical half: missing alternative text, insufficient contrast, absent form labels, invalid ARIA. What does not is every judgement — whether the alternative text is useful, whether the focus order makes sense, whether an error message tells anyone how to fix anything. So automate the first set on every build, and let that be what buys the time to do the second set properly.
import { test, expect } from '@playwright/test';
import AxeBuilder from '@axe-core/playwright';
test('checkout has no detectable violations', async ({ page }) => {
await page.goto('/checkout');
const results = await new AxeBuilder({ page })
.withTags(['wcag2a', 'wcag2aa']) // only what you commit to
.exclude('#third-party-widget') // known, tracked, not yours to fix
.analyze();
expect(results.violations).toEqual([]);
});Every real codebase has violations it cannot fix this quarter — an embedded widget, a legacy page. Excluding a specific selector, in code, with a comment, keeps the check switched on everywhere else and leaves a written record of the debt. Dropping the whole scan to level A because one page fails does the opposite.
The standard
4 entries-
WCAG 2 overview
StandardThe entry point: what the levels mean, how the documents relate, and where to go next. Read this before the specification itself, which is written for implementers.
www.w3.org -
The usable form of the standard: filterable by level and technology, each criterion with sufficient techniques and common failures. This is the page to have open while testing.
www.w3.org -
ARIA Authoring Practices Guide
StandardWorking patterns for the controls HTML does not provide — combo boxes, tabs, dialogs, tree views — with the keyboard interactions each is expected to support. Also the best available checklist for testing a custom widget.
www.w3.org -
WAI: easy checks
StandardA first review anybody can do in twenty minutes with no tools: page title, image alternatives, headings, contrast, keyboard access. Genuinely the best starting point for a tester who has never done this.
www.w3.org
Automating the part that automates
5 entries-
axe-core
ToolThe open-source engine behind most automated accessibility checking, including the browser extensions and the Lighthouse audit. The rule descriptions explain what each check means and why, which makes it a teaching tool as well as a scanner.
github.com -
Where the 57% above comes from, with the method attached: 13,000-odd page states and roughly 300,000 issues from first-time audits. Published by the vendor whose engine it measures, which is a reason to read the method rather than a reason to dismiss the number.
www.deque.com -
Playwright: accessibility testing
Official docsThe official guide to running axe inside a Playwright test, including scanning part of a page, handling known violations, and attaching the results to the report.
playwright.dev -
Lighthouse
Official docsAccessibility, performance and best-practice audits from the browser you already have, scriptable in CI. Its accessibility score is an axe subset, so treat it as a smoke check rather than the whole story.
developer.chrome.com -
Microsoft's free tool, and the reason to know it is the assessment mode: it walks you through the manual checks an automated scan cannot do, in order, recording as it goes. The bridge between the automatable third and the rest.
accessibilityinsights.io
Doing the part that does not
2 entries-
WebAIM Screen Reader User Survey
ResearchSurvey data on what assistive technology people actually use and how they navigate with it: the correction to guessing, and the answer to “which screen reader should we test with”. WebAIM run it every couple of years, so check for a newer edition before quoting a figure from this one.
webaim.org -
WebAIM: keyboard accessibility
Official docsThe single highest-value manual check: unplug the mouse and complete the journey. Everything that cannot be reached, everything with no visible focus, and every trap shows up in one pass.
webaim.org
What to take away
- Automated scanning finds a minority of the issues.Run it on every build anyway, so that manual time is spent on the majority it cannot see.
- Accessible markup is stable markup.The same change that makes a control usable by a screen reader makes it findable by
getByRole. - Unplug the mouse.Twenty minutes of keyboard-only use finds more than the first scan usually does.
Reading is the slow way to learn how a suite behaves. Record one against your own app, free, and watch what the report says.