Case Study: AI in 508 Compliance Testing
Concept Creator
Accessibility compliance, specifically Section 508 of the Rehabilitation Act, ensures that federal electronic and information technology is accessible to people with disabilities.
Traditionally, the process of ensuring compliance involves manual auditing and testing, which is time-consuming and prone to human error. With the rapid advancement of AI, there's an emerging opportunity to automate and enhance this process, improving accuracy and efficiency, provided appropriate safeguards like approval workflows are integrated.
The Opportunity:
Can AI be used to check front-end code for 508 compliance and suggest/adjust the code for full compliance, updating both the code base repository and design component library?
Before we go deeper, let’s examine the current Approach with AI.
Existing Tools
Axe DevTools by Deque: Automates accessibility testing and highlights violations to guide manual fixes.
Microsoft Accessibility Insights: Performs automated tests; offers AI-driven prioritization.
WAVE by WebAIM: Visualizes issues in live pages, aiding manual review.
Google Lighthouse: Audits many aspects, including accessibility, but relies on manual fixing.
Limitations
While these tools automate detection, they don't automatically fix issues or integrate new change with repositories or design systems without manual effort.
Axe DevTool lists issues, with code snippet, provides guidance but relies on manual fixing
Google Lighthouse highlights issues, provides guidance, but also relies on manual fixing
Accessibility Insights also list issues, with code snippet, provide guidance but again relies on manual fixing
Future approach with AI
Imagine an AI system that:
Detects and visualizes accessibility issues on live websites and or codebases.
Uses models like OpenAI Codex or GPT-4 to suggest precise fixes.
Applies those fixes then updates the code repositories and design systems through a structured approval workflow*.
*It's important to recognize that while AI can automate the detection and correction of 508 compliance issues, human oversight and approval workflows are crucial to ensure the accuracy of the correction and maintainability of the codebase and design system.
Next, let’s imagine AI's potential roles and a possible workflow for AI-driven 508 compliance.
AI 508 Compliance Review, Notification, and Change Approval Flow
Let’s follow this workflow:
1) 508 Compliance AI Triggered:
The AI reviews the codebase against 508 compliance parameters.
The AI determines the degree of severity and type of fix required (e.g., functionality, design, or content/tagging) and generates change recommendation(s).
2) Role Notification Mapping:
The AI sends notifications to the relevant roles, based on the type of issue identified.
Notifications are integrated right into the related tools (e.g., Figma for design-related changes & Storybook for component codebase-related changes).
Role mapping by severity and issue type
Mockup Example of 508 issue notification notification interface in Figma design system. The notification appears as comment and show summary and a “Review to Approve or Reject” CTA.
3) Side-by-Side Code/Design Comparison:
Approver clicks on “Review to Approve or Reject”
The system presents a side-by-side comparison of the original code/design and the AI-suggested changes, allowing reviewers to make informed decisions.
The comparison highlights the specific lines of code or design elements being modified and offers clear "Approve" or "Reject" calls to action.
Expanded Roles for AI in 508 testing
Training and Education:
Generates educational resources based on detected issues, helping developers/designers improve their accessibility knowledge.
Provides links to relevant documentation or best practices for specific types of accessibility issues.
Predictive Analysis:
Analyzes code patterns to proactively identify potential accessibility issues before implementation.
Flags potential issues in real-time as developers write code or designers create components.
Content Generation:
Generates accessible alternative text for images or creates transcripts for audio/video content.
Automatically suggests appropriate ARIA labels based on the context of the element.
Customized Recommendations:
Provides multiple fix options with different tradeoffs (e.g., performance impact, maintainability, adherence to design principles).
Considers the context of the issue (e.g., specific component, overall design system) to provide more relevant suggestions for a human reviewer to consider.
Conclusion
What is Currently Possible?
Automated detection of accessibility issues.
AI-driven suggestions to guide manual fixes.
Experimental automation for code corrections within CI/CD pipelines with human oversight.
What Can Be Explored/Achieved Soon?
Fully automated detection, suggestion, and correction workflows using AI, with integrated approval workflows.
Predictive analysis to identify potential accessibility issues proactively while components are being built.
Real-time compliance monitoring and automatic updates to code repositories and design systems, enabling continuous, scalable accessibility improvements with necessary human oversight and minimal manual intervention.
Final Thoughts
Will AI replace humans in accessibility testing? The answer is both yes and no. AI reduces workloads and automates repetitive tasks, but history shows technology transforms jobs rather than eliminates them. Like Dreamweaver, which empowered designers to collaborate better with developers, AI will augment human expertise. Tools like Figma and Storybook have reshaped design and front-end workflows, empowering teams work more effectively. AI will streamline processes for us and allow human experts to focus more on strategic and creative challenges.