Can you use AI to detect accessibility bugs automatically? Yes, and we would know.

Evinced has been an AI company since Day One. At the time that we were thinking about how to build the business, we noticed that there were only two ways to discover accessibility defects: manual checks, and syntax checks. Manual has obvious scalability problems, and syntax checks leave a lot to be desired. For example, a semantic check reads a web page and look for instances of html elements being incorrectly handled. Consider this simple (and admittedly rare) example:

Evinced has been an AI company since Day One. At the time that we were thinking about how to build the business, we noticed that there were only two ways to discover accessibility defects: manual checks, and syntax checks. Manual has obvious scalability problems, and syntax checks leave a lot to be desired. For example, a semantic check reads a web page and look for instances of html elements being incorrectly handled. Consider this simple (and admittedly rare) example:

<img src="/foo.jpg">

The problem here is that the <img> tag needs an "alt" attribute, even if that value is declared as null (alt=""). It's an easy enough thing to scan a page for instances of <img> tags that are somehow missing an alt attribute. But what if the developer didn't use the <img> tag at all? There are many ways to provide an image on web page without an <img> tag, from CSS to javascript, and a developer might have chosen one of those routes. Where does that leave our "check for img tags that are missing alt attributes" rule? Put another way, syntax checks have hard a hard time knowing what the developer's intent is.

For us, AI is what can overcome that detection gap. And it can do so in two main ways.

Computer vision is a powerful AI technique and the source of one of our earliest technical achievements, the ability to recognize interactable elements even if they are not properly characterized as such in the html of a page. It is in fact a way to piece together a developer's intent, and use that to help identify defects.

In the photo above, this computer is being trained to recognize trucks and airplanes, and the number out to the right of the label is the system's confidence level on each classification. In our case, we use computer vision to recognize, instead of trucks and planes, interactable elements on a web page -- buttons, tablets, etc. Even if they were described in the html as something completely different, or not at all. You can see more concrete examples of this in our Find More Defectsopens in a new tab page.

We can also use computer vision to find elements that are not interactable, too. For example, we can use it to recognize headers on a website and then look in the DOM -- the Document Object Model -- to see if those headers have been described properly, with the right hierarchy.

In effect, computer vision tell us what the developer wanted to do and was hoping to achieve. We can then examine, in real time, whether that objective is going to get carried across to users of assistive technology.

Machine learning is another powerful technique in the world of Artificial Intelligence. Briefly, it's the practice of teaching a computer to make inferences about new sets of data. For example, if we train a computer on 1,000 financial statements, and then show it a new financial statement where the word "Total" is spelled as "Totl", it might be able to understand that the authors of that financial statement meant to write "Total" when they were summing up the value of the assets for the company.

We use machine learning in many ways, but one key way is in helping companies determine which bugs to fix first. Our system can scan an entire web site quickly, and determine, in close to real time, what common coding practices are responsible for most critical bugs. That means we are able to infer, from the html of your entire site, what pieces of code seem to be re-used across your site. For example, if there's a login button in your header, and you haven't described that button properly for screen readers, then you are going to have a bug on every single page in which that header appears. If your site has 10,000 pages, that would get reported as 10,000 defects. But it's really one problem, and can very likely be fixed once, in a single place, and have the fix propagate to all of those 10,000 places. More on this on our Get Organizedopens in a new tab page.

Another classic use of Machine Learning for us is the ability to determine reading order and focus order on a web page, which is critical for assistive technology users.

Detection is of course only part of the problem. Next, we'll tackle how we use AI to help with fixing bugsopens in a new tab.