Project 3:  A Web Page That Speaks

Due:  Thursday, September, 21st  *in class*

In this project, you will build a web browser extension that will allow users to point at or select content and have it read aloud to them. They will also be able to have the content read to them word by word. It will be our first experience with a project that isn’t about making content accessible directly, but rather about building something that changes existing content to make it more accessible. In other words, this will be your first access technology!

The final output of this project will be an browser extension for the Chrome web browser, which you will turn in and demonstrate in class. Here’s a video of some existing
Chrome extensions for Speech Synthesis.



Jeff recorded and posed a
 video of himself demonstrating how to generate speech from the web browser, a bit about using Javascript and jQuery, and how to package all of that into a Chrome extension.

Chrome Extensions

Web browser extensions (and, plugins) have long been used as a way to give the web new features, and to make it more accessible. Browser extensions are small bits of code that are run within the context of the web browser, allowing them to interact and change web pages. Long ago, these needed to be written in languages like C++ because they really extended the code of the web browser executable. That all changed with with the introduction of Greasemonkey, a plugin that allowed users to provide short “content scripts” written in Javascript that would be applied to specified web pages that were loaded by the web browser. As Javascript has become more powerful, so have the extensions written for web browsers that use it. For example, ChromeVox, which turns your web browser into something like a screen reader is written primarily in Javascript.

Each popular web browser has a slightly different way of building extensions for them. We’ll be writing an
extension for the Chrome web browser. Specifically, you’ll be writing what’s called a content script, which is really similar to a Greasemonkey script.

Content Scripts

Content scripts in the world of Chrome extensions are somewhat like the Greasemoney scripts of yore. That is they are Javascript scripts that are inserted into specified web pages, which then run in the context of those web pages. The core of a content script extension is still Javascript, but there’s a little more structure that allows for more complicated content scripts.

You should first create a directory for your extension, and then add a folder called scripts, and a folder called css. In the scripts folder, add a blank text file named reader.js and include a copy of JQuery. In the css folder include a blank text file named reader.css.

Now add a manifest.json file to the main directory, and include the following text into it, which specifies various metadata about your extensions, e.g., its name, the files included in it, etc. For now, the extension is set to only be included on pages on the course homepage. That’s probably good for now, as you might not want it to always be there!

{

  "manifest_version": 2,

  "name": "Access Reader",

  "minimum_chrome_version": "24.0.1307.0",

  "version": "0.0.1",

  "homepage_url": "http://www.accessibilitycourse.com",

  "icons": {

  },

  "content_scripts": [

    {

      "matches": [

        "http://www.accessibilitycourse.com/*"

      ],

      "js": [

          "scripts/jquery.min.js",

          "scripts/reader.js"

      ],

      "css": ["css/reader.css"],

      "run_at": "document_idle"

    }

  ]

}

At this point, I would recommend installing your extension into your browser. To do that, type chrome://extensions into the address bar of the browser. Then click “Load Unpacked Extension,” and choose the directory of the extension that you are creating. At that point, assuming everything worked and you didn’t get any errors, you should see an entry for a new Chrome extension named Access Reader:

Making Your Extension Do Something

At this point, what you have is the basic shell of an extension. Now it’s time to make it actually do something. In our case, we’re going to make your extension highlight the content under the mouse, and speak what’s there if you press the spacebar. Fortunately, your browser has a lot of great functionality built-in that makes this much more straightforward than it used to be to do!

The main file that you’re going to edit is the reader.js file located in the scripts folder of your extension. First, let’s test that you’re able to get it to do something, and in the process explore one helpful way of debugging your extension.

First, try putting just the text
console.log(“Accessibility rocks!”); in the reader.js file. To see if this worked, takes a few steps:

  1. Save the file
  2. Reload the extension (go back to the chrome://extensions address, find the Access Reader extension, and then press “Reload”)
  3. Next, visit http://www.accessibilitycourse.com
  4. Open up the develop console. One way to do this is to right click on the web page, and choose “Inspect Element.” Then go to the Console tab.
  5. Reload the page.

If everything works, you should see “Accessibility rocks!” written on the console tab. Yay!

Now, let’s make your web page do something even more useful than spreading the good word about how accessibility rocks.

Go back to reader.js. We need to edit this so that we can be sure the web page has fully loaded before we start editing it. This is mostly taken care of us for content scripts, but it’s a little unclear whether all of our extension content has been loaded before scripts are run, so we’ll use our first bit of JQuery to ensure that the page is ready for our script.

Add the following to reader.js:

$(document).ready(function() {

}

Anything that appears between the curly braces will not be run until the whole web page has loaded. One thing we need to do once the page has loaded is tell all the elements on the page to let us know when the user mouses over them. We can do this by putting the following JQuery within the code that will be executed when the page loads.

$("*:not(body)").hover(

    function (ev) {

      … EXECUTED WHEN MOUSE ENTERS AN ELEMENT …

    },

    function (ev) {

      … EXECUTED WHEN MOUSE LEAVES ELEMENT …

    }

 );

The previous code says for all the DOM elements on the page, except the body, attach this hover event to them. It’ll call the first function when the mouse hovers over an element, and it calls the second function when the mouse stops hovering over an element. That might be a good clue for you -- we want to highlight a DOM element when the mouse is over it, and we want to stop highlighting it when the mouse stops being over it. This is where we should do that.

So, how do you highlight something? If you remember back to our slides on the web, something like highlighting would be included under style or presentation, and that means it’s a kind of CSS. You might remember we have a CSS file in our template extension. So, one way to highlight an element is to define a CSS class that would add an outline to it. The CSS for something like that would look like:

.highlight {

  outline: 3px solid red;

}

If you put that in the reader.css file we created, that class will be available to the web page that our content script is added to. That means that if you add the class to an element, it will have a red outline. Adding a class in HTML looks like, <p class=”highlight”>, but how do you do that from Javascript?

JQeury makes it pretty easy. Here’s the code to add a class to an element:
$(“#elementselector”).addClass('highlight)

And, then to remove a class, you call the analogous:
$(“elementselector”).removeClass('highlight)

One trick that might not be obvious is that in the context of the event handler functions above, you can refer to the element that the event was called on using $(this).

Once you add the lines above to the event handler functions, try it out. What do you get?  One thing you might notice is that multiple outlines are sometimes produced. This is because of the way the DOM works, your mouse can be above multiple elements. You probably only want the first element to trigger the outline, which you can do by adding the following to the end of the first event handler:

ev.stopPropagation();

You can read more about event models, and why stopping an event’s propagation would have the effect you want, but just adding it in there should do the trick. You might find you also need to erase some errant leftover highlights. You can do that easily by finding all the elements with the highlight class and removing the class from them:

$(".highlight").removeClass('highlight’)

Adding Speech

The next thing to do is to make the content of the highlight node speak!

It turns out that in modern web browsers, it’s very easy to produce speech. The following code will speak whatever string you provide:

speechSynthesis.speak(new SpeechSynthesisUtterance(text));

Equivalently, you can stop all the speech synthesis by calling:

speechSynthesis.cancel();

If you have a DOM element, JQuery will let you get a text version of the element with the following call:
$(“#elementselector”).text();

At this point, you should be able to move your mouse around and your web browser will speak whatever is under the mouse cursor.

Speaking Only When Requested

Depending on your needs, it could be annoying to always have to hear the content read as the mouse cursor is moved around. In this step, we’ll make it so the content is only spoken if the user presses the spacebar.

You can press a “key handler” to the keydown event using the following code:
$(document).keydown(function(ev) {

  if (e.keyCode == 0 || e.keyCode == 32) {

    …WHATEVER CODE YOU WANT TO RUN ON SPACEBAR GOES HERE…

  }

}

As in the previous example, you should include this code within the document ready block, to make sure that it is run after the web page has properly loaded.

When you’re done with this step, your extension should highlight elements whenever the mouse is moved, but only speak them if you press the spacebar.

You will likely notice that the web page will move when you press the spacebar. This is because in addition to your event, the spacebar moves the page down by default in the browser. You can prevent it from doing this by telling the event to stop “bubbling up” after it has triggered your response. Here’s a
StackOverflow Article on stopping it.

Speaking More Than Text

In this final section, you’ll detect if the element that the extension is attempting to read is an image, and if so you’ll read either the alt text (if available) or the filename (if alt text is not available).

To accomplish this, you’ll need to know what kind of element you’re working with, you’ll need to know if the alt text is there, and how to access it. Those tasks can be accomplished as follows:

var tagname = this.tagName;

if ($(this).attr('href')) {

  // href is not blank

} else {

  // href is blank

}

var alttext = $(this).attr(“alt”)

var srcofimg = $(this).attr(“src”)

At the end of this step, your extension should read not only text, but also the alt text of images when available, or the URL of them if it’s not.


This page and contents are copyright Jeffrey P. Bigham except where noted.