Project 7:  Single Button Input

Due: Tuesday, November 14th  *in class*

In this project, you will build a Chrome extension that will enable the user to browse the web only by pressing a single key on their keyboard. This sounds impossible, but it’s not!

Here’s the Example We Built In Class.

Yet Another Chrome Extension

The basis for this project will be a Chrome extension. To get started, copy and paste the extension that you built for a prior project. Enter its manifest to have a different name. I suggest something like, “Single Button Input,” and then load it into the browser.

Scan bars to select items on the page

The core of this project is two scan lines that will allow your users to select and click on something on the screen. When the user first presses the button, the first scan line should appear going from left to right. When the user presses the button again, the horizontal (x) location should be recorded, and then a new line should appear going from top to bottom. You should keep the vertical line on the screen. When the user hits the button again, the vertical (y) location should be recorded, and then a click should be triggered at that location, and then the scan bars should be removed.

The bars themselves can just be DIV elements styled to be the width (or height) of the page, with a corresponding thickness set to a few pixels in either the width or height. Set their CSS position to static so that they are always on the screen when you want them to be.

One of the trickiest parts of this section is getting the bars to animate. We’re going to handle the animation/paint loop manually. First, create the loop using the following:

setInterval(function() {

  paint();

}, 200);

This will cause a function that you will define called paint to be called every 200 milliseconds.

In this function, you’ll check what state the system is in, and take an appropriate action based on that. Roughly, you’ll have three states, which are
horizontal_scanning, vertical_scanning, and none. You can set and check the state using a global variable.

This will look something like the following:

var state = “none” // horizontal_scanning, vertical_scanning


function paint() {

  if(state == “horizontal_scanning”) {
   // update position of your horizontal scan bar

  } else if(state == “vertical_scanning”) {

    // update position of your vertical scan bar

  } else {

    // do nothing

  }

}

$(document).keydown(function(e) {

  if(e.key == “ “) {

    // update state here, given current state

   

    $(e).stopPropagation();

    return false;

  }

});

Triggering a “click” using the scan bars

Once an x and y location has been selected using the scan bars, you can trigger a click at that location using the following code:

var elementtoclick = document.elementFromPoint(x, y);

simulateClick(elementtoclick);

function simulateClick(element) {

  if (!element) return;

  var dispatchEvent = function (elt, name) {

    var clickEvent = document.createEvent('MouseEvents');

    clickEvent.initEvent(name, true, true);

    elt.dispatchEvent(clickEvent);

  };

  dispatchEvent(element, 'mouseover');

  dispatchEvent(element, 'mousedown');

  dispatchEvent(element, 'click');

  dispatchEvent(element, 'mouseup');

};


One trick here is that your x and y locations are relative to the window, not the document. You can calculate the appropriate x,y locations for the document by factoring in the current scroll of the page, i.e., scrollTop.

$(document).scrollTop()

Enabling the page to be scrolled

You’ll add new buttons to the document that will allow the user to scroll up and down. These can just be DIVs that have been styled to be square, position static, location in the lower right of the page, with a high z-index so they always appear on the top of everything else.


When the user clicks them (with the scan bar approach, or just with the mouse), the page should scroll up or down by 10% of the page height. You’ll find that it’s much more usable if you animate this scrolling, which jQuery allows you to do, with the following:

$('html, body').animate({

    scrollTop: $(document).scrollTop()+150

 }, 1000);

In this example, the page will scroll 150px from the current position.

Adding a software keyboard

Now you can navigate the web with just your single switch, but how will you enter text?  You’ll solve that problem now by inserting a software keyboard into the page.

The HTML and CSS for a keyboard is here:
http://accessibilitycourse.com/projects/project7/keyboard.html

http://accessibilitycourse.com/projects/project7/keyboard.css

When users press a button, you’ll show the keyboard. Then when users click on the software keyboard with the scanning clicker, you’ll enter that key in the most recently focused textbox.

Triggering keys and entering the text into a form field

When users press a key on the software keyboard, you should alter the contents of the text field so that the character represented by the key is appended to the contents of that field.

One trick to getting this to work is that your program will need to keep track of which input element keys should be sent to. To do this, record the last input element that accepts textual input to be clicked on. When the user types something using the software keyboard, you’ll add the letter to the field.

If you consider the code for clicking on an element from above,

var elementtoclick = document.elementFromPoint(x, y);

simulateClick(elementtoclick);

you’ll notice that you have access to the element that was clicked on after the first line (elementtoclick). To check if is an element that you should record as being the last one to accept text, you can use the jQuery is function, which will be something like this:

$(elementtoclick).is(“input[type=’text’],textarea”)

This will return true only if the checked element is either a <input type=”text”> or <textarea>.

Then when the user simulates a key on the software keyboard, figure out what character should be added, and then append it to whatever is already in the textbox. You can detect a click on the keyboard by either assigning a click event handler to each key, or by simply detecting the code where you simulate a click that this is your keyboard and so it should be handled specially.

You can append the character with something like this, where
newchar should be set to the value of the key, most likely by calling the jQuery text() function on the element that was clicked.

var newchar = …;

$(elem).val($(elem).val() + newchar);

CAPS LOCK is a special key that changes the mode of the keyboard. If the CAPS LOCK key has been pressed, then all characters should be made into capital letters. If you have the lowercase version of a character, you can make it into an uppercase version of the character using .toUpperCase(). Keep track of what caps mode you’re in with another global variable.

Grading

Main components of our grading will be:

Using the appropriate keyboard commands, does the following happen:


This page and contents are copyright Jeffrey P. Bigham except where noted.