Lab: Keyboard Control

Introduction

In this lab, you’ll build an alternative computer keyboard using any of the USB-native boards (the MKR series, the Due, the Leonardo, the Micro, and the Feather M0 series all fit this requirement). You’ll also learn some techniques for determining when a user takes a physical action. In the Digital Lab and Analog Lab, you learned how to read the input from a digital or analog input, but you didn’t learn how to interpret the stream of data as an event. There are a few everyday physical interactions that are fairly easy to pick out from a stream of data. You’ll see a few of them in this lab.

What You’ll Need to Know

To get the most out of this lab, you should be familiar with the following concepts. You can check how to do so in the links below:

Things You’ll Need

Below are the parts you’ll need for this exercise. Click on any image for a larger view.

A short solderless breadboard with two rows of holes along each side. There are no components mounted on the board.
A solderless breadboard.
An Arduino Leonardo. The USB connector is facing to the left, so that the digital pins are on the top of the image, and the analog pins are on the bottom.
A USB-Native Arduino board. Shown here is an Arduino Leonardo
Three 22AWG solid core hookup wires. Each is about 6cm long. The top one is black; the middle one is red; the bottom one is blue. All three have stripped ends, approximately 4 to 5mm on each end.
22AWG solid core hookup wires.
Resistors. Shown here are 220-ohm resistors. You can tell this because they have two red and one brown band, followed by a gold band.
Resistors. Shown here are 220-ohm resistors.  For this exercise, you’ll need 10-kilohm resistors (10K resistors are brown-black-orange. You’ll also need 1 220-ohm resistor. For more, see this resistor color calculator)
A pushbutton with two wires soldered one to each of the button's contacts.
Five pushbuttons. Any switches will do the job as well.

About Keyboard control

NOTE: The sketches contained in this lab will cause the microcontroller to take control of your keyboard. Make sure they’re working properly before you add the keyboard commands. The example doesn’t introduce the Keyboard commands until the end of the lab. Instead, messages are printed to the serial monitor to tell you what should happen. When you’ve run this and seen the serial messages occurring when you think they should, then you can add the mouse commands safely.

The circuit diagrams below show an Arduino Leonardo, but you can use any USB-capable board that you wish.

Prepare the breadboard

Connect power and ground on the breadboard to power and ground from the microcontroller. On the Arduino module, use the 5V and any of the ground connections as shown below in Figure 1:

An Arduino Leonardo on the left connected to a solderless breadboard, right. The Leonardo's 5V output hole is connected to the red column of holes on the far left side of the breadboard. The Leonardo's ground hole is connected to the blue column on the left of the board. The red and blue columns on the left of the breadboard are connected to the red and blue columns on the right side of the breadboard with red and black wires, respectively. These columns on the side of a breadboard are commonly called the buses. The red line is the voltage bus, and the black or blue line is the ground bus.

Figure 1: An Arduino Leonardo on the left connected to a solderless breadboard, right.

Made with Fritzing

Add Several Pushbuttons

Attach pushbuttons to digital pin 2 through 5 as shown below in Figure 2. Connect one side of each pushbutton to voltage, and the other side of the pushbuttons to 10-kilohm resistors. Connect the other end of each resistor to ground. Connect the junction where each pushbutton and resistor meet to a digital pin, using pins 2 through 6. (For more on this digital input circuit, see the Digital Input Lab).

Breadboard drawing of an Arduino Leonardo connected to five pushbuttons. The pushbuttons straddle the center divide of the breadboard in rows 4-6, 9-11, 14-16, 19-21, and 24-26, respectively. 10-kilohm resistors straddle the center divide in rows 7, 12, 17, and 22, respectively. Each resistor's row on the left center side is connected to the row above it with a wire, thus connecting the pushbuttons and the resistors. Each resistor's row on the right center side is connected to the right side ground bus with a black wire. Each pushbutton's upper row (that is, rows 4, 9, 14, 19, and 23) on the right center side is connected to the right side voltage bus with a red wire. The junction rows, that is, rows 6, 11, 16, 21, and 26, are connected to digital inputs 2 through 6 on the Leonardo, respectively, with blue wires
Figure 2: Breadboard drawing of an Arduino Leonardo connected to five pushbuttons.
Schematic drawing of five pushbuttons attached to an Arduino Leonardo. The pushbuttons is attached to digital pin 2 through 5 on one side, and to +5 volts on the other. There is also a 10-kilohm resistor attached to each of digital pins 2 through 6. The other side of each resistor is attached to ground.
Schematic drawing of five pushbuttons attached to an Arduino Leonardo.

Add an LED

Add an LED and 220-ohm resistor on pin 13 as shown below in Figure 3. You’ll use this to keep track of the state of the keyboard control.

Breadboard drawing of an Arduino Leonardo connected to five pushbuttons. The pushbuttons straddle the center divide of the breadboard in rows 4-6, 9-11, 14-16, 19-21, and 24-26, respectively. 10-kilohm resistors straddle the center divide in rows 7, 12, 17, and 22, respectively. Each resistor's row on the left center side is connected to the row above it with a wire, thus connecting the pushbuttons and the resistors. Each resistor's row on the right center side is connected to the right side ground bus with a black wire. Each pushbutton's upper row (that is, rows 4, 9, 14, 19, and 23) on the right center side is connected to the right side voltage bus with a red wire. The junction rows, that is, rows 6, 11, 16, 21, and 26, are connected to digital inputs 2 through 6 on the Leonardo, respectively, with blue wires. A 220-ohm resistor is connected to pin 13 of the board, and straddles the center divide of the breadboard in row 1. The anode of an LED is attached to the other side of the resistor, and the cathode is attached to the right side ground bus.
Figure 3: Breadboard drawing of an Arduino Leonardo connected to five pushbuttons and an LED
Schematic drawing of five pushbuttons attached to an Arduino Leonardo. The pushbuttons is attached to digital pin 2 through 5 on one side, and to +5 volts on the other. There is also a 10-kilohm resistor attached to each of digital pins 2 through 6. The other side of each resistor is attached to ground. A 220-ohm resistor is connected to pin 13. The anode of an LED is connected to the other side of the resistor, and its cathode is connected to ground.
Schematic drawing of five pushbuttons and an LED attached to an Arduino Leonardo.

In the Sensor Change Lab, you learned how to detect the state change of a digital input. You’ll use the same technique here.

You’ll use the pushbutton on pin 2 to track whether the microcontroller is acting as a keyboard or not. Pressing this button will change the state of a global variable called keyboardIsActive. It will also call the Keyboard.begin() and Keyboard.end() commands as needed to take or relinquish control of the keyboard. The other four pushbuttons are intended to imitate keystrokes. The first will send the shift key. The last three will send the letters a, s, and d. With these options, you’ll learn to send both regular keystrokes as well as key combinations.

To make this work, you’ll need to know not only when each button is pressed and released, but also when it is still being pressed. For the keyboard activator button, you’ll change the state of the keyboard whenever the button is pressed. For the shift button, you’ll send a message on press and release. For the other buttons, you’ll send a message on press and release, and when the button is still being pressed, you’ll send the keystroke repeatedly, after a short delay.

Start by including the Keyboard library, then add two global arrays to track the button states and the previous button states. Also add a boolean variable to track whether the microcontroller is acting as a keyboard or not, a variable for the key repeat rate in milliseconds, and an array holding the keys you want to press with four of the five keys:


#include <Keyboard.h>
// Global variables:
int buttonState[5];      // states of the buttons
int lastButtonState[5];  // previous states of the buttons
// whether or not the Arduino is controlling the keyboard:
bool keyboardIsActive = false;
int repeatRate = 50;    // key repeat rate, in milliseconds

// keys to be pressed by the keyboard buttons:
int key[] = {KEY_LEFT_SHIFT, 'a', 's', 'd'};

Now, in the setup() function, set all the keys to be inputs. You can do this quickly with a for loop. Set the LED pin as an output as well:

void setup() {
  // initialize serial communication:
  Serial.begin(9600);
  // set the pins as INPUT_PULLUPs:
  for (int b = 2; b < 7; b++) {
    pinMode(b, INPUT_PULLUP);
  }
  // make the LED pin an output:
  pinMode(13, OUTPUT);
}

Detect the State Change of the Buttons

Now  in your main loop() function you need to detect whether each button is pressed, released, or repeating. This is a slight variation on the digital in state change algorithm from the Sensor Change Lab. Again, you can use a for loop to do this for all the buttons like so:

void loop() {
  // iterate over the buttons:
  for (int b = 0; b < 5; b++) {
    // pin number = array index number + 2:
    buttonState[b] = digitalRead(b + 2);
    // see if the button has changed:
    if (buttonState[b] != lastButtonState[b]) {
      Serial.print("button ");
      Serial.print(b);
      if (buttonState[b] == HIGH)
      { // pressed
        Serial.println(" pressed");
      } else { // released
        Serial.println(" released");
      }
    } else {
      if (buttonState[b] == HIGH) {
        Serial.println(" still pressed");
      }
    }
    // save button's current state as previous state for next loop:
    lastButtonState[b] = buttonState[b];
  }
}

If you run the code now, you’ll see the messages “pressed”, “released” and “still pressed” for each key. You need to do something different foreach of those states for  each key, as follows:

  • Keyboard state button (pin 2): on press, toggle the keyboard state
  • Shift key button (pin 3): If keyboardIsActive is true, on press, send shift key press. On release, send shift key release. On still pressed, do nothing
  • Other key buttons:  If keyboardIsActive is true, on press, send key press. On release, send key release. On still pressed, send key press after key repeat delay

You can write a function in which you pass the button number and the state of the button to make this happen. First, add the following function after the loop() function:

void buttonAction(int buttonNumber, int buttonState) {
  // the keyboard state change button
  if (buttonNumber == 0) {
    // on press, toggle keyboardIsActive
    if (buttonState == HIGH) {
      keyboardIsActive = !keyboardIsActive;
    }
  }
  // the other three buttons:
  if (keyboardIsActive) {
    if (buttonNumber > 0 ) {
      if (buttonState == HIGH) {
        Serial.println(" pressed");
      }
      if (buttonState == LOW) {
        Serial.println(" released");
      }
    }
    // all the buttons other than the shift button and the keyboard state button:
    if (buttonNumber > 1) {
      if (buttonState == 2) { // still pressed
        Serial.println(" repeating");
        delay(repeatRate);
      }
    }
  }
}

Then replace all the Serial.println() functions in the main loop() function with calls to this function, like so:

 void loop() {
  // iterate over the buttons:
  for (int b = 0; b < 5; b++) {
    // pin number = array index number + 2:
    buttonState[b] = digitalRead(b + 2);
    // see if the button has changed:
    if (buttonState[b] != lastButtonState[b]) {
      Serial.print("button ");
      Serial.print(b);
      if (buttonState[b] == HIGH)
      { // pressed
        buttonAction(b, 0);
        //Serial.println(" pressed");
      } else { // released
        buttonAction(b, 1);
        //Serial.println(" released");
      }
    } else {
      if (buttonState[b] == HIGH) {
        buttonAction(b, 2);
        //Serial.println(" still pressed");
      }
    }
    // save button's current state as previous state for next loop:
    lastButtonState[b] = buttonState[b];
  }
}

When you run this sketch now, you should see that when you press the first button, it enables the other three to print out “pressed”, “released” and “repeating” messages.

Add commands to control the Keyboard

Finally, change the Serial.println() statements in the buttonAction() functions to Keyboard.press() or Keyboard.release() functions like so. Also add a block of code to activate or deactivate the Keyboard library and to set the LED (n.b. an audio alternative to the LED follows this code block). Note that you’re getting the key value from the key[] array, using the button number as the index:

void buttonAction(int buttonNumber, int buttonState) {
  // the keyboard state change button
  if (buttonNumber == 0) {
    // on press, toggle keyboardIsActive
    if (buttonState == HIGH) {
      keyboardIsActive = !keyboardIsActive;
      // activate the keyboard:
      if (keyboardIsActive) {
        Keyboard.begin();
      } else {
        Keyboard.end();
      }
      // change the LED to reflect the keyboard state:
      digitalWrite(13, keyboardIsActive);
    }
  }
  // the other three buttons:
  if (keyboardIsActive) {
    if (buttonNumber > 0 ) {
      if (buttonState == HIGH) {
        Keyboard.press(key[buttonNumber - 3]);
        // Serial.println(" pressed");
      }
      if (buttonState == LOW) {
        Keyboard.release(key[buttonNumber - 3]);
        // Serial.println(" released");
      }
    }
    // all the buttons other than the shift button and the keyboard state button:
    if (buttonNumber > 1) {
      if (buttonState == 2) { // pressed
        Keyboard.press(key[buttonNumber - 3]);
        // Serial.println(" repeating");
        delay(repeatRate);
      }
    }
  }
}

If you prefer an audio notification instead of the LED, put a buzzer on pin 13 and change the lines following the one that changes the keyboardIsActive variable as follows:

      keyboardIsActive = !keyboardIsActive;
      // activate the keyboard:
      if (keyboardIsActive) {
        Keyboard.begin();
        //buzz once for active:
        digitalWrite(13, HIGH);
        delay(300);
        digitalWrite(13, LOW);
      } else {
        Keyboard.end();
        //buzz twice for inactive:
        digitalWrite(13, HIGH);
        delay(300);
        digitalWrite(13, LOW);
        delay(300);
        digitalWrite(13, HIGH);
        delay(300);
        digitalWrite(13, LOW);
      }

Once you upload this change, your board will be controlling the keyboard. Open a text editor and press the keyboard activation button. The LED will light up to indicate that keyboard control is active. Then press the last three keys, and it should send a, s, and d to the text editor. Hold shift while you press the other keys and you’ll get A, S, and D.

Shouldn’t I Use the Shift Key to Modify The Behavior of Others in Code?

You may be thinking that the code above is not finished, and that you should be sending “a” when the shift key is unpressed, but “A” when it is. That is what is happening, but that’s not the microcontroller’s job. It only has to report which keys are pressed. The operating system on the receiving computer decides what to do with the results.

The Keyboard Library Commands

In the sketch above, you used the Keyboard.begin() and Keyboard.end() commands to take or relinquish control of the keyboard, and the Keyboard.press(), Keyboard.release(), and Keyboard.write() commands to send keystrokes. There is more you can do with the Keyboard library. You can use many of the same commands you’ve used in the Serial library, like print() and println(). You can even program the Arduino to reprogram itself. Check out the Keyboard library documentation for more.

The full sketch for this can be found on the phys comp github repository, called LabKeyboardControl.

Originally written on October 13, 2018 by Tom Igoe
Last modified on November 8, 2018 by Jim Schmitz