Asynchronous Serial – ITP Physical Computing

Lab: Two-Way (Duplex) Serial Communication Using An Arduino and the p5.webserial Library

Introduction

In the Introduction to Asynchronous Serial Communication lab, you learned about various methods for managing the communications between computers via asynchronous serial communication. These included formatting your data as ASCII-encoded strings or raw serial bytes and managing the flow of data using handshaking. In the P5.js WebSerial Input Lab, you sent data from one sensor to a personal computer. In this lab, you’ll send data from multiple sensors to a program in p5.js. using the p5.WebSerial library. You’ll use the data from the sensors to create a pointing-and-selecting device (i.e. a mouse).

What You Should Know

To get the most out of this tutorial, you should know what a microcontroller is and how to program them. You should also understand asynchronous serial communication between microcontrollers and personal computers. You should also understand the basics of P5.js. It would also help to go through the following labs first:

These videos might help in understanding this lab as well:

Video: Two-way Serial Communication (Call-and-Response)
These videos are explained in the Processing programming environment, but the concepts still apply:
- Video: Reading Serial Strings in Processing
- Video: Sending and Reading Multiple Serial Values in Processing

Things You’ll Need

For this lab, you’ll need the hardware below, and you’ll need the same software setup as the WebSerial Input to P5.js lab: You’ll create a p5.js sketch. You’ll also use the p5.WebSerial library. You can use the p5.js web editor or your favorite text editor for this (the Visual Studio Code editor works well).

Figures 1-5 below are the parts you’ll need for this exercise. Click on any image for a larger view.

Photo of an Arduino Nano 33 IoT module. The USB connector is at the top of the image, and the physical pins are numbered in a U-shape from top left to bottom left, then from bottom right to top right. — Figure 1. Microcontroller. Shown here is an Arduino Nano 33 IoT

Photo of flexible jumper wires — Figure 2. Jumper wires. You can also use pre-cut solid-core jumper wires.

Photo of a solderless breadboard — Figure 3. A solderless breadboard

Photo of four breadboard-mounted pushbuttons — Figure 4. A pushbutton

Photo of two potentiometers — *Figure 5. two potentiometers. You can use any two analog sensors in place of these if you prefer.*

Connect the Sensors

For this exercise, you’re going to need two analog inputs to your microcontroller, and one digital input. It doesn’t matter what they are, so use something that’s easy for you to set up. The photos and schematic in this lab show potentiometers and a pushbutton. You don’t have to use these, though. Any three sensor inputs will do the job. If you’re looking for options, consider:

Photo of a breadboard-mountable joystick, This component is mounted on a printed circuit board that's about 4cm on each side. The joystick itself is about 6cm tall, controllable by a thumb. There are five pins on one side of the PCB for mounting on the breadboard. — *Figure 6. A joystick, which consists of two potentiometers and a pushbutton*

Photo of a rotary encoder — *Figure 7. Rotary encoders, which include a built-in pushbutton*

Photo of the IMU sensor on teh Nano 33 IoT. It's the small rectangular chip above and to the left of the main processor. — *Figure 8. The built-in accelerometer on the Arduino Nano 33 IoT, which measures acceleration on three axes*

As long as you have three sensors that will output changing readings, you can follow this lab.

Connect the two analog sensors to analog pins 0 and 1 like you did in the analog input to Arduino lab. Connect a pushbutton to digital pin 2 like you did in the digital input and output with Arduino lab.

Figure 11. Breadboard view of an Arduino Nano attached to two potentiometers and a pushbutton. The potentiometers’ center pins are connected to the Arduino’s A0 and A1 inputs, respectively. Their left pins are connected to the voltage bus, and the right pins are connected to the ground bus, respectively. The pushbutton is connected from the Arduino’s voltage output to pin D2. a 10-kilohm connects the junction of the switch and pin D2 to ground.

(Diagrams made with Fritzing, a circuit design program)

Sending Multiple Serial Data using Punctuation

You’re going to program the microcontroller to read the pushbutton and two analog sensors just like you did in the Intro to Serial Communications Lab. When you have to send multiple data items, you need a way to separate them. If you’re sending them as ASCII-encoded strings, it’s simple: you can just put non-numeric punctuation bytes between them (like a comma or a space) and a unique termination punctuation at the end (like a newline and/or carriage return).

This program will send the two analog sensor values and then the pushbutton. All three will be ASCII-encoded numeric strings, separated by commas. The whole line of sensor values will be terminated by carriage return (\r, ASCII 13) and newline (\n, ASCII 10).

const int buttonPin = 2;      // digital input
 
void setup() {
  // configure the serial connection:
  Serial.begin(9600);
  // configure the digital input:
  pinMode(buttonPin, INPUT);
}
 
void loop() {
  // read the first analog sensor:
  int sensorValue = analogRead(A0);
  // print the results:
  Serial.print(sensorValue);
  Serial.print(",");
 
  // read the second analog sensor:
  sensorValue = analogRead(A1);
  // print the results:
  Serial.print(sensorValue);
  Serial.print(",");
 
  // read the button:
  sensorValue = digitalRead(buttonPin);
  // print the results:
  Serial.println(sensorValue);
}

When you run this and output it to the Serial Monitor, you should see something like this:

348,363,1
344,362,1
345,363,1
344,375,0
365,374,0
358,369,0
355,369,0
352,373,0
356,373,0

Turn the potentiometers (or tweak the analog sensors) and push the button. Now you’ve got a data format: three sensors, comma-separated, terminated by carriage return and newline. This means that you already have an algorithm for how you’re going to program p5.js to read the serial input. You’ll see that algorithm in the next section.

Receive the data in P5.js

Now write a P5.js sketch that reads the data as formatted by the Arduino program above. The setup will be the same as it was in the Serial Input to p5.js using WebSerial lab. The checklist from that lab lays out all the important parts you need.

The sketch you’re going to write will:

Read the incoming serial data into a string until a carriage return and newline appear
split the string into substrings on the commas
convert the substrings into numbers
assign the numbers to variables to change your programNow that you’ve got a plan, put it into action.

Make a P5.js sketch. If you’re using the p5.js web editor, make a new sketch. Click the Sketch Files tab, and then choose the index.html file. Edit the head of the document as you did for the other p5.webserial labs. It should look like this:

<script src="https://cdnjs.cloudflare.com/ajax/libs/p5.js/1.4.0/p5.js"></script>
     
<script src="https://unpkg.com/p5-webserial@0.1.1/build/p5.webserial.js"></script>

The setup of your sketch will initialize the P5.webserial library and define your callback functions for serial events as you did in other sketches. It should look like this:

// variable to hold an instance of the p5.webserial library:
const serial = new p5.WebSerial();
 
// HTML button object:
let portButton;
let inData;                      // for incoming serial data
let outData;                     // for outgoing data
 
function setup() {
  createCanvas(400, 300);          // make the canvas
  // check to see if serial is available:
  if (!navigator.serial) {
    alert("WebSerial is not supported in this browser. Try Chrome or MS Edge.");
  }
  // if serial is available, add connect/disconnect listeners:
  navigator.serial.addEventListener("connect", portConnect);
  navigator.serial.addEventListener("disconnect", portDisconnect);
  // check for any ports that are available:
  serial.getPorts();
  // if there's no port chosen, choose one:
  serial.on("noport", makePortButton);
  // open whatever port is available:
  serial.on("portavailable", openPort);
  // handle serial errors:
  serial.on("requesterror", portError);
  // handle any incoming serial data:
  serial.on("data", serialEvent);
  serial.on("close", makePortButton);
}
function draw() {
 
}
 
// if there's no port selected, 
// make a port select button appear:
function makePortButton() {
  // create and position a port chooser button:
  portButton = createButton('choose port');
  portButton.position(10, 10);
  // give the port button a mousepressed handler:
  portButton.mousePressed(choosePort);
}
 
// make the port selector window appear:
function choosePort() {
  serial.requestPort();
}
 
// open the selected port, and make the port 
// button invisible:
function openPort() {
  // wait for the serial.open promise to return,
  // then call the initiateSerial function
  serial.open().then(initiateSerial);
 
  // once the port opens, let the user know:
  function initiateSerial() {
    console.log("port open");
  }
  // hide the port button once a port is chosen:
  if (portButton) portButton.hide();
}
 
// read any incoming data as a byte:
function serialEvent() {
  
}
 
// pop up an alert if there's a port error:
function portError(err) {
  alert("Serial port error: " + err);
}
 
// try to connect if a new serial port 
// gets added (i.e. plugged in via USB):
function portConnect() {
  console.log("port connected");
  serial.getPorts();
}
 
// if a port is disconnected:
function portDisconnect() {
  serial.close();
  console.log("port disconnected");
}

Change the serialEvent() function to read the incoming serial data as a string until it encounters a carriage return and newline (“\r\n”). Then check to see that the resulting string has a length greater than 0 bytes. If it does, use the split() function to split it in to an array of strings. If the resulting array is at least three elements long, you have your three sensor readings. The first reading is the first analog sensor, and can be mapped to the horizontal movement using the locH variable. The second is the second analog sensor and can be mapped to the locV variable. The third is the button. When it’s 0, set the circleColor variable equal to 255 and when it’s 1, set the variable to 0. Here’s how:

function serialEvent() {
  // read a string from the serial port
  // until you get carriage return and newline:
  var inString = serial.readStringUntil("\r\n");
  //check to see that there's actually a string there:
  if (inString) {
    // split the string on the commas:
    var sensors = split(inString, ",");
    if (sensors.length > 2) {
      // if there are three elements
      // element 0 is the locH:
      locH = map(sensors[0], 0, 1023, 0, width);
      // element 1 is the locV:
      locV = map(sensors[1], 0, 1023, 0, height);
      // element 2 is the button:
      circleColor = 255 - sensors[2] * 255;
    }
  }
}

Note the mappings of sensors[0] and sensors[1]. If you’re not using potentiometers as the first two inputs on your Arduino, then you should use the input mappings for your sensors instead of 0 and 1023. If your analog values are greater than the width of the sketch or the height, the circle will be offscreen, which is why you have to map your sensor range to the screen size.

Program the draw() function to draw a circle that’s dependent on three global variables, locH, locV, and circleColor. Add these three globals to the top of the program:

// variables for the circle to be drawn:
let locH, locV;
let circleColor = 255;

Finally, here is the draw function:

function draw() {
  background(0);               // black background
  fill(circleColor);           // fill depends on the button
  ellipse(locH, locV, 50, 50); // draw the circle
}

If you run this, you should see the circle moving onscreen whenever you change your sensors. When you press the pushbutton, the circle will disappear. Okay, it’s not exactly a mouse, but you are controlling an animation from a device that you built.

Flow Control: Call and Response (Handshaking)

You’ve seen now that by coming up with a serial format (called a protocol), you can write the algorithm for receiving it even before you see any data. You can send multiple pieces of data this way, as long as you format it consistently.

Sometimes you can run into a problem when the sender sends faster than the receiver can read. When this happens, the receiver program slows down as the serial buffer fills up. You can manage this by implementing some form of flow control. The simplest way do to this is using a call-and-response method, where the sending program only sends when it’s told to do so, and the receiving program has to request new data every time it finishes reading what it’s got.

You can add handshaking to the code above fairly simply. Modify the Arduino code as follows. First, add a a new block of code in the setup() This block sends out a message until it gets a byte of data from the remote computer:

void setup() {
  Serial.begin(9600);
  while (Serial.available() <= 0) {
    Serial.println("hello"); // send a starting message
    delay(300);              // wait 1/3 second
  }
}

Now, modify the loop() by adding an if() statement to look for incoming serial data and read it.

void loop() {
  if (Serial.available() > 0) {
    // read the incoming byte:
    int inByte = Serial.read();
    // read the sensor:
    int sensorValue = analogRead(A0);
    // print the results:
    Serial.print(sensorValue);
    Serial.print(",");
 
    // read the sensor:
    sensorValue = analogRead(A1);
    // print the results:
    Serial.print(sensorValue);
    Serial.print(",");
 
    // read the sensor:
    sensorValue = digitalRead(buttonPin);
    // print the results:
    Serial.println(sensorValue);
  }
}

The rest of the sketch remains the same. When you run this and open the serial monitor, you’ll see:

hello
hello
hello
hello

Type any character in the output box and click Send. You’ll get a string of sensor values at the end of your hellos:

510,497,0

Type another character and click Send. It doesn’t matter what character you send, but the loop will always wait for an incoming byte before sending a new set of sensor values. When you write a program to receive this format, it just has to behave the same way you did:

Open the serial port
Wait for a hello
Send a byte to request data
Begin loop:
Wait for one set of data
Send a byte to request new data
end loop

Next, modify the P5.js sketch. Most of the changes are in the serialEvent() function. The initial “hello” messages will trigger this function, so when you get a “hello” or any other string, you need to send a byte back so that the Arduino has a byte available to read. Here’s the new serialEvent():

function serialEvent() {
  // read a string from the serial port
  // until you get carriage return and newline:
  var inString = serial.readStringUntil("\r\n");
  //check to see that there's actually a string there:
  if (inString) {
    if (inString !== "hello") {
      // if you get hello, ignore it
      // split the string on the commas:
      var sensors = split(inString, ",");
      if (sensors.length > 2) {
        // if there are three elements
        // element 0 is the locH:
        locH = map(sensors[0], 0, 1023, 0, width);
        // element 1 is the locV:
        locV = map(sensors[1], 0, 1023, 0, height);
        // element 2 is the button:
        circleColor = 255 - sensors[2] * 255;
        // send a byte back to prompt for more data:
        serial.print('x');
      }
    }
  }
}

You also need to add a line to the initiateSerial() function (which is inside the openPort() function) like so:

function initiateSerial() {
   console.log("port open");
   // send a byte to start the microcontroller sending:
   serial.print("x");
 }

The reason for this is that if your Arduino is still in the setup() waiting for a byte to arrive, then it needs p5.js to send something when the port is opened. If the Arduino has already broken out of the loop (let’s say you opened the Serial monitor to check), then it is waiting for a byte from p5.js to send the next block of code. Whether it’s in the initiateSerial() function or at the end of the serialEvent() function, by sending a byte when you know the port has just been opened in p5.js, you force the Arduino to send you new data.

That’s it. Your sketch should still run just as it did before, though the serial communication is managed better now, because Arduino’s only sending when P5.js is ready to receive.

You can see the sketch running on GitHub at this link. You can see the source files for copying into the p5.js editor at this link.

Advantages of Raw Binary vs. ASCII

All the examples shown here sent the sensor values as ASCII-encoded strings. As mentioned above, that means you sent three bytes to send a three-digit value. If that same value was less than 255, you could send it in one raw binary byte. So ASCII is definitely less efficient. However, it’s more readable for debugging purposes, and if the receiving program is well-suited to convert strings to numbers, then ASCII is a good way to go. If the receiver’s not so good at converting strings to numbers (for example, it’s more challenging to read a multiple byte string in Arduino than in Processing) then you may want to send your data as binary values.

Advantages of Punctuation or Call-and-Response

The punctuation method for sending multiple serial values may seem simpler, but it has its limitations. You can’t easily use it to send binary values, because you need to have a byte with a unique value for the punctuation. In the example above, you’re using the value 10 (ASCII newline) as punctuation, so if you were sending your sensor values as raw bytes, you’d be in trouble when the sensor’s value is 10. The receiver would interpret the 10 as punctuation, not as a sensor value. In contrast, call-and-response can be used whether you’re sending data as raw binary values or as ASCII-encoded values.

Sometimes the receiver reads serial data slower than the sender sends it. For example, if you have a program that does a lot of graphic work, it may only read serial data every few milliseconds. The serial buffer will get full in that case, you’ll notice a lag in response time. This is when it’s good to switch to a call-and-response method.

Build an Application of Your Own

You just duplicated the basic functionality of a mouse; that is, a device with two analog sensors that affect X and Y, and a digital sensor (mouse button). What applications can you think of that could use a better physical interface for a mouse? A video editor that scrubs forward and back when you tilt a wand? An action game that reacts to how hard you hit a punching bag? An instructional presentation that speeds up if you shift in your chair too much? A music program driven by a custom musical instrument that you design?

Create a prototype in Arduino and P5.js, Node.js, Processing, or whatever programming environment you choose. Come up with a physical interface that makes it clear what actions map to what movements and actions. Figure out which actions can and should be possible at the same time. Present a working software and hardware model of your idea.

Lab: Serial Output From p5.js Using the p5.webserial Library

In this lab you’ll learn how to send data from p5.js to a microcontroller using asynchronous serial communication.

Overview

When you use the p5.webserial library for P5.js, it uses the W3C’s WebSerial API to allow your browser to communicate with serial ports on your computer. This lab shows you how to use P5 to control a microcontroller using asynchronous serial communication. WebSerial is currently only available in the Chrome and Chromium browsers and the Microsoft Edge browser, so make sure you’re using one of those to do this lab.

To get the most out of this tutorial, you should know what a microcontroller is and how to program them. You should also understand asynchronous serial communication between microcontrollers and personal computers. You should also understand the basics of P5.js, and should have tried the WebSerial Input to P5.js lab.

Things You’ll Need

For this lab, you’ll need the hardware below,

*Figure 2. LEDs. Shown here are four LEDs. The one on the right is an RGB LED. You can tell this because it has four legs, while the others have only two legs.*

*Figure 3. Resistors. Shown here are 220-ohm resistors. You can tell this because they have two red and one brown band, followed by a gold band.*

An 8 ohm speaker with 2 wires solder to the speakers leads — *Figure 4. An 8 ohm speaker (optional).This is a good alternate to the LED if you prefer audible output.*

Prepare the breadboard

Connect power and ground on the breadboard to power and ground from the microcontroller. On the Arduino Uno, use the 5V and any of the ground connections. On the Nano, use 3.3V and the ground connections:

An Arduino Uno on the left connected to a solderless breadboard, right. The Uno's 5V output hole is connected to the red column of holes on the far left side of the breadboard. The Uno's ground hole is connected to the blue column on the left of the board. The red and blue columns on the left of the breadboard are connected to the red and blue columns on the right side of the breadboard with red and black wires, respectively. These columns on the side of a breadboard are commonly called the buses. The red line is the voltage bus, and the black or blue line is the ground bus. — *Figure 5. An Arduino Uno on the left connected to a solderless breadboard, right.*

Arduino Nano on a breadboard. — *Figure 6. Breadboard view of an Arduino Nano mounted on a breadboard.*

The +3.3 volts and ground pins of the Arduino Nano are connected by red and black wires(Figure 6), respectively, to the left side rows of the breadboard. +3.3 volts is connected to the left outer side row (the voltage bus) and ground is connected to the left inner side row (the ground bus). The side rows on the left are connected to the side rows on the right using red and black wires, respectively, creating a voltage bus and a ground bus on both sides of the board.Figure 5. Breadboard view of an Arduino Nano connected to a breadboard. The +3.3 volts and ground pins of the Arduino are connected by red and black wires, respectively, to the left side rows of the breadboard. +3.3 volts is connected to the left outer side row (the voltage bus) and ground is connected to the left inner side row (the ground bus). The side rows on the left are connected to the side rows on the right using red and black wires, respectively, creating a voltage bus and a ground bus on both sides of the board.

Made with Fritzing

Add an LED

Connect the LED and resistor to digital I/O pin 11 of the module(Figure 7-8). Alternately, you can replace the 220-ohm LED with a speaker (Figure 9-10). You’ll find code below that uses tones instead of LEDs where appropriate. For more on how to do that, see the Tone Output lab:

Schematic view of an Arduino connected to an LED. Digital pin 5 is connected to a 22-ohm resistor. The other side of the resistor is connected to the anode (long leg) of an LED. The cathode of the LED is connected to ground. — *Figure 7. Schematic view of an Arduino connected to an LED.*

Breadboard view of an Arduino connected to an LED. The +5 volts and ground pins of the Arduino are connected by red and black wires, respectively, to the left side rows of the breadboard. +5 volts is connected to the left outer side row (the voltage bus) and ground is connected to the left inner side row (the ground bus). The side rows on the left are connected to the side rows on the right using red and black wires, respectively, creating a voltage bus and a ground bus on both sides of the board. A blue wire connects Digital to a 22-ohm resistor that straddles the center divide of the breadboard in row 17. The other side of the resistor is connected to the anode (long leg) of an LED. The LED is mounted in rowsd 16 and 17 of the right side of the center section of the board. a black wire connects the cathode's row, row 16, to the ground bus on the right side of the board. — *Figure 8. Breadboard view of an Arduino connected to an LED.*

Figure 9 shows a breadboard view of an LED connected to digital pin 5 of an Arduino Nano. The Nano straddles the center of the breadboard in the first fifteen rows. The Nano’s voltage pin (physical pin 2) connects to the board’s voltage bus, and the Nano’s ground pin (physical pin 14) connects to the board’s ground bus. The LED is in the right center of the board, with its anode in one row and the cathode in the next. A 220-ohm resistor connects the LED’s anode to a wire connecting to digital pin 5. The LED’s cathode is connected to the ground bus.

Breadboard view of an Arduino Nano connected to two force sensing resistors (FSRs) and a speaker. The Nano’s 3.3 Volts (physical pin 2) and ground (physical pin 14) are connected to the voltage and ground buses of the breadboard as usual. The red positive wire of the speaker is connected to digital pin 5 of the Arduino. The black ground wire of the speaker is connected to one leg of a 100 ohm resistor. The other leg of the resistor connects to ground. — *Figure 10. Breadboard view of an Arduino Nano connected to a speaker to digital pin 5.*

Figure 10 shows a breadboard view of an Arduino Nano connected to a speaker. The Nano’s ground (physical pin 14) is connected to the ground bus of the breadboard as usual. The red positive wire of the speaker is connected to digital pin 5 of the Arduino. The black ground wire of the speaker is connected to one leg of a 100 ohm resistor. The other leg of the resistor connects to ground.

Program the Microcontroller

Program your Arduino to read the analog input as follows:

void setup() {
  Serial.begin(9600);     // initialize serial communications
  pinMode(5, OUTPUT);
}
 
void loop() {
  if (Serial.available() > 0) { // if there's serial data available
    int inByte = Serial.read();   // read it
    Serial.write(inByte);         // send it back out as raw binary data
    analogWrite(5, inByte);       // use it to set the LED brightness
    // if you're using a speaker instead of an LED, uncomment line below  and comment out the previous line:
    //  tone(5, inByte*10);     // play tone on pin 5
  }
}

Only one port at a time can access a serial port.

As you work on this any microcontroller-to-computer application, you will be switching back and forth between the app that programs the microcontroller (in this case, the Arduino IDE) and the app that the microcontroller is communicating with (in this case, p5.js in the browser). You have to keep in mind that only one of these at a time can access a serial port.

That means that when you want to reprogram your Arduino from the Arduino IDE, you should to stop your sketch in the browser window to do so. Then, restart the browser sketch when you’re done reprogramming the Arduino. You don’t need to quit the Arduino IDE each time, because it knows to release the serial port when it’s not programming. However, you do need to close the Serial Monitor in the Arduino IDE when you are using WebSerial in the browser.

The P5.js WebSerial Library

To communicate with your microcontroller serially, you’re going to use the P5.js WebSerial library. If you’re using the p5.js web editor, make a new sketch. Click the Sketch Files tab, and then choose the index.html file. In the head of the document, look for this line:

1	`<script` `src="https://cdnjs.cloudflare.com/ajax/libs/p5.js/1.4.0/p5.js"></script>`

Right after that line, add this line:

1	`<script` `src="https://unpkg.com/p5-webserial@0.1.1/build/p5.webserial.js"></script>`

The P5.js Sketch

The sketch you’re going to write will control the microcontroller’s LED from P5.js. Dragging the mouse up and down the canvas will dim or brighten the LED, and typing 0 through 9 will set the LED’s brightness in increments from off (0) through almost full brightness (9). There’s an alternate sketch that will make changing tones if you prefer that instead of a changing LED. The sketch will also receive serial input from the microcontroller just as in the WebSerial Input to P5.js lab, so that you can see that the microcontroller is getting the same values you’re sending.

The setup of your sketch will initialize the P5.webserial library and define your callback functions for serial events. Program the global variables and setup() function as follows:

// variable to hold an instance of the p5.webserial library:
const serial = new p5.WebSerial();
 
// HTML button object:
let portButton;
let inData;                            // for incoming serial data
let outByte = 0;                       // for outgoing data
 
function setup() {
  createCanvas(400, 300);          // make the canvas
  // check to see if serial is available:
  if (!navigator.serial) {
    alert("WebSerial is not supported in this browser. Try Chrome or MS Edge.");
  }
  // if serial is available, add connect/disconnect listeners:
  navigator.serial.addEventListener("connect", portConnect);
  navigator.serial.addEventListener("disconnect", portDisconnect);
  // check for any ports that are available:
  serial.getPorts();
  // if there's no port chosen, choose one:
  serial.on("noport", makePortButton);
  // open whatever port is available:
  serial.on("portavailable", openPort);
  // handle serial errors:
  serial.on("requesterror", portError);
  // handle any incoming serial data:
  serial.on("data", serialEvent);
  serial.on("close", makePortButton);
}
 
function draw() {
 
}

For now you’re leaving the draw() function empty. You’ll fill it in later. You’ll be adding some functions to read mouse dragging and key pressing as well.

Program the handler functions similarly to those in the WebSerial Input to P5.js lab:

// if there's no port selected, 
// make a port select button appear:
function makePortButton() {
  // create and position a port chooser button:
  portButton = createButton("choose port");
  portButton.position(10, 10);
  // give the port button a mousepressed handler:
  portButton.mousePressed(choosePort);
}
 
// make the port selector window appear:
function choosePort() {
  serial.requestPort();
}
 
// open the selected port, and make the port 
// button invisible:
function openPort() {
  // wait for the serial.open promise to return,
  // then call the initiateSerial function
  serial.open().then(initiateSerial);
 
  // once the port opens, let the user know:
  function initiateSerial() {
    console.log("port open");
  }
  // hide the port button once a port is chosen:
  if (portButton) portButton.hide();
}
 
// read any incoming data as a byte:
function serialEvent() {
  // read a byte from the serial port:
  var inByte = serial.read();
  // store it in a global variable:
  inData = inByte;
}
 
// pop up an alert if there's a port error:
function portError(err) {
  alert("Serial port error: " + err);
}
 
// try to connect if a new serial port 
// gets added (i.e. plugged in via USB):
function portConnect() {
  console.log("port connected");
  serial.getPorts();
}
 
// if a port is disconnected:
function portDisconnect() {
  serial.close();
  console.log("port disconnected");
}
 
function closePort() {
  serial.close();
}

Program the draw() function to display the value of any incoming serial bytes. Here it is:

function draw() {
  // black background, white text:
  background(0);
  fill(255);
  // display the incoming serial data as a string:
  text("incoming value: " + inData, 30, 30);
}

To read the mouse and keyboard, you’ll need to write functions to respond to the ‘mouseDragged’ and ‘keyPressed’ events. ‘MouseDragged’ will happen whenever you click and drag the mouse on the canvas. When that happens, read the mouseY, and map its position on the canvas to a value from 0 to 255. Convert the result to a number using the int() function. Then send it out the serial port using the serial.write() function:

function mouseDragged() {
  // map the mouseY to a range from 0 to 255:
  outByte = byte(map(mouseY, 0, height, 0, 255));
  // send it out the serial port:
  serial.write(outByte);
}

The serial.write() function is versatile. If you give it a variable or literal that’s a numeric data type, it will send it as its raw binary value. In the code above, note how you’re converting the output of the map() function to a number using the int() function. If you give it a string, however, it will send out that ASCII string. So be aware of the difference, and make sure you know whether your serial receiving device wants raw binary or ASCII-encoded data.

Program the keyPressed() function similarly to the mouseDragged() function. You want it to read the key strokes, convert them to raw bytes, and send them out the serial port. But you only want to send them if they key hit was 0 through 9. The P5.js variable key returns a numeric value, so you can do math on it and convert it like so:

function keyPressed() {
  if (key >= 0 && key <= 9) { // if the user presses 0 through 9
    outByte = (key * 25); // map the key to a range from 0 to 225
    serial.write(outByte); // send it out the serial port
  }
}

That’s all you want your sketch to do, so try running it now. You should see that the initial incoming serial value is undefined, but when you drag the mouse up and down, or type 0 through 9, it will update when the Arduino program returns what it received. The LED will also change with these actions.

You can see this sketch running on gitHub at this link. You can get the full text of it at this link.

Sending ASCII-Encoded Serial Data

When you send data from p5.js using p5.webserial, the serial.write() function works like it does in Arduino: it sends numbers as binary data. In the programs above, you’re sending binary data from p5.js and reading it as binary in Arduino.

If you want to send ASCII-encoded serial data from P5.js instead, all you have to do is to serial.print() or serial.println() your string. On the Arduino side, you can read single characters one byte at a time simply as well. However, if you want to convert multi-byte number strings to numeric values, you’ll need a new function to read ASCII encoded numeric strings called parseInt().

Program the Microcontroller Again

To start off with, load a sketch from the Arduino examples called PhysicalPixel. You can find it in the File Menu -> Examples -> Communication -> PhysicalPixel. Here’s what it looks like. Change the LED pin number to pin 5 as follows:

const int ledPin = 5; // the pin that the LED is attached to
int incomingByte;     // a variable to read incoming serial data into
 
void setup() {
  Serial.begin(9600);             // initialize serial communication
  pinMode(ledPin, OUTPUT);        // initialize the LED pin as an output
}
 
void loop() {
  if (Serial.available() > 0) { // see if there's incoming serial data
    incomingByte = Serial.read(); // read it
    if (incomingByte == 'H') {    // if it's a capital H (ASCII 72),
      digitalWrite(ledPin, HIGH); // turn on the LED
      // if you're using a speaker instead of an LED, uncomment line below  and comment out the previous line:
      //  tone(5, 440);           // play middle A on pin 5
    }
    if (incomingByte == 'L') {    // if it's an L (ASCII 76)
      digitalWrite(ledPin, LOW);  // turn off the LED
      // if you're using a speaker instead of an LED, uncomment line below  and comment out the previous line:
      // noTone(5);
    }
  }
}

When you run this, open the serial monitor and type H or L, and the LED will go on or off. Try typing h or l instead. The LED won’t change, because H and h have different ASCII values, as do L and l. But you can see from this that you don’t need to memorize the ASCII chart to check for character values in your code. Put the character you want to read in single quotes, and the Arduino compiler will automatically convert the character to its ASCII value for you. It only works for single characters, though.

Program P5.js To Control the LED

To get P5.js to control this Arduino program serially, you only need to add to the keyPressed() function to read H or L in addition to 0 through 9. Here’s your new mousePressed() function:

function keyPressed() {
  if (key >= 0 && key <= 9) {
    // if the user presses 0 through 9
    outByte = byte(key * 25); // map the key to a range from 0 to 225
    serial.write(outByte); // send it out the serial port
  }
  if (key === "H" || key === "L") {
    // if the user presses H or L
    serial.write(key); // send it out the serial port
  }
}

Because the key is already a single character, P5.js sends it out as is, and Arduino reads it as a single byte, looking for the ASCII value of H or L. Notice how the values returned to P5.js are 72 and 76, the ASCII values for H and L. For single characters like this, exchanging data is simple.

If you tried to change the LED with the mouse, you didn’t see anything happen unless your output value was 72 or 76. Why is that?

To see the sketch running on GitHub at this link. You can see the source files for copying into the p5.js editor at this link.

Processing ASCII-Encoded Strings With Arduino

It is also possible to read and interpret ASCII-encoded strings in Arduino. The String.parseInt() function reads an incoming string until it finds a non-numeric character, then converts the numeric string that it read into a long integer. This is a blocking function, meaning that String.parseInt() stops the program and does nothing until it sees a non-numeric character, or until a timeout passes. The timeout is normally one second (or 1000 milliseconds), but you can set it to a lower number of milliseconds using Serial.setTimeout(). Here’s a variation on the original Arduino sketch from above, using Serial.parseInt() this time:

void setup() {
  Serial.begin(9600);    // initialize serial communications
  Serial.setTimeout(10); // set the timeout for parseInt
  pinMode(5, OUTPUT);
}
 
void loop() {
  if (Serial.available() > 0) { // if there's serial data available
    int inByte = Serial.parseInt(); // read it
    if (inByte > 0) {
      Serial.write(inByte);      // send it back out as raw binary data
      analogWrite(5, inByte);    // use it to set the LED brightness
      // if you're using a speaker instead of an LED, uncomment line below  and comment out the previous line:
      //  tone(5, inByte*10);     // play tone on pin 5
    }
  }
}

Upload this to your microcontroller, then open the Serial Monitor and send in some ASCII numeric strings. You’ll see the character that’s represented by the string’s value. For example, 65 will return A, 34 will return “, and so forth.Notice that this version of the sketch has a conditional statement to check if the incoming byte is 0. This is because of a quirk of the parseInt() function. It returns 0 if the timeout is hit, or if the string is legitimately 0. This means you can’t really parse for a string like this: "0\n".

Program p5.js To Send a String

Now that your microcontroller is expecting a string, program P5.js to send one. This means changing the mouseDragged() function. Program it to print a string with a newline at the end using the serial.println() command like so:

1	`serial.println(outByte);`

Since the newline added at the end by serial.println() is a non-numeric character, the Serial.parseInt() function will see it and parse the string, not waiting for the timeout.

To see the sketch running on GitHub at this link. You can see the source files for copying into the p5.js editor at this link.

Conclusion

When you’re sending data between two computers using asynchronous serial communication, you have to make sure that what the sender is sending is formatted the same as what the receiver is listening for. See Table 1 to review what are suitable data formats for different types/sizes of data and which functions to use on p5.js and Arduino for serial communication.

Number of Bytes	1 Byte		Multi Bytes
Data to Send	A single number < 255	A single character	A single number > 255, multiple values
Send as:	Binary	Ascii	Ascii
p5.js ->	serial.write(integer)	serial.write(string)	serial.write (valueToSend + ",")
-> Arduino	Serial.read()		Serial.parseInt()

Table 1. Serial Communication: p5.js to Arduino

Think this out in advance before you code, then consider what functions you’ve got on both computers to convert data from strings to raw binary numbers and back. Test with fixed values at first, so you know you’re getting what you think you should. For example, sending an ASCII-encoded numeric string like this:

1023\n

Will always result in these six bytes:

49 48 50 51 10

Likewise, this text string:

Hello\n

will always be:

72 101 108 108 111 10

By sending a string you know both the ASCII and raw binary representations of, you can test your code easier, because what you’re sending won’t change. Once you know the sending and receiving works, then you can send variable strings.

The more you work with serial data, the more you’ll become familiar with the methods for handling it.

For more on serial flow control in P5.js, see the Two-Way Duplex Serial Communication Using p5.WebSerial Lab.

Lab: Serial Input to p5.js Using the p5.webserial Library

This lab uses a p5.js library called p5.WebSerial to make it easy in p5.js. In this lab, you’ll generate an analog output value from a potentiometer, then send that value via asynchronous serial communication to P5.js. You’ll use that value in P5.js to draw a graph.

Web browsers have traditionally been designed to be separate from the rest of a computer’s operating system, not able to connect to the computer’s hardware ports for security reasons. Recently, however, the W3C developed the WebSerial API in JavaScript, which allows browsers to communicate with a computer’s serial ports. WebSerial is currently only available in the Chrome and Chromium browsers and the Microsoft Edge browser, so make sure you’re using one of those to do this lab.

Once you gain an understanding of serial communication, you can use any program that can connect with your computer’s serial ports to communicate with a microcontroller. In addition to the WebSerial API you’ll see here, you can use other programming environments to communicate serially. Processing, Max/MSP, and OpenFrameworks are three other popular multimedia programming environments that can communicate via the serial ports. You can also do this with Unity, Unreal, or any other programming environment that can access the serial ports.

Things You’ll Need

For this lab, you’ll need the hardware below, and you’ll need to create a p5.js sketch. You’ll also use the p5.WebSerial library. You can use the p5.js web editor or your favorite text editor for this (the Visual Studio Code editor works well).

Figures 1-4 below show the parts you’ll need for this exercise. Click on any image for a larger view.

Prepare the Breadboard

For this exercise you’re going to attach a potentiometer as an analog input to your microcontroller, and send the sensor’s reading serially to p5.js via the p5.serialcontrol app.

Connect power and ground on the breadboard to the microcontroller. On the Arduino module, use the 5V or 3.3V (depending on your model) and any of the ground connections. Figures 5 and 6 show connections for an Arduino Uno and a Nano, respectively.

The Uno’s 5V output hole is connected to the red column of holes on the far left side of the breadboard (Figure 5). The Uno’s ground hole is connected to the blue column on the left of the board. The red and blue columns on the left of the breadboard are connected to the red and blue columns on the right side of the breadboard with red and black wires, respectively. These columns on the side of a breadboard are commonly called the buses. The red line is the voltage bus, and the black or blue line is the ground bus.

Images made with Fritzing, a circuit drawing application

The Nano is mounted at the top of the breadboard (Figure 6), straddling the center divide, with its USB connector facing up. The top pins of the Nano are in row 1 of the breadboard.

The Nano, like all Dual-Inline Package (DIP) modules, has its physical pins numbered in a U shape, from top left to bottom left, to bottom right to top right. The Nano’s 3.3V pin (physical pin 2) is connected to the left side red column of the breadboard. The Nano’s GND pin (physical pin 14) is connected to the left side black column. These columns on the side of a breadboard are commonly called the buses. The red line is the voltage bus, and the black or blue line is the ground bus. The blue columns (ground buses) are connected together at the bottom of the breadboard with a black wire. The red columns (voltage buses) are connected together at the bottom of the breadboard with a red wire.

Add a Potentiometer

Connect a potentiometer to analog in pin 0 of the module. Figure 7 shows the schematic and figures 8 and 9 show the potentiometer connected to an Arduino and Nano, respectively.

Schematic view of a potentiometer. First leg of the potentiometer is connected to +5 volts. The second leg connected to analog in 0 of the Arduino. The third leg is connected to ground. — *Figure 7. Schematic view of a potentiometer connected to analog in 0 of the Arduino*

Breadboard view of a potentiometer. First leg of the potentiometer is connected to +5 volts. The second leg connected to analog in 0 of the Arduino. The third leg is connected to ground. — Figure 8. Breadboard view of a potentiometer connected to analog in 0 of an Arduino. The potentiometer is connected to three rows in the left center section of the breadboard. The two outside pins are connected to voltage and ground. The center pin is connected to the Arduino’s analog in 0.

Breadboard view of Arduino Nano with an potentiometer input. — Figure 9. Breadboard view of a potentiometer connected to analog in 0 of an Arduino Nano. The Nano is connected as usual, straddling the first fifteen rows of the breadboard with the USB connector facing up. Voltage (physical pin 2) is connected to the breadboard’s voltage bus, and ground (physical pin 14) is connected to the breadboard’s ground bus. The potentiometer is connected to three rows in the left center section of the breadboard. The two outside pins are connected to voltage and ground. The center pin is connected to the Nano’s analog in 0.

Program the Microcontroller

Program your Arduino to read the analog input as follows:

void setup() {
  Serial.begin(9600); // initialize serial communications
}
 
void loop() {
  // read the input pin:
  int potentiometer = analogRead(A0);                  
  // remap the pot value to fit in 1 byte:
  int mappedPot = map(potentiometer, 0, 1023, 0, 255); 
  // print it out the serial port:
  Serial.write(mappedPot);                             
  // slight delay to stabilize the ADC:
  delay(1);                                            
}

Only one port at a time can access a serial port.

The P5.js WebSerial Library

1	`<script` `src="https://cdnjs.cloudflare.com/ajax/libs/p5.js/1.4.0/p5.js"></script>`

Right after that line, add this line:

1	`<script` `src="https://unpkg.com/p5-webserial@0.1.1/build/p5.webserial.js"></script>`

The p5.js Sketch

To start off, you need to know if WebSerial is supported in the browser you’re using. Open the sketch.js file and change it to the following:

// variable to hold an instance of the p5.webserial library:
const serial = new p5.WebSerial();
 
function setup() {
   // check to see if serial is available:
   if (!navigator.serial) {
    alert("WebSerial is not supported in this browser. Try Chrome or MS Edge.");
  } 
}

When you run this p5.js sketch in a browser, you’ll get a message letting you know whether or not this browser supports WebSerial. In a browser that supports WebSerial, you may want to delete the else clause.

Serial Events

JavaScript, the language on which p5.js is based, relies heavily on events and callback functions. An event is generated by the operating system when something significant happens, like a serial port opening, or new data arriving in the port. In your sketch, you write a callback function to respond to that event. The p5.webserial library uses events and callback functions as well. It can listen for the following serialport events:

noport – when there is no selected serial port
portavailable – when a serial port becomes available
data – new data arrives in a serial port
close – the serial port is closed
requesterror – something goes wrong when you request a serial port.

The WebSerial API on which the p5.webserial library is based also has connect and disconnect events for when a serial port is physically disconnected (or in the case of a USB-native device like the Nano 33 IoT, when it is reset). You’ll see those below as well, and you’ll see them run whenever the serial connection to the Arduino is reset.

To use the the webserial library’s events, you need to set callback functions for them. Change your sketch to include a port chooser button and a variable for incoming data, then in the setup() function, add callbacks for open, close, and data, and error like so:

// variable to hold an instance of the p5.webserial library:
const serial = new p5.WebSerial();
 
// HTML button object:
let portButton;
let inData;                   // for incoming serial data
let outByte = 0;              // for outgoing data
 
function setup() {
  createCanvas(400, 300);          // make the canvas
  // check to see if serial is available:
  if (!navigator.serial) {
    alert("WebSerial is not supported in this browser. Try Chrome or MS Edge.");
  }
  // if serial is available, add connect/disconnect listeners:
  navigator.serial.addEventListener("connect", portConnect);
  navigator.serial.addEventListener("disconnect", portDisconnect);
  // check for any ports that are available:
  serial.getPorts();
  // if there's no port chosen, choose one:
  serial.on("noport", makePortButton);
  // open whatever port is available:
  serial.on("portavailable", openPort);
  // handle serial errors:
  serial.on("requesterror", portError);
  // handle any incoming serial data:
  serial.on("data", serialEvent);
  serial.on("close", makePortButton);
}
 
function draw() {
  
 
 
}

The draw() function is empty for the moment. You’ll come back and fill that in later.

Now that you’ve set listeners for the events, you need to add the callback functions. Here they are. Add these after the draw() function:

// if there's no port selected, 
// make a port select button appear:
function makePortButton() {
  // create and position a port chooser button:
  portButton = createButton("choose port");
  portButton.position(10, 10);
  // give the port button a mousepressed handler:
  portButton.mousePressed(choosePort);
}
 
// make the port selector window appear:
function choosePort() {
  if (portButton) portButton.show();
  serial.requestPort();
}
 
// open the selected port, and make the port 
// button invisible:
function openPort() {
  // wait for the serial.open promise to return,
  // then call the initiateSerial function
  serial.open().then(initiateSerial);
 
  // once the port opens, let the user know:
  function initiateSerial() {
    console.log("port open");
  }
  // hide the port button once a port is chosen:
  if (portButton) portButton.hide();
}
 
// pop up an alert if there's a port error:
function portError(err) {
  alert("Serial port error: " + err);
}
// read any incoming data as a string
// (assumes a newline at the end of it):
function serialEvent() {
  inData = Number(serial.read());
  console.log(inData);
}
 
// try to connect if a new serial port 
// gets added (i.e. plugged in via USB):
function portConnect() {
  console.log("port connected");
  serial.getPorts();
}
 
// if a port is disconnected:
function portDisconnect() {
  serial.close();
  console.log("port disconnected");
}
 
function closePort() {
  serial.close();
}

Wait a Minute! Don’t I have to Set the Data Rate When I Open the Port?

In asynchronous serial communications, both computers have to set the same data rate in order to communicate. In Arduino, you set the data rate with Serial.begin(9600); In p5.webserial, 9600 bits per second is the default, so you don’t have to set the rate if you want 9600bps. But if you want to set the rate to another value, change your serial.open() call in the openPort() function as follows:

let options = { baudrate: 9600}; // change the data rate to whatever you wish
serial.open(portName, options);

There are other port options you can set using p5.webserial, including baudRate, bufferSize, dataBits, flowControl, parity and stopBits. These are all standard serialport options, and most programming environments which support serial communication will have them. You can get details of these in the p5.webserial documentation.

p5.WebSerial Sketch Checklist

Most p5.WebSerial sketches that you write will have a similar structure to this one. The main difference between them all will be how you read and interpret incoming serial data, and how and when you send and format outgoing serial data. Here’s a checklist of the pieces you’re likely to see in every sketch:

In the HTML file, include the p5.webserial library
In the global variables of the sketch,
- make a new instance of the library
- include a port selector button or some way to invoke the serial port chooser dialogue box
In the setup:
- Make sure WebSerial is supported in this browser
- Include a call to serial.getPorts() to check for available ports.
- include serial.on() listeners for these events:
  - noport
  - portavailable
  - data
  - close
  - requesterror
- include navigator listeners for connect and disconnect
Define handler functions for all of the events above. Most of these can be simple alerts or console.log messages
Customize the function that responds to the data listener (usually called serialEvent() in these examples), as you’ll see below.
Decide when and how you’ll send serial data out, as you’ll see in the other p5.webserial labs.

The last two items of this list are the ones on which you’ll spend most of your time. The rest of the items are things you’re likely to copy from one sketch to another.

Reading Incoming Serial Data

The event that that you’ll use the most is the data event, which calls the serialEvent() function. Each time a new byte arrives in the serial port, this function is called. Now it’s time to make serialEvent() do some work. Add a new global variable at the top of your sketch called inData like so:

The serialEvent() function you added above looks like this:

function serialEvent() {
  inData = Number(serial.read());
  console.log(inData);
}

It’s reading the incoming data byte by byte, and interpreting each byte as a number. That’s why the Number() function surrounds the read() function. Since you’re sending the data from the Arduino as a binary value (using the Serial.write() function), you have to interpret it in p5.js as a binary value as well.

Next, make the draw() function to print the sensor value to the screen like so:

function draw() {
   background(0);
   fill(255);
   text("sensor value: " + inData, 30, 50);
}

Figure 10. A screenshot of the sketch running in a browser. The sketch prints the sensor value in text on the screen.

When you run your sketch now, you should get something like the sketch shown in Figure 10.

To see the sketch running on GitHub at this link. You can see the source files for copying into the p5.js editor at this link.

The sensor value onscreen should change as you turn your potentiometer. Congratulations! You’ve got P5.js talking to your microcontroller.

What’s Happening Here

Every time your microcontroller sends a byte serially using Serial.write(), the computer receives it and generates a ‘data’ event. Then your serialEvent() function is called. It reads the byte as a number, and stores it in the global variable inData. The draw() method just uses the latest value of inData in the text string it displays on the screen.

You may be wondering why you’re mapping the sensor value or dividing it by 4 in the Arduino sketch above. That’s because in order to send the sensor value as a single byte, it must be between 0 and 255, or no more than 2⁸ bits.

P5.js Console.log() and Arduino delay(): a Tricky Combination

In testing this, you may have put a console.log() statement in the serialEvent() function in your P5.js sketch. When you did, you would have noticed that it causes a lag in the sketch, and the console.log() statements continue even after you stop the sketch. This is because the operating system keeps the incoming serial data in a buffer, and P5.js isn’t reading and printing it as fast as Arduino is sending it.

You might think, “Okay, then I’ll just put a delay() in my Arduino sketch to slow it down.” That’s a bad idea. When you put in a delay, it means you’re only reading your sensor when that delay is not running. You can miss critical sensor events while that delay is in progress. Even a relatively small delay, for example 30ms, can make it difficult to reliably read state changes in a switch or peaks in an analog sensor. Don’t use delays if you can avoid it. For more on how to handle the flow of serial data from Arduino to P5.js and back, see the Duplex Serial Flow using WebSerial in P5.js lab.

Draw a Graph With the Sensor Values

It would be useful to see a graph of the sensor values over time. You can do that by modifying the draw() method to draw the graph. To do this, add a new global variable at the top of your sketch called xPos. You’ll use this to keep track of the x position of the latest graph line:

let xPos = 0;                     // x position of the graph

Because of the way the graphing function below works, you can’t reset the background every time through the draw() loop. So take the background() command and put it in the setup() function instead of the draw(), as shown below. That way it runs once, then not again. As long as you’re at it, switch from black & white to a nice blue color:

function setup() {
  createCanvas(400, 300);
  background(0x08, 0x16, 0x40);

Now make a new function called graphData(). It’ll take a number value as a parameter, and it will draw a line on the screen that’s mapped to the number value. Then it will increment xPos so that the next line is drawn further along. It will also check if the xPos is at the right edge of the screen, and reset the screen by calling background() again if it is:

function graphData(newData) {
  // map the range of the input to the window height:
  var yPos = map(newData, 0, 255, 0, height);
  // draw the line in a pretty color:
  stroke(0xA8, 0xD9, 0xA7);
  line(xPos, height, xPos, height - yPos);
  // at the edge of the screen, go back to the beginning:
  if (xPos >= width) {
    xPos = 0;
    // clear the screen by resetting the background:
    background(0x08, 0x16, 0x40);
  } else {
    // increment the horizontal position for the next reading:
    xPos++;
  }
}

Finally, take everything out of the draw() function and just call graphData() from there:

function draw() {
  graphData(inData);
}

When you run the sketch now, you should get a graph, as shown in Figure 11.

Figure 11. Screenshot of the serial graph p5.js sketch. The sensor’s values are graphed on the screen

To see the sketch running on GitHub at this link. You can see the source files for copying into the p5.js editor at this link.

Reading Serial Data as a String

This works well if you want to read your sensor values as a single byte, but what if you want a larger range of numbers? What if you want the full 0 to 1023 that analogRead() can output instead of just 0 to 255? To do this, you need to send the data as an ASCII-encoded numeric string from the microcontroller, and you need to read and interpret the incoming data in P5 as an ASCII-encoded numeric string as well.

Change your Arduino program to the following:

void setup() {
 Serial.begin(9600); // initialize serial communications
}
 
void loop() {
 int potentiometer = analogRead(A0);                  // read the input pin
 int mappedPot = map(potentiometer, 0, 1023, 0, 255); // remap the pot value to fit in 1 byte
 Serial.println(mappedPot);                           // print it out the serial port
 delay(1);                                            // slight delay to stabilize the ADC
}

Now it will print the potentiometer’s value as an ASCII-encoded numeric string, and it will add a carriage return byte and a newline byte at the end, because that’s what println() does.

Once you’ve uploaded this to your Arduino, run your P5 sketch again. Try adding println(inData); at the end of your serialEvent() function. When your P5 sketch reads the data from this Arduino program, you get very low values, and every so often you see the value 10 followed by the value 13. What’s going on?

When a computer ASCII-encodes a number, it converts that number to a string of bytes, each of which is the ASCII value for a numeral in the number. For example, the number 865 gets converted to three bytes, as shown in Figure 12.

ASCII representation of a three-digit number. — Figure 12. The decimal number 865 when sent serially as ASCII is three bytes long. The first byte representing digit, 8, has the ASCII value 58. The second byte representing the digit 6 has the ASCII value 54. The third byte representing the digit 5 has the ASCII value 53.

If there’s a carriage return byte and a newline byte after this, the string is five bytes, and the last two bytes’ values are 13 (carriage return, or \r in most programming languages) and 10 (newline or \n in most programming languages), respectively.

Your P5.js sketch is reading every byte’s value and graphing it. That’s why you get a graph of very low values, with a bunch of them being 13 and 10. The Arduino is ASCII-encoding the potentiometer values, but the P5 sketch is interpreting the bytes as if they’re not encoded that way.

Now change the serialEvent() function like so:

function serialEvent() {
  // read a byte from the serial port, convert it to a number:
  inData = serial.readLine();
}

Run it again. What’s changed? Now you’re getting a graph kind of like you were before. The serial.readLine(); command reads the incoming serial data as a string, and when that string happens to be all-numeric, it converts it to a number. So you’re getting the ASCII-encoded string as a number again. But sometimes there are gaps. Why?

Remember, the ‘data’ event occurs every time a new byte comes in the serial port. Now that you’re sending an ASCII-encoded string, every potentiometer reading is several bytes long. So you only get a complete string every three to six bytes (three for “0\r\n” and six for “1023\r\n”). Sometimes, when the serialEvent() function calls serial.readLine(); it gets nothing. That’s when draw() draws the gaps. You need to change your function to check that the resulting string is actually a valid number before you put the string into inData. First, create a local variable to get the string, then check to see if the string’s length is greater than zero. If it is, then put it into inData so that the other functions in the sketch can use the new data. Here’s how you do that:

function serialEvent() {
  // read a string from the serial port:
  var inString = serial.readLine();
  // check to see that there's actually a string there:
  if (inString) {
  // convert it to a number:
  inData = Number(inString);
  }
}

Now you’re able to send in a number of any value to P5.js. You don’t have to limit your input to a 0-255 value range. See if you can modify the Arduino sketch and the P5.js sketch to exchange a potentiometer value that spans the whole range from 0 to 1023.

note: readLine() is the same as readStringUntil(‘\r\n’);

You can see the sketch running on GitHub at this link. You can see the source files for copying into the p5.js editor at this link.

Conclusion

In this lab, you saw how to connect an Arduino microcontroller to a P5.js sketch using a webSocket-to-serial server, P5.serialserver, and the P5.serialport library. You sent data from Arduino to the sketch as a raw binary value — that is, a single byte ranging from 0 to 255 — and you sent it as an ASCII-encoded numeric string with a carriage return and newline at the end. See Table 1 below to review what are suitable data formats for different types/sizes of data and which functions to use on p5.js and Arduino for serial communication.

Data to Send	A single value within 0-255 (< 2^8 = 1 byte)	Larger numbers & characters in Ascii Table, Multiple values
Send as:	Binary	Ascii
Arduino ->	Serial.write()	Serial.println()
-> p5.js	serial.read()	serial.readLine() =readStringUntil('\r\n')

Table 1. Serial Communication: Arduino to p5.js

Notes about sending ASCII-encoded data:

Using Serial.println() on Arduino and serial.readLine() on p5.js is one of many different ways of sending data from Arduino to p5.js via serial communication.
If you want to read an ASCII-encoded numeric string as a number and not a string, convert the value into number by using Number().

Understanding the difference between ASCII-encoded strings and raw binary data is central to all serial communications. For more examples of this in action, see the WebSerial Output from P5.js lab.

p5.serialport and p5.webserial Compared

Asynchronous serial communication is not part of the core of p5.js, nor is it part of the core for web browsers. To read serial in a p5.js sketch, or in a web page in general, you need some help. One approach is to open a separate application that connects to the serial port and offers a webSocket connection to the browser. This is the approach that the p5.serialport library and the p5.serialcontrol app take. Another approach is to use the browser API called WebSerial. This is the approach that the p5.webserial library takes.

The difference between them is best illustrated by Figures 1 and 2 below. With p5.seriaport, you must open another application on your computer, p5.serialcontrol, to communicate with the serial port. It is this application that handles serial communication, not the p5.js sketch in the browser. This can be more complicated for beginning users.

Diagram of three rectangles connected by arrows. The rectangle on the right represents your p5.js sketch, running in a browser. Your sketch implements the p5.serialport library. Your sketch connects to p5 serial control, the middle rectangle, via a webSocket. The p5 serial control application runs on your laptop. It connects to a serial port on your computer and listens for webSocket connections from your p5.js sketch. It passes whatever comes in the serial port through to the webSocket and vice versa. The third rectangle is your Arduino, connected to p5 serial control via the serial port. — *Figure 1. Diagram of the connection from the serial port to p5.js through p5.serialcontrol*

Diagram of two rectangles connected by arrows. The rectangle on the right represents your p5.js sketch, running in a browser. Your sketch implements the p5.webserial library. The browser connects to a serial port on your computer. It passes whatever comes in the serial port through to the webSocket and vice versa. The second rectangle is your Arduino, connected via the serial port. — *Figure 2. Diagram of the connection from the serial port to p5.js through p5.webserial*

p5.serialport/p5.serialcontrol:	p5.webserial:
advantages: * works in all browsers. * Allows your browser to connect to the serial port of a remote computer, if that computer is running a web server program. * Supports opening multiple serial ports at the same time	advantages: * connection is direct to the browser, making it easier for beginners to learn
disadvantages: * requires a third application (p5.serialcontrol), making it more difficult for beginners to learn	disadvantages: * only works in Chrome and Edge browsers * No connection to remote computers * Not sure if it supports a second serial port.

Table 1. Comparing p5.seriaport to p5.webserial

The APIs of p5.serialport and p5.webserial are similar, so it’s pretty straightforward to convert between them.

The code below is a typical p5.serialport setup. You make an instance of the p5.serialport library with the port name, then you make callback functions for the primary tasks:

let serial;          // variable to hold an instance of the serialport library
let portName = '/dev/cu.usbmodem1421';  // fill in your serial port name here
  
function setup() {
  serial = new p5.SerialPort();       // make a new instance of the serialport library
  serial.on('list', printList);  // set a callback function for the serialport list event
  serial.on('connected', serverConnected); // callback for connecting to the server
  serial.on('open', portOpen);        // callback for the port opening
  serial.on('data', serialEvent);     // callback for when new data arrives
  serial.on('error', serialError);    // callback for errors
  serial.on('close', portClose);      // callback for the port closing
  
  serial.list();                      // list the serial ports
  serial.open(portName);              // open a serial port
}

The code below is a typical p5.webserial setup. You check t see if WebSerial is available, and then you make an instance of the p5.webserial library, then you look for ports using getPorts():

// variable to hold an instance of the p5.webserial library:
const serial = new p5.WebSerial();
 // port chooser button:
let portButton;
// variable for incoming serial data:
let inData;
  
function setup() {
   // check to see if serial is available:
   if (!navigator.serial) {
    alert("WebSerial is not supported in this browser. Try Chrome or MS Edge.");
  }
  // check for any ports that are available:
  serial.getPorts();
  // setup function continues below:

Unlike with p5.serialport, you don’t pass in the port name. Instead, the user chooses the port with a pop-up menu:

// if there's no port chosen, choose one:
serial.on("noport", makePortButton);
// open whatever port is available:
serial.on("portavailable", openPort);
// setup function continues below:

Then you make callback functions for the primary tasks. These are similar to p5.serialport’s callbacks:

// handle serial errors:
serial.on("requesterror", portError);
// handle any incoming serial data:
serial.on("data", serialEvent);
serial.on("close", makePortButton)
// setup function continues below:

You also add listeners for when the USB serial port enumerates or de-enumerates. This way, if you unplug your serial device or reset it, your program automatically reopens it when it’s available:

  // add serial connect/disconnect listeners from WebSerial API:
  navigator.serial.addEventListener("connect", portConnect);
  navigator.serial.addEventListener("disconnect", portDisconnect);
} // end of setup function

The functions in each API are similar too:

p5.serialport functions (from the library’s guide)	p5.webserial functions (from the library’s guide)
`read()` returns a single byte of data (first in the buffer)	`read()` — reads a single byte from the queue (returns a number)
`readChar()` returns a single char, e,g, ‘A’, ‘a’	`readChar()` — reads a single byte from the queue as a character (returns a string)
`readBytes()` returns all of the data available as an array of bytes	`readBytes()` — reads all data currently available in the queue as a `Uint8Array`.
`readBytesUntil(charToFind)` returns all of the data available until `charToFind` is encountered	`readBytesUntil(charToFind)` — reads bytes until a character is found (returns a `Uint8Array`)
`readString()` returns all of the data available as a string
`readStringUntil(charToFind)` returns all of the data available as a string until `charToFind` is encountered	`readStringUntil(stringToFind)` — reads out a string until a certain substring is found (returns a string)
`last()` returns the last byte of data from the buffer
`lastChar()` returns the last byte of data from the buffer as a char
	`readLine()` — read a string until a line ending is found (returns a string)
`write()` – sends most any JS data type out the port.	`write()` – sends most any JS data type out the port.
	`print()`– sends string data out the serial port
	`println()` – sends string data out the serial port, along with a newline character.

Table 2. Comparison of p5.serialport and p5.webserial APIs

Both p5.serialport and p5.webserial will do the job of getting bytes from a serial device like an Arduino or other microcontroller to p5.js and vice versa. Which you use depends on what you need. p5.webserial is great if you’re connected directly to the browser’s computer via USB. For most serial applications, this is the way to go. Alternately, p5.serialport offers the advantage that you can run p5.serialcontrol app on your browser’s computer or on another computer on the same network. That way, a browser on your phone or tablet can communicate to the serial device via the computer running p5.serialcontrol.

This site contains lab examples for both p5.serialport and p5.webserial for your convenience.

Lab: Serial IMU Output to p5.js Using p5.webserial

In this exercise you’ll read the built-in Inertial Motion Unit on the Arduino Nano 33 IoT, then feed its output into a Madgwick filter to determine heading, pitch, and roll of the board. Then you’ll send the output of that serially to p5.js and use it to move a virtual version of the Nano onscreen.

Introduction

What You’ll Need to Know

To get the most out of this lab, you should be familiar with the following concepts and you should install the Arduino IDE on your computer. You can check how to do so in the links below:

What is a microcontroller
Beginning programming terms
The basics of electricity
What is a solderless breadboard and how to use one
Getting Started with Arduino Guide
The basics of serial communication, both asynchronous and synchronous.
Two-Way (Duplex) Serial Communication Using An Arduino and the p5.webserial Library
The basics of accelerometers, gyrometers, and IMUs

Things You’ll Need

The only part you’ll need for this exercise is an Arduino Nano 33 IoT and its built-in IMU, as shown in Figure 1. You can modify this exercise to work with other IMUs, however. There are details on various IMUs on the accelerometers, gyrometers, and IMUs page.

Prepare the Breadboard

because the Nano 33 IoT has a built-in IMU, there is no additional circuit needed for this exercise. However, there are two libraries you’ll need to install: the Arduino_LSM6DS3 library, which allows you to read the IMU, and the MadgwickAHRS library, which takes the raw accelerometer and gyrometer inputs and provides heading, pitch, and roll outputs. Both libraries can be found in the Library Manager of the Arduino IDE. Install them before proceeding.

Program the Microcontroller to Read the IMU

The first thing to do in the microcontroller code is to confirm that your accelerometer and gyrometer are working. Start with the code below:

#include "Arduino_LSM6DS3.h"
 
void setup() {
  Serial.begin(9600);
  // attempt to start the IMU:
  if (!IMU.begin()) {
    Serial.println("Failed to initialize IMU");
    // stop here if you can't access the IMU:
    while (true);
  }
}
 
void loop() {
  // values for acceleration and rotation:
  float xAcc, yAcc, zAcc;
  float xGyro, yGyro, zGyro;
 
  // check if the IMU is ready to read:
  if (IMU.accelerationAvailable() && IMU.gyroscopeAvailable()) {
    // read accelerometer and gyrometer:
    IMU.readAcceleration(xAcc, yAcc, zAcc);
    IMU.readGyroscope(xGyro, yGyro, zGyro);
 
    Serial.print("sensors: ");
    Serial.print(xAcc);
    Serial.print(",");
    Serial.print(yAcc);
    Serial.print(",");
    Serial.print(zAcc);
    Serial.print(",");
    Serial.print(xGyro);
    Serial.print(",");
    Serial.print(yGyro);
    Serial.print(",");
    Serial.println(zGyro);
  }
}

When you run this sketch and open the Serial Monitor, you should see a printout with six values per line. The first three are your accelerometer values, and the next three are your gyrometer values. The following reading is typical:

sensors: 0.04,-0.05,1.02,3.05,-3.72,-1.77

The Nano 33 IoT’s accelerometer’s range is fixed at +/-4G by this library, and its gyrometer’s range is set at +/-2000 degrees per second (dps). The sampling rate for both is set to 104 Hz by the library. Other IMUs may have differing ranges. You need to know at least the sampling rate when you want to use a different IMU with this exercise. If you know that information, though, it’s easy to swap one IMU for another in the Madgwick library.

Add the Madgwick Library to Get Orientation

The MadgwickAHRS library can work with any accelerometer/gyrometer combination. It expects the acceleration in Gs and the rotation in degrees per second as input, and uses the sensors’ sampling rate when you initialize it. Add a few lines to the code before your setup() as follows:

#include "Arduino_LSM6DS3.h"
#include "MadgwickAHRS.h"
 
// initialize a Madgwick filter:
Madgwick filter;
// sensor's sample rate is fixed at 104 Hz:
const float sensorRate = 104.00;
 
// values for orientation:
float roll = 0.0;
float pitch = 0.0;
float heading = 0.0;

Next, add the following line at the end of the setup() to initialize the Madgwick filter:

// start the filter to run at the sample rate:
filter.begin(sensorRate);

Now change the main loop so that you’re sending the sensor readings into the Madgwick filter. You’ll do this inside of the if statement that checks if the sensors are ready:

// check if the IMU is ready to read:
if (IMU.accelerationAvailable() &&
IMU.gyroscopeAvailable()) {
  // read accelerometer and gyrometer:
  IMU.readAcceleration(xAcc, yAcc, zAcc);
  IMU.readGyroscope(xGyro, yGyro, zGyro);
 
  // update the filter, which computes orientation:
  filter.updateIMU(xGyro, yGyro, zGyro, xAcc, yAcc, zAcc);
 
  // print the heading, pitch and roll
  roll = filter.getRoll();
  pitch = filter.getPitch();
  heading = filter.getYaw();
 
  // print the filter's results:
  Serial.print(heading);
  Serial.print(",");
  Serial.print(pitch);
  Serial.print(",");
  Serial.println(roll);
}

Now when you run the sketch, you’ll get heading, pitch, and roll instead of the raw sensor readings. Here’s a typical output you might see:

167.59,-2.50,-2.52In this case, the readings are all in degrees. The first is the heading angle, around the Z axis. The second two are the pitch, around the x axis, and roll, around the Y axis.

Add Serial Handshaking

Reading these values in p5.js will work smoother if you add handshaking, also known as call-and-response, to your serial communications protocol. Modify the loop() so that the sketch sends the latest heading, pitch, and roll whenever a new byte comes in the serial port. Here’s the final version of the loop():

void loop() {
  // values for acceleration and rotation:
  float xAcc, yAcc, zAcc;
  float xGyro, yGyro, zGyro;
 
  // check if the IMU is ready to read:
  if (IMU.accelerationAvailable() & amp; & amp;
      IMU.gyroscopeAvailable()) {
    // read accelerometer and gyrometer:
    IMU.readAcceleration(xAcc, yAcc, zAcc);
    IMU.readGyroscope(xGyro, yGyro, zGyro);
 
    // update the filter, which computes orientation:
    filter.updateIMU(xGyro, yGyro, zGyro, xAcc, yAcc, zAcc);
 
    // print the heading, pitch and roll
    roll = filter.getRoll();
    pitch = filter.getPitch();
    heading = filter.getYaw();
  }
 
  // if you get a byte in the serial port,
  // send the latest heading, pitch, and roll:
  if (Serial.available()) {
    char input = Serial.read();
    Serial.print(heading);
    Serial.print(",");
    Serial.print(pitch);
    Serial.print(",");
    Serial.println(roll);
  }
}

this link. When you have this much working, and you’ve tested it in the Serial Monitor, you can close Arduino and work on the p5.js sketch.

Program p5.js to Read the Incoming Serial Data

Now it’s time to write a p5.js sketch to read this data. The setup will be the same as it was in the Serial Input to p5.js using WebSerial lab. The checklist from that lab lays out all the important parts you need.

<script src="https://cdnjs.cloudflare.com/ajax/libs/p5.js/1.4.0/p5.js"></script>
      
<script src="https://unpkg.com/p5-webserial@0.1.1/build/p5.webserial.js"></script>

Start your sketch with some code to initialize the serial library:

// variable to hold an instance of the p5.webserial library:
const serial = new p5.WebSerial();
  
// HTML button object:
let portButton;
 
function setup() {
    createCanvas(500, 600, WEBGL);     // make the canvas
  // check to see if serial is available:
  if (!navigator.serial) {
    alert("WebSerial is not supported in this browser. Try Chrome or MS Edge.");
  }
  // if serial is available, add connect/disconnect listeners:
  navigator.serial.addEventListener("connect", portConnect);
  navigator.serial.addEventListener("disconnect", portDisconnect);
  // check for any ports that are available:
  serial.getPorts();
  // if there's no port chosen, choose one:
  serial.on("noport", makePortButton);
  // open whatever port is available:
  serial.on("portavailable", openPort);
  // handle serial errors:
  serial.on("requesterror", portError);
  // handle any incoming serial data:
  serial.on("data", serialEvent);
  serial.on("close", makePortButton);
}
function draw() {
  
}
  
// if there's no port selected, 
// make a port select button appear:
function makePortButton() {
  // create and position a port chooser button:
  portButton = createButton('choose port');
  portButton.position(10, 10);
  // give the port button a mousepressed handler:
  portButton.mousePressed(choosePort);
}
  
// make the port selector window appear:
function choosePort() {
  serial.requestPort();
}
  
// open the selected port, and make the port 
// button invisible:
function openPort() {
  // wait for the serial.open promise to return,
  // then call the initiateSerial function
  serial.open().then(initiateSerial);
  
  // once the port opens, let the user know:
  function initiateSerial() {
    console.log("port open");
    serial.write("x");
  }
  // hide the port button once a port is chosen:
  if (portButton) portButton.hide();
}
  
// read any incoming data:
function serialEvent() {
  // read a string from the serial port
  // until you get carriage return and newline:
  var inString = serial.readStringUntil("\r\n");
  console.log(inString);
}
  
// pop up an alert if there's a port error:
function portError(err) {
  alert("Serial port error: " + err);
}
  
// try to connect if a new serial port 
// gets added (i.e. plugged in via USB):
function portConnect() {
  console.log("port connected");
  serial.getPorts();
}
  
// if a port is disconnected:
function portDisconnect() {
  serial.close();
  console.log("port disconnected");
}

Save this as sketch.js, then open p5.serialcontrol. Then open the sketch in a browser. Open the JavaScript console, and you should see the first set of data printed out. This is because the initiateSerial() function sent a single byte to the Nano when the port opened, and the Nano sent one set of readings. That generated a serial data event in p5.js, and called the serialEvent() function, which printed out the results.You need this to happen repeatedly: p5.js sends a byte when it wants new data, then the Nano sends the data, then waits for another byte from p5.js.

Add Serial Handshaking

To make this happen, you need to add a few things to your p5.js sketch. You can assume that if you saw the message in the console, then you’re ready for new data. That’s when you should send a byte back to the microcontroller to request new data. Add one line to the serialEvent() function to make this happen:

// callback function for incoming serial data:
ffunction serialEvent() {
  // read a string from the serial port
  // until you get carriage return and newline:
  var inString = serial.readStringUntil("\r\n");
  if (inString != null) {
    console.log(inString);
    serial.write("x");
  }
}

When you run the sketch with this update, you should see a continuous flow of new data from the microcontroller.

Next, you need to break the string up into parts and convert them into floating point numbers so you can use them as heading, pitch, and roll. Start by adding three new variables at the top of your sketch as global variables, because you’ll need them when you draw the virtual Arduino:

// orientation variables:
let heading = 0.0;
let pitch = 0.0;
let roll = 0.0;

Next, in the serialEvent() function, use the JavaScript trim() function to get rid of any extraneous characters at the end of the message string, like carriage returns or newlines. Then use the split() function to split the string into a list of elements separated by commas. Then convert them to floating point numbers. Once you know you have three valid numbers for heading, pitch, and roll, send another byte to the microcontroller to get a new reading. Here’s what the new version of serialEvent() looks like:

function serialEvent() {
  // read from port until new line:
  let inString = serial.readStringUntil("\r\n");
  if (inString != null) {
    let list = split(trim(inString), ",");
    if (list.length > 2) {
      // conver list items to floats:
      heading = float(list[0]);
      pitch = float(list[2]);
      roll = float(list[1]);
      console.log(heading + "," + pitch + "," + roll);
      // send a byte to the microcontroller to get new data:
      serial.write("x");
    }
  }
}

When you reload the sketch after making these changes, you should be getting floating point numbers for heading, pitch and roll. Once you have these values coming in consistently, it’s a good idea to comment out the console.log() statement, as shown above, as it slows down the sketch considerably.

Now that you have serial communication working properly, it’s time to write the code to draw the virtual microcontroller.

Draw the Virtual Arduino

Add the function below to draw a virtual Arduino. It draws in three dimensions, using the WEBGL framework you chose in createCanvas() above in the setup() function.

// draws the Arduino Nano:
function drawArduino() {
   // the base board:
   stroke(0, 90, 90); // set outline color to darker teal
   fill(0, 130, 130); // set fill color to lighter teal
   box(300, 10, 120); // draw Arduino board base shape
 
   // the CPU:
   stroke(0);         // set outline color to black
   fill(80);          // set fill color to dark grey
   translate(30, -6, 0); // move to correct position
   box(60, 0, 60);    // draw box
 
   // the radio module:
   stroke(80);       // set outline color to grey
   fill(180);        // set fill color to light grey
   translate(80, 0, 0); // move to correct position
   box(60, 15, 60);  // draw box
 
   // the USB connector:
   translate(-245, 0, 0); // move to correct position
   box(35, 15, 40);   // draw box
}

You haven’t added a draw() function yet, so add it now, as follows:

function draw() {
   background(255); // set background to white
   push();          // begin object to draw
   // draw arduino board:
   drawArduino();
   pop();           // end of object
}

When you reload the sketch, you’ll see a drawing like that in Figure 4. It won’t change.

Figure 4. A virtual Arduino Nano 33 IoT, drawn in in p5.js. The Nano is seen from the side, with the USB connector on the right, and the radio on the right.

To make it change its orientation, you need to use the heading, pitch, and roll values to rotate the object. You get the sine and cosine of each angle, and use them to generate a matrix for translation. The math below was worked out by Helena Bisby based on the Madgwick algorithm. p5.js’ applyMatrix() function does the matrix math for you to rotate in all three dimensions. Modify the draw() function as shown below:

function draw() {
   // update the drawing:
   background(255); // set background to white
   push();          // begin object to draw
 
   // variables for matrix translation:
   let c1 = cos(radians(roll));
   let s1 = sin(radians(roll));
   let c2 = cos(radians(pitch));
   let s2 = sin(radians(pitch));
   let c3 = cos(radians(heading));
   let s3 = sin(radians(heading));
   applyMatrix(c2 * c3, s1 * s3 + c1 * c3 * s2,
      c3 * s1 * s2 - c1 * s3, 0, -s2, c1 * c2,
      c2 * s1, 0, c2 * s3, c1 * s2 * s3 - c3 * s1,
      c1 * c3 + s1 * s2 * s3, 0, 0, 0, 0, 1);
 
   // draw arduino board:
   drawArduino();
   pop(); // end of object
}

When you reload the sketch after making these changes, the virtual Arduino should change its position as you move the physical Arduino. That’s the whole application! Figure 5 shows the virtual Arduino in motion.

You can see the sketch running on GitHub at this link. You can see the source files for copying into the p5.js editor at this link.

Moving GIF of a virtual Arduino Nano turning in three dimensions — Figure 5. This virtual Arduino Nano, written in p5.js, moves in three dimensions as you move a real Nano connected to the sketch serially.

Conclusion

If you followed along all of the steps to this application, you probably hit a number of places where communication broke down. There are a lot of pieces to this application, and they all need to work together.

Get the Sensors Working

When you’re dealing with IMU sensors, no data will be perfect, because the sensor’s measurement is always relative. You’ll notice, for example, that the position of the virtual Arduino drifts a bit the longer you run the sketch. Heading, in particular, tends to drift. In real-world applications, this is often adjusted by using a magnetometer as a compass in addition to the accelerometer and gyrometer. It’s also wise to provide ways for a human to calibrate the system, perhaps by pressing a button when the sensor is level in order to calculate offset values for the sensors. For many interactive applications, though, even an imperfect measurement of orientation will do the job well.

Test the Hardware

If you’re using serial communication that’s ASCII-encoded like this, you can always use the Serial Monitor or another serial terminal application to test the Arduino sketch before you ever begin working on the multimedia programming. Ideally, you don’t need to change the Arduino sketch at all once your communication is working as planned.

Get the Communication Working

Whenever you’re building an application that incorporates asynchronous serial communication, it’s best to get the communication working correctly before you build the animation or other parts of the interaction. Once the communication protocol is known, you can even divide the work, with one team developing the hardware and another developing the media programming.

This exercise shows the value of using handshaking (aka call-and-response) in serial communication. Because the drawing of the microcontroller takes time, the p5.js sketch reads data less frequently than the microcontroller can send it. If you simply allow the microcontroller to send data continuously, the serial buffer on the p5.js side will fill up, and the movement of the virtual Arduino will become sluggish. This is why you only send back to the microcontroller when you know you have a set of valid data, in the serialEvent() function.

Test the Incoming Data

You don’t need to do anything with your incoming serial data to know it’s valid, if you’ve thought through the protocol well. In this case, if you see you’re getting three separate values and they’re all in a range of 0 to 360 (indicating degrees of the heading, pitch, and roll angles), you know it’ll work.

Program the Interface, Animation, etc.

Once you know the communication is good, and you’re getting accurate values, you can program the parts of your final application that use that data. In this case, you didn’t even start on the movement of the virtual Arduino until you knew you had communication working. Drawing of the virtual model was separated from moving it, using the push(), pop(), and translation functions, like applyMatrix(), in p5.js. That separation makes the programming easier to do, and easier to debug.

Lab: Intro to Asynchronous Serial Communications

In this lab, you’ll get to know serial communication from a microcontroller to your personal computer a bit more in depth, so that you’re ready to start writing programs in other languages on your computer to interact with our microcontroller.

Introduction

From the first digital I/O and analog labs, you’ve been using asynchronous serial communication in order to send data back to the Arduino Serial Monitor. In this lab, you’ll get to know serial communication from a microcontroller to your personal computer a bit more in depth.

There are many programming APIs that can communicate with your computer’s serial ports. Among them are Processing, node.js, p5.js with the p5.seriaport library and p5.serialcontrol app, or p5.js with the p5.webserial library. There are many others as well. This lab won’t introduce you to any of those just yet; instead, it’ll introduce you to serial terminal applications other than the Arduino Serial Monitor, and it will give you some background on how serial data is formatted when it’s sent from one device to another. In this lab, you’ll send data from multiple sensors from Arduino to your computer and learn how to format the data, and to manage the exchange between the two.

What You’ll Need to Know

To get the most out of this Lab, you should be familiar with how to program an Arduino, and with the basics of serial communication. If you’re not, review the links below:

Some of the examples below show you how to connect to a serial port using the Linux or Unix command line tools. Collectively, Linux, Unix, and similar systems are sometimes called POSIX systems. Unix is built in on MacOS. If you’re using Windows 10 or Window 11, you should install Windows Subsystem for Linux, which will give you a Linux command line environment on your Windows machine. Instructions and tips for setting up the environment can be found at this link: Windows Subsystem for Linux (WSL). The command line applications shown here are not part of the regular Windows OS, but they are available in wsl.

Things You’ll Need

Figures 1-5 below are the parts you’ll need for this exercise. Click on any image for a larger view.

Connect the sensors

As long as you have three sensors that will output changing readings, you can follow this lab.

(Diagrams made with Fritzing, a circuit design program)

A Serial Output Sketch

To get started, upload the following sketch to your Arduino board, then open the Serial Monitor:

void setup() {
   // start serial port at 9600 bps:
   Serial.begin(9600);
}
 
void loop() {
   int analogValue = analogRead(A0);
   Serial.println(analogValue);
}

This will read the value from the first potentiometer, and print it to the Serial Monitor. You’ve already done this in the earlier labs. Now let’s get a deeper understanding of what’s going on between the Arduino and your personal computer.

Serial Terminal Programs

The Arduino Serial Monitor is a good basic way to see your serial communications, but it’s not the only way to view incoming serial data on your computer. There are many different serial terminal programs. Serial terminal programs allow you to send and receive messages from your computer’s serial ports. There are both graphical serial applications like CoolTerm or Serial Studio, and there are ways to read from the serial port from the command line terminal of your computer. For a beginner, the Arduino Serial Monitor does the job fine, but as you get more familiar with serial communication, you may want to get to know some of these. If you want to know about a few of them, read on. If not, feel free to skip to the next section on initializing serial communication.

Connecting via the POSIX Command Line

You can use a command line interface as a serial terminal program on MacOS or Linux, or Windows Subsystem for Linux (WSL). The simplest thing you can do with serial ports on the command line is to listen for incoming serial messages. Make sure your Arduino is running a sketch with at least one Serial.print() or Serial.println() statement in your code, like the one above.

On MacOS, open the Terminal app, then get a list of your serial ports by typing ls -1 /dev/cu.*
On Linux, you might need to type ls -1 /dev/cu* or ls -1 /dev/tty* instead.
On Windows, once you’ve installed WSL, open the Start Menu and type wsl to open a linux command line. Then when the command line window opens, type ls -l /dev/tty*
You’ll get a list like this:

/dev/cu.Bluetooth-Incoming-Port /dev/cu.lpss-serial2 /dev/cu.usbmodem14131

On Windows or Linux you’ll get a list like this:

/dev/tty /dev/ttyS0 /dev/ttyS1

Each tty port in WSL corresponds to a COM port in Windows. So if your Arduino is showing up as COM7 to the Arduino IDE, then it’d be /dev/ttyS7 in wsl. It’s similar in Linux.

In MacOS, the Arduino is the one labeled usbmodemXXXX, as you’ve seen before in the Arduino IDE’s Serial Monitor. To see what the Arduino is sending out, type:

cat /dev/cu.usbmodem14131

in Windows and Linux, you’ll need to run the cat command as an admin user, so type:

sudo cat /dev/ttyS6

You’ll be prompted for your admin password. Enter it, then the command will run.

The terminal will then print out whatever the Arduino is sending. It’s using a POSIX command line application called cat, short for concatenate. This same application can be used for listing the contents of a file files, like so: cat filename.txt

To close the port and quit the cat application, type control-C. If you’ve got a continually repeating serial output, you may prefer to use the less command instead of cat. less /dev/cu.usbmodemXXXX will also print out the serial output, but it will stop after each screenful. Type the spacebar to page through multiple screens, or use the arrow keys to read up and down. Type q to exit.

Using the Command Line for Duplex (two-way) Communication

Another useful POSIX command line tool is the screen program. This allows you to both read from and write to a serial port. Type screen followed by the name of your serial port to open the interactive serial monitor called screen. Yours might look like this:

screen /dev/cu.usbmodem-1421

On Windows and Linux, remember to use sudo:

sudo screen /dev/ttyS6

The screen program will take your terminal over. You will be able to type messages to be sent out the serial port in addition to receiving them in the port. To end the screen program, type control-A control-D. This will quit the program.

NOTE: only one program can control a serial port at a time. When you’re not using a given program, remember to close the serial port. You won’t be able to re-program your Arduino module if you don’t, because the serial terminal program will have control of the serial port.

Sparkfun has a quick guide to other serial terminal connections from the command line on MacOS, Windows, and Linux as well.

GUI Serial Terminal Programs

CoolTerm is a more fully-featured GUI serial terminal program. It’s free, it’s available for OSX, Windows and Linux, and includes some features you don’t get from the Serial Monitor, like being able to open multiple ports in multiple windows, being able to view your data in ASCII (Figure 10) or hexadecimal (Figure 9) values, and more.

Serial Studio is a fully-featured serial terminal program that allows you to configure dashboards to display data via JSON, send data over a network via MQTT or UDP, and many more features. It’s more than a beginner needs, but a useful advanced user’s tool.

The Serial Monitor, CoolTerm, Serial Studio and the screen program on the command line are all examples of a serial terminal program.

The CoolTerm serial terminal application showing the hexadecimal view. The screen is filled with hexadecimal values in the center, along with the ASCII characters corresponding to those values running down the side. — Figure 9. The CoolTerm serial terminal application showing the hexadecimal view.

The CoolTerm serial terminal application showing the ASCII view. A column of text numbers runs down the screen. — Figure 10. The CoolTerm serial terminal application showing the ASCII view.

Initializing Communication

What is asynchronous serial communication, anyway? Serial means that the two computers communicating are doing so one bit at a time, by sending pulses of data back and forth. Asynchronous means that the two computers that are communicating are each keeping track of time independently. As you know from the Asynchronous Serial Communications: the Basics page, there are three things the two computers need to agree upon:

Electrical: what is the voltage of the pulses?
Data rate: How often is a bit of data sent?
Logic: Does low voltage mean the bit is a 0 or a 1?

Electrical and logic agreements are set for you when you use an Arduino. You set the data rate in your code. In your Arduino sketches so far, you’ve been setting the data rate to 9600 bits per second with this line:

Serial.begin(9600);

Whatever program you’re communicating to (whether it’s the Serial Monitor, CoolTerm, Serial Studio, the command line or another programming environment) has to set the same data rate. You can change both, as long as you change them to the same rate.

There are three connections between the two computers:

a transmit line from sender to receiver
a receive line from receiver to sender
a common ground line

The transmit (sometimes called TX) and the receive (sometimes called RX) are relative: my transmit is connected to your receive, and vice versa. On the Arduino Uno and Nano 33 IoT, digital pins 0 and 1 are used for receive and transmit. They are attached to the USB-to-serial chip on the board. When you plug the Arduino into the computer, it shows up as a USB COM device, meaning a serial communications device. When you ask for a list of serial ports, whether in the Arduino Serial Monitor, CoolTerm, or any program, the Arduino will show up as a new port.

ASCII vs. Binary: What are you sending?

In order to communicate clearly between two devices, you need to understand how the devices are encoding the data which they’re communicating. There are two main approaches most computers use. They will either send the data directly, as a series of bits (this is called raw binary data), or they will encode the information as a series of alphanumeric characters (this is called ASCII-encoded data). This section explains the difference between ASCII and raw binary data.

To begin with, just send the value from one sensor, the first analog sensor (the first axis of the accelerometer in the photos) and divide the output to give a maximum value of 1023:

void setup() {
   // start serial port at 9600 bps:
   Serial.begin(9600);
}
 
void loop() {
   int analogValue = analogRead(A0);
   Serial.println(analogValue);
}

When you open the serial monitor, you should see a number between 0 and 1023 scrolling down the window. That’s because Serial.println() formats the value it prints as an ASCII-encoded decimal number, with a linefeed at a carriage return at the end. To send the value 1023, for example, println() sents six bytes: the characters 1, 0, 2, and 3, and a carriage return byte and a newline byte. Meanwhile, the serial monitor, receiving the data, assumes it should show you the ASCII character corresponding to each byte it receives.

The output from analogRead() can’t fit in a single byte, because the microcontroller’s analog-to-digital converter (ADC) reads the input with a resolution of 10 bits, or 2¹⁰. To get the output into a single byte, map the output to a range from 0-255 like so:

void setup() {
   // start serial port at 9600 bps:
   Serial.begin(9600);
}
 
void loop() {
   int analogValue = analogRead(A0);
   int mappedValue = map(analogValue, 0, 1023, 0, 255);
   Serial.println(mappedValue);
}
﻿

Try it now, and your output should range from 0 to 255.

Try changing the Serial.println() to a Serial.write(). Now you get a range of garbage characters (Figure 11). What’s going on? The Serial.write() command doesn’t format the bytes as ASCII characters. It sends out the binary value of the sensor reading. Each sensor reading can range from 0 to 1023; in other words, it has a 10-bit range, since 2¹⁰ = 1024 possible values. Since that’s more than the eight bits that can fit in a byte, you can either divide the value by 4 in the code above or use the map() function to get a range from 0 to 255, or 2⁸ bits. For more background on this, see the notes on variables.

The Arduino IDE with the serial monitor open. There is a string of random characters in the serial monitor — Figure 11. The Arduino IDE with the serial monitor open

So, for example, if the sensor reading’s value is 234, then the Serial.write() command sends the binary value 11101010. If the reading is 255, then Serial.write() sends 11111111. If it’s 157, then the command sends 10011101. For more decimal-to-binary conversions, open your computer’s calculator and choose the Programmer view (press apple-3 on a mac, and Alt-3 on Windows). Sometimes you’ll see byte values in hexadecimal as well; for example, both CoolTerm and Serial Studio have hexadecimal modes. You can use the calculator to do these conversions too.

The garbage characters are characters corresponding to the ASCII values the Monitor is receiving. When the Serial Monitor receives a byte, it and assumes it should show you the ASCII character corresponding to that byte’s value.

For example, imagine that analogValue = 32:

Serial.println(analogValue) results in “32” with a linefeed and carriage return
Serial.write(analogValue) results in ” “, the space character, which has the ASCII value 32.

How many bytes does Serial.println(analogValue) send when analogValue = 32?

Serial.println(analogValue) actually sends FOUR bytes! It sent a byte to represent the 3, a byte to represent the 2, a byte to tell the Monitor to move the cursor down a line (newline), and a byte to move the cursor all the way to the left (carriage return). The raw binary values of those four bytes are 51 (ASCII for “3”), 50 (ASCII for “2”), 10 (ASCII for “newline”), and 13 (ASCII for “carriage return”). Check the ASCII table and you’ll see for yourself.

Send the data in many formats

Try this program and view the results in the Serial Monitor:

void setup() {
   // start serial port at 9600 bps:
   Serial.begin(9600);
}
 
void loop() {
// read analog input, map it to make the range 0-255:
int analogValue = analogRead(A0);
int mappedValue = map(analogValue, 0, 1023, 0, 255);
Serial.println(mappedValue);
 
// print different formats:
Serial.write(mappedValue);  // Print the raw binary value
Serial.print('\t');             // print a tab
// print ASCII-encoded values:
Serial.print(mappedValue, BIN); // print ASCII-encoded binary value
Serial.print('\t');             // print a tab
Serial.print(mappedValue);      // print decimal value
Serial.print('\t');             // print a tab
Serial.print(mappedValue, HEX); // print hexadecimal value
Serial.print('\t');             // print a tab
Serial.print(mappedValue, OCT); // print octal value
Serial.println();               // print linefeed and carriage return
}

You should get output like this:

â 11100010 226 E2 342
á 11100001 225 E1 341
á 11100001 225 E1 341
á 11100001 225 E1 341
à 11100000 224 E0 340
à 11100000 224 E0 340
ß 11011111 223 DF 337
ß 11011111 223 DF 337
ß 11011111 223 DF 337

This sketch is printing the raw binary value, then the ASCII-encoded binary value, then the ASCII-encoded decimal, hexadecimal, and octal values. You may never need all of these different formats, but you’ll likely need at least the decimal and the raw binary versions at some point.

What’s this \t, \r and \n Stuff, Anyway?

The ASCII table contains several characters that are “invisible”, like tab, newline, carriage return, and so forth. These characters tell the receiving program how to format the visible characters on the screen; newline tells the receiver to move down a line; carriage return tells it to move to the left edge of the screen, and so forth. In most programming languages, these characters are denoted with an escape string starting with a backslash: \. For example, newline is \n. Carriage return is \r. Tab is \t. When you see an escape string in a code sample, replace it with the appropriate ASCII value in your mind.

Send the values for all three sensors

In the first example above, using Serial.write(), you sent one byte representing one sensor’s value, over and over. When you’re sending multiple sensor values, it gets a little more complicated. You need to a way to know which value represents which sensor. For example, imagine if you sent the value of the potentiometers like so:

void loop() {
   int sensor1Value = analogRead(A0);
   Serial.print(sensor1Value);
   Serial.print(",");
   int sensor2Value = analogRead(A1);
   Serial.print(sensor2Value);
   Serial.print(",");
}
﻿

You’ll get a string like this:

452,345,416,234,534,417,325,452,231

How can you tell which value corresponds to which sensor? You don’t know which sensor is which. You could assume that if you start listening when the microcontroller starts sending that the first reading corresponds to the first sensor, but you can’t know that for sure. There are two ways to get your sensor values in order. You can use punctuation or you can use a call-and-response or handshaking method. Use whichever makes the most sense to you. They’re explained below.

Formatting Multiple Serial Data: Punctuation

One way to send the data such that it can be interpreted clearly is to punctuate each set of data uniquely. Just as a sentence ends with a period, you can end your data with a carriage return and a newline. Change the code above as shown below so that a carriage return and newline are printed at the end of each string of values.

void loop() {
   int sensor1Value = analogRead(A0);
   Serial.print(sensor1Value);
   Serial.print(",");
   int sensor2Value = analogRead(A1);
   Serial.println(sensor2Value);
}
﻿
﻿

From this loop, you’d get output like this:

452,345
234,534
325,452

This is much better. Whenever you get a newline, you know that the next value is the first sensor.

Now write a program that reads the two analog sensors on your board and the one digital switch, and prints them out in this format:

analog1, analog2, switch
analog1, analog2, switch
analog1, analog2, switch

Start by setting up a constant for the switch pin’s number. Then in the setup, initialize serial communications at 9600bps, and declare the switch pin as an input.

const int switchPin = 2;      // digital input
 
void setup() {
   // configure the serial connection:
   Serial.begin(9600);
   // configure the digital input:
   pinMode(switchPin, INPUT);
}
﻿

In the main loop, use a local variable called sensorValue to read each input. You can re-use the variable after each read because you don’t need the sensors’ values once you print them. Read the two analog inputs first, and print them with a comma after each one. Then read the digital input, and print it with a carriage return and linefeed at the end.

void loop() {
   // read the sensor:
   int sensorValue = analogRead(A0);
   // print the results:
   Serial.print(sensorValue);
   Serial.print(",");
 
   // read the sensor:
   sensorValue = analogRead(A1);
   // print the results:
   Serial.print(sensorValue);
   Serial.print(",");
 
   // read the sensor:
   sensorValue = digitalRead(switchPin);
   // print the results:
   Serial.println(sensorValue);
}
﻿

Once you’ve got a data format, all you have to do is to write a program that reads that format. You’ll see how do to that in this lab:

Lab: Two-Way (Duplex) Serial Communication Using An Arduino and the p5.webserial Library

Flow Control: Call and Response (Handshaking)

Punctuation helps keep your data in order, but because asynchronous serial communication is asynchronous, you can run into a problem when the sender sends faster than the receiver can read. When this happens, the receiver program slows down as the serial buffer fills up. You can manage this by implementing some form of flow control. The simplest way do to this is using a call-and-response method, where the sending program only sends when it’s told to do so, and the receiving program has to request new data every time it finishes reading what it’s got.

void setup() {
   Serial.begin(9600);
   while (Serial.available() <= 0) {
     Serial.println("hello"); // send a starting message
     delay(300);              // wait 1/3 second
   }
}
﻿

Now, modify the loop() by adding an if() statement to look for incoming serial data and read it.

void loop() {
   if (Serial.available()) {
      // read the incoming byte:
      int inByte = Serial.read();
      // read the sensor:
      sensorValue = analogRead(A0);
      // print the results:
      Serial.print(sensorValue);
      Serial.print(",");
 
      // read the sensor:
      sensorValue = analogRead(A1);
      // print the results:
      Serial.print(sensorValue);
      Serial.print(",");
 
      // read the sensor:
      sensorValue = digitalRead(switchPin);
      // print the results:
      Serial.println(sensorValue);
   }
}
﻿

The rest of the sketch remains the same. When you run this and open the serial monitor, you’ll see:

hello
hello
hello
hello

Type any character in the output box and click Send. You’ll get a string of sensor values at the end of your hellos:

510,497,0

Open the serial port
Wait for a Hello
Send a byte to request data
Begin loop:
  Wait for one set of data
  Send a byte to request new data
end loop
﻿

Advantages of Raw Binary vs. ASCII

All the examples shown here sent the sensor values as ASCII-encoded strings. As mentioned above, that means you sent three bytes to send a three-digit value. If that same value was less than 255, you could send it in one raw binary byte. So ASCII is definitely less efficient. However, it’s more readable for debugging purposes, and if the receiving program is well-suited to convert strings to numbers, then ASCII is a good way to go. For example, JavaScript sends data as ASCII strings almost by default, so it makes sense to use ASCII when writing in JavaScript. If the receiver’s not so good at converting strings to numbers (for example, it’s more challenging to read a multiple byte string in Arduino than in Processing) then you may want to send your data as binary values.

Advantages of Punctuation and Call-and-Response

Sometimes the receiver reads serial data slower than the sender sends it. For example, if you have a program that does a lot of graphic work, it may only read serial data every few milliseconds. For example, if you’re using P5.js, you may notice that using println() in the draw() loop will cause your sketch to slow down. The serial buffer will get full in that case, you’ll notice a lag in response time. This is when it’s good to switch to a call-and-response method.

Bonus Tip: writing Serial Data to a File

POSIX environments treat every input to the computer as a file stream, so when you’re using the cat command like you saw above, you’re telling the operating system to concatenate the serial input as if it’s a file. You can so some useful things as a result of this, like sending the serial output directly to a file.

Let’s say you want to save a few minutes of serial readings into a file. First, write a simple serial program that continually sends serial out, like you did earlier in this lab. Then, on the command line, type:

cat /dev/cu.usbmodem14131 > datalog.csv

You won’t see anything, unlike when you did it before, because now the cat program is redirecting to a file called datalog.csv, not to the command line. Type control-C to stop it, then open the file. You can do this from the Finder as you might usually, or you can just type

open datalog.csv

Since you made it a .csv file (for comma separated values), the operating system tries to open it with your spreadsheet program. This can be a quick way to get data into a structured format from your serial devices. For more on using the command line, see Karl Ward’s Introduction to Unix and the command line.

Further Work

The following labs will help you get better at serial communication:

Lab: Serial output from P5.js Using the p5.serialport Library

In this lab you’ll learn how to send data from p5.js to a microcontroller using asynchronous serial communication.

Overview

When you use the p5.serialport library for P5.js, it communicates with a webSocket server in the P5.js IDE to give you access to serial devices attached to your computer. This lab shows you how to use P5 to control a microcontroller using asynchronous serial communication.

To get the most out of this tutorial, you should know what a microcontroller is and how to program them. You should also understand asynchronous serial communication between microcontrollers and personal computers. You should also understand the basics of P5.js, and should have tried the Serial Input to P5.js lab.

Things You’ll Need

For this lab, you’ll need the hardware below,

For this lab, you’ll need the hardware below, and you’ll need the same software setup as the Serial Input to P5.js lab: You’ll create a p5.js sketch. You’ll also use the p5.serialport library and the P5.serialcontrol app. You can use the p5.js web editor or your favorite text editor for this (the Visual Studio Code editor or the Atom text editor work well).

Prepare the breadboard

Connect power and ground on the breadboard to power and ground from the microcontroller. On the Arduino UNo, use the 5V and any of the ground connections. On the Nano, use 3.3V and the ground connections:

Made with Fritzing

Add an LED

Program the Microcontroller

Program your Arduino to read the analog input as follows:

void setup() {
  Serial.begin(9600);           // initialize serial communications
}
 
void loop() {
  if (Serial.available() > 0) { // if there's serial data available
    int inByte = Serial.read();   // read it
    Serial.write(inByte);         // send it back out as raw binary data
    analogWrite(5, inByte);       // use it to set the LED brightness
    // if you're using a speaker instead of an LED, uncomment line below  and comment out the previous line:
    //  tone(5, inByte*10);     // play tone on pin 5
  }
}

The P5.js serialport library

To communicate with your microcontroller serially, you’re going to use the P5.js serialport library and the p5.serialcontrol app. The P5.js serialport library can’t access your serial ports directly when a sketch is running in a browser because the browser doesn’t have direct access to the serial port. But it can communicate with another program on your computer that can exchange data with the serialport. p5.serialcontrol is the app that connects your sketch, running in a browser, with the serial ports on your computer as shown in Figure 11.

Once you gain an understanding of serial communication, you can use any program that can connect with your computer’s serial ports to communicate with a microcontroller. Processing, Max/MSP, and OpenFrameworks are three other popular multimedia programming environments that can communicate via the serial ports. You can also do this with Unity, Unreal, or any other programming environment that can access the serial ports.

Install the P5.serialcontrol App

Download the latest version of the P5.serialcontrol application and save it in your Applications folder. When you run it, it will check serial ports on your machine. You don’t need to do anything with the app, just have it open. However, remember this most important fact:

Only one port at a time can access a serial port.

That means that when you want to reprogram your Arduino from the Arduino IDE, you need to quit p5.serialcontrol to do so. Then, reopen p5.serialcontrol when you’re done reprogramming the Arduino. You don’t need to quit the Arduino IDE each time, because it knows to release the serial port when it’s not programming. However, you do need to close the Serial Monitor in the Arduino IDE when you are using p5.serialcontrol.

You’ll need to know the name of your serial port to get started. If you’re not sure how to get this, see the Serial Input to P5.js lab for how to get a list of ports.

The P5.js Sketch

The sketch you’re going to write will control the microcontroller’s LED from P5.js. Dragging the mouse up and down the canvas will dim or brighten the LED, and typing 0 through 9 will set the LED’s brightness in increments from off (0) through almost full brightness (9). There’s an alternate sketch that will make changing tones if you prefer that instead of a changing LED. The sketch will also receive serial input from the microcontroller just as in the Serial Input to P5.js lab, so that you can see that the microcontroller is getting the same values you’re sending.

Program P5.js For Serial Communication

Make a P5.js sketch. If you’re using the p5.js web editor, make a new sketch. Click the Sketch Files tab, and then choose the index.html file. In the head of the document, look for this line:

1	`<script` `src="https://cdnjs.cloudflare.com/ajax/libs/p5.js/1.4.0/p5.js"></script>`

Right after that line, add this line:

1	`<script` `language="javascript"` `type="text/javascript"` `src="https://cdn.jsdelivr.net/npm/p5.serialserver@0.0.28/lib/p5.serialport.js"></script>`

The setup of your sketch will initialize the P5.serialport library and define your callback functions for serial events. Program the global variables and setup() function as follows:

var serial;          // variable to hold an instance of the serialport library
var portName = '/dev/cu.usbmodem1421'; // fill in your serial port name here
var inData;                            // for incoming serial data
var outByte = 0;                       // for outgoing data
 
function setup() {
  createCanvas(400, 300);          // make the canvas
  serial = new p5.SerialPort();    // make a new instance of the serialport library
  serial.on('data', serialEvent);  // callback for when new data arrives
  serial.on('error', serialError); // callback for errors
  serial.on('list', printList);       // set a callback function for the serialport list event
  serial.list();                   // list the serial ports
  
  serial.open(portName);           // open a serial port
}

You’re only using the ‘data’ and ‘error’ and ‘list’ callbacks this time, but you can add the other serial callbacks if you want them. You can also add a serialport select menu as you did in the Serial Input to P5.js lab if you wish.

Program the serialEvent() function and serialError() function similarly to those in the previous lab. They read incoming data (serialEvent()) and report any errors (serialError()), as follows:

function serialEvent() {
  // read a byte from the serial port:
  var inByte = serial.read();
  // store it in a global variable:
  inData = inByte;
}
 
function serialError(err) {
  println('Something went wrong with the serial port. ' + err);
}

Program the draw() function to display the value of any incoming serial bytes. Here it is:

function draw() {
  // black background, white text:
  background(0);
  fill(255);
  // display the incoming serial data as a string:
  text("incoming value: " + inData, 30, 50);
}

function mouseDragged() {
  // map the mouseY to a range from 0 to 255:
  outByte = int(map(mouseY, 0, height, 0, 255));
  // send it out the serial port:
  serial.write(outByte);
}

function keyPressed() {
  if (key >= 0 && key <= 9) { // if the user presses 0 through 9
    outByte = byte(key * 25); // map the key to a range from 0 to 225
  }
  serial.write(outByte); // send it out the serial port
}

Sending ASCII-Encoded Serial Data

If you want to send ASCII-encoded serial data from P5.js, all you have to do is to serial.write() your string. Sending strings is the P5.serialport’s default behavior. On the Arduino side, you can read single characters one byte at a time simply as well. However, if you want to convert multi-byte number strings to numeric values, you’ll need a new function to read ASCII encoded numeric strings called parseInt().

Program the Microcontroller Again

const int ledPin = 5; // the pin that the LED is attached to
int incomingByte;     // a variable to read incoming serial data into
 
void setup() {
  Serial.begin(9600);             // initialize serial communication
  pinMode(ledPin, OUTPUT);        // initialize the LED pin as an output
}
 
void loop() {
  if (Serial.available() > 0) { // see if there's incoming serial data
    incomingByte = Serial.read(); // read it
    if (incomingByte == 'H') {    // if it's a capital H (ASCII 72),
      digitalWrite(ledPin, HIGH); // turn on the LED
      // if you're using a speaker instead of an LED, uncomment line below  and comment out the previous line:
      //  tone(5, 440);           // play middle A on pin 5
    }
    if (incomingByte == 'L') {    // if it's an L (ASCII 76)
      digitalWrite(ledPin, LOW);  // turn off the LED
      // if you're using a speaker instead of an LED, uncomment line below  and comment out the previous line:
      // noTone(5);
    }
  }
}

Program P5.js To Control the LED

To get P5.js to control this Arduino program serially, you only need to change the keyPressed() function to read H or L instead of 0 through 9. Here’s your new mousePressed() function:

function keyPressed() {
  if (key ==='H' || key ==='L') { // if the user presses H or L
    serial.write(key);              // send it out the serial port
  }
}

If you tried to change the LED with the mouse, you didn’t see anything happen unless your output value was 72 or 76. Why is that?

Processing ASCII-Encoded Strings With Arduino

void setup() {
  Serial.begin(9600);    // initialize serial communications
  Serial.setTimeout(10); // set the timeout for parseInt
}
 
void loop() {
  if (Serial.available() > 0) { // if there's serial data available
    int inByte = Serial.parseInt(); // read it
    if (inByte >= 0) {
      Serial.write(inByte);      // send it back out as raw binary data
      analogWrite(5, inByte);    // use it to set the LED brightness
      // if you're using a speaker instead of an LED, uncomment line below  and comment out the previous line:
      //  tone(5, inByte*10);     // play tone on pin 5
    }
  }
}

Program P5.js To Send a String With a Newline Character

Now that your microcontroller is expecting a string, program P5.js to send one. This means changing the mouseDragged() function. You still need to convert it to an integer using the int() function (you could also use round()), but then you need to convert it back to a String and add a delimiter. A quick way to do this is by adding the delimiter in the serial.write() command like so:

1	`serial.write(outByte +` `'\n');`

When the command encounters the two different elements, the number and the string (‘\n’), it will convert the number into a string in in order to concatenate the two. In addition, the newline on the end will is useful on the Arduino side. Since it’s a non-numeric character, the Serial.parseInt() function will see it and parse the string, not waiting for the timeout.

The full code for all the examples in this lab can be found in this gitHub repository.

Conclusion

Number of Bytes	1 Byte		Multi Bytes
Data to Send	A single number < 255	A single character	A single number > 255, multiple values
Send as:	Binary	Ascii	Ascii
p5.js ->	serial.write(integer)	serial.write(string)	serial.write (valueToSend + ",")
-> Arduino	Serial.read()		Serial.parseInt()

Table 1. Serial Communication: p5.js to Arduino

1023\n

Will always result in these six bytes:

49 48 50 51 10

Likewise, this text string:

Hello\n

will always be:

72 101 108 108 111 10

The more you work with serial data, the more you’ll become familiar with the methods for handling it.

For more on serial flow control in P5.js, see the Two-Way Duplex Serial Communication Using P5.js Lab.

Lab: Two-Way (Duplex) Serial Communication Using An Arduino and the p5.serialport Library

In this tutorial you’ll learn how to send data using asynchronous serial between an Arduino and p5.js in both directions.

Introduction

In the Introduction to Asynchronous Serial Communication lab, you learned about various methods for managing the communications between computers via asynchronous serial communication. These included formatting your data as ASCII-encoded strings or raw serial bytes and managing the flow of data using handshaking. In the P5.js Serial Input Lab, you sent data from one sensor to a personal computer. In this lab, you’ll send data from multiple sensors to a program in P5.js. You’ll use the data from the sensors to create a pointing-and-selecting device (i.e. a mouse).

What You Should Know

These videos might help in understanding this lab as well:

Video: Two-way Serial Communication (Call-and-Response)
These videos are explained in the Processing programming environment, but the concepts still apply:
- Video: Reading Serial Strings in Processing
- Video: Sending and Reading Multiple Serial Values in Processing

Things You’ll Need

Figures 1-5 below are the parts you’ll need for this exercise. Click on any image for a larger view.

Connect the Sensors

As long as you have three sensors that will output changing readings, you can follow this lab.

(Diagrams made with Fritzing, a circuit design program)

Sending Multiple Serial Data using Punctuation

const int buttonPin = 2;      // digital input
 
void setup() {
  // configure the serial connection:
  Serial.begin(9600);
  // configure the digital input:
  pinMode(buttonPin, INPUT);
}
 
void loop() {
  // read the first analog sensor:
  int sensorValue = analogRead(A0);
  // print the results:
  Serial.print(sensorValue);
  Serial.print(",");
 
  // read the second analog sensor:
  sensorValue = analogRead(A1);
  // print the results:
  Serial.print(sensorValue);
  Serial.print(",");
 
  // read the button:
  sensorValue = digitalRead(buttonPin);
  // print the results:
  Serial.println(sensorValue);
}

When you run this and output it to the Serial Monitor, you should see something like this:

348,363,1
344,362,1
345,363,1
344,375,0
365,374,0
358,369,0
355,369,0
352,373,0
356,373,0

Read the incoming serial data into a string until a carriage return and newline appear
split the string into substrings on the commas
convert the substrings into numbers
assign the numbers to variables to change your programNow that you’ve got a plan, put it into action.

Receive the data in P5.js

Now write a P5.js sketch that reads the data as formatted by the Arduino program above. Download the latest version of the P5.serialcontrol application if you haven’t already and save it in your Applications folder. When you run it, it will check serial ports on your machine. You don’t need to do anything with the app, just have it open.

You’ll need to know the name of your serial port to get started. If you’re not sure how to get this, see the Serial Input to P5.js lab for how to get a list of ports.

The P5.js Sketch

The sketch you’re going to write will control the microcontroller’s LED from P5.js. Dragging the mouse up and down the canvas will dim or brighten the LED, and typing 0 through 9 will set the LED’s brightness in increments from off (0) through almost full brightness (9). There’s an alternate sketch that will make changing tones if you prefer that instead of a changing LED. The sketch will also receive serial input from the microcontroller just as in the Serial Input to P5.js lab, so that you can see that the microcontroller is getting the same values you’re sending.

As you saw in the Serial Input to P5.js lab, you’ll need to copy the p5.seriaport.js lbrary into your sketch’s libraries folder. You’ll need to run p5.serialcontrol, or run the p5.serialserver from the command line as you did in the Serial Input to P5.js lab as well.

Make a P5.js sketch. If you’re using the p5.js web editor, make a new sketch. Click the Sketch Files tab, and then choose the index.html file. In the head of the document, look for this line:

1	`<script` `src="https://cdnjs.cloudflare.com/ajax/libs/p5.js/1.4.0/p5.js"></script>`

Right after that line, add this line:

1	`<script` `language="javascript"` `type="text/javascript"` `src="https://cdn.jsdelivr.net/npm/p5.serialserver@0.0.28/lib/p5.serialport.js"></script>`

The setup of your sketch will initialize the P5.serialport library and define your callback functions for serial events. , as you did in other sketches

Then in the setup(), create a canvas, make an instance of the serialport library, and declare your callback functions.

var serial;          // variable to hold an instance of the serialport library
 
function setup() {
  createCanvas(800, 600);          // make canvas
  smooth();                        // antialias drawing lines
  serial = new p5.SerialPort();    // make a new instance of the serialport library
  serial.on('list', printList);    // set a callback function for the serialport list event
  serial.on('connected', serverConnected); // callback for connecting to the server
  serial.on('open', portOpen);     // callback for the port opening
  serial.on('data', serialEvent);  // callback for when new data arrives
  serial.on('error', serialError); // callback for errors
  serial.on('close', portClose);   // callback for the port closing
 
  serial.list();                   // list the serial ports
}

This time, you’ll add a serialport select menu as you did in the Serial Input to P5.js lab.First, declare a global variable to hold the port menu at the top of your sketch like so:

1	`var portSelector;` `// a select menu for the port list`

Then change the printList function like so:

// get the list of ports:
function printList(portList) {
  // make a select menu and position it:
  portSelector = createSelect();
  portSelector.position(10,10);
   
  // portList is an array of serial port names
  for (var i = 0; i < portList.length; i++) {
    // Display the list the console:
    // console.log(i + " " + portList[i]);
    // add item to the select menu:
    portSelector.option(portList[i]);
  }
  // set a handler for when a port is selected from the menu:
  portSelector.changed(mySelectEvent);
}

When the select menu’s value has changed, you can assume a serial port has been selected, so write a handler to open it like so:

function mySelectEvent() {
  let item = portSelector.value();
   // if there's a port open, close it:
  if (serial.serialport != null) {
    serial.close();
  }
  // open the new port:
  serial.open(item);
}

From now on, when you run this sketch, you’ll need to select the serial port to open the port.

The rest of the serial event handlers are all the same as you saw in the P5.js Serial Input Lab, except for the serialEvent(). Here are all but the serialEvent():

function serverConnected() {
  console.log('connected to server.');
}
 
function portOpen() {
  console.log('the serial port opened.')
}
 
function serialError(err) {
  console.log('Something went wrong with the serial port. ' + err);
}
 
function portClose() {
 console.log('The serial port closed.');
}

Program the serialEvent() function to read the incoming serial data as a string until it encounters a carriage return and newline (‘\r\n’). Then check to see that the resulting string has a length greater than 0 bytes. If it does, use the split() function to split it in to an array of strings. If the resulting array is at least three elements long, you have your three sensor readings. The first reading is the first analog sensor, and can be mapped to the horizontal movement using the locH variable. The second is the second analog sensor and can be mapped to the locV variable. The third is the button. When it’s 0, set the circleColor variable equal to 255 and when it’s 1, set the variable to 0. Here’s how:

function serialEvent() {
  // read a string from the serial port
  // until you get carriage return and newline:
  var inString = serial.readStringUntil('\r\n');
 
  //check to see that there's actually a string there:
  if (inString.length > 0 ) {
    var sensors = split(inString, ',');            // split the string on the commas
    if (sensors.length > 2) {                      // if there are three elements
      locH = map(sensors[0], 0, 1023, 0, width);   // element 0 is the locH
      locV = map(sensors[1], 0, 1023, 0, height); // element 1 is the locV
      circleColor = 255 - (sensors[2] * 255);      // element 2 is the button
    }
  }
}

Note the mappings of sensor[0] and sensor[1]. You should use the input mappings for your accelerometer instead of 0 and 1023. If your analog values are greater than the width of the sketch or the height, the circle will be offscreen, which is why you have to map your sensor range to the screen size.

Program the draw() function to draw a circle that’s dependent on three global variables, locH, locV, and circleColor. Add these three globals to the top of the program:

var locH = 0;
var locV = 0;        // location of the circle
var circleColor = 255; // color of the circle

Finally, here is the draw function:

function draw() {
  background(0);               // black background
  fill(circleColor);           // fill depends on the button
  ellipse(locH, locV, 50, 50); // draw the circle
}

If you run this, you should see the circle moving onscreen whenever you tilt the accelerometer. When you press the pushbutton, the circle will disappear. Okay, it’s not exactly a mouse, but you are controlling an animation from a device that you built.

Flow Control: Call and Response (Handshaking)

void setup() {
  Serial.begin(9600);
  while (Serial.available() <= 0) {
    Serial.println("hello"); // send a starting message
    delay(300);              // wait 1/3 second
  }
}

Now, modify the loop() by adding an if() statement to look for incoming serial data and read it.

void loop() {
  if (Serial.available() > 0) {
    // read the incoming byte:
    int inByte = Serial.read();
    // read the sensor:
    sensorValue = analogRead(A0);
    // print the results:
    Serial.print(sensorValue);
    Serial.print(",");
 
    // read the sensor:
    sensorValue = analogRead(A1);
    // print the results:
    Serial.print(sensorValue);
    Serial.print(",");
 
    // read the sensor:
    sensorValue = digitalRead(buttonPin);
    // print the results:
    Serial.println(sensorValue);
  }
}

The rest of the sketch remains the same. When you run this and open the serial monitor, you’ll see:

hello
hello
hello
hello

Type any character in the output box and click Send. You’ll get a string of sensor values at the end of your hellos:

510,497,0

Open the serial port
Wait for a Hello
Send a byte to request data
Begin loop:
Wait for one set of data
Send a byte to request new data
end loop

function serialEvent() {
  // read a string from the serial port
  // until you get carriage return and newline:
  var inString = serial.readStringUntil('\r\n');
  //check to see that there's actually a string there:
  if (inString.length > 0) {
    if (inString !== 'hello') { // if you get hello, ignore it
      var sensors = split(inString, ','); // split the string on the commas
      if (sensors.length > 2) { // if there are three elements
        locH = map(sensors[0], 0, 1023, 0, width); // element 0 is the locH
        locV = map(sensors[1], 0, 1023, 0, height); // element 1 is the locV
        circleColor = 255 - (sensors[2] * 255); // element 2 is the button
      }
    }
    serial.write('x'); // send a byte requesting more serial data
  }
}

You also need to add a line to the openPort() function like so:

function portOpen() {
  console.log('the serial port opened.')
  // send a byte to prompt the microcontroller to send:
  serial.write('x');
}

The reason for this is that if your Arduino has already broken out of the loop (let’s say you opened the Serial monitor to check), then it is waiting for a byte from p5.js to send the next block of code. By sending a byte when you know the port has just been opened in p5.js, you force the Arduino to send you new data.

That’s it. Your sketch should still run just as it did before, though the serial communication is managed better now, because Arduino’s only sending when P5.js is ready to receive.

The full code for all the examples in this lab can be found in this gitHub repository.

Advantages of Raw Binary vs. ASCII

Advantages of Punctuation or Call-and-Response

Build an Application of Your Own

Lab: Serial Input to P5.js Using the p5.serialport library

n this lab, you’ll generate an analog output value from a potentiometer, then send that value via asynchronous serial communication to P5.js. You’ll use that value in P5.js to draw a graph.

Overview

In this lab, you’ll generate an analog output value from a potentiometer, then send that value via asynchronous serial communication to P5.js. You’ll use that value in P5.js to draw a graph.

Serial communication to a web page in a browser isn’t something you see every day. Web browsers don’t usually have access to a computer’s serial ports. In order to get your browser-based applications to communicate with a microcontroller serially, you need a program that can both serve HTML/JavaScript pages, and communicate with the serial port. When you’re making projects with P5.js, you can achieve this by using the P5.serialport library and the P5.serialcontrol app by Shawn Van Every (updated by Jiwon Shin). When you use the p5.serialport library, it communicates with the p5.serialcontrol app, a WebSocket server that gives you access to serial devices attached to your computer. This lab shows you how to do that.

To get the most out of this tutorial, you should know what a microcontroller is and how to program them. You should also understand asynchronous serial communication between microcontrollers and personal computers. You should also understand the basics of P5.js. This video on serial from Arduino to p5.js may be useful.

Things You’ll Need

For this lab, you’ll need the hardware below, and you’ll need to create a p5.js sketch. You’ll also use the p5.seri alport library and the P5.serialcontrol app. You can use the p5.js web editor or your favorite text editor for this (the Visual Studio Code editor or the Atom text editor work well).

Figures 1-4 below show the parts you’ll need for this exercise. Click on any image for a larger view.

Prepare the Breadboard

For this exercise you’re going to attach a potentiometer as an analog input to your microcontroller, and send the sensor’s reading serially to p5.js via the p5.serialcontrol app.

Images made with Fritzing, a circuit drawing application

The Nano is mounted at the top of the breadboard (Figure 6), straddling the center divide, with its USB connector facing up. The top pins of the Nano are in row 1 of the breadboard.

Add a Potentiometer

Connect a potentiometer to analog in pin 0 of the module. Figure 7 shows the schematic and figures 8 and 9 show the potentiometer connected to an Arduino and Nano, respectively.

Program the Microcontroller

Program your Arduino to read the analog input as follows:

void setup() {
  Serial.begin(9600); // initialize serial communications
}
 
void loop() {
  // read the input pin:
  int potentiometer = analogRead(A0);                  
  // remap the pot value to fit in 1 byte:
  int mappedPot = map(potentiometer, 0, 1023, 0, 255); 
  // print it out the serial port:
  Serial.write(mappedPot);                             
  // slight delay to stabilize the ADC:
  delay(1);                                            
}

The P5.js Serialport Library

Install the p5.Serialcontrol App

Only one port at a time can access a serial port.

Program P5.js to List the Available Serial Ports

Now you’re ready to make a P5.js sketch. If you’re using the p5.js web editor, make a new sketch. Click the Sketch Files tab, and then choose the index.html file. In the head of the document, look for this line:

1	`<script` `src="https://cdnjs.cloudflare.com/ajax/libs/p5.js/1.4.0/p5.js"></script>`

Right after that line, add this line:

1	`<script` `language="javascript"` `type="text/javascript"` `src="https://cdn.jsdelivr.net/npm/p5.serialserver@0.0.28/lib/p5.serialport.js"></script>`

The p5.js Sketch

To start off, your programming environment needs to know what serial ports are available in the operating system. Open the sketch.js file and change it to the following:

let serial; // variable to hold an instance of the serialport library
 
function setup() {
  serial = new p5.SerialPort(); // make a new instance of the serialport library
  serial.on('list', printList); // set a callback function for the serialport list event
 
  serial.list(); // list the serial ports
}
 
// get the list of ports:
function printList(portList) {
  // portList is an array of serial port names
  for (var i = 0; i < portList.length; i++) {
    // Display the list the console:
    console.log(i + portList[i]);
  }
}

When you run this p5.js sketch in a browser, you’ll get a list of the available serial ports in the console. This list will look just like the list of serial ports you see in the Arduino Tools menu. Find the name of your port in the list. Later, you’ll assign that name to a global variable called portName.

Now you’re ready to listen for some incoming serial data.

Serial Events

JavaScript, the language on which p5.js is based, relies heavily on events and callback functions. An event is generated by the operating system when something significant happens, like a serial port opening, or new data arriving in the port. In your sketch, you write a callback function to respond to that event. The serialport library uses events and callback functions as well. It can listen for the following serialport events:

list – the program asks for a list of ports.
connected – when the sketch connects to a webSocket-to-serial server
open – a serial port is opened
close – a serial port is closed
data – new data arrives in a serial port
error – something goes wrong.

You’re already using a callback for the ‘list’ event in the code above. You set a callback for the ‘list’ event, then you called it with serial.list(). Generally, you should set your callbacks before you use them like this.

To use the rest of the serialport library’s events, you need to set callback functions for them as well. Add a new global variable called portName and initialize it with the name of your serial port that you got from the listPorts() function before. Then change your setup() function to include callbacks for open, close, and error like so:

let serial;          // variable to hold an instance of the serialport library
let portName = '/dev/cu.usbmodem1421';  // fill in your serial port name here
 
function setup() {
  serial = new p5.SerialPort();       // make a new instance of the serialport library
  serial.on('list', printList);  // set a callback function for the serialport list event
  serial.on('connected', serverConnected); // callback for connecting to the server
  serial.on('open', portOpen);        // callback for the port opening
  serial.on('data', serialEvent);     // callback for when new data arrives
  serial.on('error', serialError);    // callback for errors
  serial.on('close', portClose);      // callback for the port closing
 
  serial.list();                      // list the serial ports
  serial.open(portName);              // open a serial port
}

Notice the final line of the setup(). It’s going to generate an ‘open’ event, which will be handled by a function called portOpen().

Wait a Minute! Don’t I have to Set the Data Rate When I Open the Port?

In asynchronous serial communications, both computers have to set the same data rate in order to communicate. In Arduino, you set the data rate with Serial.begin(9600); In p5.js, 9600 bits per second is the default, so you don’t have to set the rate if you want 9600bps. But if you want to set the rate to another value, do it like this:

let options = { baudrate: 9600}; // change the data rate to whatever you wish
serial.open(portName, options);

Now add new functions to respond to the callbacks you just declared. These come after your setup() function:

function serverConnected() {
  console.log('connected to server.');
}
 
function portOpen() {
  console.log('the serial port opened.')
}
 
function serialEvent() {
 
}
 
function serialError(err) {
  console.log('Something went wrong with the serial port. ' + err);
}
 
function portClose() {
  console.log('The serial port closed.');
}

The function that matters the most, though, is serialEvent(), the one that responds to new data. Each time a new byte arrives in the serial port, this function is called. Now it’s time to make serialEvent() do some work. Add a new global variable at the top of your sketch called inData like so:

let serial;          // variable to hold an instance of the serialport library
let portName = '/dev/cu.usbmodem1421';  // fill in your serial port name here
let inData;                             // for incoming serial data

Then modify the serialEvent() function like so:

function serialEvent() {
  inData = Number(serial.read());
}

Next, make the draw() function to print the sensor value to the screen. Start by adding a createCanvas() call to the top of your setup() like so:

function setup() {
createCanvas(400, 300);

Then here’s your draw() function:

function draw() {
   background(0);
   fill(255);
   text("sensor value: " + inData, 30, 50);
}

When you run your sketch now, you should get something like the sketch shown in Figure 13.

The sensor value onscreen should change as you turn your potentiometer. Congratulations! You’ve got P5.js talking to your microcontroller.

What’s Happening Here

P5.js Console.log() and Arduino delay(): a Tricky Combination

Adding A Serial Port Select Menu

If you don’t want to have to remember the serial port name every time you run the sketch, you can add a drop-down menu to select the port. Add a global variable at the top of your sketch called portSelector like so:

// HTML Select option object:
let portSelector;

Then replace the printList() function with the following:

// make a serial port selector object:
function printList(portList) {
  // create a select object:
  portSelector = createSelect();
  portSelector.position(10, 10);
  // portList is an array of serial port names
  for (var i = 0; i < portList.length; i++) {
    // add this port name to the select object:
    portSelector.option(portList[i]);
  }
  // set an event listener for when the port is changed:
  portSelector.changed(mySelectEvent);
}

Then add an extra function that will get called when the port list is changed. This function will get the name of the port you select, close any port that’s open, and open the port you asked for:

function mySelectEvent() {
  let item = portSelector.value();
   // if there's a port open, close it:
  if (serial.serialport != null) {
    serial.close();
  }
  // open the new port:
  serial.open(item);
}

If you choose this approach, you can delete the global portName variable at the top of your sketch, and the line in your setup() that says:

1	`serial.open(portName);` `// open a serial port`

Instead of opening the serial port once at the beginning of your code, you’re now opening and closing the port every time you select from this menu.

Draw a Graph With the Sensor Values

let xPos = 0;                     // x position of the graph

function setup() {
  createCanvas(400, 300);
  background(0x08, 0x16, 0x40);

function graphData(newData) {
  // map the range of the input to the window height:
  var yPos = map(newData, 0, 255, 0, height);
  // draw the line in a pretty color:
  stroke(0xA8, 0xD9, 0xA7);
  line(xPos, height, xPos, height - yPos);
  // at the edge of the screen, go back to the beginning:
  if (xPos >= width) {
    xPos = 0;
    // clear the screen by resetting the background:
    background(0x08, 0x16, 0x40);
  } else {
    // increment the horizontal position for the next reading:
    xPos++;
  }
}

Finally, take everything out of the draw() function and just call graphData() from there:

function draw() {
  graphData(inData);
}

When you run the sketch now, you should get a graph, as shown in Figure 14.

If you’d like to see a fancier graphing sketch, check out this sketch, which adds chart.js to make a better graph. Chart.js is a great tool for graphing data in JavaScript, and it integrates well with p5.js.

Reading Serial Data as a String

Change your Arduino program to the following:

void setup() {
 Serial.begin(9600); // initialize serial communications
}
 
void loop() {
 int potentiometer = analogRead(A0);                  // read the input pin
 int mappedPot = map(potentiometer, 0, 1023, 0, 255); // remap the pot value to fit in 1 byte
 Serial.println(mappedPot);                           // print it out the serial port
 delay(1);                                            // slight delay to stabilize the ADC
}

Now it will print the potentiometer’s value as an ASCII-encoded numeric string, and it will add a carriage return byte and a newline byte at the end, because that’s what println() does.

Now change the serialEvent() function like so:

function serialEvent() {
  // read a byte from the serial port, convert it to a number:
  inData = serial.readLine();
}

function serialEvent() {
  // read a string from the serial port:
  var inString = serial.readLine();
  // check to see that there's actually a string there:
  if (inString.length > 0 ) {
  // convert it to a number:
  inData = Number(inString);
  }
}

Here is a link to the complete serial in p5.js sketch.

note: readLine() is the same as readStringUntil(‘\r\n’);

The full code for all the examples in this lab can be found in this gitHub repository.

Optional: Accessing a Serial Sketch From Another Computer

You know that p5.js can run in any browser, but what happens to a sketch using p5.serialport.js when it’s running on someone else’s computer, or on your phone or tablet? What serial port does it connect to? That depends on what you do when you initialize it.

When you call new p5.SerialPort() with no parameter between the parentheses, the library attempts to connect to p5.serialcontrol app on the computer that’s running the sketch. If you’re running the sketch on your laptop’s browser, then the library connects to the your laptop’s ports through p5.serialcontrol. But imagine you run the sketch on your phone, but you want the sketch to connect to an Arduino connected to a serial port on your laptop. To do this, the your laptop has to be the web host for your sketch, as well as the computer running p5.serialcontrol. In addition, your laptop and the device running the sketch have to be on the same local network.

note: this will not work if your sketch is in the p5.js web editor. You’ll need to download the sketch and edit it on your laptop to make it work.

Launch the p5.serialcontrol app. It will display the IP address of the computer on which it’s running. Copy it, and modify the new p5.SerialPort() line at the beginning of your setup() function, adding the IP address like so:

serial = new p5.SerialPort('10.17.34.128'); // fill in your own IP address in place of the one shown here

Save the sketch, then open a command line interface on your computer (the Terminal app in MacOS, for example). Change directories to the directory where your sketch lives (for example, cd ~/Documents/p5_sketches/mySerialSketch) and run a simple web server like so:

$ python -m SimpleHTTPServer 8080

Now open a browser, either on your computer or your phone or tablet, and enter the following address:

http://your.ip.address:8080

To close this server, type control-C. You can modify the sketch as much as you want, and reload it in the browser without having to re-start the server.

The sketch should run, and operate just as you saw earlier. This approach is handy if you want to use a tablet or phone as a multimedia device to control sounds or videos, controlled by physical sensors on a microcontroller.

Note: the IP address of your computer will change as you move from one network to another (for example, from school to home). If you want to get the IP address dynamically, you can use this: window.location.hostname. This will always return the address of the host computer that served the webpage. So, for example, changing the line above to read as follows will automatically adjust the hostname each time.

// use the hostname of the computer that served this page:
serial = new p5.SerialPort(window.location.hostname); 

For more detail on this, see the video Screens: Communicating from a mobile device to a microcontroller using p5.js serialControl.

Conclusion

Data to Send	A single value within 0-255 (< 2^8 = 1 byte)	Larger numbers & characters in Ascii Table, Multiple values
Send as:	Binary	Ascii
Arduino ->	Serial.write()	Serial.println()
-> p5.js	serial.read()	serial.readLine() =readStringUntil('\r\n')

Table 1. Serial Communication: Arduino to p5.js

Notes about sending Ascii data:

Using Serial.println() on Arduino and serial.readLine() on p5.js is one of many different ways of sending Ascii data from Arduino to p5.js with serial communication.
If you want to use the value as a numeric value not a string, convert the value into number by using Number().

Understanding the difference between ASCII-encoded strings and raw binary data is central to all serial communications. For more examples of this in action, see the Serial Output from P5.js lab.

Lab: Arduino to Digital Audio Workstation

This lab covers the process of taking MIDI messages sent from the Arduino and creating sound with them via a Digital Audio Workstation (DAW) such as Ableton LIVE, Logic or Garageband.

Introduction

This lab covers the process of taking MIDI messages sent from the Arduino and creating sound with them via a Digital Audio Workstation (DAW) such as Ableton LIVE, Logic or Garageband.

What You’ll Need to Know

To get the most out of this lab, you should be familiar with the following concepts. You can check how to do so in the links below:

How to Solder a connector
Digital Input with Arduino
Analog Input with Arduino
What is an Arduino Library
How to communicate serially with the Arduino and how to send values serially in different formats
How to send MIDI Output using Arduino

Things You’ll Need

An Arduino Uno. The USB connector is facing to the left, so that the digital pins are on the top of the image, and the analog pins are on the bottom. — Figure 3. An Arduino Uno or…

Photo of a 5-pin MIDI socket. Three wires protrude from the back. — Figure 5. A 5-pin MIDI socket (for the Uno version only)

Photo of a handful of 220-ohm resistors. — Figure 6. 220-ohm resistors (for the Uno version only)

Force-sensing resistor (FSR). These sensors have a resistive rubber inside that changes its resistance depending on the force with which you press on the sensor. The one shown is a flat disc about 5cm in diameter — Figure 7. Force-sensing resistor

Photo of a handful of 10-kilohm resistors — Figure 8. 10-kilohm resistors.

MIDI approaches: Serial, SoftwareSerial, or MIDIUSB

There are three approaches you can take to MIDI output, depending on the board you’re using and the application you have in mind.

If you’re communicating with a MIDI sound module like a synthesizer or sample, you’ll need to use either Serial or SoftwareSerial output. On the Uno, SoftwareSerial is best. On most other Arduino models, there is a second hardware serial port, Serial1, that you can use for MIDI output.

If you’re communicating with a MIDI program like Ableton, GarageBand, or a soundFont synth like Sforzando, either on a laptop or mobile device, then MIDIUSB is the way to go. The Uno can’t communicate using MIDIUSB, but the Nano 33 IoT, the MKR series, the Leonardo, Micro, or Due can.

SoftwareSerial Approach

If you’re using an Uno or any board with only one serial port, the SoftwareSerial library is your best bet. This section describes how to wire and program your board for SoftwareSerial.

Build the MIDI Circuit

Use the MIDi circuit from the MIDI Output lab, or any variation on it you want, as long as it sends MIDI:

Schematic view of an Arduino connected to a voltage divider and a switch. The switch and voltage divider are connected as shown above. the MIDI connector's pin 2 is connected to a 220-ohm resistor, and the other side of the resistor is connected to +5 volts. Pin 3 of the MIDI connector is connected to ground. Pin 4 of the MIDI connector is connected to the Arduino's digital pin 3. — Figure 11. Schematic view of an Arduino connected to a voltage divider and a switch, with a MIDI connector as well.

Breadboard view of an Arduino connected to a voltage divider, a switch, and a MIDI connector. This view builds on the breadboard view above. The voltage divider and the switch are connected to analog pin 0 and digital pin 10 as described above. The MIDI connector's pin 2 is connected row 19 in the right center section of the breadboard. A red wire connects it to row 21. From there, a 220-ohm resistor connects to the right side voltage bus. The MIDI connector's pin 3 connects to the right side ground bus. The connector's pin 4 connects to row 14 in the right center section, and a blue wire connects from there to digital pin 3 on the Arduino. — Figure 12. Breadboard view of an Arduino connected to a voltage divider, a switch, and a MIDI connector.

This circuit doesn’t actually match the MIDI specification, but it works with all the MIDI devices we’ve tried it with. This circuit includes an analog and a digital sensor to allow for physical interactivity, but those aren’t necessary to send MIDI data.

Photo of an Arduino connected to a breadboard. The breadboard contains a MIDI output circuit like the one shown above. — Figure 13. An Arduino MIDI circuit ready to play

Here we have a little Arduino instrument we whipped up using the MIDI Output Lab. It features two photocells, one controlling the pitch (or note being sent) and the other controlling the volume. Now that we are sending MIDI messages, we need to get them into the computer.

Photo of a USB-to-MIDi interface. — Figure 14. The MIDISport 2×2 USB-to-MIDI interface

USB-to-MIDI Interfaces

To get the MIDI messages into the computer, you will need a USB MIDI interface. A MIDISport 2×2 like the one pictured can be found in the Equipment Room. For more information see the technical details on the MIDISport. The interface connects to your computer via the USB cable and to the Arduino via a MIDI cable. Connect the Arduino to the MIDI In port of the interface.

Alternatively, you can use the MIDIUSB library to send MIDI via USB if you are using an ARM-based Arduino board like the MKR series. Doing so will eliminate the need for a USB-to-MIDI adapter like the MIDISport. Your Arduino will communicate directly with the computer as a MIDI device over USB.

Note on MIDIUSB and Serial Ports

The MIDIUSB library on the SAMD boards (MKRZero, MKR1xxx, Nano 33IoT) has an unusual behavior: using it changes the serial port enumeration. When you include the MIDIUSB library in a sketch, your board’s serial port number will change. For example, on MacOS, if the port number is /dev/cu.usbmodem14101, then adding the MIDIUSB library will change it to /dev/cu.usbmodem14102. Removing the MIDIUSB library will change it back to /dev/cu.usbmodem14101. Similarly, if you double-tap the reset button to put the board in bootloader mode, the serial port will re-enumerate to its original number.

Windows and MIDIUSB

You may have trouble getting these MIDI sketches to work on Windows. On Windows, the default Arduino drivers must be uninstalled so the system can recognize the Arduino as both serial device and MIDI device. Read this issue and follow the instructions there.

Audio MIDI Setup

To connect with your device on MacOS you’ll need the Audio MIDI Setup application. It’s in the Utilities directory of your Applications directory.

If you’re not using the MIDIUSB library, you will need to download drivers for the MIDI interface you are using. Go to the manufacturer’s website (for example, M-Audio’s driver page, where you can find the MIDISport under MIDI Interfaces) and download the drivers that correspond to your interface and operating system. Once the drivers are installed, the interface should be recognized by your computer. To check this, open up Audio MIDI Setup. If you only see the Audio setup, click Window in your task bar and select Show MIDI Studio. You should then see the window pictured here and the interface should appear colored in. Interfaces that are not connected will appear faded as shown above.

Preferences in your DAW Software

Screenshot of the MIDI Sync Control Panel in Ableton Live, showing the MIDI ports turned on. — Figure 16. The MIDI Sync control panel in Ableton Live

Now that your computer recognizes the MIDI interface, you need to set up your preferences in the DAW. In this case, we are using Ableton Live so open up the preferences window. Ableton will recognize the interface but the ports need to be set. Find your interface and where it says Input, turn on the button listed under Track. Now Live will receive MIDI messages on these ports.

Screenshot of the Ableton Live Devices window, showing the MIDISport enabled for a given track. — Figure 17. Screenshot of the Ableton Live Devices window

Select a MIDI track and add an instrument. Live has some instruments built in or you can use a third party plugin as a sound generator. In the screenshot above, the Operator FM synthesizer is selected. If you only have one device going in to Live, it is fine to leave the track’s MIDI input set to “All Ins.” If you have more than one, you will probably want to select a specific port as shown in the image above.

Ableton Live Screenshot of an track armed to take input from a MIDI input. — Figure 18. Arm your tracks with your MIDI inputs

The last step is to Arm the track by clicking the Arm button on the bottom of the channel strip. Now you are ready to go. Start sending MIDI from the Arduino and you should be hearing your lovely new instrument.

MIDI Monitor App

The Snoize MIDI monitor is a useful application to have if you do lots of MIDI input to your computer. You can download it from Snoize’s download page.

When running the application, MIDI Monitor will show you all the information that is coming in through the MIDI interface, such as the source, channel, message and note values. This is especially useful for debugging your MIDI instrument.