Lab: Serial Input to P5.js Using the p5.serialport library

Overview

In this lab, you’ll generate an analog output value from a potentiometer, then send that value via asynchronous serial communication to P5.js. You’ll use that value in P5.js to draw a graph.

Serial communication to a web page in a browser isn’t something you see every day. Web browsers don’t usually have access to a computer’s serial ports. In order to get your browser-based applications to communicate with a microcontroller serially, you need a program that can both serve HTML/JavaScript pages, and communicate with the serial port. When you’re making projects with P5.js, you can achieve this by using the P5.serialport library and the P5.serialcontrol app by Shawn Van Every (updated by Jiwon Shin). When you use the p5.serialport library, it communicates with the p5.serialcontrol app, a WebSocket server that gives you access to serial devices attached to your computer. This lab shows you how to do that. 

To get the most out of this tutorial, you should know what a microcontroller is and how to program them. You should also understand asynchronous serial communication between microcontrollers and personal computers. You should also understand the basics of P5.js. This video on serial from Arduino to p5.js may be useful.

Things You’ll Need

For this lab, you’ll need the hardware below, and you’ll need to create a p5.js sketch. You’ll also use the p5.seri alport library and the P5.serialcontrol app. You can use the p5.js web editor or your favorite text editor for this (the Visual Studio Code editor or the  Atom text editor work well).

Figures 1-4 below show the parts you’ll need for this exercise. Click on any image for a larger view.

Prepare the Breadboard

For this exercise you’re going to attach a potentiometer as an analog input to your microcontroller, and send the sensor’s reading serially to p5.js via the p5.serialcontrol app.

Connect power and ground on the breadboard to the microcontroller. On the Arduino module, use the 5V or 3.3V (depending on your model) and any of the ground connections. Figures 5 and 6 show connections for an Arduino Uno and a Nano, respectively.

The Uno’s 5V output hole is connected to the red column of holes on the far left side of the breadboard (Figure 5). The Uno’s ground hole is connected to the blue column on the left of the board. The red and blue columns on the left of the breadboard are connected to the red and blue columns on the right side of the breadboard with red and black wires, respectively. These columns on the side of a breadboard are commonly called the buses. The red line is the voltage bus, and the black or blue line is the ground bus.

Images made with Fritzing, a circuit drawing application

The Nano is mounted at the top of the breadboard (Figure 6), straddling the center divide, with its USB connector facing up. The top pins of the Nano are in row 1 of the breadboard.

The Nano, like all Dual-Inline Package (DIP) modules, has its physical pins numbered in a U shape, from top left to bottom left, to bottom right to top right. The Nano’s 3.3V pin (physical pin 2) is connected to the left side red column of the breadboard. The Nano’s GND pin (physical pin 14) is connected to the left side black column. These columns on the side of a breadboard are commonly called the buses. The red line is the voltage bus, and the black or blue line is the ground bus. The blue columns (ground buses) are connected together at the bottom of the breadboard with a black wire. The red columns (voltage buses) are connected together at the bottom of the breadboard with a red wire.

Add a Potentiometer

Connect a potentiometer to analog in pin 0 of the module. Figure 7 shows the schematic and figures 8 and 9 show the potentiometer connected to an Arduino and Nano, respectively.

Program the Microcontroller

Program your Arduino to read the analog input as follows:

void setup() {
  Serial.begin(9600); // initialize serial communications
}

void loop() {
  // read the input pin:
  int potentiometer = analogRead(A0);                  
  // remap the pot value to fit in 1 byte:
  int mappedPot = map(potentiometer, 0, 1023, 0, 255); 
  // print it out the serial port:
  Serial.write(mappedPot);                             
  // slight delay to stabilize the ADC:
  delay(1);                                            
}

The P5.js Serialport Library

To communicate with your microcontroller serially, you’re going to use the P5.js serialport library and the p5.serialcontrol app. The P5.js serialport library can’t access your serial ports directly when a sketch is running in a browser because the browser doesn’t have direct access to the serial port. But it can communicate with another program on your computer that can exchange data with the serialport. p5.serialcontrol is  the app that connects your sketch, running in a browser, with the serial ports on your computer as shown in Figure 10.

Once you gain an understanding of serial communication, you can use any program that can connect with your computer’s serial ports to communicate with a microcontroller. Processing, Max/MSP, and OpenFrameworks are three other popular multimedia programming environments that can communicate via the serial ports. You can also do this with Unity, Unreal, or any other programming environment that can access the serial ports.

Install the p5.Serialcontrol App

Download the latest version of the P5.serialcontrol application and save it in your Applications folder. When you run it, it will check serial ports on your machine. You don’t need to do anything with the app, just have it open. However, remember this most important fact:

Only one port at a time can access a serial port.

That means that when you want to reprogram your Arduino from the Arduino IDE, you need to quit p5.serialcontrol to do so. Then, reopen p5.serialcontrol when you’re done reprogramming the Arduino. You don’t need to quit the Arduino IDE each time, because it knows to release the serial port when it’s not programming. However, you do need to close the Serial Monitor in the Arduino IDE when you are using p5.serialcontrol.

Program P5.js to List the Available Serial Ports

Now you’re ready to make a P5.js sketch. If you’re using the p5.js web editor, make a new sketch. Click the Sketch Files tab, and then choose the index.html file. In the head of the document, look for this line:

<script src="https://cdnjs.cloudflare.com/ajax/libs/p5.js/1.4.0/p5.js"></script>

Right after that line, add this line:

<script language="javascript" type="text/javascript" src="https://cdn.jsdelivr.net/npm/p5.serialserver@0.0.28/lib/p5.serialport.js"></script>

The p5.js Sketch

To start off, your programming environment needs to know what serial ports are available in the operating system. Open the sketch.js file and change it to the following:

let serial; // variable to hold an instance of the serialport library

function setup() {
  serial = new p5.SerialPort(); // make a new instance of the serialport library
  serial.on('list', printList); // set a callback function for the serialport list event

  serial.list(); // list the serial ports
}

// get the list of ports:
function printList(portList) {
  // portList is an array of serial port names
  for (var i = 0; i < portList.length; i++) {
    // Display the list the console:
    console.log(i + portList[i]);
  }
}

When you run this p5.js sketch in a browser, you’ll get a list of the available serial ports in the console. This list will look just like the list of serial ports you see in the Arduino Tools menu. Find the name of your port in the list. Later, you’ll assign that name to a global variable called portName.

Now you’re ready to listen for some incoming serial data.

Serial Events

JavaScript, the language on which p5.js is based, relies heavily on events and callback functions. An event is generated by the operating system when something significant happens, like a serial port opening, or new data arriving in the port. In your sketch, you write a callback function to respond to that event. The serialport library uses events and callback functions as well. It can listen for the following serialport events:

• list – the program asks for a list of ports.
• connected – when the sketch connects to a webSocket-to-serial server
• open – a serial port is opened
• close – a serial port is closed
• data – new data arrives in a serial port
• error – something goes wrong.

You’re already using a callback for the ‘list’ event in the code above. You set a callback for the ‘list’ event, then you called it with serial.list(). Generally, you should set your callbacks before you use them like this.

To use the rest of the serialport library’s events, you need to set callback functions for them as well. Add a new global variable called portName and initialize it with the name of your serial port that you got from the listPorts() function before. Then change your setup() function to include callbacks for open, close, and error like so:

let serial;          // variable to hold an instance of the serialport library
let portName = '/dev/cu.usbmodem1421';  // fill in your serial port name here

function setup() {
  serial = new p5.SerialPort();       // make a new instance of the serialport library
  serial.on('list', printList);  // set a callback function for the serialport list event
  serial.on('connected', serverConnected); // callback for connecting to the server
  serial.on('open', portOpen);        // callback for the port opening
  serial.on('data', serialEvent);     // callback for when new data arrives
  serial.on('error', serialError);    // callback for errors
  serial.on('close', portClose);      // callback for the port closing

  serial.list();                      // list the serial ports
  serial.open(portName);              // open a serial port
}

Notice the final line of the setup(). It’s going to generate an ‘open’ event, which will be handled by a function called portOpen().

Wait a Minute! Don’t I have to Set the Data Rate When I Open the Port?

In asynchronous serial communications, both computers have to set the same data rate in order to communicate. In Arduino, you set the data rate with Serial.begin(9600); In p5.js, 9600 bits per second is the default, so you don’t have to set the rate if you want 9600bps. But if you want to set the rate to another value, do it like this:

let options = { baudrate: 9600}; // change the data rate to whatever you wish
serial.open(portName, options);

Now add new functions to respond to the callbacks you just declared. These come after your setup() function:

function serverConnected() {
  console.log('connected to server.');
}

function portOpen() {
  console.log('the serial port opened.')
}

function serialEvent() {

}

function serialError(err) {
  console.log('Something went wrong with the serial port. ' + err);
}

function portClose() {
  console.log('The serial port closed.');
}

The function that matters the most, though, is serialEvent(), the one that responds to new data. Each time a new byte arrives in the serial port, this function is called. Now it’s time to make serialEvent() do some work. Add a new global variable at the top of your sketch called inData like so:

let serial;          // variable to hold an instance of the serialport library
let portName = '/dev/cu.usbmodem1421';  // fill in your serial port name here
let inData;                             // for incoming serial data

Then modify the serialEvent() function like so:

function serialEvent() {
  inData = Number(serial.read());
}

Next, make the draw() function to print the sensor value to the screen. Start by adding a createCanvas() call to the top of your setup() like so:

function setup() {
createCanvas(400, 300);

Then here’s your draw() function:

function draw() {
   background(0);
   fill(255);
   text("sensor value: " + inData, 30, 50);
}

When you run your sketch now, you should get something like the sketch shown in Figure 13.

The sensor value onscreen should change as you turn your potentiometer. Congratulations! You’ve got P5.js talking to your microcontroller.

What’s Happening Here

Every time your microcontroller sends a byte serially using Serial.write(), the computer receives it and generates a ‘data’ event. Then your serialEvent() function is called. It reads the byte as a number, and stores it in the global variable inData. The draw() method just uses the latest value of inData in the text string it displays on the screen.

You may be wondering why you’re mapping the sensor value or dividing it by 4 in the Arduino sketch above. That’s because in order to send the sensor value as a single byte, it must be between 0 and 255, or no more than 2⁸ bits.

P5.js Console.log() and Arduino delay(): a Tricky Combination

In testing this, you may have put a console.log() statement in the serialEvent() function in your P5.js sketch. When you did, you would have noticed that it causes a lag in the sketch, and the console.log() statements continue even after you stop the sketch. This is because the operating system keeps the incoming serial data in a buffer, and P5.js isn’t reading and printing it as fast as Arduino is sending it.

You might think, “Okay, then I’ll just put a delay() in my Arduino sketch to slow it down.” That’s a bad idea. When you put in a delay, it means you’re only reading your sensor when that delay is not running.  You can miss critical sensor events while that delay is in progress. Even a relatively small delay, for example 30ms, can make it difficult to reliably read state changes in a switch or peaks in an analog sensor. Don’t use delays if you can avoid it. For more on how to handle the flow of serial data from Arduino to P5.js and back, see the Duplex Serial Flow in P5.js lab.

Adding A Serial Port Select Menu

If you don’t want to have to remember the serial port name every time you run the sketch, you can add a drop-down menu to select the port. Add a global variable at the top of your sketch called portSelector like so:

// HTML Select option object:
let portSelector;

Then replace the printList() function with the following:

// make a serial port selector object:
function printList(portList) {
  // create a select object:
  portSelector = createSelect();
  portSelector.position(10, 10);
  // portList is an array of serial port names
  for (var i = 0; i < portList.length; i++) {
    // add this port name to the select object:
    portSelector.option(portList[i]);
  }
  // set an event listener for when the port is changed:
  portSelector.changed(mySelectEvent);
}

Then add an extra function that will get called when the port list is changed. This function will get the name of the port you select, close any port that’s open, and open the port you asked for:

function mySelectEvent() {
  let item = portSelector.value();
   // if there's a port open, close it:
  if (serial.serialport != null) {
    serial.close();
  }
  // open the new port:
  serial.open(item);
}

If you choose this approach, you can delete the global portName variable at the top of your sketch, and the line in your setup() that says:

 serial.open(portName);              // open a serial port

Instead of opening the serial port once at the beginning of your code, you’re now opening and closing the port every time you select from this menu.

Draw a Graph With the Sensor Values

It would be useful to see a graph of the sensor values over time. You can do that by modifying the draw() method to draw the graph. To do this, add a new global variable at the top of your sketch called xPos. You’ll use this to keep track of the x position of the latest graph line:

let xPos = 0;                     // x position of the graph

Because of the way the graphing function below works, you can’t reset the background every time through the draw() loop. So take the background() command and put it in the setup() function instead of the draw(), as shown below. That way it runs once, then not again. As long as you’re at it, switch from black & white to a nice blue color:

function setup() {
  createCanvas(400, 300);
  background(0x08, 0x16, 0x40);

Now make a new function called graphData(). It’ll take a number value as a parameter, and it will draw a line on the screen that’s mapped to the number value. Then it will increment xPos so that the next line is drawn further along. It will also check if the xPos is at the right edge of the screen, and reset the screen by calling background() again if it is:

function graphData(newData) {
  // map the range of the input to the window height:
  var yPos = map(newData, 0, 255, 0, height);
  // draw the line in a pretty color:
  stroke(0xA8, 0xD9, 0xA7);
  line(xPos, height, xPos, height - yPos);
  // at the edge of the screen, go back to the beginning:
  if (xPos >= width) {
    xPos = 0;
    // clear the screen by resetting the background:
    background(0x08, 0x16, 0x40);
  } else {
    // increment the horizontal position for the next reading:
    xPos++;
  }
}

Finally, take everything out of the draw() function and just call graphData() from there:

function draw() {
  graphData(inData);
}

When you run the sketch now, you should get a graph, as shown in Figure 14.

If you’d like to see a fancier graphing sketch, check out this sketch, which adds chart.js to make a better graph. Chart.js is a great tool for graphing data in JavaScript, and it integrates well with p5.js.

Reading Serial Data as a String

This works well if you want to read your sensor values as a single byte, but what if you want a larger range of numbers?  What if you want the full 0 to 1023 that analogRead() can output instead of just 0 to 255?  To do this, you need to send the data as an ASCII-encoded numeric string from the microcontroller, and you need to read and interpret the incoming data in P5 as an ASCII-encoded numeric string as well.

Change your Arduino program to the following:

void setup() {
 Serial.begin(9600); // initialize serial communications
}

void loop() {
 int potentiometer = analogRead(A0);                  // read the input pin
 int mappedPot = map(potentiometer, 0, 1023, 0, 255); // remap the pot value to fit in 1 byte
 Serial.println(mappedPot);                           // print it out the serial port
 delay(1);                                            // slight delay to stabilize the ADC
}

Now it will print the potentiometer’s value as an ASCII-encoded numeric string, and it will add a carriage return byte and a newline byte at the end, because that’s what println() does.

Once you’ve uploaded this to your Arduino, run your P5 sketch again. Try adding println(inData); at the end of your serialEvent() function. When your P5 sketch reads the data from this Arduino program, you get very low values, and every so often you see the value 10 followed by the value 13. What’s going on?

When a computer ASCII-encodes a number, it converts that number to a string of bytes, each of which is the ASCII value for a numeral in the number. For example, the number 865 gets converted to three bytes, as shown in Figure 15.

If there’s a carriage return byte and a newline byte after this, the string is five bytes, and the last two bytes’ values are 13 (carriage return, or \r in most programming languages) and 10 (newline or \n in most programming languages), respectively.

Your P5.js sketch is reading every byte’s value and graphing it. That’s why you get a graph of very low values, with a bunch of them being 13 and 10. The Arduino is ASCII-encoding the potentiometer values, but the P5 sketch is interpreting the bytes as if they’re not encoded that way.

Now change the serialEvent() function like so:

function serialEvent() {
  // read a byte from the serial port, convert it to a number:
  inData = serial.readLine();
}

Run it again. What’s changed? Now you’re getting a graph kind of like you were before. The serial.readLine(); command reads the incoming serial data as a string, and when that string happens to be all-numeric, it converts it to a number. So you’re getting the ASCII-encoded string as a number again. But now there are gaps. Why?

Remember, the ‘data’ event occurs every time a new byte comes in the serial port. Now that you’re sending an ASCII-encoded string, every potentiometer reading is several bytes long. So you only get a complete string every three to six bytes (three for “0\r\n” and six for “1023\r\n”). Sometimes, when the serialEvent() function calls serial.readLine(); it gets nothing. That’s when draw() draws the gaps. You need to change your function to check that the resulting string is actually a valid number before you put the string into inData. First, create a local variable to get the string, then check to see if the string’s length is greater than zero. If it is, then put it into inData so that the other functions in the sketch can use the new data. Here’s how you do that:

function serialEvent() {
  // read a string from the serial port:
  var inString = serial.readLine();
  // check to see that there's actually a string there:
  if (inString.length > 0 ) {
  // convert it to a number:
  inData = Number(inString);
  }
}

Here is a link to the complete serial in p5.js sketch.

Now you’re able to send in a number of any value to P5.js. You don’t have to limit your input to a 0-255 value range. See if you can modify the Arduino sketch and the P5.js sketch to exchange a potentiometer value that spans the whole range from 0 to 1023.

note: readLine() is the same as readStringUntil(‘\r\n’);

The full code for all the examples in this lab can be found in this gitHub repository.

Optional: Accessing a Serial Sketch From Another Computer

You know that p5.js can run in any browser, but what happens to a sketch using p5.serialport.js when it’s running on someone else’s computer, or on your phone or tablet? What serial port does it connect to? That depends on what you do when you initialize it.

When you call new p5.SerialPort() with no parameter between the parentheses, the library attempts to connect to p5.serialcontrol app on the computer that’s running the sketch. If you’re running the sketch on your laptop’s browser, then the library connects to the your laptop’s ports through p5.serialcontrol. But imagine you run the sketch on your phone, but you want the sketch to connect to an Arduino connected to a serial port on your laptop. To do this, the your laptop has to be the web host for your sketch, as well as the computer running p5.serialcontrol. In addition, your laptop and the device running the sketch have to be on the same local network.

note: this will not work if your sketch is in the p5.js web editor. You’ll need to download the sketch and edit it on your laptop to make it work.

Launch the p5.serialcontrol app. It will display the IP address of the computer on which it’s running. Copy it, and modify the new p5.SerialPort() line at the beginning of your setup() function, adding the IP address like so:

serial = new p5.SerialPort('10.17.34.128'); // fill in your own IP address in place of the one shown here

Save the sketch, then open a command line interface on your computer (the Terminal app in MacOS, for example). Change directories to the directory where your sketch lives (for example, cd ~/Documents/p5_sketches/mySerialSketch) and run a simple web server like so:

$ python -m SimpleHTTPServer 8080

Now open a browser, either on your computer or your phone or tablet, and enter the following address:

To close this server, type control-C. You can modify the sketch as much as you want, and reload it in the browser without having to re-start the server.

The sketch should run, and operate just as you saw earlier. This approach is handy if you want to use a tablet or phone as a multimedia device to control sounds or videos, controlled by physical sensors on a microcontroller.

Note: the IP address of your computer will change as you move from one network to another (for example, from school to home). If you want to get the IP address dynamically, you can use this: window.location.hostname. This will always return the address of the host computer that served the webpage. So, for example, changing the line above to read as follows will automatically adjust the hostname each time.

// use the hostname of the computer that served this page:
serial = new p5.SerialPort(window.location.hostname); 

For more detail on this, see the video Screens: Communicating from a mobile device to a microcontroller using p5.js serialControl.

Conclusion

In this lab, you saw how to connect an Arduino microcontroller to a P5.js sketch using a webSocket-to-serial server, P5.serialserver, and the P5.serialport library. You sent data from Arduino to the sketch as a raw binary value — that is, a single byte ranging from 0 to 255 — and you sent it as an ASCII-encoded numeric string with a carriage return and newline at the end. See Table 1 below to review what are suitable data formats for different types/sizes of data and which functions to use on p5.js and Arduino for serial communication.

Notes about sending Ascii data:

• Using Serial.println() on Arduino and serial.readLine() on p5.js is one of many different ways of sending Ascii data from Arduino to p5.js with serial communication.
• If you want to use the value as a numeric value not a string, convert the value into number by using Number(). 

Understanding the difference between ASCII-encoded strings and raw binary data is central to all serial communications. For more examples of this in action, see the Serial Output from P5.js lab.