Lab: Serial Input to P5.js

Overview

Serial communication to a web page in a browser isn’t something you see every day. Web browsers don’t usually have access to a computer’s serial ports. In order to get your browser-based applications to communicate with a microcontroller serially, you need a program that can both serve up HTML/JavaScript pages, and communicate with the serial port. When you’re making projects with P5.js, you can achieve this by using the P5.serialport library and P5.serialserver and the P5.serialcontrol app by Shawn Van Every. When you use the p5.serialport library, it communicates with a the serialserver, a WebSocket server written in node.js to give you access to serial devices attached to your computer. This lab shows you how to do that. In this lab, you’ll generate an analog output value from a potentiometer, then send that value via asynchronous serial communication to P5.js. You’ll use that value in P5.js to draw a graph.

To get the most out of this tutorial, you should know what a microcontroller is and how to program them. You should also understand asynchronous serial communication between microcontrollers and personal computers. You should also understand the basics of P5.js. This video on serial from Arduino to p5.js may be useful.

Things You’ll Need

For this lab, you’ll need the hardware below, and you’ll need to download the P5.js complete library. You’ll also need to install node.js and the P5.serialserver (instructions below). You can use the P5.js web editor or your favorite text editor for this (the Atom text editor works well).

An Arduino Uno. The USB connector is facing to the left, so that the digital pins are on the top of the image, and the analog pins are on the bottom.
An Arduino Uno.
Potentiometer. The one shown here has three legs spaced 0.1 inches apart and can be therefore mounted on a solderless breadboard.
Potentiometer

Assemble the circuit

Assemble the following circuit. It’s the same circuit from the Analog Input lab.

Connect power and ground on the breadboard to power and ground from the microcontroller. On the Arduino module, use the 5V and any of the ground connections:

An Arduino Uno on the left connected to a solderless breadboard, right. The Uno's 5V output hole is connected to the red column of holes on the far left side of the breadboard. The Uno's ground hole is connected to the blue column on the left of the board. The red and blue columns on the left of the breadboard are connected to the red and blue columns on the right side of the breadboard with red and black wires, respectively. These columns on the side of a breadboard are commonly called the buses. The red line is the voltage bus, and the black or blue line is the ground bus.
An Arduino Uno on the left connected to a solderless breadboard, right.

Made with Fritzing

Add a Potentiometer

Connect a potentiometer to analog in pin 0 of the module:

Related Video: Potentiometer schematic

Schematic view of a potentiometer. First leg of the potentiometer is connected to +5 volts. The second leg connected to analog in 0 of the Arduino. The third leg is connected to ground.
Schematic view of a potentiometer connected to analog in 0 of the Arduino
Breadboard view of a potentiometer. First leg of the potentiometer is connected to +5 volts. The second leg connected to analog in 0 of the Arduino. The third leg is connected to ground.
Breadboard view of a potentiometer connected to analog in 0 of an Arduino

 


 

Program the Microcontroller

Program your Arduino to read the analog input as follows:

void setup() {
 Serial.begin(9600); // initialize serial communications
}

void loop() {
 int potentiometer = analogRead(A0);                  // read the input pin
 int mappedPot = map(potentiometer, 0, 1023, 0, 255); // remap the pot value to fit in 1 byte
 Serial.write(mappedPot);                             // print it out the serial port
 delay(1);                                            // slight delay to stabilize the ADC
}

The P5.js serialport library

To communicate with your microcontroller serially, you’re going to use the P5.js serialport library and the P5.serialserver (command line version) or p5.serialcontrol (GUI version). The P5.js serialport library can’t access your serial ports directly when a sketch is running in a browser because the browser doesn’t have direct access to the serial port. But it can communicate with a server program on your computer that can exchange data with the serialport.P5.serialserver (or p5.serialcontrol) is  the server that connects your sketch, running in a browser, with the serial ports on your computer as shown below:

Diagram of three rectangles connected by arrows. The rectangle on the right represents your p5.js sketch, running in a browser. Your sketch implements the p5.serialport library. Your sketch connects to p5 serial control, the middle rectangle, via a webSocket. The p5 serial control application runs on your laptop. It connects to a serial port on your computer and listens for webSocket connections from your p5.js sketch. It passes whatever comes in the serial port through to the webSocket and vice versa. The third rectangle is your Arduino, connected to p5 serial control via the serial port.
Diagram of the connection from the serial port to p5.js through p5.serialcontrol

 

Once you gain an understanding of serial communication, you can use any program that can connect with your computer’s serial ports to communicate with a microcontroller. Processing, Max/MSP, and OpenFrameworks are three other popular multimedia programming environments that can communicate via the serial ports.

Install the P5.serialcontrol App

Download the latest version of the P5.serialcontrol application and save it in your Applications folder. When you run it, it will check serial ports on your machine. Click “list ports” to refresh the list, and you should see the serial port of your Arduino board show up. You can now move onto using the P5.js editor making sure that the p5.serialport.js file is inside your project folder. You can skip the next section unless you want to run p5.serialserver from the command line instead from the p5.serialcontol app.

Program P5.js to list the available  serial ports

Now you’re ready to make a P5.js sketch. If you’re using the p5.js web editor, make a new sketch. Download this file: p5.serialport.js. Then add it to your sketch by clicking the file navigation arrow on the left of the screen, then click the down arrow to add files. Choose the downloaded p5.serialport.js file and upload it to your sketch.

p5_editor_file_nav_screenshot,p5_editor_add_file_screenshot

p5 editor showing file navigation arrow, p5 editor showing add file arrow

Whether you are running the P5.serialserver via command line or using the P5.serialcontrol app, you will need to add one line to the index.html file. After this line:

<script language="javascript" type="text/javascript" src="libraries/p5.js"></script>

Add this line:

 <script language="javascript" type="text/javascript" src="p5.serialport.js"></script>

Save index.html. Now you’re ready to edit the sketch.

To start off, your programming environment needs to know what serial ports are available in the operating system. Open the sketch.js file and change it to the following:

var serial; // variable to hold an instance of the serialport library

function setup() {
 serial = new p5.SerialPort(); // make a new instance of the serialport library
 serial.on('list', printList); // set a callback function for the serialport list event

 serial.list(); // list the serial ports
}

// get the list of ports:
function printList(portList) {
 // portList is an array of serial port names
 for (var i = 0; i < portList.length; i++) {
 // Display the list the console:
 println(i + " " + portList[i]);
 }
}

When you run this P5 sketch in a browser, you’ll get a list of the available serial ports in the console. This list will look just like the list of serial ports you see in the Arduino Tools menu. Find the name of your port in the list. Later, you’ll assign that name to a global variable called portName.

 Serial events

JavaScript, the language on which P5.js is based, relies heavily on events and callback functions. An event is generated by the operating system when something significant happens, like a serial port opening, or new data arriving in the port. In your sketch, you write a callback function to respond to that event. The serialport library uses events and callback functions as well. It can listen for the following serialport events:

  • list – the program asks for a list of ports.
  • connected – when the sketch connects to a webSocket-to-serial server
  • open – a serial port is opened
  • close – a serial port is closed
  • data – new data arrives in a serial port
  • error – something goes wrong.

You’re already using a callback for the ‘list’ event in the code above. You set a callback for the ‘list’ event, then you called it with serial.list(). Generally, you should set your callbacks before you use them like this.

To use the rest of the serialport library’s events, you need to set callback functions for them as well. Add a new global variable called portName and initialize it with the name of your serial port. Then change your setup() function to include callbacks for open, close, and error like so:

var serial;          // variable to hold an instance of the serialport library
var portName = '/dev/cu.usbmodem1421';  // fill in your serial port name here

function setup() {
  serial = new p5.SerialPort();       // make a new instance of the serialport library
  serial.on('list', printList);  // set a callback function for the serialport list event
  serial.on('connected', serverConnected); // callback for connecting to the server
  serial.on('open', portOpen);        // callback for the port opening
  serial.on('data', serialEvent);     // callback for when new data arrives
  serial.on('error', serialError);    // callback for errors
  serial.on('close', portClose);      // callback for the port closing

  serial.list();                      // list the serial ports
  serial.open(portName);              // open a serial port
}

Notice the final line of the setup(). It’s going to generate an ‘open’ event, which will be handled by a function called portOpen().

Now add new functions to respond to the callbacks you just declared. These come after your setup() function:

function serverConnected() {
  println('connected to server.');
}

function portOpen() {
  println('the serial port opened.')
}

function serialEvent() {

}

function serialError(err) {
  println('Something went wrong with the serial port. ' + err);
}

function portClose() {
  println('The serial port closed.');
}

Most of these functions just provide notification. The error function is a bit more useful because it tells you what went wrong if something went wrong. If you didn’t change the portName variable to the name of your microcontroller’s serial port, for example, you probably got this error in the command line window where P5.serialserver is running:


Something went wrong with the serial port. Couldn't open port: /dev/cu.usbmodem1421

The function that matters the most, though, is serialEvent(), the one that responds to new data. Each time a new byte arrives in the serial port, this function is called. Now it’s time to make serialEvent() do some work. Add a new global variable at the top of your sketch called inData like so:

var serial;          // variable to hold an instance of the serialport library
var portName = '/dev/cu.usbmodem1421';  // fill in your serial port name here
var inData;                             // for incoming serial data

Then modify the serialEvent() function like so:

function serialEvent() {
  inData = Number(serial.read());
}

Although you’re reading the incoming data, you’re not displaying it anywhere yet. Add a draw() function and print the sensor value to the screen. Start by adding a createCanvas() to the top of your setup() like so:

function setup() {
  createCanvas(400, 300);

Then here’s your draw() function:

function draw() {
  background(0);
  fill(255);
  text("sensor value: " + inData, 30, 30);
}

When you run your sketch now, you should get something like this:

readSerial-screenshot

The sensor value onscreen should change as you turn your potentiometer. Congratulations! You’ve got P5.js talking to your microcontroller.

What’s Happening Here

Every time your microcontroller sends a byte serially using Serial.write(), the computer receives it and generates a ‘data’ event. Then your serialEvent() function is called. It reads the byte as a number, and stores it in the global variable inByte. The draw() method just uses the latest value of inByte in the text string it displays on the screen.

You may be wondering why you’re mapping the sensor value or dividing it by 4 in the Arduino sketch above. That’s because in order to send the sensor value as a single byte, it must be between 0 and 255, or no more than 28 bits.

Draw a Graph With the Sensor Values

It would be useful to see a graph of the sensor values over time. You can do that by modifying the draw() method to draw the graph. To do this, add a new global variable at the top of your position called xPos. You’ll use this to keep track of the x position of the latest graph line:

var serial;          // variable to hold an instance of the serialport library
var portName = '/dev/cu.usbmodem1421';  // fill in your serial port name here
var inData;                             // for incoming serial data
var xPos = 0;                           // x position of the graph

Because of the way the graphing function below works, you can’t reset the background every time through the draw() loop. So take the background() command and put it in the setup() function instead of the draw(), as shown below. That way it runs once, then not again. As long as you’re at it, switch from black & white to a nice blue color:

function setup() {
  createCanvas(400, 300);
  background(0x08, 0x16, 0x40);

Now make a new function called graphData(). It’ll take a number value as a parameter, and it will draw a line on the screen that’s mapped to the number value. Then it will increment xPos so that the next line is drawn further along. It will also check if the xPos is at the right edge of the screen, and reset the screen by calling background() again if it is:

function graphData(newData) {
  // map the range of the input to the window height:
  var yPos = map(newData, 0, 255, 0, height);
  // draw the line in a pretty color:
  stroke(0xA8, 0xD9, 0xA7);
  line(xPos, height, xPos, height - yPos);
  // at the edge of the screen, go back to the beginning:
  if (xPos >= width) {
    xPos = 0;
    // clear the screen by resetting the background:
    background(0x08, 0x16, 0x40);
  } else {
    // increment the horizontal position for the next reading:
    xPos++;
  }
}

Finally, take everything out of the draw() function and just call graphData() from there:

function draw() {
  graphData(inData);

When you run the sketch now, you should get a graph, like so:

graph-screenshot

Try changing the Arduino sketch by adding delay(100); at the end of the loop. Note how it changes what happens in the graph.

Reading Serial Data as a String

This works well if you want to read your sensor values as a single byte, but what if you want a larger range of numbers?  What if you want the full 0 to 1023 that analogRead() can output instead of just 0 to 255?  To do this, you need to send the data as an ASCII-encoded numeric string from the microcontroller, and you need to read and interpret the incoming data in P5 as an ASCII-encoded numeric string as well.

Change your Arduino program to the following:

void setup() {
 Serial.begin(9600); // initialize serial communications
}

void loop() {
 int potentiometer = analogRead(A0);                  // read the input pin
 int mappedPot = map(potentiometer, 0, 1023, 0, 255); // remap the pot value to fit in 1 byte
 Serial.println(mappedPot);                           // print it out the serial port
 delay(1);                                            // slight delay to stabilize the ADC
}

Now it will print the potentiometer’s value as an ASCII-encoded numeric string, and it will add a carriage return byte and a newline byte at the end, because that’s what println() does.

Once you’ve uploaded this to your Arduino, run your P5 sketch again. Try adding println(inData); at the end of your serialEvent() function. When your P5 sketch reads the data from this Arduino program, you get very low values, and every so often you see the value 10 followed by the value 13. What’s going on?

When a computer ASCII-encodes a number, it converts that number to a string of bytes, each of which is the ASCII value for a numeral in the number. For example, the number 865 gets converted to three bytes, like so:

ascii-numeral

If there’s a carriage return byte and a newline byte after this, the string is five bytes, and the last two bytes’ values are 13 (carriage return, or \r in most programming languages) and 10 (newline or \n in most programming languages), respectively.

Your P5.js sketch is reading every byte’s value and graphing it. That’s why you get a graph of very low values, with a bunch of them being 13 and 10. The Arduino is ASCII-encoding the potentiometer values, but the P5 sketch is interpreting the bytes as if they’re not encoded that way.

Now change the serialEvent() function like so:

function serialEvent() {
  // read a byte from the serial port, convert it to a number:
  inData = serial.readLine();
}

Run it again. What’s changed? Now you’re getting a graph kind of like you were before. The serial.readLine(); command reads the incoming serial data as a string, and when that string happens to be all-numeric, it converts it to a number. So you’re getting the ASCII-encoded string as a number again. But now there are gaps. Why?

Remember, the ‘data’ event occurs every time a new byte comes in the serial port. Now that you’re sending an ASCII-encoded string, every potentiometer reading is several bytes long. So you only get a complete string every three to six bytes (three for “0\r\n” and six for “1023\r\n”). Sometimes, when the serialEvent() function calls serial.readLine(); it gets nothing. That’s when draw() draws the gaps. You need to change your function to check that the resulting string is actually a valid number before you put the string into inData. First, create a local variable to get the string, then check to see if the string’s length is greater than zero. If it is, then put it into inData so that the other functions in the sketch can use the new data. Here’s how you do that:

function serialEvent() {
  // read a string from the serial port:
  var inString = serial.readLine();
  // check to see that there's actually a string there:
  if (inString.length > 0 ) {
  // convert it to a number:
  inData = Number(inString);
  }
}

Now you’re able to send in a number of any value to P5.js. You don’t have to limit your input to a 0-255 value range. See if you can modify the Arduino sketch and the P5.js sketch to exchange a potentiometer value that spans the whole range from 0 to 1023.

note: readLine() is the same as readStringUntil(‘\r\n’);

The full code for all the examples in this lab can be found in this gitHub repository.

Optional: Accessing a Serial Sketch From Another Computer

You know that p5.js can run in any browser, but what happens to a sketch using p5.serialport.js when it’s running on someone else’s computer, or on your phone or tablet? What serial port does it connect to? That depends on what you do when you initialize it.

When you call new p5.SerialPort() with no parameter between the parentheses, the library attempts to connect to p5.serialcontrol app on the computer that’s running the sketch. If you’re running the sketch on your laptop’s browser, then the library connects to the your laptop’s ports through p5.serialcontrol. But imagine you run the sketch on your phone, but you want the sketch to connect to an Arduino connected to a serial port on your laptop. To do this, the your laptop has to be the web host for your sketch, as well as the computer running p5.serialcontrol. In addition, your laptop and the device running the sketch have to be on the same local network.

note: this will not work if your sketch is in the p5.js web editor. You’ll need to download the sketch and edit it on your laptop to make it work.

Launch the p5.serialcontrol app. It will display the IP address of the computer on which it’s running. Copy it, and modify the new p5.SerialPort() line at the beginning of your setup() function, adding the IP address like so:

serial = new p5.SerialPort('10.17.34.128');  // fill in your own IP address in place of the one shown here

Save the sketch, then open a command line interface on your computer (the Terminal app in MacOS, for example). Change directories to the directory where your sketch lives (for example, cd ~/Documents/p5_sketches/mySerialSketch) and run a simple web server like so:

$ python -m SimpleHTTPServer 8080

Now open a browser, either on your computer or your phone or tablet, and enter the following address:

http://your.ip.address:8080

To close this server, type control-C. You can modify the sketch as much as you want, and reload it in the browser without having to re-start the server.

The sketch should run, and operate just as you saw earlier. This approach is handy if you want to use a tablet or phone as a multimedia device to control sounds or videos, controlled by physical sensors on a microcontroller. For more detail on this, see the video Screens: Communicating from a mobile device to a microcontroller using p5.js serialControl

Conclusion

In this lab, you saw how to connect an Arduino microcontroller to a P5.js sketch using a webSocket-to-serial server, P5.serialserver, and the P5.serialport library. You sent data from Arduino to the sketch as a raw binary value — that is, a single byte ranging from 0 to 255 — and you sent it as an ASCII-encoded numeric string with a carriage return and newline at the end. Understanding the difference between ASCII-encoded strings and raw binary data is central to all serial communications. For more examples of this in action, see the Serial Output from P5.js lab.

Originally written on October 4, 2015 by Tom Igoe
Last modified on October 9, 2018 by Tom Igoe