Tele

Anything Next To Anything (Anywhere)

Now by adding telecommunication, any element in one person’s computer can be next any element on another person’s computer.  We talked about how the random access nature of electronic storage allows any element in memory to be instantly juxtaposed with any other element.  Rearranging data into different versions is cheap and fast, which makes looking at things new ways easier, as when you change categories on this blog.  Creating new versions is democratized as compared with movable type or film where new versions take much more effort and expense.  Now your ideas can be more easily be juxtaposed with the ideas on any one else’s computer on the planet into new versions.  This radically accelerates the possibilities for ideas being in the right place at the right time.

For now we will talk between the Arduino and your laptop.   But the principles will be the same later when we connect any two computers on the internet.

Goal

You are going to try to get some numbers or letters  from one of your sensors in Arduino out the USB cable into one of the variables controlling your display in P5 .  You might also try to go the other way get a variable in P5out the USB cable to the Arduino to say move the servo motor.  Sending one number is not so bad.  Sending multiple readings will require you to make a protocol for sending the readings separated by delimiters and then on arrival splitting and storing them into arrays.  That is a nice challenge for you.

Circuit

Wire up a potentiometer to your arduino. Using analogRead() and your Serial Monitor.  Double check that you are getting sensor readings from your potentiometer:

You should see something like this:

serialMonitor

Protocol

Communication is hard when you make up your own words.  Both sides have to agree on a protocol, that is a the conventions of the communication, ahead of time.  Because you will be controlling both sides of the communication, Arduino and P5, you really only have to agree with yourself.  There is a whole stack of electrical agreements that you don’t have to worry about because they were set for you, for instance that there is one wire for transmit and one wire for receive in the USB cable and electrical pulses get sent one right after another (serially).  The conventions of your protocol will mostly be about formatting the message that you are sending.

Baud Rate

The only important agreement about the electrical signal that you control is the speed in which the pulses come along.  This is called the baud rate.  The most common baud rate is 9600 baud which will send a byte (a letter or a number between 0-255) every millisecond.  9600 is pretty common and you will probably never change this one so this is not very interesting.  Sorry.

Sending on the Arduino Side

You have already done the Arduino side of this when you were debugging by sending from your Arduino to your laptop using Serial.begin(9600) and then Serial.println(myVar).  Back on your laptop you had to click on the “Serial Monitor” and make sure it was agreeing with the 9600 baud.

But now we’re going to try the same thing with a small difference.

Notice there is one small change compared to the analog debugging output example.  Instead of “Serial.println”  We are sending out information using “Serial.write” .  IF you were to open your Serial Monitor now, the output will look like this:

Serial Write Output

Do not be alarmed.  Even though it looks like garbled nonsense, this is actually what we are expecting to see.  But why?

 We’ll get to it later.

Listening on the P5 Side

Listening in P5 is a little more involved.  You have to:

  1. Download the latest p5.serialcontrol app , save it to your applications
  2. In the p5 editor: Import the  p5js serialport Library
  3. Open the p5.serialcontrol app
  4. Make a Serial Object Variable
  5. Find the Right USB port in the Setup Function and use new to make a serial object.
  6. Make a Callback Function (several actually….) to Listen to Incoming Messages.

Import the Serial Library

Because USB serial port is different on every different kind of computer (eg mac, pc),  is not part of the core of P5 (or javascript).  Instead you have to extend P5 with the appropriate library for doing it on your particular hardware platform.  Luckily for you, the library for your machine downloaded with P5 when you installed it.  So all you have to do is let P5 know adding the p5.serialport.js file to your libraries folder (there is even a menu option that does it for you).

importserial2 seriallib

Serial Object

Welcome to object oriented programming!  Until now we have been using variables to store number or letters.  Now we are going to use a variable to store an object.  An object not only has numbers and letters (data) but also functions for doing things (behavior).  So at the top you declare a variable of type “Serial” meaning you are going to store a Serial object in it.

Initializing Objects: We typically use the new command with some parameters, in this case which port (more on that in a minute) and it returns an object for you to store in your variable.  This is usually done in setup.

Start the Serial Server

To communicate with your microcontroller serially, you’re going to use the P5.js serialport library and the P5.js editor. The P5.js serialport library can’t access your serial ports directly when a sketch is running in a browser or in the P5 sketch window. But it can communicate with a server program on your computer that can exchange data with the serialport. The P5 editor acts as the server that connects your sketch, running in a browser or sketch window, with the serial ports on your computer as shown below:

 

The P5.js serial server uses the node.js programming environment to create the serial-to-webSocket server that allows you to connect your sketch to the serial port.

The P5.serialport library can only work with a serial-to-webSocket server. The p5.serialControl app will start and stop your serial server.  In order to run a sketch that utilizes serial communication make sure you start your Serial Server before you run your sketch:

 

Pick a Port in Setup

The P5 version of the “setup” is slightly more complicated than Arduino because there can be more than one USB cable plugged into your laptop.  Most of the time this code will just work because your Arduino will be the first (zeroth) USB serial port that P5 looks for.  Other times you will have to run your program once and then look in the output box at the list of ports that result from “println(i+” “+portList[i])”   The port names are not very friendly especially on the mac they look like “/dev/tty.usbmodem1421” and on the PC some like “COM1”   It is not easy to find the one that belongs to the Arduino but you can try unplugging the arudino and see which one disappears.  You already set this in the Arudino software so you can also look there under the “Tools>Port” menu.  You can either supply the string like “/dev/tty.usbmodem1421” or you can pull the name by number from the list of ports as happens in the sample code here.

Baud Rate (again)

This has to match your Arduino’s baud rate setting in order for P5 to communicate correctly.

Events and Callbacks

JavaScript, the language on which P5.js is based, relies heavily on events and callback functions. An event is generated by the operating system when something significant happens, like a serial port opening, or new data arriving in the port. In your sketch, you write a callback function to respond to that event. The serialport library uses events and callback functions as well. It can listen for the following serialport events:

  • list – the program asks for a list of ports.
  • connected – when the sketch connects to a webSocket-to-serial server
  • open – a serial port is opened
  • close – a serial port is closed
  • data – new data arrives in a serial port
  • error – something goes wrong.

You’re already using a callback for the ‘list’ event in the code above. You set a callback for the ‘list’ event, then you called it with serial.list(). Generally, you should set your callbacks before you use them like this.

Now add new functions to respond to the callbacks you just declared. These come after your setup()function:

Most of these functions just provide notification. The error function is a bit more useful because it tells you what went wrong if something went wrong. If you didn’t change the portName variable to the name of your microcontroller’s serial port, for example, you probably got this error:

The function that matters the most, though, is serialEvent(), the one that responds to new data. Each time a new byte arrives in the serial port, this function is called. Now it’s time to make serialEvent() do some work. Add a new global variable at the top of your sketch called inData like so:

Then modify the serialEvent() function like so:

Although you’re reading the incoming data, you’re not displaying it anywhere yet. Add a draw() function and print the sensor value to the screen. Start by adding a createCanvas() to the top of your setup() like so:

Then here’s your draw() function:

When you run your sketch now, you should get something like this:

Sending on the Arduino Side (again)

You have already done the Arduino side using Serial.write. You’ve also done this when you were debugging by sending from your Arduino to your laptop using Serial.begin(9600) and then Serial.println().  Back on your laptop you had to click on the “Serial Monitor” and make sure it was agreeing with the 9600 baud.  Let’s try using Serial.print and Serial.println again.

 

Notice there is one small change compared to the analog debugging output example.  The “ln” is missing at the end of “println“.  The println function automatically appends a carriage return character ‘\r‘ and new line character ‘\n‘ to whatever message we want to print.   In the code above we are using Serial.print to explicitly send out those characters.

Try printing out your potentiometer readings both ways, try the code above and see what the serial monitor reads.  Then replace the three Serial.print() commands with just one Serial.println().

In either case you should see something like this:

serialMonitor

The next step is really just replacing the Serial Monitor on your laptop with P5 on your laptop.  You might still test everything out with the Serial Monitor first to make sure the Arduino side is right before trying with P5.  Make sure you close the Serial Monitor because it should not be open at the same time as P5.

 

Listening is Harder Than Talking

In Software, as in person, it is easy to know you have to pay attention when speaking.  Because you don’t know exactly when someone else is going to speak you have to always be ready to hear which is difficult. The technical term here is Asynchronous Communication.  You could use a “blocking function” which stop the whole program to watch for something coming in the port.  Computers have few tricks to save you from dedicating your whole program to waiting for the next thing to come in.

The first feature is called a buffer which is a little waiting room where the computer seats incoming bytes (in order) for you until your program has time to grab them an use them.  The serial buffer is made for you automatically in the setup.

The second feature is called a control character.  This is a character that is added to the the end of our communication to signal the end of our transmission

We will use callbacks to check when we have received information as well as when we should send information

CALL BACK METHOD: We want P5 to call a function when there is some action on the USB cable.  By making the serial object in setup, behind the scenes you subscribed to a service that will call our serialEvent function whenever something comes into the buffer.  You have already seen this concept of callbacks with MousePressed() and KeyPressed() for user input which is also asynchronous, meaning you never know exactly when it is going to happen.

Also instead of printing the number like before, lets use that number to move an ellipse:

 

Control Characters

Congratulations you just made a protocol.  You are saying there will be a ‘\r’ and a ‘\n’ character between each reading.  ‘\r’ is a carriage return and ‘\n’ is the new line character.  The serial monitor uses these characters to put each reading on it’s own line.  This is the extra bit that “println” sends as opposed to “print.”  You will notice in the previous arduino example I am using just print and sending the carriage return and new line characters more explicitly with print(‘\r’) print(‘\n’) instead of println().   The new line is what is called a control character.  You see the results of it but the actual character is invisible so to give it a manifestation we use the single quotes followed by a back slash and the letter n for new line.  ‘\t’ is the way your represent a tab, another invisible control character.  In general the back slash in text string is called a the escape character which tells the machine not to take the next character (in this case “n” or “t” ) at face value.  One other thing you will notice here is sometimes we use double quote and sometimes single quotes.  Because it is a single character we use a single quote in this case.  Hopefully you will get used to this stuff.

ReadStringUntil(‘\r\n’)

This  is a really great function that looks in the buffer to see if the whole message has arrived yet.   If the ‘\r\n’ characters are not in the buffer it means your full message has not arrived yet and it puts null or undefined  into the variable.  If it the carriage return and new line characters are in there that means that the full message has arrived and this function call  pulls it out of the waiting buffer and puts it in your variable.

Trim()

This cleans off the invisible control characters that will screw things up.

Number()

This converts the incoming message from a String to a number (much more on this below).

You Don’t Really Need to Know This: (How the Computer get bigger numbers from a group of 1’s and 0’s which is all it knows) As we have seen before these computers are really only capable of 1’s and 0’s.  Now we are really just sending a serial stream of pulses of yes and no over the wire.  To express bigger numbers we group a bunch of 1’s or 0’s (aka bits)  into groups of 8 bits (aka bytes).  We are always sending bytes back and forth.  Then each unique permutation of the 8 bits represents a number between 0 and 255 (there are 256 different permutations of eight 1’s and 0’s). The actual method for getting the decimal from a eight ones and zeros is something you may never need to know but I will tell you anyway.  Basically each place has a weight and you add up the weight of all the places where there was a one.

Position Weights: 1   2   8   16   32  64  128
Number: 1  0  0  1   0    0    0   = 17
Number: 0  1  1  0   0   1    0    =  74

Sending and Receiving Numbers

The simplest way to communicate would have been to use the .write() and .read() functions (same on both Arduino and P5).  These send numbers as single bytes back and forth which would seem to make sense because often the message begins its life as a number (eg analogRead()) and completes its mission as a number (eg ellipse(xpos,10,10,10)).  But we probably won’t be doing this in sending from Arduino to P5.

Text from Numbers

Humans prefer to communicate more with words than numbers so we often make one further conversion from numbers to letters using an ASCII table.  The first 32 characters are those invisible control characters we were just talking about. You can also refer to them by number, for instance, 10 for ‘\n’ (although sometimes it is 10  and 13).


 

Sending and Receiving Text

Usually I recommend that you go the extra step of turning your numbers into text using print() (intend of write()) and read them as text using readString() (instead of read()) as I have been doing in the  above  examples (except the polling example).  Here are the reasons it is better to send numbers as text:

  1. Debugging is easier with text.   Tools like the Serial Monitor can only really display text.  Numbers  get translated by the ASCII table by the Serial Monitor anyway and end up looking like all sorts of nonsense characters.
  2. It is easier to send arbitrarily long numbers with text because sending each number as a byte limits you to numbers between 0-255.  With text you can just keep adding a byte for every new digit as new letter in the string.
  3. It is is easier to send multiple numbers as text because you can use text things like comma’s and control characters as separators (delimiters) between the numbers.

I am Confused How Can a Letter Be A Number?

You will see in the ascii chart that character number 48-58 are “alphanumeric characters.”   When you send out a number using the Serial.print() the Arduino uses the ASCII table to look for the number of the alphanumeric character for each digit.  For instance to send “1008” it would send four bytes, 49,48,48,56.  If you were using println it would send a fifth character 49,48,48,56,10.  If you see a lot of numbers in this range coming by in your debug windows it probably means you are interpreting numbers sent as text using the print command when sending but then interpreting them as numbers using the read() command on receiving.  Being consistent about interpreting things as either numbers or text on both sides of the conversation is probably the most important protocol decision you will make.  When talking from Arduino to P5 I recommend you send and receive as text using the print and readString commands (if you want to).

Sending Multiple Things From Arduino to P5

serialPotSwitch_bb serialPotSwitch_schem

Suppose you wanted to make a game controller with a force sensitive resistor for controlling your movement and switch for firing.  You would want to send two numbers from Arduino to P5.  We were already using the ‘\n’ character to delimit between messages.  You need a new delimiter to separate the values from the two different sensors.  Luckily because you are only using characters 48-58 in the ASCII chart for alpha numeric characters, all of  the others are available for another delimiter.  The most common ones are commas or another control character, tab ‘\t’.  Now your protocol is getting a little more detailed.  Whatever you pick as a delimiter, it just can’t show up in the delimiter.  Later on you might look at HTML code where they use <> or json where they use {} and : for delimiters.

Here is the Arduino Code.  I added a comma and a second sensor (a switch) and sent everything out separated by commas.

On the P5 side both sensors arrive in the order they were sent as one long string of text.  We break it into parts using the split() function.  We need something called an array variable that can hold multiple things in one variable.  Arrays have one name like the usual variables but then have multiple numbered slots for storage.  The tell tale sign of an array variable is the square brackets.  You will see this parts variable is declared as String[ ] instead of String.  This kind of parsing of text based on delimiters is the most common task in computer communication so get used to it.

 

Sending From P5 to Arduino

For talking back from P5 to Arduino, I suggest you just send numbers instead of letters using the write()  from P5 and read() in Arduino  (not print() and readStringUntil()).

Synchronizing:

Sometimes you just want to send a character back to Arduino to say you are ready for a new message.  If your Arduino sent stuff faster than P5 can use it, data will get backed up in the buffer.  We are using the if statement to check if something is available, meaning P5 sent something before we send something back.   In this case the content of the message is unimportant.

Note we are also using a while loop in our setup() to check if the Serial port is open.  So as long as the serial port is NOT open, the arduino will print “hello” every 300 milliseconds to initiate contact.  Once the port is open, P5 will (finally) receive the “hello” message and we can proceed to the loop.

And still, in the loop we are using an if statement to continue checking if P5 is still ready to communicate.  So once the port is open we still want to continue to check if P5 is ready for communication.

 

Now you have to make sure to send something to Arduino to feed the “if available”. It is easy to wind up in the situation where both sides are locked up waiting for something to be sent by the other side. You should send in serialEvent so Arduino is prodded to send a new value every time P5 gets a new one.  We call this call and response or handshaking.  But you should also send a character right in setup in P5 to prime the pump so you don’t get into a situation where both sides are waiting to hear from the other.

the important parts:

full code:

Sending Values from P5 to Arduino and from Arduino Back to P5 (NOW WITH A MOTOR)

servo_bb servo

Sometimes you actually want to send a real value from P5 to Arduino not just a nonsense byte for synchronization.  Here is an example where I am using the mouseX in P5 to move a servo motor in Arduino.  And just for kicks lets also use P5 to visualize at what angle the motor is currently positioned.  Here is the P5 sending side:

 

Here is the Arduino receiving side:

 

The Internet

Microcontrollers are moving away from being connected by USB towards being connected by WIFI.  Devices like the Arduino Yun, the Rasberry PI, the Beagle Bone all feel more like nodes on the internet.  Regardless of whether you use a wire or note,  the principles of sending bytes back and forth will be the same will be the same.

Helpful links

ITP physcomp lab serial from Arduino to P5

ITP physcomp lab serial from P5 to Arduino

ITP phsycomp lab two-way serial communication btwn P5 and Arduino

Leave a Reply