{"id":6306,"date":"2019-08-24T17:11:56","date_gmt":"2019-08-24T21:11:56","guid":{"rendered":"https:\/\/itp.nyu.edu\/physcomp\/?page_id=6306"},"modified":"2022-11-06T18:18:01","modified_gmt":"2022-11-06T23:18:01","slug":"lab-serial-imu-output-to-p5-js","status":"publish","type":"page","link":"https:\/\/itp.nyu.edu\/physcomp\/labs\/lab-serial-imu-output-to-p5-js\/","title":{"rendered":"Lab: Serial IMU Output to p5.js Using p5.webserial"},"content":{"rendered":"\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Introduction\"><\/span>Introduction<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<p>In this exercise you&#8217;ll read the built-in Inertial Motion Unit on the Arduino Nano 33 IoT, then feed its output into a Madgwick filter to determine heading, pitch, and roll of the board. Then you&#8217;ll send the output of that serially to p5.js and use it to move a virtual version of the Nano onscreen.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"What_Youll_Need_to_Know\"><\/span>What You\u2019ll Need to Know<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<p>To get the most out of this lab, you should be familiar with the following concepts and you should install the Arduino IDE on your computer. You can check how to do so in the links below:<\/p>\n\n\n\n<ul class=\"wp-block-list\"><li><a href=\"https:\/\/itp.nyu.edu\/physcomp\/lessons\/serial-communication-the-basics\/\">What is a microcontroller<\/a><\/li><li>Beginning&nbsp;<a href=\"https:\/\/itp.nyu.edu\/physcomp\/lessons\/programming-terms-and-programming-environments\/\">programming terms<\/a><\/li><li>The basics of&nbsp;<a title=\"Electricity: the Basics\" href=\"https:\/\/itp.nyu.edu\/physcomp\/lessons\/electronics\/electricity-the-basics\/\">electricity<\/a><\/li><li>What is a&nbsp;<a href=\"https:\/\/itp.nyu.edu\/physcomp\/labs\/breadboard\/\">solderless breadboard<\/a>&nbsp;and how to use one<\/li><li><a href=\"https:\/\/docs.arduino.cc\/hardware\/nano-33-iot\">Getting Started with Arduino Guide<\/a><\/li><li>The basics of serial communication, both <a href=\"https:\/\/itp.nyu.edu\/physcomp\/lessons\/serial-communication-the-basics\/\">asynchronous<\/a> and <a href=\"https:\/\/itp.nyu.edu\/physcomp\/lessons\/synchronous-serial-communication-the-basics\/\">synchronous<\/a>. <\/li><li><a href=\"https:\/\/itp.nyu.edu\/physcomp\/labs\/labs-serial-communication\/lab-two-way-duplex-webserial-communication\/\">Two-Way (Duplex) Serial Communication Using An Arduino and the p5.webserial Library<\/a><\/li><li>The basics of <a href=\"https:\/\/itp.nyu.edu\/physcomp\/lessons\/accelerometers-gyros-and-imus-the-basics\/\">accelerometers, gyrometers, and IMUs<\/a><\/li><\/ul>\n\n\n\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Things_Youll_Need\"><\/span>Things You\u2019ll Need<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<p>The only part you&#8217;ll need for this exercise is an Arduino Nano 33 IoT and its built-in IMU, as shown in Figure 1. You can modify this exercise to work with other IMUs, however. There are details on various IMUs on the <a style=\"font-size: revert;\" href=\"https:\/\/itp.nyu.edu\/physcomp\/lessons\/accelerometers-gyros-and-imus-the-basics\/\">accelerometers, gyrometers, and IMUs<\/a> page.<\/p>\n\n\n\n<div class=\"wp-block-image\"><figure class=\"alignleft\"><a href=\"https:\/\/itp.nyu.edu\/physcomp\/wp-content\/uploads\/pcomp-kit-f2019-arduino-nano-33-iot.jpg\"><img loading=\"lazy\" decoding=\"async\" width=\"150\" height=\"150\" src=\"https:\/\/itp.nyu.edu\/physcomp\/wp-content\/uploads\/pcomp-kit-f2019-arduino-nano-33-iot-150x150.jpg\" alt=\"Photo of an Arduino Nano 33 IoT module. The USB connector is at the top of the image, and the physical pins are numbered in a U-shape from top left to bottom left, then from bottom right to top right.\" class=\"wp-image-5921\"\/><\/a><figcaption>Figure 1. Microcontroller. Shown here is an Arduino Nano 33 IoT.<\/figcaption><\/figure><\/div>\n\n\n\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Prepare_the_Breadboard\"><\/span>Prepare the Breadboard<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<p>because the Nano 33 IoT has a built-in IMU, there is no additional circuit needed for this exercise. However, there are two libraries you&#8217;ll need to install: the Arduino_LSM6DS3 library, which allows you to read the IMU, and the&nbsp;MadgwickAHRS library, which takes the raw accelerometer and gyrometer inputs and provides heading, pitch, and roll outputs. Both libraries can be found in the Library Manager of the Arduino IDE. Install them before proceeding.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Program_the_Microcontroller_to_Read_the_IMU\"><\/span>Program the Microcontroller to Read the IMU<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<p>The first thing to do in the microcontroller code is to confirm that your accelerometer and gyrometer are working. Start with the code below:<\/p>\n\n\n<div class=\"wp-block-syntaxhighlighter-code \"><pre class=\"brush: arduino; title: ; notranslate\" title=\"\">\n#include \"Arduino_LSM6DS3.h\"\n\nvoid setup() {\n  Serial.begin(9600);\n  \/\/ attempt to start the IMU:\n  if (!IMU.begin()) {\n    Serial.println(\"Failed to initialize IMU\");\n    \/\/ stop here if you can't access the IMU:\n    while (true);\n  }\n}\n\nvoid loop() {\n  \/\/ values for acceleration and rotation:\n  float xAcc, yAcc, zAcc;\n  float xGyro, yGyro, zGyro;\n\n  \/\/ check if the IMU is ready to read:\n  if (IMU.accelerationAvailable() && IMU.gyroscopeAvailable()) {\n    \/\/ read accelerometer and gyrometer:\n    IMU.readAcceleration(xAcc, yAcc, zAcc);\n    IMU.readGyroscope(xGyro, yGyro, zGyro);\n\n    Serial.print(\"sensors: \");\n    Serial.print(xAcc);\n    Serial.print(\",\");\n    Serial.print(yAcc);\n    Serial.print(\",\");\n    Serial.print(zAcc);\n    Serial.print(\",\");\n    Serial.print(xGyro);\n    Serial.print(\",\");\n    Serial.print(yGyro);\n    Serial.print(\",\");\n    Serial.println(zGyro);\n  }\n}\n<\/pre><\/div>\n\n\n<p>When you run this sketch and open the Serial Monitor, you should see a printout with six values per line. The first three are your accelerometer values, and the next three are your gyrometer values. The following reading is typical:<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\">sensors: 0.04,-0.05,1.02,3.05,-3.72,-1.77<\/pre>\n\n\n\n<p>The Nano 33 IoT&#8217;s accelerometer&#8217;s range is fixed at +\/-4G by this library, and its gyrometer&#8217;s range is set at +\/-2000 degrees per second (dps). The sampling rate for both is set to 104 Hz by the library. Other IMUs may have differing ranges. You need to know at least the sampling rate when you want to use a different IMU with this exercise. If you know that information, though, it&#8217;s easy to swap one IMU for another in the Madgwick library.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Add_the_Madgwick_Library_to_Get_Orientation\"><\/span>Add the Madgwick Library to Get Orientation<span class=\"ez-toc-section-end\"><\/span><\/h3>\n\n\n\n<p>The MadgwickAHRS library can work with any accelerometer\/gyrometer combination. It expects the acceleration in Gs and the rotation in degrees per second as input, and uses the sensors&#8217; sampling rate when you initialize it. Add a few lines to the code before your <code>setup()<\/code> as follows:<\/p>\n\n\n<div class=\"wp-block-syntaxhighlighter-code \"><pre class=\"brush: arduino; title: ; notranslate\" title=\"\">\n#include \"Arduino_LSM6DS3.h\"\n#include \"MadgwickAHRS.h\"\n\n\/\/ initialize a Madgwick filter:\nMadgwick filter;\n\/\/ sensor's sample rate is fixed at 104 Hz:\nconst float sensorRate = 104.00;\n\n\/\/ values for orientation:\nfloat roll = 0.0;\nfloat pitch = 0.0;\nfloat heading = 0.0;\n<\/pre><\/div>\n\n\n<p>Next, add the following line at the end of the <code>setup()<\/code> to initialize the Madgwick filter:<\/p>\n\n\n<div class=\"wp-block-syntaxhighlighter-code \"><pre class=\"brush: arduino; title: ; notranslate\" title=\"\">\n\/\/ start the filter to run at the sample rate:\nfilter.begin(sensorRate);\n<\/pre><\/div>\n\n\n<p>Now change the main loop so that you&#8217;re sending the sensor readings into the Madgwick filter. You&#8217;ll do this inside of the if statement that checks if the sensors are ready:<\/p>\n\n\n<div class=\"wp-block-syntaxhighlighter-code \"><pre class=\"brush: arduino; title: ; notranslate\" title=\"\">\n\/\/ check if the IMU is ready to read:\nif (IMU.accelerationAvailable() &amp;&amp;\nIMU.gyroscopeAvailable()) {\n  \/\/ read accelerometer and gyrometer:\n  IMU.readAcceleration(xAcc, yAcc, zAcc);\n  IMU.readGyroscope(xGyro, yGyro, zGyro);\n\n  \/\/ update the filter, which computes orientation:\n  filter.updateIMU(xGyro, yGyro, zGyro, xAcc, yAcc, zAcc);\n\n  \/\/ print the heading, pitch and roll\n  roll = filter.getRoll();\n  pitch = filter.getPitch();\n  heading = filter.getYaw();\n\n  \/\/ print the filter's results:\n  Serial.print(heading);\n  Serial.print(\",\");\n  Serial.print(pitch);\n  Serial.print(\",\");\n  Serial.println(roll);\n}\n<\/pre><\/div>\n\n\n<p>Now when you run the sketch, you&#8217;ll get heading, pitch, and roll instead of the raw sensor readings. Here&#8217;s a typical output you might see:<\/p>\n\n\n\n<p>167.59,-2.50,-2.52In this case, the readings are all in degrees. The first is the heading angle, around the Z axis. The second two are the pitch, around the x axis, and roll, around the Y axis.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Add_Serial_Handshaking\"><\/span>Add Serial Handshaking<span class=\"ez-toc-section-end\"><\/span><\/h3>\n\n\n\n<p>Reading these values in p5.js will work smoother if you add handshaking, also known as call-and-response, to your serial communications protocol. Modify the <code>loop()<\/code> so that the sketch sends the latest heading, pitch, and roll whenever a new byte comes in the serial port. Here&#8217;s the final version of the <code>loop()<\/code>:<\/p>\n\n\n<div class=\"wp-block-syntaxhighlighter-code \"><pre class=\"brush: arduino; title: ; notranslate\" title=\"\">\nvoid loop() {\n  \/\/ values for acceleration and rotation:\n  float xAcc, yAcc, zAcc;\n  float xGyro, yGyro, zGyro;\n\n  \/\/ check if the IMU is ready to read:\n  if (IMU.accelerationAvailable() & amp; & amp;\n      IMU.gyroscopeAvailable()) {\n    \/\/ read accelerometer and gyrometer:\n    IMU.readAcceleration(xAcc, yAcc, zAcc);\n    IMU.readGyroscope(xGyro, yGyro, zGyro);\n\n    \/\/ update the filter, which computes orientation:\n    filter.updateIMU(xGyro, yGyro, zGyro, xAcc, yAcc, zAcc);\n\n    \/\/ print the heading, pitch and roll\n    roll = filter.getRoll();\n    pitch = filter.getPitch();\n    heading = filter.getYaw();\n  }\n\n  \/\/ if you get a byte in the serial port,\n  \/\/ send the latest heading, pitch, and roll:\n  if (Serial.available()) {\n    char input = Serial.read();\n    Serial.print(heading);\n    Serial.print(\",\");\n    Serial.print(pitch);\n    Serial.print(\",\");\n    Serial.println(roll);\n  }\n}\n<\/pre><\/div>\n\n\n<p>this link. When you have this much working, and you&#8217;ve tested it in the Serial Monitor, you can close Arduino and work on the p5.js sketch.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Program_p5js_to_Read_the_Incoming_Serial_Data\"><\/span>Program p5.js to Read the Incoming Serial Data<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<p>Now it&#8217;s time to write a p5.js sketch to read this data.&nbsp; &nbsp;The setup will be the same as it was in the&nbsp;<a href=\"https:\/\/itp.nyu.edu\/physcomp\/labs\/labs-serial-communication\/lab-webserial-input-to-p5-js\/#p5WebSerial_Sketch_Checklist\">Serial Input to p5.js using WebSerial&nbsp;<\/a>lab. The checklist from that lab lays out all the important parts you need.<\/p>\n\n\n\n<p>Make a P5.js sketch. If you\u2019re using the&nbsp;<a href=\"http:\/\/alpha.editor.p5js.org\/\">p5.js web editor<\/a>, make a new sketch. Click the Sketch Files tab, and then choose the&nbsp;<code>index.html<\/code>&nbsp;file.&nbsp;Edit the head of the document as you did for the other p5.webserial labs. It should look like this:<\/p>\n\n\n<div class=\"wp-block-syntaxhighlighter-code \"><pre class=\"brush: xml; title: ; notranslate\" title=\"\">\n&lt;script src=&quot;https:\/\/cdnjs.cloudflare.com\/ajax\/libs\/p5.js\/1.4.0\/p5.js&quot;&gt;&lt;\/script&gt;\n     \n&lt;script src=&quot;https:\/\/unpkg.com\/p5-webserial@0.1.1\/build\/p5.webserial.js&quot;&gt;&lt;\/script&gt;\n\n<\/pre><\/div>\n\n\n<p>Start your sketch with some code to initialize the serial library:<\/p>\n\n\n<div class=\"wp-block-syntaxhighlighter-code \"><pre class=\"brush: jscript; title: ; notranslate\" title=\"\">\n\/\/ variable to hold an instance of the p5.webserial library:\nconst serial = new p5.WebSerial();\n \n\/\/ HTML button object:\nlet portButton;\n\nfunction setup() {\n  \tcreateCanvas(500, 600, WEBGL);     \/\/ make the canvas\n  \/\/ check to see if serial is available:\n  if (!navigator.serial) {\n    alert(\"WebSerial is not supported in this browser. Try Chrome or MS Edge.\");\n  }\n  \/\/ if serial is available, add connect\/disconnect listeners:\n  navigator.serial.addEventListener(\"connect\", portConnect);\n  navigator.serial.addEventListener(\"disconnect\", portDisconnect);\n  \/\/ check for any ports that are available:\n  serial.getPorts();\n  \/\/ if there's no port chosen, choose one:\n  serial.on(\"noport\", makePortButton);\n  \/\/ open whatever port is available:\n  serial.on(\"portavailable\", openPort);\n  \/\/ handle serial errors:\n  serial.on(\"requesterror\", portError);\n  \/\/ handle any incoming serial data:\n  serial.on(\"data\", serialEvent);\n  serial.on(\"close\", makePortButton);\n}\nfunction draw() {\n \n}\n \n\/\/ if there's no port selected, \n\/\/ make a port select button appear:\nfunction makePortButton() {\n  \/\/ create and position a port chooser button:\n  portButton = createButton('choose port');\n  portButton.position(10, 10);\n  \/\/ give the port button a mousepressed handler:\n  portButton.mousePressed(choosePort);\n}\n \n\/\/ make the port selector window appear:\nfunction choosePort() {\n  serial.requestPort();\n}\n \n\/\/ open the selected port, and make the port \n\/\/ button invisible:\nfunction openPort() {\n  \/\/ wait for the serial.open promise to return,\n  \/\/ then call the initiateSerial function\n  serial.open().then(initiateSerial);\n \n  \/\/ once the port opens, let the user know:\n  function initiateSerial() {\n    console.log(\"port open\");\n    serial.write(\"x\");\n  }\n  \/\/ hide the port button once a port is chosen:\n  if (portButton) portButton.hide();\n}\n \n\/\/ read any incoming data:\nfunction serialEvent() {\n  \/\/ read a string from the serial port\n  \/\/ until you get carriage return and newline:\n  var inString = serial.readStringUntil(\"\\r\\n\");\n  console.log(inString);\n}\n \n\/\/ pop up an alert if there's a port error:\nfunction portError(err) {\n  alert(\"Serial port error: \" + err);\n}\n \n\/\/ try to connect if a new serial port \n\/\/ gets added (i.e. plugged in via USB):\nfunction portConnect() {\n  console.log(\"port connected\");\n  serial.getPorts();\n}\n \n\/\/ if a port is disconnected:\nfunction portDisconnect() {\n  serial.close();\n  console.log(\"port disconnected\");\n}\n<\/pre><\/div>\n\n\n<p>Save this as <code>sketch.js<\/code>, then open p5.serialcontrol. Then open the sketch in a browser. Open the JavaScript console, and you should see the first set of data printed out. This is because the <code>initiateSerial()<\/code> function sent a single byte to the Nano when the port opened, and the Nano sent one set of readings. That generated a serial data event in p5.js, and called the <code>serialEvent()<\/code> function, which printed out the results.You need this to happen repeatedly: p5.js sends a byte when it wants new data, then the Nano sends the data, then waits for another byte from p5.js.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Add_Serial_Handshaking-2\"><\/span>Add Serial Handshaking<span class=\"ez-toc-section-end\"><\/span><\/h3>\n\n\n\n<p>To make this happen, you need to add a few things to your p5.js sketch. You can assume that if you saw the message in the console, then you&#8217;re ready for new data. That&#8217;s when you should send a byte back to the microcontroller to request new data. Add one line to the <code>serialEvent()<\/code> function to make this happen:<\/p>\n\n\n<div class=\"wp-block-syntaxhighlighter-code \"><pre class=\"brush: jscript; title: ; notranslate\" title=\"\">\n\/\/ callback function for incoming serial data:\nffunction serialEvent() {\n  \/\/ read a string from the serial port\n  \/\/ until you get carriage return and newline:\n  var inString = serial.readStringUntil(\"\\r\\n\");\n  if (inString != null) {\n    console.log(inString);\n    serial.write(\"x\");\n  }\n}\n<\/pre><\/div>\n\n\n<p>When you run the sketch with this update, you should see a continuous flow of new data from the microcontroller.<\/p>\n\n\n\n<p>Next, you need to break the string up into parts and convert them into floating point numbers so you can use them as heading, pitch, and roll. Start by adding three new variables at the top of your sketch as global variables, because you&#8217;ll need them when you draw the virtual Arduino:<\/p>\n\n\n<div class=\"wp-block-syntaxhighlighter-code \"><pre class=\"brush: jscript; title: ; notranslate\" title=\"\">\n\/\/ orientation variables:\nlet heading = 0.0;\nlet pitch = 0.0;\nlet roll = 0.0;\n<\/pre><\/div>\n\n\n<p>Next, in the <code>serialEvent()<\/code> function, use the JavaScript <code>trim()<\/code> function to get rid of any extraneous characters at the end of the message string, like carriage returns or newlines. Then use the <code>split()<\/code> function to split the string into a list of elements separated by commas. Then convert them to floating point numbers. Once you know you have three valid numbers for heading, pitch, and roll, send another byte to the microcontroller to get a new reading. Here&#8217;s what the new version of <code>serialEvent()<\/code> looks like:<\/p>\n\n\n<div class=\"wp-block-syntaxhighlighter-code \"><pre class=\"brush: jscript; title: ; notranslate\" title=\"\">\nfunction serialEvent() {\n  \/\/ read from port until new line:\n  let inString = serial.readStringUntil(&quot;\\r\\n&quot;);\n  if (inString != null) {\n    let list = split(trim(inString), &quot;,&quot;);\n    if (list.length &gt; 2) {\n      \/\/ conver list items to floats:\n      heading = float(list&#x5B;0]);\n      pitch = float(list&#x5B;2]);\n      roll = float(list&#x5B;1]);\n      console.log(heading + &quot;,&quot; + pitch + &quot;,&quot; + roll);\n      \/\/ send a byte to the microcontroller to get new data:\n      serial.write(&quot;x&quot;);\n    }\n  }\n}\n\n<\/pre><\/div>\n\n\n<p>When you reload the sketch after making these changes, you should be getting floating point numbers for heading, pitch and roll. Once you have these values coming in consistently, it&#8217;s a good idea to comment out the <code>console.log()<\/code> statement, as shown above, as it slows down the sketch considerably.<\/p>\n\n\n\n<p>Now that you have serial communication working properly, it&#8217;s time to write the code to draw the virtual microcontroller.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Draw_the_Virtual_Arduino\"><\/span>Draw the Virtual Arduino<span class=\"ez-toc-section-end\"><\/span><\/h3>\n\n\n\n<p>Add the function below to draw a virtual Arduino. It draws in three dimensions, using the WEBGL framework you chose in <code>createCanvas()<\/code> above in the <code>setup()<\/code> function.<\/p>\n\n\n<div class=\"wp-block-syntaxhighlighter-code \"><pre class=\"brush: jscript; title: ; notranslate\" title=\"\">\n\/\/ draws the Arduino Nano:\nfunction drawArduino() {\n   \/\/ the base board:\n   stroke(0, 90, 90); \/\/ set outline color to darker teal\n   fill(0, 130, 130); \/\/ set fill color to lighter teal\n   box(300, 10, 120); \/\/ draw Arduino board base shape\n\n   \/\/ the CPU:\n   stroke(0);         \/\/ set outline color to black\n   fill(80);          \/\/ set fill color to dark grey\n   translate(30, -6, 0); \/\/ move to correct position\n   box(60, 0, 60);    \/\/ draw box\n\n   \/\/ the radio module:\n   stroke(80);       \/\/ set outline color to grey\n   fill(180);        \/\/ set fill color to light grey\n   translate(80, 0, 0); \/\/ move to correct position\n   box(60, 15, 60);  \/\/ draw box\n\n   \/\/ the USB connector:\n   translate(-245, 0, 0); \/\/ move to correct position\n   box(35, 15, 40);\t  \/\/ draw box\n}\n<\/pre><\/div>\n\n\n<p>You haven&#8217;t added a <code>draw()<\/code> function yet, so add it now, as follows:<\/p>\n\n\n<div class=\"wp-block-syntaxhighlighter-code \"><pre class=\"brush: jscript; title: ; notranslate\" title=\"\">\nfunction draw() {\n   background(255); \/\/ set background to white\n   push();          \/\/ begin object to draw\n   \/\/ draw arduino board:\n   drawArduino();\n   pop();           \/\/ end of object\n}\n<\/pre><\/div>\n\n\n<p>When you reload the sketch, you&#8217;ll see a drawing like that in Figure 4. It won&#8217;t change.<\/p>\n\n\n\n<div class=\"wp-block-image\"><figure class=\"aligncenter\"><a href=\"https:\/\/itp.nyu.edu\/physcomp\/wp-content\/uploads\/VirtualNano.png\"><img loading=\"lazy\" decoding=\"async\" width=\"399\" height=\"252\" src=\"https:\/\/itp.nyu.edu\/physcomp\/wp-content\/uploads\/VirtualNano.png\" alt=\"A virtual Arduino Nano 33 IoT, drawn in in p5.js. \" class=\"wp-image-6313\"\/><\/a><figcaption>Figure 4. A virtual Arduino Nano 33 IoT, drawn in in p5.js. The Nano is seen from the side, with the USB connector on the right, and the radio on the right.<\/figcaption><\/figure><\/div>\n\n\n\n<p>To make it change its orientation, you need to use the heading, pitch, and roll values to rotate the object. You get the sine and cosine of each angle, and use them to generate a matrix for translation. The math below was worked out by Helena Bisby based on the Madgwick algorithm. p5.js&#8217; <a href=\"https:\/\/p5js.org\/reference\/#\/p5\/applyMatrix\">applyMatrix()<\/a> function does the matrix math for you to rotate in all three dimensions.&nbsp; Modify the <code>draw()<\/code> function as shown below:<\/p>\n\n\n<div class=\"wp-block-syntaxhighlighter-code \"><pre class=\"brush: jscript; title: ; notranslate\" title=\"\">\nfunction draw() {\n   \/\/ update the drawing:\n   background(255); \/\/ set background to white\n   push();          \/\/ begin object to draw\n\n   \/\/ variables for matrix translation:\n   let c1 = cos(radians(roll));\n   let s1 = sin(radians(roll));\n   let c2 = cos(radians(pitch));\n   let s2 = sin(radians(pitch));\n   let c3 = cos(radians(heading));\n   let s3 = sin(radians(heading));\n   applyMatrix(c2 * c3, s1 * s3 + c1 * c3 * s2,\n      c3 * s1 * s2 - c1 * s3, 0, -s2, c1 * c2,\n      c2 * s1, 0, c2 * s3, c1 * s2 * s3 - c3 * s1,\n      c1 * c3 + s1 * s2 * s3, 0, 0, 0, 0, 1);\n\n   \/\/ draw arduino board:\n   drawArduino();\n   pop(); \/\/ end of object\n}\n<\/pre><\/div>\n\n\n<p>When you reload the sketch after making these changes, the virtual Arduino should change its position as you move the physical Arduino. That&#8217;s the whole application! Figure 5 shows the virtual Arduino in motion. <\/p>\n\n\n\n<p>You can see the sketch running on GitHub at&nbsp;<a href=\"https:\/\/itpnyu.github.io\/physcomp\/Labs\/LabIMUs\/MadgwickVisualizer\">this link<\/a>. You can see the source files for copying into the p5.js editor at&nbsp;<a href=\"https:\/\/github.com\/ITPNYU\/physcomp\/tree\/main\/Labs\/LabIMUs\/MadgwickVisualizer\">this link<\/a>.<\/p>\n\n\n\n<div class=\"wp-block-image\"><figure class=\"aligncenter\"><a href=\"https:\/\/itp.nyu.edu\/physcomp\/wp-content\/uploads\/VirtualNanoMotion.gif\"><img loading=\"lazy\" decoding=\"async\" width=\"480\" height=\"291\" src=\"https:\/\/itp.nyu.edu\/physcomp\/wp-content\/uploads\/VirtualNanoMotion.gif\" alt=\"Moving GIF of a virtual Arduino Nano turning in three dimensions\" class=\"wp-image-6317\"\/><\/a><figcaption>Figure 5. This virtual Arduino Nano, written in p5.js, moves in three dimensions as you move a real Nano connected to the sketch serially.<\/figcaption><\/figure><\/div>\n\n\n\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Conclusion\"><\/span>Conclusion<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<p>If you followed along all of the steps to this application, you probably hit a number of places where communication broke down. There are a lot of pieces to this application, and they all need to work together.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Get_the_Sensors_Working\"><\/span>Get the Sensors Working<span class=\"ez-toc-section-end\"><\/span><\/h3>\n\n\n\n<p>When you&#8217;re dealing with IMU sensors, no data will be perfect, because the sensor&#8217;s measurement is always relative. You&#8217;ll notice, for example, that the position of the virtual Arduino drifts a bit the longer you run the sketch. Heading, in particular, tends to drift. In real-world applications, this is often adjusted by using a magnetometer as a compass in addition to the accelerometer and gyrometer. It&#8217;s also wise to provide ways for a human to calibrate the system, perhaps by pressing a button when the sensor is level in order to calculate offset values for the sensors. For many interactive applications, though, even an imperfect measurement of orientation will do the job well.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Test_the_Hardware\"><\/span>Test the Hardware<span class=\"ez-toc-section-end\"><\/span><\/h3>\n\n\n\n<p>If you&#8217;re using serial communication that&#8217;s ASCII-encoded like this, you can always use the Serial Monitor or another serial terminal application to test the Arduino sketch before you ever begin working on the multimedia programming. Ideally, you don&#8217;t need to change the Arduino sketch at all once your communication is working as planned.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Get_the_Communication_Working\"><\/span>Get the Communication Working<span class=\"ez-toc-section-end\"><\/span><\/h3>\n\n\n\n<p>Whenever you&#8217;re building an application that incorporates asynchronous serial communication, it&#8217;s best to get the communication working correctly before you build the animation or other parts of the interaction. Once the communication protocol is known, you can even divide the work, with one team developing the hardware and another developing the media programming.<\/p>\n\n\n\n<p>This exercise shows the value of using handshaking (aka call-and-response) in serial communication. Because the drawing of the microcontroller takes time, the p5.js sketch reads data less frequently than the microcontroller can send it. If you simply allow the microcontroller to send data continuously, the serial buffer on the p5.js side will fill up, and the movement of the virtual Arduino will become sluggish. This is why you only send back to the microcontroller when you know you have a set of valid data, in the <code>serialEvent()<\/code> function.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Test_the_Incoming_Data\"><\/span>Test the Incoming Data<span class=\"ez-toc-section-end\"><\/span><\/h3>\n\n\n\n<p>You don&#8217;t need to do anything with your incoming serial data to know it&#8217;s valid, if you&#8217;ve thought through the protocol well. In this case, if you see you&#8217;re getting three separate values and they&#8217;re all in a range of 0 to 360 (indicating degrees of the heading, pitch, and roll angles), you know it&#8217;ll work.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Program_the_Interface_Animation_etc\"><\/span>Program the Interface, Animation, etc.<span class=\"ez-toc-section-end\"><\/span><\/h3>\n\n\n\n<p>Once you know the communication is good, and you&#8217;re getting accurate values, you can program&nbsp; the parts of your final application that use that data. In this case, you didn&#8217;t even start on the movement of the virtual Arduino until you knew you had communication working. Drawing of the virtual model was separated from moving it, using the <code>push()<\/code>, <code>pop()<\/code>, and translation functions, like <code>applyMatrix()<\/code>, in p5.js. That separation makes the programming easier to do, and easier to debug.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>In this exercise you&#8217;ll read the built-in Inertial Motion Unit on the Arduino Nano 33 IoT, then feed its output into a Madgwick filter to determine heading, pitch, and roll of the board. Then you&#8217;ll send the output of that serially to p5.js and use it to move a virtual version of the Nano onscreen.<\/p>\n","protected":false},"author":2,"featured_media":0,"parent":11,"menu_order":801,"comment_status":"closed","ping_status":"closed","template":"","meta":{"footnotes":""},"categories":[42,11,15,31,78,7,9],"tags":[],"class_list":["post-6306","page","type-page","status-publish","hentry","category-asynchronous-serial","category-code","category-lab","category-p5-js","category-p5-webserial","category-sensors","category-serial-communication"],"_links":{"self":[{"href":"https:\/\/itp.nyu.edu\/physcomp\/wp-json\/wp\/v2\/pages\/6306"}],"collection":[{"href":"https:\/\/itp.nyu.edu\/physcomp\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/itp.nyu.edu\/physcomp\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/itp.nyu.edu\/physcomp\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/itp.nyu.edu\/physcomp\/wp-json\/wp\/v2\/comments?post=6306"}],"version-history":[{"count":25,"href":"https:\/\/itp.nyu.edu\/physcomp\/wp-json\/wp\/v2\/pages\/6306\/revisions"}],"predecessor-version":[{"id":10741,"href":"https:\/\/itp.nyu.edu\/physcomp\/wp-json\/wp\/v2\/pages\/6306\/revisions\/10741"}],"up":[{"embeddable":true,"href":"https:\/\/itp.nyu.edu\/physcomp\/wp-json\/wp\/v2\/pages\/11"}],"wp:attachment":[{"href":"https:\/\/itp.nyu.edu\/physcomp\/wp-json\/wp\/v2\/media?parent=6306"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/itp.nyu.edu\/physcomp\/wp-json\/wp\/v2\/categories?post=6306"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/itp.nyu.edu\/physcomp\/wp-json\/wp\/v2\/tags?post=6306"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}