I had an initial idea to create a musical instrument based around a heart rate sensor and a few other sensors and a Smile detection program in Processing to create layers of music as the user interacts. The concept behind the project was to create something from the transference of life. The shape of the object in mind was inspired by organic shapes of blown glass such as these from artist Gabriella Bisetto.
( http://www.flinders.edu.au/artmuseum/include/popup_view.cfm?photoid=379 )
I like the irregular shapes because they are very nature inspired and look fluid. Searching high and low, the closest materials I found that would fit my sensors and Arduino within my time limit were these half-spheres from Canal Plastics.
Here are the two sensors I decided to connect to the Arduino:
I packed up the Arduino, breadboard, along with the sensors and lots of wiring into another, smaller set of chrome looking half-spheres. I routed the wires for the heart rate sensor to the exterior of the orb, and used conductive copper tape to reinforce them. Soon enough, my project started to take on a life of its own.
Suddenly it looked to me more like a sci-fi orb than something out of nature; a bit more like one of these Tesla Orbs…
It reminded me of something Dr. Frankenstein would use to bring his monster to life. That then gave me the idea of doing just that. So, I designed a little robot and using Processing put him together…
Watch the video to see what happens when I transfer him my heartbeat…
It takes about 8 seconds for the heart rate sensor to initially read the pulse. Did you see and hear my heart rate increase when his eyes lit up?
On a side note, I spent a lot of time trying to find the right sound libraries for Processing. I started importing some edited sound files using the Minim library. I soon discovered that some features such as changing the file’s sample rate (to be able to vary speed in proportion to heart rate) is not possible with Minim. I then spent some time with the Sonia library, which lets to alter the sample rate. I calculated heart rate using millis within Processing and mapped it to a sound file’s speed, but I wasn’t too happy with the outcome because the change was either not too apparent, or not very appealing. So I eventually reverted back to my original Minim library.
I am currently adding the functionality in Processing to move his arms as you turn the ball from side to side (read using the accelerometer inside). I am also collecting more sounds to add layers of sounds and music to the experience. I would also like to use Smile Detection to be able to transfer the smile to him using your own smile (detected by the built in camera on the laptop).
Although my project did not end up what I initially concepted, I really learned a lot about serial communication between Arduino and Processing using several sensors, learned a great deal about the available sound libraries out there, experienced some possibilities and limitations of material fabrication, spent seemingly endless nights becoming more comfortable with programming, and most importantly had fun creating an interactive project I never imagined 3 months ago I could create by myself.
Source Code within Processing:
import processing.serial.*; // import the Processing serial library
Serial myPort; // The serial port
import ddf.minim.*;
// A Heartbeat object (that will trigger the sound)
Heartbeat drums;
Heartbeat music;
Heartbeat music2;
Heartbeat music3;
Heartbeat music4;
Heartbeat piano2;
Heartbeat piano3;
Heartbeat recorded;
Minim minim;
// import robot image files
PImage img;
PImage img_robot;
PImage img_heart;
PImage img_left_arm;
PImage img_right_arm;
PImage img_eyes;
PImage img_lips;
PImage img_bgleft;
PImage img_bgright;
int beatcounter = 0;
int heart_alpha = 0;
int eyes_alpha = 0;
int lips_alpha = 0;
float angle;
boolean sensorPressed = false;
boolean beatswitch = false;
boolean musicboolean = false;
boolean musicboolean2 = false;
boolean musicboolean3 = false;
void setup() {
// —– Arduino related code starts here ———
println(Serial.list()); // List all the available serial ports
String portName = Serial.list()[0];
myPort = new Serial(this, portName, 9600);
// —– Arduino related code ends here ———–
size(1000,750);
background(255);
img_robot = loadImage(“robot.gif”);
image(img_robot,330,10);
// start up Minim
minim = new Minim(this);
// Some sound files
drums = new Heartbeat(“beat2.mp3″);
music = new Heartbeat(“ding1.mp3″);
music2 = new Heartbeat(“ding2.mp3″);
music3 = new Heartbeat(“ding3.mp3″);
music4 = new Heartbeat(“ding4.mp3″);
piano2 = new Heartbeat(“piano2.mp3″);
piano3 = new Heartbeat(“piano3.mp3″);
recorded = new Heartbeat(“record.mp3″);
smooth();
// —– Arduino related code starts here ——–
// read bytes into a buffer until you get a linefeed (ASCII 10):
myPort.bufferUntil(‘\n’);
// —– Arduino related code ends here ———-
}
void draw() {
img_heart = loadImage(“heart.gif”);
img_left_arm = loadImage(“left_arm.gif”);
image(img_left_arm,312,270);
img_right_arm = loadImage(“right_arm.gif”);
image(img_right_arm,590,270);
img_eyes = loadImage(“eyes.gif”);
img_lips = loadImage(“lips.gif”);
img_bgleft = loadImage(“bg_left.jpg”);
img_bgright = loadImage(“bg_right.jpg”);
if (beatswitch == true)
{
beatcounter = beatcounter +1;
beatswitch = false;
}
if (musicboolean==true) {
piano2.ring();
musicboolean = false;
}
if (beatcounter >=15)
{
heart_alpha = heart_alpha +5;
tint(255, heart_alpha);
image(img_heart,450,270);
musicboolean = true;
}
if (beatcounter >=17)
{
musicboolean = false;
}
if (musicboolean2==true) {
piano3.ring();
musicboolean2 = false;
}
if (beatcounter >=25)
{
eyes_alpha = eyes_alpha +5;
tint(255, eyes_alpha);
image(img_eyes,417,22);
musicboolean = false;
musicboolean2 = true;
}
if (beatcounter >=33)
{
recorded.ring();
}
if (beatcounter >=35)
{
lips_alpha = lips_alpha +5;
tint(255, lips_alpha);
image(img_lips,437,118);
musicboolean2 = false;
musicboolean3 = true;
image(img_bgleft,0,0);
image(img_bgright,695,0);
}
if (beatcounter >=40)
{
//image(img_bgleft,0,0);
musicboolean3 = false;
}
}
// Close the sound files
public void stop() {
drums.close(); // The Heartbeat object must close its sound.
super.stop();
}
class Heartbeat {
// An AudioPlayer object is used to store the sound.
AudioSnippet dingdong;
// load music file into a new AudioPlayer
dingdong = minim.loadSnippet(filename);
}
void ring() {
if (!dingdong.isPlaying()) {
// rewind() ensures the sound starts from the beginning.
dingdong.rewind();
dingdong.play();
}
}
}
// —– Arduino related code starts here ———-
void serialEvent(Serial myPort) {
String myString = myPort.readStringUntil(‘\n’); // read the serial buffer
if (myString != null)
{
myString = trim(myString);
int sensors[] = int(split(myString, ‘,’)); // split the string at the commas and convert the sections into integers
for (int sensorNum = 0; sensorNum < sensors.length; sensorNum++) // print out the values you got {print(“Sensor ” + sensorNum + “: ” + sensors[sensorNum] + “\t”); } println(); if (sensors[1] >= 570) { // accelerometer x-axis
music4.ring();
}
else if (sensors[1] >= 550 && sensors[1] = 535 && sensors[1] = 520 && sensors[1] < 540) { // accelerometer x-axis music.ring(); //display_arms(); } if (sensors[2] >= 200) { // accelerometer y-axis
//music2.ring();
}
if (sensors[3] == 1) { // heart rate sensor digital input
drums.ring();
beatswitch = true;
}
}
else {
beatcounter =0;
}
}
// —– Arduino related code ends here ———-
Source Code within Arduino:
const int HRM = 2; // digital input
const int ledPin = 9; // LED output
const int ledPin2 = 10; // LED 2 output
const int ledPin3 = 11; // LED 2 output
int fade = 0;
int ledState = LOW;
long lastBeat; // a variable that gets assigned to the last time an event happened
int lastReading; // a variable that has only 2 states/tracks the diff states
void setup() {
// configure the serial connection:
Serial.begin(9600);
// configure the digital input:
pinMode(HRM, INPUT); // the heart rate module input to digital pin
pinMode(ledPin, OUTPUT);
pinMode(ledPin2, OUTPUT);
pinMode(ledPin3, OUTPUT);
}
void loop() {
// read the sensor: (force sensor)
int forcesensor = analogRead(A2); // in Processing, sensors[2]
// print the results:
Serial.print(forcesensor);
Serial.print(“,”);
if (forcesensor >= 100) {
digitalWrite(ledPin, HIGH);
digitalWrite(ledPin2, HIGH);
digitalWrite(ledPin3, LOW);
// read the sensor: (accelerometer x) —————-
int sensorValue = analogRead(A0);
// print the results:
Serial.print(sensorValue);
Serial.print(“,”);
// read the sensor: (accelerometer y) —————-
int sensorValue2 = analogRead(A1);
// print the results:
Serial.print(sensorValue2);
Serial.print(“,”);
// read the sensor: (heart rate sensor) ————–
int pulse = digitalRead(HRM);
Serial.println(pulse);
if (pulse == HIGH) {
digitalWrite(ledPin3, HIGH);
}
else {
digitalWrite(ledPin3, LOW);
}
}
else {
digitalWrite(ledPin, LOW);
digitalWrite(ledPin2, LOW);
digitalWrite(ledPin3, LOW);
}
}