Thought Patterns
brainwaves and the environment
heather dewey hagborg, zach layton, leif krinkle, toshi ozawa
PROJECT: build a homemade EEG machine to interface with a bluetooth enabled phone, as a means of producing audio signals and as a way to trigger and manipulate video and static images using brainwave data.
see: http://openeeg.sourceforge.net/doc/
journal:
12/7 10:35pm...some photos of the progress thus far...
the eeg boards in their new box
bluetooth circuit using bluesmirf model
active electrode
todd holubeck meditating wearing headphones producing binaural beats
heather monitors the data with eyes open
heather and zach monitor data together
screenshot of data visualization built in max. Raw data on left, smoothed data on right, FFT display on right.
12/2/11:01 pm
toshi - soldered the pin connectors and tried to get a better connection for the serial and active electrodes...next steps: get plugs instead (1/4"?). Then hooked up electrodes into zterm and saw that packets were in fact being sent.
heather got the bluetooth to work, the baud rate should be 57600...the data had to be inverted so she built a hex inversion circuit.
then we met with tom and showed him the setup. Seemed to be working pretty well, he encouraged Toshi to blink and get into varying states of mind and the electric guru software was responding. We did this using a serial cable...the bluetooth was not functioning the way way we were hoping. We determined with Tom that what we're getting in electric guru is a combined wave which is representing all the varying wavelengths that are emitted from the nervous system. Tom suggested that we'll need to run this information through an FFT so we can isolate the individual waves.
Zach began to develop the max patch further. The atmel chip is sending out a 17 byte packet:
5a, a5, 2, counter and then 6 low and high bytes in big endian (and a stop byte).
After talking with Luke Dubois we were able to concatenate the bytes in max so we can now represent them as a 16 bit number. The data looked acceptable, but need to be smoothed. Zach extended the patch a bit more, smoothing the data and it now looks very much like an EEG.
next steps:
heather is working on the bluetooth slave/master issue so that the sparkfun bluetooth device can be set to discoverable mode. Toshi is going to do some research on the mac bluetooth issue and we'll look into purchasing some power and mini plugs for the cables. Also, we'll need to get some more flexible shielded cable. Also getting the components into a project box (specifically a metal box for the analog board to reduce interference).
Toshi will experiment with refining the electrodes to cut down on impedance as much as possible. Zach will investigate the FFT issue and try to build that into the max patch so we can see the component waves individually. Once this is accomplished we can extend the patch into jitter using openGL.
12/1/05 10:31 pm
Today we made a great deal of progress. Heather and Zach sat with Dano to discuss the bluetooth issue and the problems we had been having with serial. Dano recommended that we set up a master/slave command setup with the bluetooth device using AT commands. Blue Radios recommends the Dial as Slave command:
EXAMPLE SLAVE:
TYPE: ATDS<cr>
REPLY: <cr_lf>OK<cr_lf> or <CONNECT, 00A0961F008F<cr_lf>
We'll see if this helps us recieve.
On a more exciting note, we were able to finally get stable readings from the 17 byte data sent from the AVR chip in our Modular EEG board. The data at first seemed random and unstable, but now makes much more sense. This is how it is looking right now:
byte 0 = A5 (header byte)
byte 1 = 5A
byte 2 = version 2 (2)
byte 3 = counter
4 = low byte
5 = high byte
6 = low
7 = high
8 ...
..17 = stop
So, at this time what we need to do with this data is to represent this data in binary...then convert it into a text string...then run an operation to reverse that string (assuming this big endian data means it needs to be reversed)...then concatenate the high and low bytes with high in the first positon and then convert that binary data back into decimal.
Right now Toshi is working on developing the active electrodes, so hopefully very soon we'll be seeing some actual biophysiological data.
11:22pm
we're discovering that the active electrodes are not getting enough power from the analog board (which we're powering with 9V), meaning we're going to have to power the electrodes separately. We'll have to set up a breadboard, for now, to send 5 volts to the electrodes. Easy enough.
11/29/05 10:19 pm
Tonight we tried to tackle a coulple of different tasks at once. Heather was working on the Bluetooth chip and we were able to get it to send serial data into her Electric Guru software (PC only). The macs are able to recognize the bluetooth device but we're not able to get the serial data into bluetooth or max. Zach was working with a max patch that takes incoming serial data and was parsing it into groups of 17. The data seemed somewhat erratic. He noticed that there seemed to be patterns of 3 occuring, so he changed the grouping to reflect this and got a much more stable data parsing it in 3s. Toshi worked on developing the active electrodes which give a much larger voltage output. Leif has been experimenting with auralization of galvanic skin response with the relaxomat.
hacking the relaxomat
experimenting with galvanic skin response
todd digging the relaxomat box
wiring passive electrodes to shielded cable
hooking up the modular EEG...pictured here sending serial data, the clips on the ATMEL chip running into a hex inverter circuit
EEG into serial port
data displayed in RealTerm. Looks way too random. Mostly noise?
data displayed in electric guru software, compatible with modular eeg
zach and toshi experimenting with electrodes
heather
11/11/05
Progress:
Got bluetooth working on the phone today. I can now send data out from my PC, caprture it on the phone and save it to a file on the memory card. Combined with previous work on controlling the camera I think I am about 50% done with the cellphone side of this app.
Next steps:
-Implement threads on the phone? Or figure out a workaround so bluetooth capture and picture capture can happen simultaneously.
-Figure out how high resolution I can have for the pictures and what memory card I should get to store all the information.
-Wrap everything in a simple GUI
-Start on the PC end of the program
11/7/05
Description/Scenario:
part 1 - absorption:
I am the user. I wear a portable device which logs my brain activity (via EEG) and records the world around me for a 24 hour period. Everything I do during this period is monitored and the associations between my brain waves and my environment are carefully indexed.
part 2 - performance:
I am the performer. Seated on the side of a stage, strapped back in to the EEG my brainwaves are monitored again. It is a collaborative performance with Zach Layton performing sound with Max MSP. Connected to the indexed record of my previous day, as brain wave frequencies re-occur they trigger video projections of the images associated with those frequencies from the day before. The idea is to see if connections and correlations emerge, bringing the audience directly into my thought patterns.
Algorithm:
Zach is building the portable EEG device which will spit out a bitstream via bluetooth. I am programming a Nokia 6630 cell phone to log the EEG data, snap and timestamp pictures. All the information will be recorded onto a 512MB memory card. At the end of the day I will load the info from the card onto my laptop and a program will index and connect the EEG stream to the snapshots. This will probably be uploaded onto the web and transformed into a format Zach can use for the final Max/Jitter performance. The last programming step is creating the glue between the EEG device and either Max or Jitter for the live video association.
Resources:
Open EEG project on Sourceforge
Accomplished:
-J2ME and python running on cell phone
-automatic camera capture working
Next Steps:
-Get a larger memory card and test automatic picture capture for a trial period
-Get a bluetooth adapter for my PC and code cell-bluetooth communication module
-Evaluate using Java vs. Python as a cellphone development environment
-PC program for handling captured data
-Max MSP/ Jitter integration