Attempts, successes & setbacks.
The initial idea for this project was to create a tangible clay interface for sculpting digital clay or the features of a digital image.
The seam carving algorithm could be really amazing, if applied to specific areas of an image.
I started by looking at how to expand images. I expected to be able to add seams back in with just a bit of a tweak from removing seams. My approach was to go into the Dynamic Programming library and add a function to the seam carving library that would add seams rather than remove them.
The first attempt was successful in adding seams, but the algorithm looks for the path of least consequence, or the one that costs the least in terms of crossing highly varied pixel neighbors.
This is great for deleting seams. Once the seam is removed, the algorithm is forced to find a new seam to remove. For adding seams, this approach wouldn’t work.
Here’s the original image I started practicing with.
Here’s the image resized by removing seams using the seamCarving library.
To add seams, there was not a function ready to use. I brought in the java files to the sketch and dug into the seamCarving file.
Here’s the first attempt at writing a function into the seamCarving library based on the removeColumn function:
//how can I put each used seam into an array so that they will no longer be chosen?
//approach: when a seam is chosen as the seam to add, it is simultaneously deleted from the score image, then the new seam is chosen from the score image as if it were shrinking the image rather than expanding the image.
By adding one pixel to the width of the image, then copying the seam of least consequence, the image grows by one pixel. Of course, now the seam that costs the least is still in the image- twice! so the seamCarver has to return to the same seam over and over again stretching the image at the exact same seam.
To solve this problem – I tried to simultaneously delete the seam from the energy image, while adding it to the main image. That way, as the new seam of least cost was located, it would look for a new seam. This also proved to be messy and ineffective.
With guidance, the approach that I ended up with was to export the seam data to a csv file from deleting seams. The data set could then be loaded back into the sketch and each seam could be redrawn.
Here’s that approach with a test csv file from another image.
seamCarving expansion test 1 from Claire Mitchell on Vimeo.
Not exactly what I was hoping for. So now using the appropriate image to create a csv file -.
allowFullScreen>
seamCarving expansion test 2 from Claire Mitchell on Vimeo.
It kind of works! But it wasn’t that amazing for the clay interface that I imagined. I was interested in adjusting only facial features, but the seamCarver looks for the path of least cost which would avoid any drastic changes between pixel neighbors. Could this be reversed?
I decided to try using the full image as a background image, and the features separated in a png file.
The idea was to erase the features from the original image, so that those areas would be picked up by the seam carver more often, then to use that image to create the energyImage. The features would be used to perform the actual seam removal and addition with the original as the background.
This wasn’t giving me the exact effect that I was hoping for, but I do want to try to go further with it. It’s also kind of creepy – so…I decided to forget about trying to use a face.
And so – I decided to use a simple pig instead.
There is a pressure sensor on the platform under the clay. As force is applied to the clay, the sensor gets the readings into the processing sketch through an arduino shrinking the image.
Calibrating the sensor so that the sensitivity worked with the pressure applied to the clay took a lot of playing with.
There is also a stretch sensor used to stretch the image.
Both sensors are read simultaneously. The shrinking is mapped to a much higher value than the stretching because the pressure sensor is able to go higher. In future iterations, I would avoid this short cut and separate the serial values to isolate the data coming in from each sensor. Right now, it makes the program run more slowly, and is not as efficient as it could be in controlling the the sketch.
Here are some tests.
Pig Clay Test from Claire Mitchell on Vimeo.
Clay Interface for Molding Imags from Claire Mitchell on Vimeo.
These tests were great for figuring out how to move forward if I want to continue trying to build this kind of interface. I would use other sensors potentially- or embed the pressure sensor in a way that the applied force could be more predictable.
The stretch sensor could be attached to a smoothing tool, or other accessory that would work well for the medium. Right now it’s a bit out of place and doesn’t feel like a part of the piece.
So future development:
• Play with other sensors. Conductive/Resistive clay?
• Resize images vertically & horizontally simultaneously
• Isolate parts of the image to be resized