After months of procrastination this site has now officially been moved to a new location – julioterra.com/journal.
Journal Moving to JULIOTERRA.COM/JOURNAL
August 23rd, 2010Seesaws and Puredata
July 21st, 2010During the past several weeks I’ve been working with a New York-based interactive media artist named GH Hovagyam. I’ve been helping out GH with the programming for his latest installation, which he has been building at the Lower Manhattan Community Council artist studio in Governor’s Island.
The piece is titled Seesaw. It is an interactive installation that features a seesaw in a room next to two video projections that play clips from the movie “Two for the Seesaw”. People’s interactions with the seesaw control the video projections.
There will be an open studio weekend from this Friday through Sunday. If you are in New York stop by to check it out (I will probably be there on Friday only). I will make sure to capture some video to share here – it will actually be my first time playing on this seesaw.
Pure Data to the Rescue
I got involved in this project through Hans-Christoph Steiner, the instructor from my Introduction to Dataflow Audio Programming class at ITP. Hans referred GH to me, after I told him that I wanted to continue building my Puredata skills.
When I got started, GH had already build an initial prototype that worked for about 30 minutes at a time. My task was to create another version of this patch that would work without crashing for extended periods of time, and that was more streamlined. This task was easier said then done – especially considering that I had no previous experience using GEM (PD’s graphics engine).
GH chose to use Puredata because it is a powerful open source dataflow programming language. Though PD’s video engine is a bit less stable than Max/Jitter, it can be used without any concerns regarding licensing.
The Patch Architecture
I divided the application into a few core components. Some of these components were created as sub-patches, others were created as abstractions:
- Application controller (main patch): Launches the application window and starts the video. It also houses all of the abstractions and sub-patches listed below.
- Midi reader (sub-patch): Reads incoming MIDI messages and processes these messages to determine whether to turn or off video projection one and two.
- Clip selector (abstraction): Selects a new clip and outputs (using the send object) the path and file names for the new video and audio clips.
- Clip controller (abstraction): Loads and plays new clips based on messages from the clip selector and turns on and off video and based on messages from Midi reader.
Running the Patch
In order to get this patch running on your computer there are a few settings that you need customize first. Notice: If you don’t customize these settings the patch may crash your computer.
First, open the pd window_settings subpatch that is located in the Create & Destroy Video section of the main patch. In this patch, update the window dimension message that is highlighted in red.
Next, you need to update the path of the video and audio files and the file name database in the clip-selector patch. I recommend that you open this patch directly to make these changes. First update the path for video and audio clips. Make sure to append “%s.mov” to the end of the movie file path, and “%s.aif” to the end of the audio file path (as this is what makes sure the file names are inserted appropriately).
Last, update the database that is stored in the “coll” object. For the patch to work properly the video and audio clips need to share the same file names. Update the data in this file but make sure to keep to the proper format (“index number, path;”). To finish up, update the random object to make sure it is set to the number of items that exist in the database.
The Activities and Thread Classes – Learning Java for Android
July 12th, 2010It has been a long time since I last posted to my journal. I’ve had a few hectic weeks as I’ve been working to finish the Android games course, I’ve started working with GH Hovagymian on his seesaw project, and had to prepare to share my second semester project from ITP at the Sony Wonderlab.
Today I am going to provide additional information regarding the structure of Android applications. A few weeks ago I wrote a post about the Java View class, which provides the basic building blocks for creating user interfaces. Today I will focus on providing an overview of the Java Activity and Thread classes.
Activities
An activity is a focused thing that an application can do. Almost all activities enable some type of interaction with the user. It is important to note that an activity is not an application. Applications are referred to a as a task in Android. Tasks usually contains multiple activities and often other components such as views, services, etc.
All activities live inside a window on the screen. Activities can be presented to the user as full-screen windows, as floating windows or embedded inside of another activity. There are two main methods that almost all Activity subclasses will implement. These methods play crucial roles in the processes of setting up and leaving activities:
- onCreate(Bundle) is where you initialize your activity. Most importantly, here you will usually call setContentView(int) with a layout resource defining your UI, and using findViewById(int) to retrieve the widgets in that UI that you need to interact with programmatically.
- onPause() is where you deal with the user leaving your activity. Most importantly, any changes made by the user should at this point be committed (usually to the ContentProvider holding the data).
Activities are managed via an activity stack. The most recent activity is always added to the top of the stack. The previous activities remain on the stack and can be reactivated simply by removing the new activity from the stack.
The startActivity(Intent) method starts new activities by placing them at the top of the activity stack. It takes a single argument, an Intent, which describes the activity to be executed.
Activities ultimately have three main different states: foreground activity, visible activity, background activity. Foreground activities live on top of the activity stack. Users are only able to interact directly with activities that are in this state.
Visible activities can be viewed by the user but they are not at the top of the stack. The user usually cannot interact directly with these activities until those that are located higher in the stack are removed from the stack.
Background activities are lower down in the stack and are not visible to the user. These are the first types of activities to be killed when an Android device needs additional space.
To switch between these various states seven activity methods can be implemented in an Activity subclass (two of which I already discussed above). Here is the full list:
- protected void onCreate(Bundle savedInstanceState);
- protected void onStart();
- protected void onRestart();
- protected void onResume();
- protected void onPause();
- protected void onStop();
- protected void onDestroy();
Here is a graphical representation of the Android Activity lifecyle, from the Android Developer website.

Activities can be killed by the OS at any time if they are not in the foreground state. The following methods can be called when an activities location in the stack is changed: onPause(), onStop(), and onDestroy() – however, only the onPause() is guaranteed to be called before a process is killed. Therefore, it is recommended that you should use onPause() to store any data when you are removing an activity from the foreground.
Threads
Threads enable android applications to run more than one activity at a time. Threads can be created in two ways: by extending the thread class and overriding its run() method; or creating a class the implements the runnable interface. In both cases, the start() method is called within an activity to actually execute the new Thread.
For my game I leveraged the code example provided by Matt. He created an extension of the Thread class called GameThread that includes the required methods (run, start, and stop). It also features a few functions that are meant to be overwritten by the main activity when it creates the thread during runtime. I will post the current code and screenshots of my game by early next week.
Battle of Brooklyn at Come Out and Play Festival
June 22nd, 2010On a sunny and hot Sunday earlier this month, June 6th to be exact, we played version 1.0 of our game, Battle of Brooklyn, at the Come Out & Play festival in New York. I’m happy to report that we had a successful play session. All players (I mean soldiers) seemed to enjoy themselves as they ran around the Boathouse battling each other for control over bridges, tunnels and other strategic locations. We also captured some really good feedback that will serve as inspiration for future improvements to the game.
Here are some pictures from the play session.
That is not to say that we did not have any problems. Here is a quick overview of our challenges: first, the day began with me running to Kmart and then Target to procure some bandanas. Next, when we arrived at the game location we were notified that 45 people had signed up to play our game even though we only had props for 36 players. After finding a solution to accommodate more players only 12 people actually showed up to play the game (this ended up being good news because I got to jump into the game as well).
Feedback and Opportunities
After the game we got a chance to capture feedback from the players from both teams – we were lucky because Eric Zimmerman (the guy who literally wrote the book about game design) was one of the judges assigned to play our game. Here is an overview of the most crucial or promising ideas we captured:
Team selection and balance: create a system that helps ensure teams are evenly balanced when selected. In this round of play we ended up using an ad hoc system that allowed friends to opt to join the same team. Unfortunately, this led to uneven distribution of skills between the two teams. Need to find a better way for doing this in the future.
Kill flag location: explore new locations to secure the kill flag from each soldier. One promising suggestion was to attach the kill flags to a wrist band so that whenever someone attacked the soldier from the other team they were also putting their own lives at risk. This would likely help equalize the balance of the game as well.
Territory capture using kill flags: explore different mechanisms to capture territory. One promising suggestion was to make it so that armies need to spend kill flags in order to capture a territory. This would solve the current issue with the game, where the army that has captured the most kill flags can always capture all territories since they are able to use the same kill flags to capture multiple locations.
Defending captured territories: we have been wanting to find a solution for running this game without the need for volunteers to work at each battlefield. One suggestion that came up was to change the rules so that an army needs to always leave at least one soldier to defend a conquered territory. Territory battles could be decided based on the number of kill flags + live soldiers – for example, if a soldier from the blue army is defending a territory with 2 kill flags, then to conquer the territory the other team must arrive with 3 kill flags and 1 soldier, 2 soldiers and 2 kills flags, or 1 kill flag and 3 soldiers.
Game Rules and Props
Now finally here is a detailed outline of the final game as it currently stands. The full game rules are posted as an image below next to the complete list of required props, and guidance regarding locations.
To play this game you will need the following props: each team will need two to six “nation” flags and a few maps that feature strategic battlefields. Individual players need bandanas and a flag football-style belt and flag system in their army’s color (extra flags will be needed).
Over the past few months we have tested this game in several locations. From this fieldwork we gained two important insights: First, the playing field needs to be right sized to the number of players. Since there is a lot of running in this game the field cannot be too large otherwise players get exhausted early. Secondly, it is important to find a place that has easily identifiable structures that can be used as readily-identifiable battlefields.
Thanks for the Support
Thanks to everyone who came out to play the game during the past several months. We truly appreciate your support, without it this game would not exist. A special thanks to Morgen, my partner in crime on this game.
The View Class – Learning Java for Android
June 8th, 2010The view class represents the basic building block for user interface components. A View occupies a rectangular area on the screen and is responsible for drawing and event handling. Subclasses of the view class also offer fully implemented UI objects like text fields and buttons; these are often called widgets.
A View object handles its own measurement, layout, drawing, focus change, scrolling, and key/gesture interactions for the rectangular area of the screen in which it resides. As an object in the user interface, a View is also a point of interaction between the user and the activity(s) that is reacting to the interaction events.
Multiple views can be grouped together in view groups. These view groups can also contain other view groups. This arrangement can be visualized as a hierarchical tree-structure where the group view nodes are branches, and the view nodes are leaves (refer to image to right, taken from android developer guide).
One important thing to remember is that a view can refer to the canvas that contains all of the graphics on a screen, or to a single button within a menu. This structure is enables parent views to influence child views. This is important considering that the android OS runs on so many devices with such divergent specs.
Standard Views
Android applications can be built using standard views or custom views, or they can contain both. Standard views are composed of buttons and other standard elements. These types of views are easy to create but provide limited opportunities for customization. Therefore, their use is limited in the realm of mobile games.
Standard views are created using XML layout templates that are saved in the layout folder in the package resources. These standard view definitions are usually simple documents that include only a short list of attributes such as screen orientation, width and height, and text. When an activity is linked to a standard view, the attributes in the XML document determine how objects (buttons, text, etc) from that activity appear on the screen.
Sometimes classes are created to extend the functionality of a standard view. However, I will not explore any examples of this practice here.
Custom Views
Custom views can be created in many different ways. Unlike standard views, they are always defined using both XML code and class extensions. For my application I created a custom view by extending the SurfaceView class, and defining a simple view layout using XML. I initially selected this approach because I was using Matt’s sample application as my guide. Upon further investigation I’ve confirmed that this is the best approach.
Here is an overview of how this approach works. It is pretty complex, so here is a link to an official description from the android site.
SurfaceView is a special subclass of View that offers a dedicated drawing surface to an application’s secondary thread, so that the secondary thread can draw to its own Canvas at its own pace (without needing to wait for the main thread).
Here is how you set-up an application to use this class. First, create a class that extends SurfaceView and implements the SurfaceHolder.Callback interface (named GameView in my app). This class will serve as a link between the thread class that will perform all the drawing procedures and the canvas itself.
This new view class needs to include an instance of the secondary thread (named GameThread in my app), which contains all drawing procedures. Then, to enable the secondary thread to access the drawing surface, create an instance of the SurfaceHolder object. This object functions as the connector between the view and thread classes.
Next initialize the SurfaceHolder variable so that the new view class receives callbacks related to surface activity. Lastly, we need to override all the callback methods from the SurfaceHolder.Callback interface. I know this is a lot of information to keep straight, and we’re not done yet.
Now that you’ve set-up the new view class, here is what you need to do in the thread so that it can draw to the canvas/surface. First, we need to pass the SurfaceHandler object from the view class to the thread class. Then, each time we want to draw to the canvas we need to retrieve the canvas using the lockCanvas() method from the SurfaceHandler object. Then the thread can draw to this canvas.
Once done drawing, we need to pass the updated canvas object back to the SurfaceHolder object using the callunlockCanvasAndPost() method. The Surface will now draw the Canvas.
To clear the previous state of the Canvas you need to fill it with a color using drawColo() or set a background image with drawBitmap(). Otherwise, you will see traces of the drawings that were previously performed.
The Maze on Android
June 7th, 2010Here is an update regarding my android, currently titled The Maze. Last week I spent a lot of time porting my code from processing to java for android. I’m happy to report that I was able to make it work, for the most part at least (I still need to fix my buggy collision tracking).
To get my application to work I repurposed a lot of code from Matt’s android application example that he shared in class [add link here]. For the most part I had to copy the structure of Matt’s application. I leveraged his GameView, GameThread, GameElement, and Coods classes.
I was easily able to port over my data model classes (Shapes, Rectangles, Circles, and Maze) and activity class (Game). The updates included switching out calls to any Processing specific classes and methods, such as PVector and all draw methods, and changing the main application to be an extension of activity rather than the pApplet class.
I also used Matt’s code as the basis of my investigation into how android applications are structured. I was specifically interested in understanding the following concepts: activities, views, threads, handlers, listeners, and interfaces. I’ll start today with a brief overview regarding some of these concepts.
Over the next week I will write a series of posts that cover all of these topics. I will use the Android Developers Guide as my main source of information.
Tasks, Activities, Threads & Views
What users experience as an application android calls a task. Tasks include various components. Simple Android applications (or tasks) are usually composed of Views and Activities. There are other Android application components such as services, broadcast receivers, and content providers play important roles in more complex apps.
Android devices can run multiple tasks at once. Some application components are designed to run in the foreground, such as activities, while others run in the background, services, broadcast listeners and content providers.
Views define the visual interface of Android applications, while the activities usually define the processes that govern how foreground applications run, under the hood. An application can feature multiple different views, and many different activities. Within an application multiple activities can run simultaneously in separate processes called threads.
The Maze Game
May 28th, 2010Our first major assignment for the Games with Android class was to create a simple game in Processing using only simple shapes such as circles and rectangles. Since Android applications are built using Java, we were tasked with creating our games in Eclipse. Here is a brief overview of the initial concept I selected, followed by a description of how I have structured my solution (along with the initial prototype), and the problems and improvements that I have planned for the next iteration.
The Maze Game
After considering racing games, obstacle games, and space invader-type games I choose to create a maze-based adventure. My initial idea, which has been implemented in a rough a prototype, is described below.
The player is trapped in a maze and must navigate his way to safety. He/she is represented by a circle that is always located in the middle of the screen. The movement of the player through the maze is governed by a set of limited dynamics.
Once the player starts moving he/she will move continuously. The direction of movement can only be changed by bumping the game piece into a column or wall. When the game piece hits a rectangular object it reverses direction. When the game piece hits a circular object it changes direction by 90-degrees.
The player can steer his game piece in a limited manner. The steering enables the game piece to stray slightly from its current trajectory slightly. It is controlled by using the arrow keys or mouse.
As you will notice if you attempt to play the game, it still needs a lot of work. The collision tracking is not working well, and the game mechanics also need to be worked on. More on the planned improvements below.
Developing the Game
I spent a good number of hours working on this project last week. I wanted to lay a solid foundation so that it would make it easier to port the game to Android, which is the next step in this assignment. With this in mind, I decided to design a basic class structure before beginning to write code.
The main consideration guiding my design was a desire to write object-oriented code that would facilitate debugging, porting, and future improvements to the game mechanics. The initial architecture evolved during the implementation process into the structure depicted in the diagram below.
In order to support multiple levels, I have created a function in the maze class that can read a text file made up of 0s, 1s, 2s, and 3s. It converts these text files into a maze made up of space, walls, round columns, and exits.
Planned Improvements
After playing the initial prototype I realized that I still have to do a lot of work. I also became even more excited about the possibilities of creating a good maze game. Here is a list of the planned improvements:
- Change controls: simplify the game controls and improve the visual feedback. I plan to change the game controls in one of the following ways: make it Pac Man-like, where the controls change the direction in which the game piece is moving; or keep it similar to the current design but make the game piece move up or down one full row when the controls are pressed.
- Change objective: rather than use small mazes with clear end points and time limits I am considering more open-ended possibilities. One idea that is loosely inspired by the 1980’s movie Tron, is to create a never ending maze that contains various different objects. When the player collides with one of these objects they would either gain benefits such as bonus points, special powers, or suffer punishment, such as death.
- Design updates: Overall I need to think about how to make this game more compelling form a visual perspective. For now I plan to stay within the bounds of the assignment (limiting my shapes to rectangles and circles). That said, I want to explore how I can use color palettes and other design attributes to make the game look more appealing.
Casual Games for Android
May 21st, 2010Earlier this week summer school started. During the first summer session I will be taking a course entitled “Introduction to Mobile Games for Android“. This is my first ever foray into the world of programming mobile applications.
I had originally planned to take a course that focused on development for the iPhone and iPad. Unfortunately, that course was cancelled so I decided to focus my studies on Android (for iPad and iPhone we have created a summer study group that will meet up weekly to learn the basics of Cocoa and Objective C).
The “Games for Android” course is being taught by Matt Parker. Matt is a resident at ITP who has done some pretty cool work over the past several years. Here is a short list of my favorite of his projects:
- Recurse is an awesome game where you have to twist and move your body in order to score points. It provides a simple but addictive experience that has caused me to waste many hours on the ITP floor.
- Lumarca is a volumetric display that Matt created for his final project at ITP. This interesting prototype is built using strings arranged in a three dimensional array, you have see it to understand what the hell I mean.
- Lastly, Proclipsing is an extension that Matt wrote for the Eclipse IDE that enables Processing users to easily set-up their sketches in Eclipse (Thank You Matt!)
During our first week of classes our focus has been on getting up and running with Eclipse and agreeing on a practical definition for casual gaming. For the definition we identified the key areas where casual games differ from more immersive game experience as presented in Jesper Juul’s recent book A Casual Revolution. I am extremely excited about finally being able to understand the basics of how Eclipse works. It would have saved me many a headache to be able to use this IDE for my latest project, Finger Paint Mandala. Here is a link to the full syllabus for the course.
From a technical perspective, we are going to focus our coding efforts on Android version 1.6 (the latest version is 2.1). I was surprised to find out that the Android operating system for a given phone is defined by the carrier rather than the user. This is different than the iPhone, where the user can update his/her phone’s OS independently of the service provider.
Finger Paint Mandala: Prototype V1.1 Complete
May 20th, 2010Last week, I showed this project at the ITP show. Overall I got a very positive response, especially from kids and people who work with kids and education. Unfortunately, I was not able to get any good footage from the show itself as I was stuck behind the display for the entire time. I know that some other people captured some good footage and I will post links to it here once I find out where they are available.
After the show I decided to re-write the Processing sketch using a different TUIO library – I switched from the TuioZones library to the Tuio library (from the reacTVision team). I decided to make this leap because I was unhappy with the performance of the first draft of my sketch. The main issues that I was encountering were that the sketch seemed to run rather slow and the color selection was not working well (users were not able to select multiple colors at once). I’m happy to report that the new version of the sketch works better on both of these accounts.
Unfortunately, I still have some work to do to improve the performance of the multitouch display itself. First and foremost, I need to take apart the display to improve the finger tracking resolution. I suspect that I forgot to remove one or two of the filter layers from the original LCD (I hope this is the case as this would be relatively easy to solve). The other issue that I need to address is to procure a new glass mirror that will not bend out of shape, like my current plexi mirror.
FarmBridge: 2010 Design Expo Winner
May 19th, 2010I am happy to announce that our team and our project, FarmBridge, has been selected to represent ITP at the Microsoft Design Expo conference this summer. This means that Cindy, Noah, Tianwei, and I are going to spend several days with Nancy Hechinger in Seattle this July.

Here is a long overdue update regarding our project and our final presentation. Over the past several weeks we worked hard to pull together initial design mock-ups and get our presentation ready for prime time. Here is a brief overview of the flow from our final presentation. Unfortunately, we are not able to post the entire presentation because it contains some content that we would rather not share online yet. That said, we have included a brief video that walks through some key features of the design.
Our Inspiration: We all live in New York and are passionate about food. But even in this city of extreme wealth, entire communities do not have access to the kind of fresh, healthy that we care so much about. The question that motivates our project is how can we use technology to get locally farmed produce to more of the city’s residents.
Our Product: FarmBridge is an online platform that makes it easier for neighbors to form groups and gain access to locally farmed food. It provides two main sets of tools: 1) Management tools for community leaders 2) Social software for community as a whole.
Why CSAs: CSAs are grassroots community that are run by local communities that are dedicated to making healthy foods available in their neighborhoods. We decided to work with CSAs for three reasons:
- It is the only truly grassroots approach to the problems that the local food movement aims to address
- It is the model most able to get healthy food to neighborhoods in the city that need it
- Our research points to a tremendous demand from for foods from these communities
In the past 1-2 years there has been an explosion of interest in CSAs. The volunteers that run these groups are already responsible for organizing larger communities. They are also trying to bring more food offerings from more local producers, and add more payment options and subscription packages to allow a more diverse group of city residents to join their communities.
User Research: To inspire and guide our design we needed to gain a deep understanding of how these communities work. So we attended the annual CSA conference in New York, and held one-on-one conversations with more than 20 organizers and members from these communities. There are two main take aways from these conversations:
- First, local community organizers need tools that help them manage memberships, subscriptions, volunteers, communications and transitions.
- Second, we realized that running a successful community is too much work for just one person. These communities need social tools that enable organizers to actively engage members and farmers in the day-to-day activities of the community.
Initial Design & Experience: I’ve put together a brief video that walks through the design and experience slides from our presentation.
Here is a link to a post from Cindy regarding farmbridge.


















