ASL Learning Game (attempt)

So I decided to build upon my ASL project that I did earlier this semester. Originally I was going to do a search engine using the ASL alphabet. But during the user testing session Anna had an idea to make it into an ASL learning game. What would happen is at the top of the screen images of the handshapes of letters would appear. The object is that you would copy the handshapes which would input a query (which you wouldn’t see) and then it would pull up the first google image of what you signed. Then you would type into an answer box what you think the word you signed meant. If you got it correct you’d move onto the next level, if not you’d have to guess again. When I started putting all of these ideas together I ran into many road blocks though. When I first started I was trying to use a LeapMotion but it wasn’t working with Processing at all. Then when I started with the flex sensors I couldn’t get the Arduino to work (turns out I had the wrong numbers put into the program, but whatever). Then I got the Arduino to work but I had to find all the various numbers for the letters again, and it wasn’t working as nicely as the last time I ran the ASL program. Next I just tried to get Processing to print the letter on the screen, but I couldn’t figure out how to get readings from the Arduino to be slower and I couldn’t figure out how to make the letters into one string. Because I couldn’t figure out this I wasn’t able to think of a way to get the letters into a query. So I don’t have the entire project all together… But I do have all the pieces of the code (except for the ASL Handshape pictures).

Here is the Processing code for getting the letters to show up on the screen.

And here is the query code:


Leave a Reply