Author Archive

Social Search

Monday, December 14th, 2009

The topic of my research relates to internet search and smart recommendation systems. The popularization of social media is changing traditional methods of discovering, consuming, producing and sharing content. Internet users contribute a staggering amount of information to a network that is becoming increasingly aware of our behaviors. For instance, Facebook knows “who you may know”; Amazon knows your buying habits and can recommend products; iTunes knows what we are listening to.

For my mid-term project, I focused primarily on how search engines work. I was able to successfully implement a robust search engine that utilized: index tables, text analysis and calculated weights to produce relevant ‘course-related’ search results. For my final project, I expanded my search engine to query instructor profiles using similar techniques. In addition, I built a search engine based on recommendations that capitalize on multiple social media platforms. In a sense, these platforms redefine the term “peer reviewed”, as their resources are often reviewed by community at large. Fortunately, many applications provide an API that enables developers to interact with their services. The search engine that I created can obtain data from: Delicious, Digg, YouTube, Amazon Books, Slideshare, Scribd, Twitter, Google Blogs, Google News, Wikipedia, Ted, Social Syllabus, FriendFeed, Google Scholar and Flickr. This iteration currently displays results in a ‘wall’ view. This provides instructors using Social Syllabus with a bird’s eye view of relevant materials to incorporate into their teaching curriculum.

The Illusion of Community

Monday, November 16th, 2009

“The Web is more a social creation than a technical one. I designed it for a social effect – to help people work together – and not as a technical toy.”  – Tim Berner’s Lee (1999)

Tim Berners-Lee, a graduate of Oxford University, wrote a proposal for the World Wide Web in 1989. He wrote the first web client and server in 1990, enabling a mass adoption of using hypertext to connect disparate documents to one another.

As the size of the network increased, the potential to use the internet as a medium for self expression, collaboration, research and networking encouraged businesses and educators to explore new modes of communication and personalization. The personal web started with simple web pages enabling anyone with a modem and a basic understanding of using HTML to broadcast their content to the world. But this was only the beginning.

In 1996, Larry Page and Sergey Brin began working on the Stanford Digital Library Project, whose mission was “to develop the enabling technologies for a single, integrated and universal digital library.” Page explored the notion of representing the link structure established on the World Wide Web as a large graph. He focused on measuring the importance of a web page, based on the number of other web pages that linked to it. This research led to the development of the PageRank algorithm.  Larry Page and Sergey Brin formally incorporated their company, Google Inc, on September 4, 1998. The algorithm that they had created served as the backbone for the Google search engine. The Google search engine provided a convenient method of finding information online and supplemented traditional hypertext browsing methods.

At the same time, the adoption of CGI (Common Gateway Interface) began changing the way that people communicated using the internet. Larry Wall, a linguist working as a systems administrator for NASA, developed a programming language called Perl that was designed make report processing easier. However, the language proved to be a powerful way to allow users to enter information into web forms and storing it for later use.

This advancement in internet-based communication is significant because websites were not limited to the traditional one-to-many broadcast model. Instead, the audience could talk back – adopting a many-to-many communication pattern. This shift spawned a mass adoption in regards to using discussion boards to talk about niche topics and enabled online communities to be formed. The transition from a static web to a platform that supported dynamic content provided a fertile base for a more ‘personalized’ web.

In 2001, Jimmy Wales and Larry Sanger launched Wikipedia, a free online encylopedia that could be edited by anyone in the world. Today, Wikipedia hosts more than 14 million articles in 260 languages, contributed by millions of users. The project was originally called Nupedia and only accepted articles from highly qualified volunteers. An elaborate multi-step peer review process hindered the website’s initial growth, generating only 12 articles in its first year. In an effort to encourage a more rapid growth, Wales and Sanger decided to develop an open system that allowed anyone to submit articles using a wiki as a vehicle for producing and editing content. Sanger writes, “No, this is not an indecent proposal. It’s an idea to add a little feature to Nupedia. Jimmy Wales thinks that many people might find the idea objectionable, but I think not. (…) As to Nupedia’s use of a wiki, this is the ULTIMATE “open” and simple format for developing content. We have occasionally bandied about ideas for simpler, more open projects to either replace or supplement Nupedia. It seems to me wikis can be implemented practically instantly, need very little maintenance, and in general are very low-risk. They’re also a potentially great source for content. So there’s little downside, as far as I can determine.” This decision caused an exponential increase in the number of articles submitted by users. Wikipedia has become one of the most successful collaborative projects in human history. Its success can be attributed to providing users with an easy way to contribute as little or as much content relating to their topic of interest.

The power of the social web is enabled by the ability for users to consume, produce, categorize and re-factor information without needing to understand the complex networks and scripting technologies used to host these types of applications.

Flickr, a community based photo sharing website encourages users to organize their photos by ‘tagging’ them (also known as Folksonomy).  The idea of using this sort of metadata to categorize documents provides a method of indexing that could not be attained using traditional automated web crawlers. Flickr also provides a mechanism that allows users to comment on photos. This fosters a sense of community and conversation – but more importantly, it reminds contributors that other people are viewing their photos. In other words, the idea of presenting to an ‘audience’ becomes a significant motivator to share work. YouTube, the popular video hosting website also operates on these same assumptions.

In 2003, Joshua Schachter founded Delicious, a social bookmarking web service used to store, share and discover websites. It also uses a classification based on tags. When a user discovers a new website that is ‘bookmark worthy’, the user can submit the website to Delicious, providing: a title, tags and a note. While this makes it easy for the user to categorize their own bookmarks, the act of community-based bookmarking enables the most popular websites (the ones that many people choose to bookmark) to bubble to the surface. In a sense, this gives new meaning to the term, “peer reviewed”. In addition, the service provides a platform that enables users to view the bookmarks of similarly-minded individuals – helping the discovery of noteworthy websites even more transparent. The notion of collective intelligence is afforded by the aggregation of many users performing simple tasks.

In 2004, Mark Zuckerberg, a student at Harvard University began development on a social networking platform called ‘The Facebook” (later renamed to Facebook). “Everyone’s been talking a lot about a universal face book within Harvard,” Zuckerberg explains,  “I think it’s kind of silly that it would take the University a couple of years to get around to it. I can do it better than they can, and I can do it in a week.” Within a short time after its release, Facebook was well adopted by the students at Harvard University and was extended to Stanford and Yale. Today, the Facebook community is not limited exclusively to college students. Facebook offers its services to anyone in the world, making it the most popular online social networking platform in the world.

The popularization of social media tools represents a fundamental shift in the way that we communicate. The increase in globalization influences how we consume and produce information. The micro-blogging tool, Twitter, is an example that shows how large groups can communicate based on the idea of subscribers (more commonly known as followers). Twitter enables its users to send and read messages known as tweets (text-based snippets up to 140 characters). Created in 2006 by Jack Dorsey, Twitter has often been described as the “SMS of the Internet”. It allows users to obtain a birds-eye view of what niche communities are doing in real-time (without the baggage of a full-blown social network).

The sense that “we are all in this together” is being perpetuated further by emerging platforms. In 2007, Time Magazine elected the person of the year award to: ‘You. Yes, you. You control the Information Age. Welcome to your world’.  Since then, we have become increasingly networked. We continue to feed the network with more and more information about us. Facebook knows “who we may know”. Amazon knows our buying habits can make recommendations for us. If the internet already knows this much about us, imagine what it will know in twenty years from now. Social media platforms such as Justin.tv are making is so that everyone can have their own ‘Truman Show’. And this is still… just the beginning.

Smart Search

Monday, October 26th, 2009

ssscreenshot

Social Syllabus is a teaching tool that supports the open learning initiative. It enables instructors to author and share course materials. It utilizes a number of behaviors commonly found in popular social media platforms.

Imagine that you are an instructor and you have been asked to teach a web design course. If you’ve never taught a web design course, you would need to invest a lot of energy in creating a course structure that pulls from many different resources (books, websites, videos, current events, etc). Wouldn’t it be great if you could browse a hundred other web design courses to help you get started?

This week, I built a weighted ‘search’ engine to find courses based on keywords. The weights are defined as follows:

Course title: 5pts
Course description: 4pts
Course tags: 4pts
Calendar entry title: 3pts
Calendar entry text: 2pts
Course Textbook: 2pts
Course Assignments: 2pts
General Info: 2pts

The script queries each table to find the number of occurrences of a ‘keyword’. It then multiplies the number of occurrences by the defined weights. Each result is added and an overall weight is for a course is calculated. A list of courses is displayed to the user by a context driven ‘relevance’ search.

For instance, if a user types in ‘web design’, a list of courses that relate to web design are displayed.

My prototype shows a graph that explains how the weighted categories are being calculated. In production, this would not be displayed to the user and would simply occur behind the scenes. At this time, only a single course is displayed because the application hosts one course (a web design class that I am teaching). As more courses are uploaded into the system, this search functionality will prove to be more effective.

http://socialsyllabus.com/index.php?state=searchcoursebyweight

The next step is to automatically create course tags for each course based on this same principal. This would help to suggest ‘similar courses’ without requiring an external action from the user.

Bird Watch

Monday, October 19th, 2009

birdwatch

Last semester, I spent a great deal of time working through Objective-C to create iPhone applications. While I was successful in building a few simple applications, I found that the coding environment was difficult. Part of the reason had to do with the fact that I was much more familiar using web-based tools such as html, css, javascript, php and flash. However, this week I discovered that I could more easily ‘mobilize’ my applications using familiar web technologies that render on the iPhone. HTML5 allows developers to build sophisticated applications that target the iPhone platform.

More specifically, HTML5 allows:

  1. Local SQL storage: Using Javascript, developers can create dedicated client-side databases that support common SQL commands such as: SELECT, INSERT, UPDATE and DELETE.
  2. Offline capabilities: Even if the iPhone is disconnected from the internet, an application can selectively ‘cache’ my application so that it can operate offline.
  3. Canvas: Graphics can be drawn to the screen using an environment similar to Processing. Using a bit of jQuery and Canvas, developers can create interactive games, simulations and graphs.
  4. CSS3: Developers can leverage the flexibility of CSS3 to make web applications appear as native iPhone applications. CSS3 supports gradients, drop shadow and fonts.
  5. Geolocation: Developers can write applications that record the physical location of the user.

While not specific to HTML5, developers can easily construct ‘classic’ iPhone animations using jQuery. Similarly, jQuery can handle ajax requests to get and receive data from a server. In fact, it is also possible to run your application in ‘full screen’ mode, which removes the address bar from Safari. Developers can also upload a graphic that appears on the home screen that allows users to access your application as they would any other iPhone application. View http://building-iphone-apps.labs.oreilly.com/ch01.html for more details.

In an effort to learn how to use HTML5 to build an iPhone application, I decided to create a simple bird watching tool. Users can add new birds to a local database (by typing in the name of the bird), update the quantity of a particular type of bird (by tapping on its name in the list) and view the distribution of birds on a graph. The application appears as though it is a native iPhone application and does require a network connection.

Watch

Monday, October 12th, 2009

I go to an antique flea market near Lambertville, NJ every Sunday morning. Much of what is sold is ‘old junk’ – which is exactly what I go there to buy. I like to stuff ‘old junk’ with new electronics (I like old radios in particular). This Sunday I went to the flea market with the intention of buying an old stopwatch because I’ve been playing a lot with the 555 timers this week. However, they were a bit expensive and it seemed as though it would be difficult to cram in a circuit board into a stopwatch. Instead, I found a small ‘emergency’ light that could be retrofitted as a watch. The box cost $1.00.

The watch that I built performs more like a stopwatch or an alarm. Most watches are small, compact and hardly noticeable. Not this one. This one is large, heavy and looks ridiculous. A large apparatus clinging to the wrist could arguably increase awareness of time. To operate the watch there’s a switch on the side that resets the timer. Once the timer has been reset, the watch counts for one minute. After one minute, the watch will glow and make an annoying squealing/buzzing noise. While it’s not particularly a practical project, but it’s fun. This would be easy to build using a microcontroller and a block of code. However, I’m a bit more interested in controlling the timer using some analog components because I feel as though it’s a bit more challenging.

Color Recognition and Awareness

Monday, October 5th, 2009

People are aware of the colors that surround them. But to what degree? Are we cognoscente of the subtle variations of color that exist in a photograph of a bumble bee versus that of a sunflower? I devised a simple experiment to find out. This experiment examines color awareness. The question being asked is: Can a person depict the subject matter of a scene by examining its color palette?

The experiment contains ten images and color palettes. To obtain a color palette for each image, a simple program was used to extract the most common colors in that image. The procedure for the experiment is as follows:

1.    User is presented with a color palette and two words. Using an adjustable knob, the user chooses the degree of certainty they feel that a color palette corresponds with one of those words. When they are finished, they press a big red button on the controller to submit their choice.
2.    The correct image that corresponds with the color palette is displayed to the user.
3.    After ten iterations (repeating step 1 and 2), the results of the test are displayed as a graph.

After performing the experiment with a few friends, I have found that people can depict the subject matter of a scene by examining a color palette about 85% of the time (when given two similar choices).

Wrist braces

Monday, September 28th, 2009

wristbraces

The doctor tells me, “Any time spent on a computer is too much time.” I explain that as a web developer, much of my day is spent in front of my computer. “How much time?” she asks. “Typically, somewhere around 12 hours each day.” She shakes her head in disapproval and recommends that I wear wrist braces. “These will help with your carpal tunnel. Wear them when you are on the computer and when you go to sleep at night.”

I have to admit that I don’t wear my wrist braces as much as I should. Aside from being uncomfortable, I have found that I can’t type as quickly with them on.

This week, I devised an experiment to show how well wrist braces stabilize movement and if their use affects the speed at which I am able to type.  To do this, I created an apparatus that utilizes a flex sensor, an Arduino microcontroller and pair of wrist braces. All logging and plotting was performed using Flash.

Two tests were necessary in calculating wrist activity and typing speed. The first trial was performed without the use of wrist braces; the second trial included the use of wrist braces. The testing process followed the procedures listed below:

1.    Start logging application
2.    Start timer (two minutes)
3.    Type passage into Microsoft Word until timer runs out
4.    Stop logging application
5.    Calculate and display results

The passage included words not commonly found in the English language. This helped to obfuscate the presence of ‘remembered words’ that could affect the typing speed during the second trial. A snippet from the passage includes, “Lorem ipsum dolor sit amet, consectetur adipiscing elit. Ut ac mauris nulla. Pellentesque habitant morbi tristique senectus et netus et malesuada fames ac turpis egestas.”

The results of the test were not surprising – my original hypothesis was correct. The use of a wrist brace helps to stabilize movement. The speed at which I am able to type decreases with the use of wrist braces.

The screenshot below shows wrist activity without the use of braces. The graph shows fluctuations in wrist movement over time. The plots colored in red capture instances when the wrist is angled in less than ideal ergonomic conditions. The score measures the stability of movement (100% is the maximum possible score). I was able to type 42 words per minute during this trial.

Picture 9

The readings for the second trial show how wrist braces enforce stability in wrist movement over time. I was able to type 31 words per minute during this trial. The screenshot from this trial is listed below:

Picture 10