alex dodge (°_°)

Recommendation Engines

While there is no doubt a need for recommendation within information systems that are growing in volume and traffic as fast as they are. Recommendation can be thought of in a sense as subjective filtering. Taking the common example of Netflix as a model. Offering approximately 100,000 titles on DVD, each on average 2 hours long.. works out to about 23 years of continuous viewing were you to try to personally evaluate each title… which would be impractical for even the most devoted cinephile. The interesting part of the problem is that within any recommendation system we are attempting an algorithm that can subjectively filter a volume of information that is itself not static (that is, it is constantly being updated with new releases and added titles) toward an individual that is also not static… people’s behavior can change over time. So accounting for what is a massively dynamic and changing system is no small feat. Assuming that you could account for multi-nodal change efficiently… being able to make suggestions to users on new or previously unavailable information… accounting for how that information might effect successive iterations of interest and recommendation… I would be inclined to say that such a system, in its optimal form, would accelerate cultural consumption toward a bottle-neck of cultural production. A good problem to have if you are in the business of producing culture.

What is more accurately the case is that at present our recommendation systems make some significant assumptions about their users… namely: that users don’t change very much (this is arguable on different scopes of human behavior), that people know what they want… or more appropriate to the subject of recommendation… they know what they like. This can be taken in at least one of two ways… You could take the vantage from high above like that of Edward Bernays (“father of public relations” and modern advertising) –”It is sometimes possible to change the attitudes of millions but impossible to change the attitude of one man.” It’s hard to say whether the advent of the internet would have made him reconsider the latter statement. Inspired by his uncle’s (Sigmund Freud) ideas about human desire and the subconscious mind…  his was a view in which people could be manipulated to desire things that they might not have previously by sublimation and association. But possibly more valuable in an ever more stratified media landscape is considering how an individual user can be understood in algorithmic terms. Tendencies can drift toward either trait based analysis and attribution or situational models entirely… known as Fundamental attribution error… Only a system that accounts for an individual user’s situation and his or her specific traits holistically, neither of which remain static.. not even their genetic code, will be truly effective. How active the system is in attempting to alter a user’s immediate situation or exploit known traits toward 3rd party interests places that system somewhere nearer or further away from Bernay’s style manipulation.

Where I tend to fixate however is the idea of discovery and the psychology of it. The feeling of having ownership over something because it was in some contrived instance perhaps seen first by you. I recall during the days of alternative and “grunge” that there was often a notion of an elusive threshold, that once crossed, a band’s contingent… the very same people that had “discovered” them and instilled them with exclusive value could no longer maintain their grasp… alas as the band hurled forth transforming into a “sell out”. We dance with this ephemeral quality of scarcity. A question that I often ask is whether or not moments of discovery could be effectively simulated.. and what that might mean for the user and their experience of information. On a somewhat flamboyant level Wikileaks might be doing this.

Returning to Netflix for a moment… I have to admit that I was pleasantly surprised once with their recommendation, discovering a film that I would easily put in my top 10: Primer by Shane Carruth.

A variation on recommendation that centers on the effects of choice in different conditions is also relevant. See Sheena Iyengar‘s Ted talk “The Art of Choice” or book by the same title. The idea of customization as a means of making choices more personally meaningful is discussed.

See also Century of the Self by Adam Curtis.

Clustering data from IPExplore

As an exercise I chose to use a database log from a previous project called IPExplore that visits IP addresses randomly. The full data comprised over 10,000 IP addresses of which roughly 600 were “hits” or actual websites of some kind and the others logged as network timeouts. I was interested in seeing if any discernable patterns would emerge from the “hit” IP’s vs. the timeouts. Below are two very simple sketches, one with 600 “hit” IP addresses and then the other with 600 timeouts. In the “hits” data, clear stratification can be seen which is encouraging as I work toward a more intelligent implementation of IPExplore for a final project.

While the clusters are present below in the “timeout” data the distribution, as expected, is fairly even. Something to note however is that the way IPExplore can sub-explore a “hit” IP range, essentially iterating through lower byte ranges successively up through the scope of the IP. This probably accounts for some of the consistency seen above but not all of it. This feature in the current build is not automated, but as I move IPExplore toward “Deep Search” this will be automated as the search component will no longer be interactive.

You Are Not A Gadget (Chapters 1 – 3)
By Jaron Lanier

Lanier Begins a section in his book with the statement: “The most important thing about a technology is how it changes people.”…In relative terms technology itself could be defined as that which reinvents its inventor, be it spoken language, a neolithic stone axe, or a marginalizing database of personal attributes on a social network. That is… that technologies, whether they are abstract systems or specialized physical tools, redefine a society or individual by enabling a path that was obstructed or impassible without it. In this sense technologies can be thought of as both imperatives to a species’ survival in an absolute sense, but also as rifts or deterministic events that are often not perceived until long after they have occurred. It might also be worth saying that once these paths are perceived, there is something uniquely human in the inability not to pursue them; that is that once we imagine that something might be technically possible it is never a question of if but when.

The scope of Lanier’s book, at least the first three chapters of it, distills the conundrum of standardization and plasticity in information systems / individual agency and crowd based models in those systems.

Lanier describes a precipice of technological standardization, which he terms “Lock-in” as the often unforeseeable consequences of social adherence to a system standard. He gives the example of MIDI, a system developed by Dave Smith in the 1980′s to control multiple synthesizers. MIDI, for lack of any significant alternatives at the time soon became the standard, not only for the instruments it was designed for but for instrumentation it was not. In this sense the accepted standard has a potentially negative determinism by marginalizing against things that don’t adhere to it well and by offering the benefits to things that do. In the case of MIDI it unintentionally enabled specific genres of music and arguably disabled others. Had MIDI allowed for better manipulation of other instrumentation… 80′s music may have sounded different.

Other examples might be Apple’s App Store forcing developers to adhere to specific standards and rules, or the acceptance of Romanized keyboard input for Asian languages. In the case of the App Store the peril might be the inability to be innovative within restricted hardware guidelines or censorship. With Asian language input, inability to write one’s language without a text input device is a growing problem, as people forget how to manually write characters and increasingly rely on automatic conversion from romanized phonetics.

Within this larger problem seems the principle that, given no competing alternative, convenience always wins, and when there is an alternative the most convenient option wins regardless of how superior in quality, plasticity, or how well suited it is for future needs. The real core of the problem can be seen when the level of investment in a given standard outweighs the benefit or cost of revising or replacing it. More often than not this is the case, though with something like the App Store the standard is an artificial market-driven strategy and could revised at any time. In the case of western keyboard input or Linux, revision or replacement is next to impossible due to the number of dependencies on them. The social impact of this paradigm is the focus of Lanier’s criticism.

Similarly Lanier describes the origins of anonymous agency on the net as being formed in the early days of Usenet, and explains the unfortunate consequences that would eventually result in things like Trolling. While Lanier argues that it would not have been so difficult for Usenet to have employed a system where user’s identities were known, it could be countered that it was simply that convenient not to… again convenience always wins.

Lanier describes a  process of “self reductionism” specific to social networking, a model in which an individual begins to define them self based on more or less arbitrary criteria set in place by a social networking site like Facebook rather than established choices in the “real world”. While it remains speculative as to where models like Facebook will go if anywhere at all, the same issue presents itself perhaps more stealthily in the form of personalized search.

Though the problem is varied, we can attempt to simplify it, the notion of personalized search is certainly in the same class as the marketing of a”personal graph” and other peer group advertising models, but because search acts as point of mediation, a primary and definitive point of user experience and access to content, the implications are perhaps more impacting. While Lanier speculates on the viability of Facebook’s business model, citing the failures of things like Beacon, though there may be a larger Personalized search aims at enriching personal experience of the internet and does so successfully in certain terms; that is it can provide a filter for more or less desired content. The problem arrises when that model of personalization is a cumulative one, or when the user profile is a compilation of past behavior, which even with the best predictive models to date, it only can be. In this sense the user builds a confined version of the internet unknowingly through self censorship or self segregation. The danger is that the user would find it hard to detect this effect… how would you miss what you don’t know about?