Traditionally, researchers have tried to model either statutory laws (e.g., an expert system that determines whether a law has been broken), or to codify case law (precedent) in an effort to model legal reasoning (e.g., what precedent would apply to a particular situation and how is a court likely to decide). I propose that instead of focusing on modeling the law (which will still be necessary, but simplified), we should attempt to model individual (or corporate or state or other “entity) behavior in reaction to laws. Using an “Agent-Based Modeling” approach, individuals, vehicles or corporations would be modeling much in the same way that they are modeled for other behaviors (population growth, traffic flow, disease spread).

Using an agent-based system (e.g., the Logo system), individual agents would be given certain motivations (acquire food, avoid arrest, cross the street) and the law, including law enforcement, act as incentives and penalties or modifiers to the motivations. Another way to visualize this would be invisible “semi-permeable” barriers that permit passage for individuals, but at a cost. Each agent must determine their “path” based on their perceptions of those costs and benefits.

One of the most important teaching tools I hope to demonstrate through this approach is the “law of unintended consequences” — particularly perverse consequences.

I am working in RePast to do this modeling. The other part of this will be to discuss the process of incorporating RePast into Processing. This may warrant a secondary paper or a side-note approach to the paper.

The following is an overall summary of all my work to dates, together with notes on issues.

Research – I have completed a fairly exhaustive review of ABMS literature and feel comfortable with the principles, modeling approaches, etc. I have a strong bibliography and literature review and feel I could write a survey/intro paper without too much difficulty.

Model – I have completed what I believe is a conceptually sound model. I am starting far simpler than I had hoped, but have fleshed out a basic “jay-walking” model using shorthand logic and some predicate calculus.

Programming – This is problematic. Repast is apparently written using a Design Patterns approach. I was unaware of this and did not find any indication of this in any of the documentation. Fortunately, Heather Dewey-Hagborg was able to help me understand this.

“Design Patterns” are an approach to writing Java (and presumably other object-oriented) applications. It is not so much a framework as a design philosophy, perhaps like RESTful design for web applications. Design Patterns require development of object frameworks. The most common appear to be “Observers” and “Factories”.

It appears in Repast that an Observer pattern is the fundamental pattern to work with. That is, everything is built in a “context” within RePast — all agents reside in the context. The context gets built with a context “factory”. I *believe* that the context is the subject in the Observer pattern, and that it is then necessary to add an actual observer.

There are also a set of external XML files that contain parameters that are passed to the runtime. For example, a simple model would have a “grid” “projection” and a “space” “projection” — you create a “context” and then add projections. While these would be created using Factories, it is also necessary to identify them as “child” projections in an XML file. I’m not sure why this is, I believe it’s so that you can modify the runtime as it is used. This is not so much of a problem. The XML files are just in another folder and you need to label and format them correctly.

The developers have also built “launchers” in Eclipse that replace the “main” Java method. Instead of executing main, Eclipse looks for launcher files. The launcher files are also XML files that contain a pointer to the ultimate “repast.main” method. Very confusing to me and not too well documented, but I think I can live with that or try to replace. In looking at repast.main, it appears to just do some error trapping and then have a set of parameters to be passed.

The system also uses “schedulers” that run continuously and can be used to monitor and execute certain agents or their methods. So, some agents can be “scheduled” to run continuously, while some methods of an agent can be set to run only if a variable is changed in another agent — so, you can “schedule” zombies to run all the time, but “humans” to run only when a zombie has “moved” (a variable set in the zombie class).

I originally thought the Java scripting language “Groovy” was also required, but based on consultation with the developers, that  does not appear to be necessary.

The Problems I’m Facing - (1) I don’t really understand Design Patterns yet. I’m working my way through the book Head First Design Patterns, but the more serious issue is (2) I still need to “detach” the java code from the runtime and run it just as a standard java library that can hopefully pass things to the Processing canvas. In theory, I can ask any agent to report back what and where it is and I can also tell the system to continuously “check” every location on the grid or on the continuous space and instruct it to “run” a method for each agent that occupies that space. Unfortunately, I can’t get even the simplest “pure java” app to work. I have some guidance from Argonne, but it doesn’t really help (attached below).

Guidance from Argonne National Laboratory:

(1) The /bin and lib/*.jar from repast.simphony.core and the same from repast.simphony.relogo.runtime and repast.simphony.runtime, although for the latter the lib jars are probably sufficient.

(2) You might also need repast.simphony.relogo.ide. With respect to the core /lib jars you won’t need the geotools ones if you are not doing any geographical work which you won’t if you are focusing on ReLogo. These are the jars prefixed with gt2-.

(3) You’ll also need the groovy-all.jar for ReLogo. There’s probably a version of this as an eclipse plugin if you are using the same eclipse that runs Repast, but you can get a stand alone version from the groovy website. You’ll want version 1.8.  [NOTE – THIS DOES NOT APPEAR TO BE NECESSARY]

(4) Lastly, I’d recommend you use the code from our git repository on sourceforge. This will give you the latest and greatest and we hope to release with that very soon. git://repast.git.sourceforge.net/gitroot/repast/repast.simphony

(5) NOTE – THE LAST RELEASE OF REPAST IS NOW OUT.

(6) There are some additional steps you’ll need to take to initialize an observer. These are pretty much what you find in the SimBuilder.groovy code that is part of every ReLogo project. I have attached a sample one to this email. The relevant code is:

int minPxcor = p.getValue("default_observer_minPxcor");

int maxPxcor = p.getValue("default_observer_maxPxcor");

int minPycor = p.getValue("default_observer_minPycor");

int maxPycor = p.getValue("default_observer_maxPycor");

RLWorldDimensions rLWorldDimensions = new RLWorldDimensions(minPxcor, maxPxcor, minPycor, maxPycor);

LinkFactory lf = new LinkFactory(UserLink);

TurtleFactory tf = new TurtleFactory(UserTurtle);

PatchFactory pf = new PatchFactory(UserPatch);

ReLogoWorldFactory wf = new ReLogoWorldFactory(context,"default_observer_context", rLWorldDimensions, tf, pf, lf);

ObserverFactory oF = new ObserverFactory(“default_observer”,UserObserver,wf);

            Observer dO = oF.createObserver();

For the world dimensions (i.e. the minPxcor etc.) you should substitute your own hardcoded values, for now. I believe the defaults are -16,-16 and 16, 16. For the various Factories, add “.class” on to the arguments (i.e., new TurtleFactory(UserTurtle.class). That you don’t see it in the above is a groovy artifact — it will be necessary in Java. You’ll need to do the same with the UserObserver in the ObserverFactory. Then use that factory to create your observer. When you’ve done that, methods like createObserver should work

In addition, the single argument constructors for LinkFactory and TurtleFactory below are convenience constructors, which make use of some assumptions on the structure and loading of regular Repast Simphony projects. They automatically find all the necessary link and turtle types. You would likely need to use the non-automated version, which simply require you to explicitly specify any additional link and turtle types (if any) you’d like to use.

So this could be as simple as:

 LinkFactory lf = new LinkFactory(UserLink.class, new ArrayList<Class<? extends BaseLink>>());

TurtleFactory tf = new TurtleFactory(UserTurtle.class, new ArrayList<Class<? extends BaseTurtle>>());

This would for specifying no additional custom link or turtle classes.

To something like:

List<Class<? extends BaseTurtle>> customTurtles = new ArrayList<Class<? extends BaseTurtle>>();

customTurtles.add(MyCustomTurtle1.class);

customTurtles.add(MyCustomTurtle2.class);

TurtleFactory tf = new TurtleFactory(UserTurtle.class, customTurtles);

For specifying MyCustomTurtle1 and MyCustomTurtle2 as additional custom turtle classes (I’ve omitted the link bit).

(8) The ContextBuilder’s build method, as used in a regular Repast Simphony simulation workflow, expects a non-null (i.e., instantiated) Context object. That looks like what’s going wrong here. In this “stripped down” approach, you’d need to instantiate your own.

(9) Given the use of the ContextBuilder, I’m not sure what level of integration with Repast you want. If its just using Repast classes (i.e. create a context, and run the schedule), then something like the following should work.

 //-- Initialization --  Create the context

 DefaultContext<MyAgent> context = new DefaultContext<MyAgent>();

 // Add any projections to it

 GridFactory gridFactory = GridFactoryFinder.createGridFactory(null);

Grid<MyAgent> grid = gridFactory.createGrid("grid", context, new GridBuilderParameters<MyAgent>(

        new WrapAroundBorders(), new SimpleGridAdder<MyAgent>(), true, 50, 50));

 // Add agents

 for (int i = 0; i < count; i++) {

      int energy = RandomHelper.nextIntFromTo(4, 10);

      context.add(new MyAgent(energy));

}

 // Create a schedule

 ISchedule schedule = new Schedule();

 // Add actions to the schedule

 private static class MyAction implements IAction {

     Context<MyAgent> context;

     public MyAction(Context<MyAgent> context) {

      this.context = context;

    }

     @Override

    public void execute() {

      // probably want to use the random iterator here

        for (MyAgent agent : context) {

            // do something with the agent

            // e.g. move it in the grid etc.

        }

   }

schedule.schedule(ScheduleParameters.createRepeating(1, 1), new MyAction(context));

  // -- Running -- Call execute on the schedule during Processing's event loop

 schedule.execute()

(10) You could also use a ScheduleRunner to run the schedule, but that would integrate less well with Processing. See ScheduleRunner and its implementations in the repast source for more details.

(11) ReLogo in Java. Just to clarify — you should be able to use the various primitives in Java — BaseTurtles etc. provided the world is set up as we described in a previous email. These are just Java classes so they should work.

In the meantime, following the advice of my thesis advisor, Kathy Wilson, I have also begun drafting out a paper independent of the development work. Perhaps the biggest hurdle to my thesis is not so much the technical problems (although they are significant), as how to model the law effectively. By definition, human laws (as opposed to laws of science) are abstract and subject to enormous interpretation. As a result, representing them in a computer simulation has always been extremely difficult.

The classic example that highlights this issue is the rule that “No vehicles are permitted in the park.” This should seem like an easy issue to model — test all “objects” that enter the part and, if they are vehicles, then “flag” this as a violation of the law. The problem is that what happens when someone decides to place a boat in the park as a statue? Now the rules must be expanded. What happens when a helicopter flies overhead? The problem becomes one of constant ad-hoc rule making.

I believe that in order to simulate the law, it would be more effective use a different approach. Namely, to model the behavior of human beings (or corporations) and view laws as (1) positive and negative influences on human behavior, and (2) barriers and entry-ways that are analogous to real, physical structures.

In other words, think of the law as a set of incentives and disincentives for behavior AND as a structure that actors must navigate. Those structures would be “semi-permeable” meaning that actors can choose whether to move around them or through them.

This also potentially leads to a predicate logic representation of the law and human behavior. Specifically, that law and human behavior could be represented as a series of predicates such as:

∀x: P(x) ∃Q(x): Q(x) ¬ L (for a population P(x) there exists a sub-population Q(x) that does not obey the law).

I’m not fully sure this a helpful approach, but it does lend itself to modeling.

A very simple example – a “Jaywalking” sim – this would be a extremely simple simulation of illegal pedestrian traffic crossing and the impact that regulatory law (the rule against jaywalking), torts (the penalties for injuring someone) and law enforcement would have on group behavior. The three classes of actors would be: (a) pedestrians, (b) vehicles, and (c) police. The rules would be modifiable by the user and could include the ability to change (a) the penalties for getting caught jaywalking, (b) the penalties for hitting a pedestrian, (c) the number of police allocated to watching for jaywalking. As a hypothesis, there would be an optimality point, above which, increased penalties or law enforcement would not be effective. The purpose of the model would be a simple demonstration of the “there ‘ought to be  a law” fallacy (e.g., by creating a law, we can always affect individual behavior).

A very complex example – an Intellectual Property simulation — actors would include (a) content creators (artists), (b) content consumers, (c) content owners, (d) content aggregators / distributors, (e) enforcement entities (public and private). Rules would be, again, modifiable by users and could include (a) ease of “protecting” ideas, (b) ease of “pirating” ideas, (c) cost of generating ideas, (d) penalty for being caught stealing ideas.

Progress – I have been attempting to build a simple RePast model in Eclipse without the using full RePast Simphony / Eclipse build. In other words, a simple RePast model that runs completely in Java. From there, I hope to move to Processing. Unfortunately, I have been having some difficulty with this. Although I have copied ALL of the RePast JARS into a new, clean Eclipse build, I have not been able to move execute within Eclipse, primarily because I can’t seem to understand where the MAIN class is built — I have tried using the context_constructor classes as the foundation for MAIN without success.

Next Steps –  I could use some help moving to straight Java. From there, I will need to understand the RePast classes better, but I believe that the best step would be to combined the Continuous Space and the Grid classes together to be initialized as a part of the size(x,y) class that Processing uses. Since Processing is essentially a screen based tool, (a) pulling appropriate x and y out of the GRID class and (b) mapping it to the screen would be critical.

I have also been working on building a mental model for a very simple agent based simulation of a legal process. The simplest I can think of would be an examination of “jay-walking” (e.g., cross a street against a traffic light).

There would like be three classes of actors:

Pedestrians – Pedestrians want to cross the street as quickly as possible, but do not want to get hit by a car or fined by the police. Pedestrians will check for the presence of police, the presence of cars, the condition of the light (a “2 state light” versus a “3 state light” may be an interesting component), and the presence of other pedestrians in the cross walk.

Cars – Cars want to pass through the traffic light without hitting a pedestrian or getting fined by the police. They want to go as fast as possible. Cars may be interested in running red lights, but this adds another layer of depth to the simulation.

Police – Police want to prevent pedestrians from crossing against the light and cars from hitting pedestrians.

The rules that affect the behavior of the pedestrians, from a purely legal standpoint, will be the severity of the punishment for breaking the law, the likelihood of getting caught and the likelihood of being hit by a car.

Dr. North was amazingly supportive and helpful with the initial questions and issues I have for my thesis. He has offered to put me in touch with additional colleagues at the labs and is quite interested in a Processing library that would allowing developers to access RePast through Processing (similar, perhaps to JBox to Box2D). He shared a number of additional resources and recommendations.

One of Dr. North’s suggestions was to look more closely at the LOGO implementation of RePast. “ReLogo,” as it is called, is a GUI based implementation of LOGO that opens up the full range of ABM methods available in RePast without the complexity of Java. The turtle metaphor is still extremely strong and implementing just those features in Processing might be a really good first step.

He also indicated that in order to get Processing to work with RePast, I will need to do additional research on whether the Processing environment can do “joint compilation.” This, unfortunately, is something beyond me right now, and will require consultation with Dan Shiffman and Heather Dewey-Hagbourg.

The structure of development in RePast is either to use: (1) a true GUI to generate Groovy, or (2) use Groovy to generate Java that is then compiled within RePast, or (3) to use Java directly. Although I’m not familiar with Groovy, I understand it is a scripting language built within Java that is ultimately compiled into normal Java. As I understand it, Groovy has a simpler and “more forgiving” syntax than Java and so can be used for rapid prototyping.

Dr. North has kindly agreed to offer me what resources he can, including access to other developers. I have shared my library of articles with him through Mendeley, the free academic papers and citations system, and we have tentatively planned a conference call for two weeks from now.

One insight I had that distinguishes my project from some of the traditional “artificial intelligence & law” work that has been done to date is to use laws as a “real” but “invisible” barrier or obstacle that actors (agents) must interact with the same way they would interact with streets, buildings and other actors. As a “semi-permeable barrier” simple laws, like “don’t jay-walk” could be decisions that actors, who want to cross the street, would have to navigate. Some may decide it’s not worth the effort, others will persevere. The “permeability” of the barrier will correspond directly to factors such as (1) how strongly the law is enforced, (2) how easy it is to break the law, (3) how expensive the cost of enforcement is, (4) how expensive the cost of violation is, etc. Actors (the simulated human agents) behave as they normally would (needs, desires, limitations), but in addition to navigating the “real” world (streets, buildings, etc.) they must also navigate the “invisible” world of the law.

*NOTE: Ideally, I (and Mike) would like to coax Dan Shiffman and/or Heather Dewey-Hagbourg onto the call to discuss the project.*

I was a little unpleasantly surprised by the MOMA exhibit “Talk to Me” — ALthough the pieces were generally good, in many respects, the entire point of the overall exhibit — interactivity as art, seemed very lost. While some of the pieces did have real interactivity, even that seemed to be diminished relative to what I would expect from a museum presentation.

To start with, those pieces that had no interactivity at all. One example would be “Dwarf Fortress“. Dwarf Fortress is apparently a strategy game that has significant complexity and relies entirely on ASCII based characters to create the world. In spite (or perhaps because) of being extremely “retro” it apparently has a strong following. The art piece was a wall-sized depiction of one world in the game. The problem is, it was entirely static. It was a very large, imposing and engaging *static* image. Even a time-lapse video of the piece would have been more engaging.

Another example would be the “art game ”Passage“. Passage is an “art game” in which the player has to explore an extremely pixelated world, looking for treasure and/or a mate. If the player finds a mate, there are certain areas that are open to the player that are not open to a single player and vice versa. The game itself is intended as a metaphor about life. As time passes for the player, the player moves further to the right of the screen, representing the passing of time and mortality. Although the piece is not to everyone’s taste, it is notable for its high degree of pixelation and yet continued ability to tell story, and the fact that it is a representation of an individuals mortality. As such, the effect is completely lost on a normal-sized screen with an ordinary joystick. There is little difference in playing the game at the exhibit as compared to playing the game at home. This is an example where it would have been far more effective to large static images than a smaller moving image. Obviously a much larger moving image would have been the best solution.

Finally, there were a number of pieces of interactive design (a unique keyboard) or human prosthetics or augmentation, that were all displayed behind glass. Given their nature, it was extremely disappointing not to be able to touch the pieces, even if they were only replicas of the original. Overall, given the subject, it seemed the exhibit was trapped in a pre-interactive age rather than embracing the subject it portrayed.

There is significant information on providing information to people where there is relatively binary decision that does not require a prior value judgement. For example there is voluminous information  on the best way to design traffic signs or voting booths so that drivers and voters can get to where they want for vote for whom they want. These are situations where the “user” has typically made a decision *before* the actual process is presented — “I know I want to go to Denver, I just need to know where to turn off” — “I know I want to vote for the independents, I just need to know what boxes to check.” And, of course, the Metro and stove examples in the book.

Similarly, there is an enormous amount of information on how to effectively influence individuals to a make decision in a manner that you would like. The arts of marketing, advertising, lobbying and even effective management are all built around the notion of influencing users in a direction you would like to steer them. This is the whole issue of “influence”.

However, I have yet to see a really meaningful answer to the question “How much information does a user need in order to make a rational decision?” (the “mappings” question raised in Nudge. The medical choices issue is a good one, but highlights that there are relatively few doctors a patient will deal with who do not have a bias — surgeons vs. radiation treatment (and presumably in chemotherapy). As Nudge points out, there are no experts in “wait and see.” Nor are there likely to be. Similarly, a simple comparative cell service pricing chart is likely to be simple and effective but is unlikely to ever effectively communicate the “softer” issues such as customer service. More importantly, no cell carrier would *ever* willingly participate in such a plan and would use all their own influence to prevent it.

But let’s take a simpler issue — food labeling. I was struck this weekend by the difference in food labeling in countries when we purchased a large quantity of Coca Cola imported from Mexico. It still has sugar in it, rather than corn syrup. The bottles looked virtually identical to US bottles with the exception of (1) the “made in mexico” label, and (2) an ugly, obvious sticker with nutritional information added in the US to comply with FDA laws and obviously not required in Mexico.

My question, of course, is how useful are such labels and how do you make them “good enough” that people make “the right decision”? Forgetting for a moment that a “right decision” is extremely relative, we’re going to play god for a moment and assume that the “right” decision is that Coca Cola is fine is moderate amounts (1 or 2 bottles a week) for people who do not have diabetes, sensitivity to caffeine or other ingredients, are taking care of their teeth regularly and do not have serious weight issues.

A label with calories and nutritional information does not convey that. The US cigarette label comes close, but has no comparative feature.

We know how to label poisons — “This Will Kill You” — We know, to a less extent, how to label dangerous but less-than-lethal but otherwise dangerous substances — “Do Not Operate Heavy Machinery” — We don’t seem to know how to manage beyond that — “This is moderately enjoyable, but also somewhat addictive and a certain percentage of people get sick or die each year as a result of its use. That percentage is small in the individual case and larger in the aggregate.”

In short, how do we communicate enough, clearly and not too much and yet not incorporate too much bias into the communication?

People are just plain bad at assessing risks. Some data:

“… your chances of dying in an airplane crash? A one-year risk of one in 400,000 and one in 5,000 lifetime risk. … Drowning? A one-year risk of one in 88,000 and a one in 1100 lifetime risk. In a fire? About the same risk as drowning. Murder? A one-year risk of one in 16,500 and a lifetime risk of one in 210. … And the proverbial being struck by lightning? A one-year risk of one in 6.2 million and a lifetime risk of one in 80,000.”

These numbers seem very very flawed to me. A lifetime of murder as 1 in 210? And yet, this piece suggests similar numbers.

So, no answers here, just the question, how on earth do you get people to recognize that they are 25 times more likely to be murdered than to do in a plane crash? (no jokes in poor taste here). I have no idea.

This analysis is amazingly clear and useful in the question of design. In looking at Apple products, it’s easy to see that one of the reasons that they are so successful is that they tend to address all three qualities — they are attractive (and therefore we have an emotional reaction to them beforehand), they are typically easy to learn and use and therefore the behavioral aspect is positive, and, because of (1) their general ease of use, and (2) cultural/societal “approval” of Apple devices, we typically feel good or “reflect positively” on having used devices.

This is also a really useful tool in analyzing why some designs fail, and also in recognizing that design does not have to successfully address all three aspects. For example, a number of the teapots that Norman refers to in his book have little practical use but are extremely attractive. Their visceral and reflective design is extremely good therefore, while their behavioral design is poor. This does not mean they have failed if the designers goal was *not* to make a “useful” teapot. On the other hand, user interfaces (such as the early incarnations of Android and Windows) had relatively good *initial* visceral appeal — people wanted to play with them. However, once users began to spend time with the devices or interfaces, they quickly found them frustrating. The visceral design was there but the behavioral design was poor. Consequently, the reflective design was poor too. Few people felt particularly good about being foiled an operating system.

This does raise one question I have about Norman’s approach: Although I can see it’s possible to design something that is heavily visceral or behavioral (attractive but not easy to use, not attractive, but easy to use), I wonder if there are any devices or designs, other than his example of momentos, that are heavily reflective. That is, is it possible to design something that is neither attractive or easy to use, but which creates a good emotional reaction having used it? While this may sound glib, in fact, the only thing that immediately comes to mind is something like the UNIX operating system. I don’t think anyone would argue that UNIX is either particularly attractive, nor that it’s particularly easy to use (assuming we’re not talking about an X-Window interface or the like). However, at least personally, I found it *enormously* rewarding to have actually wrested something out of UNIX. This may be pure masochism on my part, but actually having worked in a UNIX command line, while neither viscerally pleasing nor behaviorally pleasing, is at least something I can take some pride in. However, I suspect this was not a design consideration. Heathkits were actually more rewarding than Norman suggests, in that there was a great deal of learning accomplished and the design and layout of the instractions were usually a delight. The last suggestion of reflective design, fashion, sadly, remains a total mystery to me, but then I can’t imagine wearing anythign but the most comfortable shoes social strictures would allow.

I wondered how this related to the notion of “your first choice typically being the correct one”, at least on multiple-choice exams. A fast good search reveals that most teachers and test preparation advisors recommend *not* changing your choice (intuition / emotional decisions win). However, at least one study I found indicates that the opposite is actually trued. “Counter-factual” thinking leads people to believe that their first choice is usually better, but in fact, based on the studies, most test-takers are better off changing their answers. This surprised me and appears to be contrary to Lehrer’s premise. You could argue that the decision to fire a missile is not comparable to a multiple-choice exam, but in fact, it is still a time-pressured decision making. It does make me wonder whether Lehrer’s premise (at least with respect to emotional decision making) has some shortcomings. It is, in many respects, similar to Gladwell’s hypothesis in Blink.

It’s my sense that there is a definite bias, at least in our culture to “rely on intuition.” Popular psychology regularly talk about human beings having unique decision making talents that operate at a subconcious or “pre-cognitive” level as a result of evolutionary forces (the fastest, best decision maker was most likely to survive ). And there is certainly an enormous amount of evidence associated with neural pathways developing for decision making. However, I do wonder if instinct, intuition and “emotional decision making” were really as effective as people like Lehrer and Gladwell suggest, why logic and decision making tools (game theory, occams razor, etc.) evolved. In fact, the whole unique frontal cortext that human beings possess is an argument that while emotional decision making can be valuable, it is not a deciding factor in species survival.

A few links that I found particularly interesting:

Recent Washington Post Article

Paper from the Naval Warfare College.

Info from Defense Update.

My initial thoughts —  Erosion / Evolution of US Military Man-in-the-Middle Doctrine and the changing face of warfare

There is an excellent argument that technology drives the way wars are fought. Many have argued that machine guns and powerful mobile artillery ended the primacy of the infantry / cavalry charge and lead to the trench warfare of World War I. Similarly, effective mobile armor and air power ended trench warfare and lead to the “blitzkrieg” model of warfare. Nuclear weapons are arguably the reason that there has not been a “total war” (Ludendorff) since World War II and lead directly to “low intensity” “guerilla” and “insurgency” warfare. Although the US attempted to counter this threat through a number of approaches including targeted killings (the Phoenix Program), aggressive propaganda (the “hearts and minds” program) and electronic intelligence (the “electronic battlefield”), success was limited and a paradigm shift did not occur as demonstrated by the Allied success in the first Gulf War (using traditional blitzkrieg warfare techniques) and the problems in the second Gulf War (given the huge number of casualties caused by IED’s and other insurgency / low intensity tactics).

A new warfare paradigm, however, may be emerging. Remotely operated vehicles and weapons actually were relatively well known as far back as World War II (the Germans had a large arsenal of such weapons and even the Allies had remotely piloted planes used as flying bombs). However, all such weapons have always had a human operator, or a “man-in-the-middle”. With the exception of land-mines and sea-mines, essentially all weapons (at least in the US) have required human intervention in order to kill. This doctrine most likely [research needed] arose as result of the nuclear cold war and severe concerns about a nuclear power “launching on warning” – essentially turning some degree of decision making over to machines.

However, over the last [20] years, truly autonomous weapons have begun to emerge. These are weapons that are permitted to “kill” without intervening human action. Unlike land and sea mines, which are typically mechanically triggered and are generally considered “defensive” weapons (that is, they are typically used to defend a position rather than being used to attack a position, although this is subject to debate). Autonomous weapons, including those that identify and discriminate between tanks and other vehicles, weapons that rely on facial and iris recognition are starting to emerge into the battlefield. I would like to analyze this, identify the trend, try to determine whether “man-in-the-middle” doctrine is changing and consider whether this is a fundamental shift in warfare akin to machine guns, blitzkrieg and nuclear weapons.

—

Unsurprisingly, what sounds at first glance like a relatively dry and academic point is actually a tremendously political issue and one that has serious implications for many people.

Human beings are “programmed” to identify patterns. This was an incredibly important evolutionary development. The human beings that only saw random stripes or dots in the tall grass were far less likely to reproduce and pass along their genes than the human beings who were able to “connect the dots” into something with a long tail and big teeth. Pattern recognition is a survival function.

However, sometimes patterns don’t always exist. Evolutionarily speaking, it’s better to think you saw a tiger and be wrong that fail to see a tiger that’s actually there. So evolution favors false positives over false negatives.

This means that we often simply see patterns that just aren’t there. If this is an issue of marketing a new soda to a demographic that actually doesn’t have any interest in an exciting new soft drink, that just means lost money to a company. If clustering suggests that a new drug may increase the risk of heart attacks, then, unfortunately, an otherwise useful drug may not be brought to market. When clustering suggests to us that there is something causing cancer in a population, you suddenly have a very very serious issue.

Cancer clustering is one of the more controversial issues in oncology and law right now.

And to be clear, the issue is not one of protecting companies who may have been polluting the environment and causing cancer. The issue is one of resources for the rest of the medical community.

Human beings are very good at seeing patterns, even if none exist. Lawyers are similarly very good at convincing people that patterns exist .

As shown below, random dots will cluster.  This graphic comes from the State of Delaware Department of Public Health. It’s a graphic intended to demonstrate the potentially misleading appearance of clusters.

Random Dot Pattern from Delaware DPH

The following is an interesting article from Oxford University Press on the same issue. In short, clusters are real. They are also, at times, incorrectly identified. The resources (and here, I’m not concerned about the companies so much as the EPA, the CDC, etc.) are enormous.

However, I simply have no idea of how you would go about telling people who have sick family members that it’s a statistical anomaly, or that there are inadequate resources to pursue it. And there are always lawyers lawyers lawyers looking to intervene in both sides of the argument.

On a totally unrelated note, there are amazing studies that have been done with clustering of the Enron email corpus. The data is readily available and quite bit of fun to play with. Here are some interesting images links:

Enron Visualization by J.Heer

Enron Visualization by J.Heer

http://hci.stanford.edu/jheer/projects/enron/

http://hci.stanford.edu/jheer/projects/enron/v1/

http://hci.stanford.edu/jheer/projects/enron/v2/

http://bailando.sims.berkeley.edu/enron_email.html

http://infosthetics.com/archives/2006/10/enron_explorer_data_visualization.html

http://snap.stanford.edu/data/email-Enron.html

http://www.cs.cmu.edu/~enron/

http://sgi.nu/enron/

http://www.isi.edu/~adibi/Enron/Enron.htm

http://edrm.net/resources/data-sets/edrm-enron-email-data-set-v2

Spell-Check / Autocorrect — Everywhere, everyday. Occasionally useful in word processing, less useful and more annoying in GMail, genuinely distracting and painful in SMS and mobile mail.

GoogleAds and other auto-generated Ad services – I use AdBlock+ for Firefox. Once I turned it off, it was, surprisingly, not so overhwelming as I expected. I think I’ve generated an internal filter to this stuff. As long as the ads are not either (1) offensive/pornographic, or (2) those insane popups that still show up on some high end sites, I can’t really see them. They’re just more noise. That said, I use the “readability” plug-in, or Instapaper to help cut down serious infringements on reading stuff that matters.

Amazon recommendations — I buy a lot of books. Amazon is consistently one of the most useless recommendation systems out there. I’ve also tried, in no particular order: www.librarything.com, BookArmy, WhichBook, BookLamp, and What Should I Read Next?, www.goodreads.com, and www.bookseer.com. They all want me to read Harry Potter. I don’t have anything personally against Harry, but it’s not my cup of tea. 

 Finally, there’s LinkedIn’s and FaceBook’s “you may know” recommendations. It’s rare that I actually do know any of those people. Much rarer still that I would want to link to them.

I realize now that these suggestion services are, for the most part, something of a cross between a dynamic “filler” and mass-marketing / spam. That is, once upon a time, sites used to regularly restructure their layout with the same content so that when people came back to the site it looked new. Thankfully, people don’t do that anymore. And mass-marketing / spam has really a tiny tiny percentage of successful hits. But it’s so cheap, it’s worth it to the websites and vendors to continuously bombard us with this noise. And I’ve seen no evidence that any of the recommendation services have improved significantly over the last 10 years, so, for me, it’s simply another form of graphic noise to ignore.

In truth, this appears to be EXACTLY how the PageRank system works. There’s even a great quote from the paper on this very subject: “The average web page quality experienced by a users is higher than the quality of the average web page. This is because the simplicity of creating and publishing web pages results in a large fraction of low quality web pages that users are unlikely to read.” In short, if it’s read, it’s better than if it’s not read. And for much of the web, this is true. The whole “wisdom of crowds” thing. The whole “efficiency of the market” thing. Good, valuable, clear, dense information will win out over bad information. That’s the concept.

And now, I wonder. This may be true if you wish to get information the good restaurants in New York City. Or the best places to go bird watching in Central Park. People will search. They will find and link to and refer others to the best places. And those will rise up with PageRank algorithm.

That’s true for those things that can, by their nature, have validity and usefulness determined by consensus. Which is fine. But there are many many many things that consensus is very very bad at. It’s the reason why we have the Supreme Court in the United States and decisions like Brown v. Board of Ed.  Because lots of times, consensus is wrong, or just wrong-headed.

And that means that looking for information on cancer treatments using PageRank is a very very questionable assumption. Because most people are not experts on cancer treatment. And yet, many people are affected by cancer and by worry for their loved ones and themselves and so look for the best information they can find. Unfortunately, for something like cancer treatment, the information that everyone finds is not necessarily the best information. And so PageRank fails.

PageRank is also, I suspect, extremely susectible to manipulation by individuals or organizations adept at generating interest in a subject. People talented in PR and grass roots initiatives frequently have a natural talent for drawing attention to something that, perhaps, was not always of interest to people. But if enough “buzz” is generated, then, again, the PageRank algorithm will bubble up the pages.

I know there are literally an endless number of ways to approach searching. From purely keyword, to purely popularity to heavily analytical, using linguistic formulas. At the end of the day though, I suspect that expertise is the only true factor in really determining value, at least where there are issues that are not simply consensus issues (e.g., ”great movie” is probably a consensus issue, while atomic weight of Cessium is most definitely not). So, do you insert a “person in the middle” much the way wikipedia does? I’m not sure. I suspect that, in fact, the very reason why google has not eliminated wikipedia and the like is because editorialized expertise has as much of a place as consensus and therefore cannot be eliminated.

Lastly, WOW. How about this –> “”Currently, the predominant business model for commercial search engines is advertising. The goals of the advertising business model do not always correspond to providing quality search to users. … In general, it could be argued from the consumer point of view that the better the search engine is,the fewer advertisements will be needed for the consumer to find what they want. This of course erodes the advertising supported business model of the existing search engines. However, there will always be money from advertisers who want a customer to switch products, or have something that is genuinely new. But we believe the issue of advertising causes enough mixed incentives that it is crucial to have a competitive search engine that is transparent and in the academic realm.”

WOW. Can’t really add to that. Wow.

It was only the creation of publicly available schooling that really started to alter the spelling landscape. That and Samuel Johnson’s dictionary, which was essentially the first of its kind.

Bayesian analysis is, honestly, a little bit like magic. It is for me, very reminiscent of Arthur C. Clarke’s quotation that inspired my blogs name — “Any sufficiently advanced technology is indistinguishable from magic.” Bayesian     analysis, even though it is, in many respects, quite accessible once you have read through the formula a few times. At the same time, when actually working with a good training set and a good test set, it’s ability to identify patterns (parts of speech, spam, etc.) is really nothing short of magical. I’m always reminded of the silly question “how does the aspirin know where the headache is?” But it does seem that way. The impact of probability on our ability to perform these sorts of analysis is transformational.

However, I also want to note that I have been working with variants of these technologies for many many years, and the expression about “almost only being acceptable for horseshoes and hand grenades” remains equally true with grammar, spelling, OCR, writing and speech recognition.

I have been working, as an extension of my professional responsibilities, with OCR, grammar checking, handwriting recognition and speech recognition for more than 20 years. I have tried every commercially available OCR package, grammar checking package (yes, there used to be more than 1), handwriting recognition package (LiveScript currently makes the best on the market, but people have completely forgotten the Apple Newton and the early Palm recognition applications. Similarly, Kurzweill’s speech recognition systems were originally hardware based and there were multiple other packages out there.

OCR, in particular, is critical in the legal profession. The core of litigation is to conduct “discovery” wherein you review *enormous* quantities of documents looking for materials relevant to the matter. Historically, this has meant paper. Boxes and boxes and boxes of paper. Warehouses full of paper that had to be OCR’d. For many years, one couldn’t expect an accuracy of much more than 75% – 80%. That means one in every 4 or 5 words is wrong. Not just wrong, but *completely* wrong. When you’re reviewing or searching through hundreds of thousands of documents, this is a serious issue. Speech recognition was about the same and handwriting recognition was an unmitigated disaster.

Over the last 10 years or so, the accuracy has gone up significantly, so that many vendors claim 99%+ on “good OCR” and 95% on speech and handwriting recognition.

I feel comfortable in saying that (1) these numbers are still wildly inflated and (2) even if they are accurate, you need to look at a broader context. Namely, documents tend to average 250 to 500 words per page. A typically “bankers box” for paper (a standard box for storing papers) holds between 5000 and 15000 pages. This means, on average, there will be 12,500 and 75,000 spelling mistakes for a single box of documents. If you are relying on word and phrase search to review these documents (which you must), then this is a major problem. There are no better solutions, but OCR is still really on a mediocre technology at best. When it attempts to handle formatting and the like, it tends to fail utterly.

Speech recognition and handwriting recognition are, at best, 95% accurate. Again, a number that I believe to be highly overstated. Considering that the average human being speaks at between 80 and 120 words per minute in normal conversation, this means for every minute of dictating or transcribed dialog, there will be at least 4 to 6 completely incomprehensible spelling errors. Worse, they will not be misspellings but completely wrong words. If you are going to dictate for 20 minutes, you are going to spend at least another 20 minutes identifying the misinterpretations that the system has generated.

In short, I feel all of these technologies have a significant improvement left to be generated.

Getting the code to work was, frankly, quite a bit more difficult than previous work, and unfortunately, I still don’t have anything to show.

press-releases

text-release-generated

10q_mda

generated

When a search string is put together, the parser that takes the search string will often remove these terms.

This is usally a good thing, but not always. I find myself amazed that I have real-world experience with this. One of my responsibilites in my current job is to act as an “electronic discovery liaison”, which essentially means acting as an attorney, but, in part, also crafting or assisting in crafting queries to gather electronic documents for discovery purposes.

In really abbreviated terms, “discovery” is the process where one side in a legal dispute has the right to request documents and other materials from the other side as a part of the litigation, regulatory inquiry or the like. Often this involves very very broad requests — for example, “all documents relating to home mortgages”. At one of my former employers, roughly 1 billion unique email messages were generated every year. This means that putting together and parsing an appropriate full-text query is a very challenging process and requires real research on the subject and frequently interviews with the experts.

Unfortunately, sometimes stop words can be a hindrance. For example, if one was search for information on “Subordinated Convertible Debentures, 2008 Series A” (as opposed to series B), removal of the letter “A” would be problematic.

Apparently Google has recognized this issue and has, sadly, obtained a patent on their solution to it–> http://www.seobythesea.com/?p=1109. In simple terms, it appears that Google runs searches both with and without stop words, analyzes the results and determines whether the stop words were effective.

Image from http://www.seobythesea.com/?p=2795

Stop words are obviously an important search engine optimization (“SEO”, or “how do I get my page to show up first?”) issue, and apparently much work has been done by SEO analysts to determine how the major search engines approach this issue. A really interesting analysis was done here –> http://www.seobythesea.com/?p=2795, which, among other things, offered the attached following graph.

Ironically, a simple search of Google, Yahoo and Bing all resulted in basic *and* huge stop word lists.

http://www.lextek.com/manuals/onix/stopwords1.html

http://armandbrahaj.blog.al/2009/04/14/list-of-english-stop-words/

http://www.textfixer.com/resources/common-english-words.txt

http://www.ranks.nl/resources/stopwords.html

I opted for a relatively small stop word list and modified the sample code by Heather, together with some random text pulled from –> http://watchout4snakes.com/CreativityTools/RandomParagraph/RandomParagraph.aspx which generated the following paragraph:

The birthday accommodates stop words on top of an explanatory lyric. The degenerate overlooks stop words throughout the league. The originator precedes search engines underneath an eccentric injustice. How will stop words orbit without a clock? Stop words discriminates search engines within an appearance. The rose figures stop words into each regardless obstruction.

The result set follows:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40

START   END TOKEN
4    12  |birthday|
13    25  |accommodates|
26    30  |stop|
31    36  |words|
40    43  |top|
50    61  |explanatory|
62    67  |lyric|
73    83  |degenerate|
84    93  |overlooks|
94    98  |stop|
99   104  |words|
105   115  |throughout|
120   126  |league|
132   142  |originator|
143   151  |precedes|
152   158  |search|
159   166  |engines|
167   177  |underneath|
181   190  |eccentric|
191   200  |injustice|
211   215  |stop|
216   221  |words|
222   227  |orbit|
228   235  |without|
238   243  |clock|
245   249  |stop|
250   255  |words|
256   269  |discriminates|
270   276  |search|
277   284  |engines|
285   291  |within|
295   305  |appearance|
311   315  |rose|
316   323  |figures|
324   328  |stop|
329   334  |words|
340   344  |each|
345   355  |regardless|
356   367  |obstruction|

And the code:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44

import java.util.Set;
 
import com.aliasi.tokenizer.EnglishStopTokenizerFactory;
import com.aliasi.tokenizer.IndoEuropeanTokenizerFactory;
import com.aliasi.tokenizer.LowerCaseTokenizerFactory;
import com.aliasi.tokenizer.StopTokenizerFactory;
import com.aliasi.tokenizer.TokenizerFactory;
import com.aliasi.util.CollectionUtils;
import com.lingpipe.book.tok.DisplayTokens;
 
public class SimpleStopTokenizer_v2 {
 /* Dave Boyhan - david.boyhan@gmail.com 2/18/11 
 * A small modification of the SimpleStopTokenizer written by Heather Dewey-Hagborg here --> http://itp.nyu.edu/varwiki/uploads/SimpleStopTokenizer
 * the Indo-European tokenizer will tokenize, the resulting tokens will be converted to lower case, and 
 then stop words will be removed.
 */
 
 public static void main(String[] args) {
 String text = "The birthday accommodates stop words on top of an explanatory lyric. " +
 "The degenerate overlooks stop words throughout the league. The originator precedes search engines underneath " +
 "an eccentric injustice. How will stop words orbit without a clock? Stop words discriminates search engines" +
 " within an appearance. The rose figures stop words into each regardless obstruction. ";
 
 //Note that punctuation was added to the stop list as well
 
 Set<String> stopSet = CollectionUtils.asSet(".", "?", "a", "able", "about", "across", "after", "all", "almost", "also", "am", 
 "among", "an", "and", "any", "are", "as", "at", "be", "because", "been", "but", "by", "can", "cannot", 
 "could", "dear", "did", "do", "does", "either", "else", "ever", "every", "for", "from", "get", "got", "had", 
 "has", "have", "he", "her", "hers", "him", "his", "how", "however", "i", "if", "in", "into", "is", "it", 
 "its", "just", "least", "let", "like", "likely", "may", "me", "might", "most", "must", "my", "neither",
 "no", "nor", "not", "of", "off", "often", "on", "only", "or", "other", "our", "own", "rather", "said", 
 "say", "says", "she", "should", "since", "so", "some", "than", "that", "the", "their", "them", "then", 
 "there", "these", "they", "this", "tis", "to", "too", "twas", "us", "wants", "was", "we", "were", "what", 
 "when", "where", "which", "while", "who", "whom", "why", "will", "with", "would", "yet", "you", "your");
 TokenizerFactory f1 = IndoEuropeanTokenizerFactory.INSTANCE;
 TokenizerFactory f2 = new LowerCaseTokenizerFactory(f1);
 TokenizerFactory f3 = new StopTokenizerFactory(f2,stopSet);
 //Upon trying the EnglishStopTokenizerFactory below, I received a poorer result set
 //TokenizerFactory f3 = new EnglishStopTokenizerFactory(f2);
 
 DisplayTokens.displayTokens(text,f3);
 }
 
}