Predicting Entailment

Gavin Hackeling

My thesis is a web service that predicts what one text means in relation to another, and a mobile app that uses this service to answer natural language yes-or-no questions.

gavinmh.github.io/HelloTablet



My system predicts if one English text is entailed by, or "follows from", another text.
"Everybody made a great thesis" entails "everybody made a thesis.""Everybody made an artsy thesis" is not entailed by either of the previous statements, as people could make theses that are great but not artsy. "Nobody made a thesis" is not entailed by any of the previous statements.
I built a mobile app that demonstrates one use of this system. The user asks the app a yes-or-no question. The app retrieves relevant texts from the internet, and predicts whether or not they entail the user's question. The app responds by answering "yes" or "no", and provides a justification of its answer.

Background
I have implemented a proof-of-concept for the Entailment API and an application that uses it.

Formally, the system is based on Bill MacCartney's dissertation, "Natural Language Inference." (http://nlp.stanford.edu/~wcmac/papers/nli-diss.pdf)

Web services built in Python and Java comprise my system.

Audience
The mobile app is designed in anticipation of devices with small or no screens; people using these devices will require natural language-based interactions as their devices' displays won't be able to support virtual keyboards or long pages of search results.

User Scenario
The user addresses the device by saying its name. In my demo the device is named Tablet. The device acknowledges the user. The user may then ask a question. The device processes the question and speaks back an answer.

Implementation
I have built a web service that predicts what one text means in relation to another, and a mobile app that uses this service to answer yes-or-no questions.



Specifically, my web service predicts if an entailment relation exists between the two texts. Entailment is the relation that holds when one text "follows from" another. For example, the sentence "I ate food" follows from the sentence "I had pizza for lunch."



Answering natural language yes-or-no questions is a capability that complements many search engines' and personal assistants' abilities to answer factoid questions. When asked a question like "are cats crepuscular?" or "is Dallas the capital of Texas?", my app responds with a prediction and justification, like "Yes; I found this: rabbits and cats are both crepuscular" or "I don't think so. I found this: the capital of Texas is Austin."

Conclusion
I learned about an unsolved problem in natural language processing. My system does not solve it, but it works well enough to have value in constrained applications.
I learned about designing and managing a complex, compute-intensive product that consists of many services and a large code base. I learned about web services and best practices for version control.
I learned the importance of continuous user testing and rapidly iterating based on user feedback.
The project is open source. My next steps are to refactor and document the project so that it is more accessible to other people.