Why do I care so much about opinion summarization? Because I believe strongly in “wisdom of the crowd” effects —  that we can often get better solutions to problems by synthesizing information from individuals — and yet, I believe that we sometimes lack good mechanisms for information synthesis. Wikis have proven to be an excellent way to synthesize factual information. Numerical averaging works reasonably well to synthesize opinions about numerical values. Voting and surveys help us synthesize opinions about decisions when there are a small, discrete number of alternatives. And there are other solutions too, such as prediction markets, Digg-style voting, etc.

Yet I believe we are missing good mechanisms for synthesizing qualitative opinions. By that, I mean opinions about people’s feelings, attitudes, likes, dislikes, desires, reasons, etc. What do people think about Obama? Well, there are generally two ways to find out. First, if you check the presidential approval polls you’ll see that nearly 70% of Americans are currently confident in Obama as a president. That’s definitely a synthesis of opinions, but it leave out all the details: it says nothing about why Americans like Obama, what they like or dislike about him, and more importantly whether youshould approve of Obama or not. The second major way to find out what people think is to listen to or read opinions from a variety of sources (op-ed articles, twitter, blogs, your friends, pundits on TV, etc.). There are a ton of opinions on Obama out there, roughly 10 created every minute on Twitter alone, but it would take an inordinate amount of time to read them.

Wouldn’t it be nice if you could click a button and immediately get a summary of all those opinions? For example, click a button have your computer report that the biggest single topic of discussion is Obama’s economic plan, and that though most americans are supportive, many of them apprehensive about the massive federal debt that will be created.  (Ok, I admit that’s a fictional example, those are just my opinions). If you had this button, you could take a stack of opinions on any topic and immediately get the gist of what people have said, the points where they agreed on, and the points where they disagreed.

Last year, I launched Pluribo with my friend Samidh Chakrabarti as modest initial solution to the problem. Pluribo is an NLP-driven tool to summarize user reviews for certain product categories. We began with electronics reviews and it worked pretty well. We then tried to expand to other categories and we started to run into problems. Yes, our solution worked for other categories, but  each category required so much calibration and training that it turned out to be inefficient to cover all categories, one by one. A hotel version of Pluribo is in the works, but for the most part I have become convinced that a fundamentally different solution is necessary. Rather than beginning with an algorithm for one category and then trying to extend the approach to other categories, I’m now interested in algorithms that are domain-neutral from the start.

What I want is universal tool for opinion synthesis, a tool that can accept multiple written opinions in any format, on any topic, and automatically provide a coherent summary. I do think this is possible. I am convinced the NLP technology we developed at Pluribo is not the right foundation for this, and that a fundamentally different approach is needed. Perhaps the new solution doesn’t use much NLP at all. Perhaps it mainly uses human computation to effectively decompose the tasks of aggregating, analyzing and synthesizing opinions; or perhaps there is a relatively simple machine learning solution that merely requires massive amounts of training data. Of course, what we consider to be a solution depends a lot on what we consider an acceptable summary to be.

I’m working on a human computation approach to qualitative summarization now. The approach is expensive to run, but it is domain neutral and could make sense for certain applications such as helping government agencies listen to their constituents, or helping brands listen to their customers. I’m curious to see where it will lead.