Metavana Mix: Social Complexity, SparkScore Simplicity

Metavana is a new-on-the-scene semantic-analysis vendor whose core science invokes a supposed universal descriptive pattern, the Maximum Information Principle. MIP, Metavana explains, describes the distribution of galaxy sizes and, as exploited by Metavana’s software, the distribution of multi-term, natural-language “n-plets.”

Interesting, but there’s plenty of computational-linguistics and semantic-science mojo in a host of established, competing text and sentiment analysis offerings, developed by smart people. The real question is this one: Does MIP make for great “solutions that measure customer satisfaction,” capable of “taming the chaos of the social Web”?

The answer is unclear, not yet established, despite an argument by authority, that the company was “founded by a renowned physicist,” Dr. Minh Duong-van (who speaks about the science and technology in a long video posted by Dana Stanley at his Research Access site, which Metavana CMO Romi Mahajan blogs for), and given only a couple of customer wins.

One very notable industry partnership does speak in Metavana’s favor, an alliance with Satmetrix to produce a simple, consumable number, a SparkScore, that seeks to lend new life to SatMetrix’ venerable but tired Net Promoter Score (NPS). Metavana, however, is only the latest of several text analysis companies that complement NPS with text-extracted information — Attensity, Clarabridge, Kana, Lexalytics, and others have supported the practice for years — and I even have a speaker lined up on the topic for the next Sentiment Analysis Symposium (which I organize), Bill Tuohig from J.D. Power and Associates. Yet the SatMetrix link is significant and will certainly help Metavana make sales. Read Dana Stanley’s GreenBook blog interview with Satmetrix CEO Richard Owen for detailed background. It’s interesting reading, but do keep in mind that Stanley apparently has a business connection with Metavana has a business association with a Metavana insider.

Metavana’s published materials lack detail that would allow for assessment of company claims, and there’s no testable public interface. Further — a bad sign — Metavana co-founder Spencer Trask, a private-equity firm that is seeking to sell a $6 million Metavana stake, is disseminating information that unfairly slurs competitors and contains a number of factually incorrect statements. But getting back to what can be learned, let’s examine my earlier question, restated —

Is Metavana that innovative?

Metavana’s “search for meaning” technology extracts entities, attributes, topics, and sentiment from text. The company compares its Data Feed offering to Thomson Reuters’ streaming of news and financial information (without providing information on content or coverage) and offers (or will soon offer) an as-a-service sentiment scoring engine, which Erick Watson, Metavana director of product management compared, in a June 25 briefing, to ViralHeat’s. Our briefing centered on Watson’s showing me Metavana’s self-service social-intelligence application. It’s a graphical interface for exploration of social-harvested reviews and opinions, with the ability to associate polar (positive/negative) sentiment to star ratings and hooks for a pending influence measure. See it for yourself in a demo recorded by Watson.

The MIP approach seems quite reminiscent of natural-language understanding efforts that date to the late 1950s. Metavana’s application of the Maximum Information Principle (described in a company white paper) applies the same term-frequency-as-an-indicator-of-significance seen in Hans-Peter Luhn’s 1958 “The Automatic  Creation of Literature Abstracts.” Other, latter-day technologies break text into n-grams, which are multi-word sequences. Metavana uses the words “singlet,” “doublet,” and “triplet” where a linguist would refer to an “n-gram” (n=1, 2, or 3), and then Metavana assembles its n-gram-equivalents into unordered sets, borrowing the term “n-plet” from the physics world of the company’s founding scientist. Frankly, I don’t see a meaningful, practical difference between Metavana’s “n-plets” and the sets of terms that variations of a well-established probabilistic technique, latent semantic analysis, will adduce as typical of different text clusters. Documents that contain “dog,” “cat,” and “goldfish” might typify one cluster while “mustang,” “prius,” and “escalade” typify another.

According to President Michael Tupanjanin, Metavana applies machine-learning technology to refine classifications, and also creates models specific to different business domains such as smartphones, printers, hotels, and airlines. (Tupanjanin presented a 5-minute lightning talk at the May 8, 2012 Sentiment Analysis Symposium.)

There must be nuances to Metavana’s mixture-decomposition approach, but is the company’s approach better? I don’t know, but I do know that it’s not as good as asserted. Michael Tupanjanin states, “the algorithms that we have written have taken accuracy to a whole new level, up to over 95%.” This claim is repeated by founder Dr. Minh Duong-van, who puts Metavana sentiment resolution accuracy is “95, 96 per cent accurate,” in the video I linked to above. It is not supportable. Metavana’s measurement method is flawed, as I described in Never Trust Sentiment Accuracy Claims. Do not accept that 95% figure, but refocusing, let me restate the main point of that article, that —

Accuracy isn’t enough

The insights delivered by any worthwhile data analysis have to be useful and consumable. In my opinion, sentiment analysis that assigns a positive/negative/neutral score, at the document, sentence, or phrase level — that’s what Metavana does — are rarely sufficiently useful. Those crude tools are accurate enough to help you hit the “broad side of the barn” — I’ll give them that much — but they’re of little help when your business decision-making requires guidance that is highly specific, when the broad side of the barn isn’t good enough.

Metavana targets the customer-experience space. Better tools in that space (and in others), for instance from Clarabridge, extract sentiment at the feature level — for names of companies, products, and brands and for features such as, using hospitality as an example, a hotel’s cleanliness, staff friendliness, location, comfort, and value — and further analyze sentiment according to categories such as emotion (e.g., angry, sad, happy), or even in Crimson Hexagon‘s case, categories set up by the end-user business analyst, which may be much more useful than positive/negative assignments.

Metavana’s materials, devoid of detail, do not explain or justify ing the software’s apparent limitations. Given Metavana’s ability to extra topics from documents, why is the company content with cruder sentiment analysis, capable only of sentence- and phrase-level resolution and of extraction of only positive/negative/neutral sentiment? Should it boast about high-precision but crude sentiment analysis when rivals resolve sentiment at the more-granular (and higher-recall) feature level, and of polar sentiment scoring when others can handle arbitrary categorizations, such as emotion categories, that are more business-outcome aligned? But maybe all this detail doesn’t matter. Perhaps the answer is that, so often and in Metavana’s case —

Simplicity sells

You can check out the SatMetrix-Metavana SparkScore via a demo site. According to that site, “The SparkScore Sentiment Engine, powered by Metavana, combines the methodology of Net Promoter with customer sentiments from the social graph to deliver an integrated view of customer experience without the noise.” Metavana compares the SparkScore with the Klout score, which Metavana (very justifiable) derides as limited and simplistic, but which provides an attractively simple mechanism for scoring social influence.

In the end, Metavana’s messaging is similarly, attractively simple: Authority, science, and in SparkScore, a special insights-delivery capability and route to market. Will a reliance on idiosyncratic accuracy claims sell product? Blogger Vikki Chowney of eConsultancy is skeptical is skeptical of SparkScore, and so am I. I further see the apparent crudeness of the Metavana’s sentiment resolution as a competitive disadvantage that may discourage adoption of Metavana’s as-a-service and social-intelligence workbench offering, that is, unless the company fights a price war to wrest the quick-and-dirty market from rivals such as ViralHeat and the plethora of low-end social-intelligence dashboard vendors.

Will a haze of science and authority nonetheless create magic that convinces other skeptics? We’ll see.

Leave a Reply