[This article first appeared in the Clarabridge Bridgepoints newsletter, Q3 2009.]

My take-away from this year’s Fifth Annual Text Analytics summit –held in early June – is that the text-analytics community is more vibrant than ever. (I’m summit chair.) Summit attendance was down only slightly from last year despite harsh economic conditions; my perception is that fewer service start-ups were present, new companies seeking to build media-monitoring and consumer-sentiment services around text-analytics technology. Otherwise, there was a strong end-user presence with some very illuminating case-study presentations. Those presentations, and the always-popular End User Panel (ably moderated by Katey Wood of the 451 Group), were surely helpful to the not-a-few summit attendees who are evaluating text-analytics solutions.

A Strong Market

My impressions of a strong market were affirmed by the findings of a study I conducted this last spring with the support of Clarabridge and other leading vendors. (You can download my study report, “Text Analytics 2009: User Perspectives on Solutions and Providers,” at clarabridge.com.) Fully 73% of respondents who currently use text analytics rated themselves satisfied or better with a dissatisfaction rate of only 4%. (Hurwitz & Associates analyst Fern Halper found in her own 2009 survey that “all of the companies that had deployed text analytics stated that the implementations either met or exceeded their expectations. And, close to 60% stated that text analytics had actually exceeded expectations.”)

My study and the Text Analytics Summit targeted business users (as opposed to researchers and analysts working in life sciences and intelligence and at academic institutions). The summit, my study, and conversations I’ve had with text-analytics users make it clear that solutions are delivering solid business value in a spectrum of business domains. The technology is seen as critical to initiatives that range from social-media analysis – a major theme at this year’s summit – to by-now well-established customer-satisfaction and customer-experience management programs. Further, there’s a growing (and correct) perception that text analytics will be a critical tool in the on-going transformation of media and publishing and in the creation of the emerging Semantic Web. As text-analytics pioneer Ronen Feldman said in 2006, “text analytics [is] driving the Semantic Web.” Content mark-up, the creation of RDF triples and Linked Data publishing, the generation of ontology’s — all the good stuff behind the Semantic Web — is simply too tedious and time-consuming without the information extraction, link analysis, and classification capabilities offered by emerging text-analytics tools.

Measuring Text Mining ROI

The Semantic Web is in the future. Text analytics is delivering business value here and now, nowhere more significantly than in customer experience initiatives. As I reported in my summit opening address and in my study report, my spring survey found that the top four, current or planned text-analytics ROI measures are:

• increased sales to existing customers,
• higher satisfaction ratings,
• improved new-customer acquisition, and
• higher customer retention/lower churn.

Not every user is totally won over, but the users who are tend to be enthusiastic with study comments that include:

“Text analytics allows us to gain new customer and market insights as well as better competitive intelligence: higher report frequency, automated reporting, lower cost, finer granularity.”

“We are highly satisfied. Costs were lower than expected due to high degree of automation. Expectations were exceeded. More timely and more fine grained customer insight and market intelligence and competitive intelligence than ever before.”

“Very satisfied – state-of-the-art in text analytics is advancing at a very rapid pace and text-analytics based solutions are able to demonstrate business value addition/ROI.”

To add perspective, however, I’ll relate a comment made at the summit by Clarabridge’s customer, Chris Jones of Intuit. Chris reinforced that it’s very important to manage user expectations, that text analytics is instrumental in efforts to uncover issues and investigate root causes, but that it’s up to the user to interpret and act on findings.

There are lots more comments, as well as a full set of charts reporting survey findings, in my study report, which, again, you can download at clarabridge.com.

State of the Art

Noting the third comment above: What is the state-of-the-art in text analytics? Applications in social-media analysis, per the very broad summit discussion of that topic, as well as overlapping applications to customer experience are clearly part of that state-of-the-art. A number of technologies underlie those two application areas, most notably search and sentiment analysis.

We were quite fortunate to have summit keynotes from knowledge-discovery (data-mining process) pioneer Usama Fayyad and Univ. of Illinois Professor Bing Liu, who each spoke about leading-edge text applications:

• Mr. Fayyad offered “A Tale of Two Search Engines – The Evolution of Search Technology and the Role of Social Networking in Marketing,” a long title for a big topic with the key thought that text analytics can help search(ers) “get things done.” The point is to extend search beyond information retrieval to a platform for actually conducting business.
• Prof. Liu discussed Sentiment Analysis, the “computational study of opinions, sentiments and emotions expressed in text.” He offered up a great view of enhancements to sentiment that will further expand the accuracy, granularity, and actionability of sentiment within text analytics solutions. Download Presentation

A third technology, question answering, came up at the summit, mostly due to release of the Wolfram Alpha system in early May, a few weeks before the summit. Immediate comparisons were to Google, Yahoo, and other search engines. (Microsoft’s Bing was unveiled just days before the summit and was too new to have occasioned much comment beyond jokes about its having been named for summit speaker Bing Liu.) By the summit, it had become clear that Wolfram Alpha’s applicability is actually fairly narrow, nonetheless, this genre of text-analytics fueled information system is a great demonstration of the technology’s potential to deliver value beyond established business-analytics applications.

The text analytics state-of-the-art is advancing at a very rapid pace, and with plenty of challenges yet to be met. Text analytics is an exciting field to be in!