Next month’s LT-Accelerate conference will be the third occasion I’ve invited Lipika Dey to speak at a conference I’ve organized. She’s that interesting a speaker. One talk was on Goal-driven Sentiment Analysis, a second on Fusing Sentiment and BI to Obtain Customer/Retail Insight. (You’ll find video of the latter talk embedded at the end of this article.) Next month, at LT-Accelerate in Brussels, she’ll be speaking on a particular topic that’s actually of quite broad concern, E-mail Analytics for Customer Support Centres.
As part of the conference lead-up, I interviewed Lipika regarding consumer and market analytics, and — given her research and consulting background — techniques that best extract practical, usable insights from text and social data. What follows are a brief bio and then the full text of our exchange.
Dr. Lipika Dey is a senior consultant and principal scientist at Tata Consultancy Services (TCS), India with over 20 years of experience in academic and industrial R&D. Her research interests are in content analytics from social media and news, social network analytics, predictive modeling, sentiment analysis and opinion mining, and semantic search of enterprise content. She is keenly interested in developing analytical frameworks for integrated analysis of unstructured and structured data.
Lipika was formerly a faculty member in the Department of Mathematics at the Indian Institute of Technology, Delhi, from 1995 to 2006. She has published in international journals and refereed conference proceedings. Lipika has a Ph.D. in Computer Science and Engineering, M.Tech in Computer Science and Data Processing, and 5 Year Integrated M.Sc in Mathematics from IIT Kharagpur.
Our interview with Lipika Dey —
Q1: The topic of this Q&A is consumer and market insight. What’s your personal background and your current work role, as they relate to these domains?
Lipika Dey: I head the research sub-area of Web Intelligence and Text Mining at Innovation Labs, Delhi of Tata Consultancy Services. Throughout my academic and a research career, I have worked in the areas of data mining, text mining and information retrieval. My current interests are focused towards seamless integration of business intelligence and multi-structured predictive analytics that can reliably and gainfully use information from multitude of sources for business insights and strategic planning.
Q2: What roles do you see for text and social analyses, as part of comprehensive insight analytics, in understanding and aggregating market voices?
Lipika Dey: The role of text in insight analytics can be hardly over-emphasized.
Digital transformation has shifted control of the consumer world to consumers from providers. Consumers — both actual and potential — are demanding, buying, reviewing, criticising, influencing others, and thereby controlling the market. The decreasing cost of smart gadgets is ensuring that all this is not just for the elite and tech-savvy. Ease of communicating in local languages on these gadgets is also a contributing factor to the increased user base and increased content generation.
News channels and other traditional information sources have also adopted social media for information dissemination, thereby paving the way for study of people’s reactions to policies and regulations.
With so much expressed and exchanged all over the world, it is hard to ignore content and interaction data to gather insights.
Q3: Are there particular tools or methods you favor? How do you ensure business-outcome alignment?
Lipika Dey: My personal favourites for text analytics are statistical methods and imprecise reasoning techniques used in conjunction with domain and business ontologies for interpretation and insight generation. Statistical methods are language agnostic and ideal for handling noisy text. Text inherently is not amenable to be used within a crisp reasoning framework. Hence use of imprecise representation and reasoning methodologies based on fuzzy sets or rough sets is ideal for reasoning with text inputs.
The most crucial aspect for text analytics based applications is interpretation of results and insight generation. I strongly believe in interactive analytics platforms that can aid a human analyst comprehend and validate the results. Ability to create and modify business ontology with ease and view the content or results from different perspectives is also crucial for successful adoption of a text analytics based application. Business intelligence is far too entrenched in dashboard-driven analytics at the moment. It is difficult to switch the mind-set of a whole group at once. Thus text analytics at this moment is simply used as a way to structure the content to generate numbers for some pre-defined parameters. A large volume of information which could be potentially used is therefore ignored. One possible way to practically enrich business intelligence with information gathered from text is to choose “analytics as a service” rather than look for a tool.
As a researcher I find this the most exciting phase in the history of text analytics. I see a lot of potential in the yet unused aspects of text for insight generation. We are at the confluence where surface level analytics has seen a fair degree of success. The challenge now is to dive below the surface and understand intentions, attitudes, influences, etc. from stand-alone or communications text. Dealing with ever-evolving language patterns that are also in turn influenced by the underlying gadgets through which content is generated just adds to the complexity.
Q4: A number of industry analysts and solution providers talk about omni-channel analytics and unified customer experience. Do you have any thoughts to share on working across the variety of interaction channels?
Lipika Dey: Yes, we see many business organizations actively moving towards unified customer experience. Omni-channel analytics is catching up. But truly speaking I think at this point of time it is an aspirational capabilty. A lot of information is being pushed. Some of it is contextual. But I am not sure whether the industry is still in a position to measure its effectiveness or for that matter use it to its full potential.
It is true that a multitude of sources help in generating a more comprehensive view of a consumer, both as an individual as well as a social being. Interestingly, as data is growing bigger and bigger, technology is enabling organizations to focus on smaller and smaller groups, almost to the point of catering to individuals.
As a researcher I see exciting possibilities to work in new directions. My personal view is that the success of omni-channel analytics will depend on the capability of data scientists to amalgamate domain knowledge and business knowledge with loads and loads of information gathered about socio-cultural, demographic, psychological and behavioural factors of target customers. Traditional mining tools and technologies will play a big role, but I envisage an even greater role for reasoning platforms which will help analysts play around with information in a predictive environment, pick and choose conditional variables, perform what-if analysis, juggle around with possible alternatives and come up with actionable insights. The possibilities are endless.
Q5: To what extent does your work involve sentiment and subjective information?
Lipika Dey: My work is to guide unstructured text analytics research for insight generation. Sentiments are a part of the insights generated.
The focus of our research is to develop methods for analysing different types of text, mostly consumer generated, to not only understand customer delights and pain-points but also to discover the underlying process lacunae and bottlenecks that are responsible for the pain-points. These are crucial insights for an enterprise. Most often the root cause analysis involves overlaying the text analytics results with other types of information available in the form of business rules, enterprise resource directory, information exchange network etc. for generating actionable insights. Finally it also includes strategizing to involve business teams to evaluate insights and convert the insights into business actions with appropriate computation of ROI.
Q6: How do you recommend dealing with high-volume, high-velocity, diverse data — to ensure that analyses draw on the most complete and relevant data available and deliver the most accurate results possible?
Lipika Dey: Tata Consultancy Services has conducted several surveys across industry over the last two years to understand organizational big data requirements. The findings are published in several reports available online. (See the Tata Consultancy Services Web site, under the Digital Enterprise theme.) One of the key findings from these surveys was that many business leaders saw the impending digital transformation as siloed components affecting only certain parts of the organization. We believe that this is a critical error.
The digital revolution that is responsible for high volumes of diverse data arriving at high velocity does not impact only a few parts of business — it affects almost every aspect. Thus our primary recommendation is to harness a holistic view of the enterprise that encompasses both technology and culture. Our focus is to help organizations achieve total digital transformation through an integrated approach that spans sales, customer service, marketing, and human resources, affecting the entire universe of business operations. The message is this: Business processes need to be rethought. The task at hand is to predict and prioritize the most likely and extreme areas of impact.
Q7: So what are the elements of that rethinking and that prioritization?
Lipika Dey: We urge our clients to consider the four major technology shifters under one umbrella. Big data initiatives should operate in tandem with social-media strategy, mobility plans, and cloud computing initiatives. I’ve talked about big data. The others —
Social media has tremendous potential for changing both business-to-business and business-to-consumer engagement. It is also a powerful way to build “crowdsourcing” solutions among partners in an ecosystem. Moving beyond traditional sales and services, social media also has tremendous role in supply-chain and human resource management.
Mobile apps are here to transform the way business operated for ages. They are also all set to change the way employees use organizational resources. Thus there is a pressure to rethink business rules and processes.
There will also soon be a need for complete infrastructure revision to ward off the strains imposed in meeting data needs. While cloud computing initiatives are on the rise, we still see them signed up by departments rather than enterprises. The fact that cloud offerings are typically paid for by subscription makes them economical when signed up by enterprises.
Having said that we also believe there is no “one size fits all” strategy. Enterprises may need to redesign their workplaces where business will work closely with IT to redesign its products and services, mechanisms for communicating with customers, partners, vendors and employees, business models and business processes.
Q8: Could you say more about data and analytical challenges?
Lipika Dey: The greatest challenges while dealing with unstructured data analytics for an enterprise is to measure accuracy, especially in absence of ground truths and also effectiveness of measures taken. To check effectiveness of actionable insights, one possibility is to use the A/B testing approach. It is a great way to understand the target audience and evaluate different options. We also feel it is always better to start with internal data — something that is assumed to be intuitively understood. If results match known results, well and good — your faith in the chosen methods increase. If they don’t match — explore, validate and then try out other alternatives, if not satisfied.
Q9: Could you provide an example (or two) that illustrates really well what your organization and clients have been able to accomplish via analytics, that demonstrate strong ROI?
Lipika Dey: I will describe two case studies. In the first one, one of our clients wanted to analyze calls received over a particular at their toll-free call-center. These calls were of unusually high duration. The aim was to reduce operational cost for running the call center without compromising on customer satisfaction. The calls were transcribed into text. Analysis of the calls revealed several insights that could be immediately transformed into actionable insights. The different types of analyses carried out and insights revealed were broadly categorized into different buckets as follows:
(a) Content based analysis identified that these calls contained queries pertaining to existing customer accounts, queries about new products or services, status updates about transactions, and eventually requests for documents.
(b) Structural analysis revealed that each call requested multiple services and for different clients, which eventually led to several context switches for search of information, thereby leading to high duration. It also revealed that calls often landed at wrong points and had to be redirected several times before they could be answered.
Based on the above findings, a restructuring of the underlying processes and call-center operations were suggested with an estimated ROI based on projected reduction in number of calls requesting for status updates or documents to be dispatched etc. based on available statistics.
In the second case study, analysis of customer communications for the call-center of an international financial institution, done periodically over an extended period, revealed several interesting insights about how customer satisfaction could be increased from their current levels. The bank wished to obtain aggregated customer sentiments around a fixed set attributes related to their products, staff, operating environment, etc. We provided those, and the analysis also revealed several dissatisfaction root causes that were not captured in the fixed set of parameters. Several of these issues were not even within the bank’s control since those were obtained as external services. We correlated sentiment trends for different attributes with changes in customer satisfaction index to verify correctness of actions taken.
In this case, strict monetary returns were not computed. Unlike in retail, computing ROI for financial organizations require long-term vision, strategizing, investment and monitoring of text analytics activities.
Q10: I’m glad you’ll be speaking at LT-Accelerate. Your talk is titled “E-mail Analytics for Customer Support Centres — Gathering Insights about Support Activities, Bottlenecks and Remedies.” That’s a pretty descriptive title, but is there anything you’d like to add by way of a preview?
Lipika Dey: A support centre is the face of an organization to its customers and emails remain the life-line of support centres for many organizations. Hence organizations spend a lot of money on running these centres efficiently and effectively. But unlike other log-based complaint resolution systems, when all communication within the organization and with the customers occur through emails, analytics becomes difficult. That’s because a lot of relevant information about the type of problems logged, the resolution times, the compliance factors, the resolution process, etc. remains embedded within the messages and that too not in a straight forward way.
In this presentation we shall highlight some of the key analytical features that can generate interesting performance indicators for a support centre. These indicators can in turn be used to measure compliance factors and also characterize group-wise problem resolution process, inherent process complexities and activity patterns leading to bottlenecks — thereby allowing support centers to reorganize their mechanisms. It also supports a predictive model to incorporate early warnings and outage prevention.
Thanks Lipika, for sharing insights in this interview and in advance for your December presentation.