A bit of pseudo-wisdom misattributed to industrial engineer Edwards Deming says “You can’t manage what you can’t measure.” I’d put it differently and perhaps more correctly: “You can’t manage, or measure, what you can’t model.”
Models can be formal or practical, exact or imperfect, descriptive, predictive, or prescriptive. Whatever adjectives describe the models you apply, those models should derive from observation, with a strong dose of considered judgment, and aim to produce usable insights. Among the most sought-after insights today: Individuals’ attitudes, emotions, and intents.
Scott Nowson is Global Innovation Lead at Xerox, stationed at the Xerox Research Centre Europe. He holds a Ph.D. in informatics from the University of Edinburgh, works in machine learning for document access and translation, and is interested in “personal language analytics,” “a branch of text mining in which the object of analysis is the author of a document rather than the document itself.”
Scott has consented to my interviewing him about his work as a teaser for his presentation at the up-coming LT-Accelerate conference, which I co-organize and which takes place November 23-24, 2015 in Brussels. His topic is customer modelling, generally of “anything about a person that will enable us to provide a more satisfactory customer experience,” and specifically of —
Language Use, Customer Personality, and the Customer Journey
Seth Grimes> You’re the global lead at Xerox Research for customer modeling. What customers, and what about them are you modeling? What data are you using and what insights are you searching for?
Scott Nowson> Xerox has a very large customer care outsourcing business with 2.5 million customer interactions per day, wherein, among other things, we operate contact centres for our clients. So the starting point for our research work in this area is the end-customer: the person who phones a call centre looking for help with a billing inquiry, or who uses social media or web-chat to try to solve a technical issue.
We’re interested in modelling anything about a person that will enable us to provide a more satisfactory customer experience. This includes, for example, automatically determining their level of expertise so that we can deliver a technical solution in the way that’s easiest — and most comfortable — for them to follow: not overly complex for beginners, nor overly simplified for people with experience. Similarly, we want to understand aspects of a customer’s personality and how we can tailor communication with each person to maximise effectiveness. For example, some personality types require reassurance and encouragement, while others will respond to more assertive language in conversations with no “social filler” (e.g. “how are you?”).
We learn from many sources, including social media — which is common in this field. However, we can also learn about people from direct customer care interactions. We are able, for example, to run our analyses in real-time while a customer is chatting with an agent.
There are “customers” in this sense — individuals at the end of a process or service — across many areas of Xerox’s business: transportation, healthcare administration, HR services, to name just a few. So while customer care is our focus right now, this personalisation — this individualised precision — is important to Xerox at many levels.
Seth> Your LT-Accelerate talk, titled “Language Use, Customer Personality, and the Customer Journey,” concerns multi-lingual technology you’ve been developing. Does your solution apply a single modeling approach across multiple languages, then? Could you please say a bit about the technical foundations?
Scott> There are applications for which only low-level processing is required, so that we may use a common, language-agnostic approach, particularly for rapid-prototyping. However, for much of what we do, a much greater understanding of the structure and semantics of language used is required. Xerox, and the European Research Centre in particular, has a long history with multi-lingual natural language processing research and technology. This is where we use our linguistic knowledge and experience to develop solutions which are tuned to specific languages, which can harness their individual affordances. There are languages in which the gender of a speaker/writer is morphologically encoded. In Spanish, for example to say “I am happy” a male would say “Yo estoy contento” whereas a female would say “Yo estoy contenta.” We would overlook this valuable source of information if we merely translated an English model of gender prediction.
On this language-specific feature foundation, the analytics we build on top can be more generally applied. Having a team that is constantly pushing the boundary of machine learning algorithms means that we always have a wide variety of options to use when it comes to the actual modelling of customer attributes. We will conduct experiments and benchmark each approach looking for the best combination of features and models for each task in context.
Is this research work or is it (also) deployed for use in the wild?
The model across the Xerox R&D organization is to drive forward research, and then use the cutting edge techniques we create to develop prototype technology. We will typically then transfer these to one of the business groups within Xerox who will take them to the market. Our customer modelling work can be applied across many businesses within our Xerox services operations, although, as I mentioned customer care is our initial focus. We are currently envisioning a single platform which combines our multiple strands of customer focused research, though we expect to see aspects incorporated into products within the next year. So the advanced customer modelling is currently research, but hopefully running wild soon.
How do you decide which personality characteristics are salient? Does the choice vary by culture, language, data source or context (say a Facebook status update versus an online review), or business purpose?
That’s a good question, and it’s certainly true that not all are salient at one time. Much of the work on computational personality recognition has dealt with the Big 5 — Extraversion, Agreeableness, Neuroticism (as opposed to Emotional Stability), Openness to Experience, and Conscientiousness. This is largely the most well accepted model in Psychology, and has its roots in language use so the relationship with what we do is natural. However, the Big 5 is not the only model: Myers-Briggs types are commonly used in HR, while DiSC is commonly referenced in sales and marketing literature. The use of these in any given situation varies.
We’re currently undertaking a more ethnographically driven program of research to understand which traits would be most suitable in which given situation. Adapting to which traits (or indeed other attributes) will have the most impact on the customer experience.
At the same time, in our recent research we’ve shown that personality projection through language varies across data source. We’ve shown for example, that the language patterns which convey aspects of personality in, say, video blogs, are not the same as in every day conversation. Similarly in different languages, it’s not possible to simply translate cues. This may work in sentiment — you might lose subtlety, but “happy” is a positive word in just about any language — but just as personalities vary between cultures, so do their linguistic indicators.
How do you measure accuracy and effectiveness, and how are you doing on those fronts?
Studies have traditionally divided personality traits — which are scored on a scale — into classes: high-scorers, low-scorers, and often a mid-range class. However, recent efforts such as the 2015 PAN Author profiling challenge have returned the task to regression: calculating where the individual sits on the scale, determining their trait score. We participated in the PAN challenge, and were evaluated on unseen data alongside 20 other teams on four different languages. The ranking was based on mean-squared error, how close our prediction was to the original value. Our performance varied across the languages of the challenge, from 3rd on Dutch Twitter data to 10th on English – on which the top 12 teams scored similarly, which was encouraging. Since submission we’ve continued to make improvements to our approach, using different combinations of feature sets and learning algorithms to significantly lower our training error rate.
Is there a role for human analysts, given that this is an automatic solution? In data selection, model training, results interpretation? Anywhere?
Our view, on both the research and commercial fronts, is that people will always be key to this work. Data preparation for example — labelled data can be difficult to come by when you consider personality. You can’t ask customers to complete long, complex surveys. One alternative approach to data collection is the use of personality perception — wherein the personality labels are judgments made by third parties based on observation of the original individual. This has been shown to strongly relate to “real” self-reported personality, and can be done at a much greater scale. It also makes sense from a customer care perspective: humans are good at forming impressions, and a good agent will try to understand the person with whom they are talking as much as possible. Thus labelling data with perceived personality is a valid approach.
Of course this labelling need not be done by an expert, per se. Typically the judgements are made by completing a typical personality questionnaire but from the point of view of the person being judged. The only real requirement is cultural: there’s no better judge of the personality of, say, a native French speaker than that of another French person.
Subsequently, our approach to modelling is largely data-driven. However, there is considerable requirement on human expertise in the use and deployment of such models. How we interpret the information we have about customers – how we can use this to truly understand them – requires human insight. We have researchers from more psychological and behavioural fields with whom we are working closely. This extends naturally to the training of automated systems in such areas.
We will always require human experts — be they in human behaviour, or in hands-on customer care — to help train our systems, to help them learn.
To what extent do you work with non-textual data, with human images, speech, and video and with behavioral models? What really tough challenges are you facing?
Our focus, particularly in the European labs, has been language use in text. For our purposes this is important because it’s a relatively impoverished source of data. Extra-linguistic information such as speech rate or body language is important in human impression-making. However, one of our driving motivations is supporting human care agents establish relationships with customers when they use increasingly popular text-based mediums such as web chat. It’s harder to connect with customers in the same way as on the phone, and our technology can help this.
However, we are of course looking beyond text. Speech processing is a core part of this, but also other dimensions of social media behaviour, pictures etc. We’re also looking at automatically modelling interests in the same way.
Perhaps our biggest concern in this work is back with our starting point, the customer, and understanding how this work will be perceived and accepted. There is a lot of debate right now around personalization versus privacy, and it’s easy for people to argue “big brother” and the creepiness factor, particularly when you’re modelling at the level of personality. However, studies have shown that people are increasingly comfortable with the notion that their data is being used and in parallel are expecting more personalised services from the brands with which they interact. Our intentions in this space are altruistic — to provide an enriched, personalised customer experience. However, we recognise that it’s not for everyone. Our ethnographic teams I mentioned earlier are also investigating the appropriateness of what we’re doing. By studying human interactions in real situations, in multiple domains and cultures (we have centres around the world) we will understand the when, how, and for whom of personalisation. The bottom line is a seamless quality customer experience, and we don’t want to do anything to ruin that.
Thanks Scott!
Xerox’s Scott Nowson will be speaking at the LT-Accelerate conference, 23-24 November in Brussels. The program features brand, agency, researcher, and solution provider speakers on the application of language technologies — in particular, text, sentiment, and social analytics — to a range of business and governmental challenges. Join us there!