Treating consumers as individuals is a problem for AI to solve
In the startup world building new technology first and looking for a problem to solve with it after is a common mistake. Before I co-founded Automat, I got my PhD in Computer Science and had worked as a researcher at various AI labs, including at Nuance the world leader in speech recognition where I collaborated with IBM on Watson. Over a decade of such experience, I began to get frustrated by working on isolated toy problems that were evaluated on made-up data sets. This left me feeling disconnected from making an impact and solving real problems, and I knew something had to change.
All of this led me to make the jump into the startup world and build a research team whose mantra is “the best researchers fall in love with problems, not with solutions”. Not your typical research lab, I’m proud to say we’re squarely focused on developing technology that works in the real-world under real-world conditions and has a real impact for actual end-users.
Personalized conversations at scale – a problem worth solving
As my co-founder Greg shared in a recent blog post, there’s a lot of power in getting to know your customers directly rather than making assumptions about them based on spying on their online behavior. At Automat, we think the best way for brands to learn about and build relationships with their customers is by having conversations with them over messaging channels. Having personalized conversations with each consumer opens the door to getting to know them as an individual, learn what they care about, and help them find the products, services, and content they want and need. It can also avoid wasted marketing spending on substandard customer communications – collectively billions of dollars thrown away across the industry.
But, from a technology perspective, how do you make personalized conversations with customer possible at scale? In response to Greg’s post, a former colleague commented on LinkedIn: “I think you guys are on to something really important. The piece you didn’t address in the article is the issue that happens when marketers start to use this level of data gathering…the complexity of their campaigns goes through the roof.” Of course, he’s absolutely right, the effort required to identify hyper-personalized groups of consumers and talk to them in the way they prefer isn’t really possible to achieve manually. Obviously this is where AI comes into play, but first let’s understand the problem a little better.
Three key challenges to address for marketers to make personalized conversations a reality
I believe in taking a first principles approach to problem solving. As we’ve thought deeply about the challenges that marketers face while trying to implement personalized conversations at scale, we distilled things down to three simple questions that every marketer needs to answer:
- Who should I talk to?
- How should I talk to them?
- When is the best time to talk to them?
At face value, these appear to be relatively straightforward questions. But, once you start to collect more than a few pieces of information about your customers, the amount of work required to group them, build customized campaigns, and schedule optimal outreach times becomes more than mere mortals can handle on their own, no matter how talented they are. Without getting into the math behind combinatorics, I’ll more generally observe that most people lack intuition around complexity growth models and totally underestimate the impact of combinatorial explosion.
As one simple example, if you’re a marketer who knows 10 things about consumers which each have combinations of 5 possible values, you would need to analyze over 20,000 unique segments. If you only knew 250 things about each individual consumer, the combinations of all these consumer insights would be comparable to the number of atoms in the universe. Clearly that’s more complexity than any of us can deal with on our own without the help of technology.
#1. Who should I talk to?
The first step in personalization is identifying consumer segments so marketers can understand different profiles and behaviours of the target audience. Deep understanding of segments that have responded positively to previous conversations (either by purchasing something, providing positive feedback, etc) can help figure out a content strategy that works for a given consumer segment and can either re-engage them in similar ways or acquire new consumers that are similar. On the other hand, understanding the profiles of consumers who didn’t respond as positively may help identify new opportunities for other forms of engagement.
In traditional web and email marketing, this is usually done with customer relationship tools by filtering consumers based on monitoring their online movements, search history and social media activity. While the current vendors providing these tools would have you believe that their segmentation and targeting is very precise, we know from our own consumer experience with poorly targeted display ads, spam email, and other overly broad marketing that we are faced with every day that the current approaches don’t really segment consumers that well. And how could they, our online behavior doesn’t define us fully or completely as people.
In response to this key challenge, Automat’s research team developed a set of tools that use a series of unsupervised machine learning and clustering techniques to analyze tens of thousands of micro-segments to distill out the most important consumer attributes that will help marketers achieve a range of different marketing objectives. Importantly these micro-segments are based on information obtained when talking to a consumer not inferring who they are by watching their web usage. Using our platform, marketers can systematically analyze the evolving nature of consumer behavior and quickly decide who they should converse with in order to optimize key performance indicators.
#2. How should I talk to them?
Communication is the art of transmitting information from one person to another. Conversations provide a very rich medium to do that. Compared to email and web sites that are based mostly on static layouts and one-size-fits-all content, conversations allow bi-directional interactions to personalize the communication at each turn. Of course, that opportunity intensifies the importance of defining the right way to talk to each consumer to ensure successful outcomes.
A/B testing is the traditional method for testing a handful of different layouts with consumers to optimize a given success metric. Conversations are very different from web interfaces in three major ways:
- Aiming for multiple, possibly competing objectives. The success of a conversation is hard to pin down to a single metric. For instance, driving towards a sale could motivate a short and focused conversation. At the same time, the user might benefit from longer conversations to learn more and receive recommendations over time to keep them motivated. Delivering a conversation that achieves these two competing goals is a hard task that requires multi-objective optimization.
- Exploiting the sequential nature of conversations. Conversations are by their nature bi-directional and multi-turn interactions. That means that success can only be measured after having a series of interactions with the consumer where each interaction may have a positive or negative effect. In addition, not every interaction has the same weight of importance on the result. For instance, the selection of the salutation form (“hi”, “hello”, “what’s up?”) may not be as important as the quality of recommendations that are made. As a result, simple statistics that treat each part of the conversation as equally important aren’t helpful enough to identify the best way to talk to a person overall.
- Knowing who you’re talking to. Chatbots always know who they are talking to. That is a huge benefit compared to websites where a lot of browsing and interactions are anonymous or pre-login. As a result, chatbots have greater opportunities for personalization throughout the entire customer journey from beginning to end.
Exploiting the full potential of conversations to optimize marketing efforts requires exploring multiple variations of interactions with consumers, understanding the impact of every interaction with respect to the marketing goals, and learning which ones work best for which segments. To solve that problem, our product leverages reinforcement learning, a machine learning technology that is largely used today to teach robots how to accomplish complicated tasks, to allow every conversational variation to run as a self-optimizing experiment.
#3. When is the best time to talk to them?
Reaching out at the right moment is as important as the content you show to increase the chances of getting a consumer’s complete attention. The asynchronous nature of emails reduces the tension on timing. We usually read our emails when we have time. Messaging is more about taking small actions synchronously to accomplish longer tasks asynchronously in multiple turns. In that sense, sending a message to a consumer is closer to calling them to ask a few questions. If it is done at a bad moment, the consumer may just ignore it and the message can get lost among other missed notifications.
To address this final challenge for marketers, our research team is using game theory to learn when is the best moment to contact every consumer based on past interactions. This technology allows marketers to send personalized messages and rely on the system to deliver them at the best time of the day.
AI isn’t always the right solution… but it is in this case
At Automat, we see AI as a pragmatic tool, not magic. We want our customers to understand the problem it solves for them and we aim to educate our customers, not confuse them with buzzwords. In that regard and as the leader of an AI research team, I feel it’s my responsibility to point out that artificial intelligence, despite all the hype, isn’t always the right or best solution to every problem. There are still plenty of problems that are better solved with simpler or more traditional approaches. For example, we don’t yet advocate the use of AI to generate brand content, we still think that’s a job for creative humans. AI can help you figure out who to talk to, help optimize which content is sent to which consumer, and fully automate sending the messages at the right time, but brand still matters and we aim to amplify what human marketers can do, not replace it. This mission is something that will keep my team and I engaged for the foreseeable future motivating us to keep developing new AI technologies to serve marketers. What a great problem to have.