Identifying and measuring chatbot KPIs

[cosettej]

Chatbots are hardly a new technology, but their popularity has experienced significant growth over the past few years. The promise of hands-free customer care and internal communication was so enticing that many business leaders jumped the gun on integration when they saw chatbot technology become a trending tool among major corporations. Although chatbots can be a useful and albeit “cool” solution to remedy common bandwidth problems in a business, many companies are experiencing issues with the technology due to a lack of a strategy to accurately measure and objectively evaluate the success of their chatbots before they create them. 27 percent of businesses can’t even find an ISP they like so you can imagine how a hasty decision on something as time intensive as a chatbot could ultimately disappoint.

A previous article I wrote for the SAP Community that touted the use of chatbots for internal communications inspired criticism from a few commenters who’d had bad experiences with chatbots. I wrote a follow-up piece with a few pointers for building a chatbot that “doesn’t suck” to drive home the point that chatbots are only as good as their execution. In retrospect, I think it might have been important to first address the importance of defining clear goals to measure chatbot performance before a team even writes the first line of code for a chatbot.

As an editor over a guest column for bots and AI topics, I often see stories of failed bots that give the technology a bad rap (the publication I guest edit for is VentureBeat). Hopefully, more careful creation and execution of chatbots now will help businesses achieve higher success rates with this technology in the future.

With all of that said, here are some important metrics business leaders should consider as they create and execute chatbot strategies.

User satisfaction

This metric begs a simple question: Do users even like the bot? Sure, there are growing pains with every new technology, but this doesn’t mean companies should ignore user feedback during the adoption period.

Ask users to rate their experience with the bot following their interactions. Pay attention to the ratings and leave room for comments to identify particular pain points. Listen to your users and make adjustments where possible to provide a better experience.

Return visits

Introducing chatbot technology will help you streamline communication, but the potential bandwidth savings won’t be worth losing the interest of your customers and/or employees. This is why it’s important to check your retention rate. How many users are returning to your bot after their first interaction? Keep track of the percentage of users who come back to use your bot within a given period of time after their first use.

Accuracy

Acquiring users is one thing, but making sure the users utilize the bot for its intended purpose is key. This is where the activation metric comes in. Does the user respond to your bot’s opening message with relevant questions? If not, how can you adjust the message to make sure users understand how to use the bot?

Pain points

As mentioned in the first point, it will be important to understand where things get hairy with your bot’s user experience. You could chalk slow adoption up to users’ apprehension to change, but there will likely be at least a few bugs you can fix to make the process run more smoothly following your launch date.

Check your analytics to find out where users drop off. Dive in to find out where the bot gets confused and identify ways to remedy that.

Goal completion

What is the call to action (CTA) you have in mind for your chatbot? Do you want the bot to complete an entire sale or simply field users to the appropriate service department? Define one or two clear, ultimate goals you want your chatbot to achieve and track them. This will help you identify how successful your bot really is in helping your business increase its bottom line.

Honest reflection

This isn’t a proper KPI, but it’s worth noting that you should go into the creation of your chatbot ready to make significant changes or even scrap the whole idea if it doesn’t work. Be honest with yourself and ask others at your company who were not involved in the creation of the bot to provide honest feedback from an unbiased perspective.

It can be hard to admit that a project you’ve worked on isn’t going to plan, but you can use these KPIs and outside perspectives to keep yourself in check and make the best decision for your business should the bot end up being a dud.

Putting your plan into action

Hopefully, these tips will help you define clear KPIs for your chatbot strategy before you dive head first into creating a bot to field customer service questions or streamline internal communications. A few options for platforms you can check out to measure your analytics are Google’s Chatbase, Botanalytics, Dashbot, and of course, Facebook Analytics for Messenger bots. Each offer different benefits for different use cases, so be sure to do your research before choosing an analytics platform to track your chatbot strategy.

As with any integration of new technology, chatbot success rides on the creation of an effective strategy beforehand. Although chatbots might seem like a shiny new technology worth diving right into, treat them like you would any other business decision and take your time to understand your goals for the bot and how you will achieve them before you start development.