With A/B Testing, measure and optimize your chatbot’s actions
1. What is A/B Testing
A/B testing consists in comparing several versions of an image, a video, a text, a button label in order to verify which one is the most efficient. These variations are presented in linear or random ways to your chatbot users.
Botnation offers 3 rotation modes:
- Linear: the elements rotate in a loop always in the same order. Recommended from 2 elements.
- Random: the elements are displayed randomly. Recommended from 3 elements.
- AI Optimized: Botnation’s Machine Learning automatically displays the best performing element based on its powerful learning algorithm to maximize your button actions.
A real-time statistical analysis allows you to see the effectiveness of the different versions according to the rate of clicks on actions (buttons). In other words, you can check which version triggers the most clicks, subscriptions, purchases, etc. The results then help determine the best strategy to adopt.
In this example we see that :
- the menu has been displayed 10 times in your chatbot
- the buttons were clicked 6 times (so in 60% of the cases)
- the “Choice 1” button was clicked in 10% of the cases
- the “Choice 2” button was clicked in 50% of the cases
So we can easily deduce that the “Choice 2” button has a much better performance than the first button, it would deserve to be more prominent or the first button would deserve a new and more “catchy” title.
2. Which elements to use in A/B testing?
- The texts
- The images
- The videos
- and especially the wording of the buttons and quick replies, because these are the ones that allow you to measure the action rate.
However, Botnation’s A/B testing allows you to test a much larger number of combinations, for which you need to couple the AB testing statistics to the rotations.
Rotations allow you to add several elements of the same type (text, image, video, etc.) in alternation. Depending on the type of rotation (linear, random or automatically optimized), Botnation will choose one of the elements to send to the user of your chatbot.
In this example we see that:
- Botnation will display every second version of this menu linearly
- That the first version always generates 60% of actions (clicks) on the buttons
- That the second version generates 75% of shares!
- That the wording “This way” of the first button of the second version also generates 50 actions
We can therefore conclude that a 3rd version integrating the wording “This way” on the first button and the wording “Choice 2” on the second button would eventually generate many more actions.
And indeed it is confirmed this 3rd version of the menu now generates 88% of actions on the buttons thanks to an optimization linked to Botnation’s real time A/B testing statistics and the possibility to rotate several elements.
3. Take advantage of Machine Learning on your A/B Testing
Botnation’s Machine Learning automatically displays the best performing item based on its powerful learning algorithm to maximize your button actions.
The AI Optimizer will prioritize the version that triggers the most clicks, subscriptions, purchases, etc… The Optimizer determines in real time the best strategy to adopt instead of selecting an element at random.
To do this, simply activate the “Optimized” option in the rotation of your elements.
BOTNATION ‘s goal is to help you make your chatbot more efficient. TheAI Optimizer will be efficient from the first second because it has already trained on the most popular chatbots made on Botnation having generated more than 50 Million clicks on action buttons!
So feel free to create many variations of your Buttons, Quickreplies, and Carousels elements.
4. Reset A/B Testing
Once your measurements are saved, you may need to reset them. For example, if you completely change your chatbot after the first few returns, the old results may no longer be relevant.
In this case, go to Settings then Advanced Settings, at the bottom right, the block “Reset AB testing” allows you to reset the counters to zero. To do this, click on “Permanently delete AB Testing data”.
⚠️ Beware, this is irreversible!
We wish you good optimizations of your chatbot on Botnation!