2.1 Keywords and Expressions
To properly configure your AI, you have to understand how a chatbot analyzes what users enter. We based our NLP on the detection of isolated or associated keywords.
Definition of an Intent:
In the context of Artificial Intelligence, Intent refers to the goal the user has in mind when entering text.
Exemple: If the user enters “I am looking to own a mansion” their Intent is “buying a house”. |
Definition of a Keyword:
This is a word that triggers a response from the chatbot. You have to select words that will be most frequently used by users to express an intent. You have to think about synonyms and sometimes conjugations.
Exemple : “buy”, “purchase”, “acquire” |
Definition of an Expression:
It is a combination of keywords that usually distinguish two similar Intents. When using an Expression in an AI rule, the rule is triggered only if all the keywords for the expression are present. However, the order in which these words were entered by the user or if they are separated by other words does not influence the triggering of the rule.
Exemple: The Expression "buy house" triggers a response even if the user typed "I am looking for a house to buy." |
Definition of an AI rule:
It is a set of Keywords and Expressions corresponding to an Intent that trigger a response from the chatbot.
The chatbot does not understand a whole sentence, but it will recognize certain words. It is by comparing these words with those that have been configured in the rules of his AI that he will be able to associate them with the correct answer.
The selection of the right Keywords and Expressions is therefore essential for an effective AI.
You have to look for the differentiators between the rules and translate the Intents you want your chatbot to understand into relevant Keywords.
Exemple: Imagine a chatbot specializing in the sale (and only the sale) of real estate.
We want the chatbot to react to the desire to buy a house, which is differentiated from the desire to buy an apartment. There are two Intents: the type of transaction (purchase) and the type of property (house).
As this is a sales-centric chatbot, there is no need to detect the type of transaction. It is the type of property that is the differentiator. So all you have to do is make rules about synonyms for house and apartment.
“Buying House” Rule: “Buying Apartment” Rule |
You must always enter Keywords with the correct spelling. Botnation algorithms will handle most typos or plural errors. However, when the user makes too many mistakes in a single word, the chatbot can sometimes fail to understand and refer to the Default Answer Sequence. The misunderstood words will then appear in the Recommendation Engine. Only these errors should be added to your AI rules. We will deal with this point later.
Exemple: the keyword "purchase" will be recognized even when it is written: But it will not be understood if it is written: |
Also don't worry about typographical cases, whether your keywords and phrases are in upper or lower case the algorithm will treat them the same.
Use an Expression only if one of the words is common to another rule.
Be careful, the AI only triggers a rule if and only if all the words of an Expression are contained in the user's sentence. So the more words there are in the Expression, the less likely the user is to use that combination.
Expressions of more than 2 words must be limited to strict necessity.
Do not include pronouns, conjunctions, prepositions, definite articles or contractions: the, it, you, and, or, if, etc.
Exemple: Now imagine our real estate chatbot also wants to offer the sale of property. The chatbot must react to the intent to buy or sell a house or an apartment. That makes four possibilities so you have to create four AI rules. And when we think about it in keywords, we realize that there are going to be words common to several rules. We will not be able to use isolated keywords, we must necessarily make associations to differentiate the intentions.
First, you have to imagine the different ways in which users might express purchase intention: Et pour l’intention de vente on a :
When we isolate the differentiating elements, this gives: For the type of property we have:
Which gives the following rules: “Buying House” Rule “Buying Apartment” Rule “Selling House” Rule “Selling Apartment” Rule |
If in the same rule you have an isolated Keyword that is found in an Expression, this Expression is useless because the rule is triggered anyway as soon as this keyword appears in the sentence.
Exemple: If we have the following rule: “Bank Account Problems” Rule Since we have "banking" and “bank” as single keywords, the rule is triggered when the user enters any sentence containing those words. This includes the sentences containing “banking problem”, “bank account” and “banking ban” and therefore makes these Expressions unnecessary.
In the end we have: “Bank Account Problems” Rule |
Between two conflicting rules, the AI will choose the one that has been used the most statistically and not the one that is potentially the most relevant (see the paragraph on Machine Learning).
Optimizing AI also means using as few rules as possible with as few Keywords and Phrases as possible. The fewer there are, the more responsive the chatbot will be in finding the right answer.
Online help:
2.2 Negative Expressions
Definition:
This is an Expression that excludes a keyword. Just add the - (minus) sign in front of the keyword you want to exclude.
Exemple: “house -brighton” is triggered when the sentence entered contains the Keyword “house” AND does not contain the word “brighton” |
This is a useful feature when you want to distinguish two rules that might overlap.
Exemple: In our real estate chatbot, we want to distinguish between requests for houses with a swimming pool and those without a swimming pool.
We therefore make two rules: “House without swimming pool” Rule “House with swimming pool” Rule The problem here is that “house” is common to both rules. The AI won't know what to choose between the two rules at first. Then with our learning algorithms, the AI will choose the most popular answer of the two (see paragraph on Machine Learning). But then we have a one in two chance of giving the wrong answer.
One of the ways to solve this problem is to use a Negative Expression excluding "pool" from the first rule.
“House without swimming pool” Rule “House with swimming pool” Rule |
When using a Negative Expression, it must be active on all the keywords and phrases in a rule.
Exemple: For our rule about houses without a pool: “House without swimming pool” Rule |
We see here one of the limits of Negative Expressions because if we have two rules with several keywords, we must make all the combinations. In addition, our algorithms will not apply to negative keywords (for reasons of response time optimization) and will not detect input errors on them.
Exemple: For our rule about houses with and without a pool:
“House without swimming pool” Rule “House with swimming pool” Rule |
We will see later a more efficient way on Botnation to resolve this kind of conflict of rules with the Priority Keywords.
We will therefore reserve Negative Expressions for chatbots whose AI is not very complex.
2.3 Priority Keywords (or Exit Keywords)
Definition :
It is a Keyword that will trigger a rule, even if the phrase the user enters contains other keywords (or Expression) that could have triggered other rules. To activate this option on a Keyword, it is preceded by an _ (underscore).
Exemple: If we take the example above and want a rule to be triggered when the Key Word “House” is used alone in a sentence but another rule is triggered when the user enters “I'm looking for a house with a pool". We therefore have the same problem as before but by putting “pool” in Priority Key Word, the solution is really much simpler.
“House without swimming pool” Rule “House with swimming pool” Rule |
The Priority Keywords unfortunately have their limit because they do not work on Expressions. And in a complex chatbot, rather than creating a complicated set of rules, we prefer to use the advanced AI features of Botnation, i.e. Contexts and Tunnels.
Online help: