All you need to know about Artificial Intelligence on Botnation: Basic Features

Part 4 of 8

2.1 Keywords and Expressions

To properly configure your AI, you need to understand how a chatbot analyzes what users are typing. We based our NLP on the detection of isolated or associated keywords.

Definition of an Intent:
In the context of Artificial Intelligence, intent refers to the goal that the user has in mind when entering text.

Example :
If the user enters “I am looking to own a villa” his intent is “to buy a house”.

Definition of a Keyword:
It is a word that triggers a response from the chatbot. You need to select words that will be most frequently used by users to express an intention. Think about synonyms and sometimes conjugations.
EX: “to buy”, “purchase”, “to acquire

Definition of an Expression:
It is a combination of keywords that generally distinguish two similar intentions. When an Expression is used in an AI rule, the rule is only triggered if all the keywords in the expression are present. However, the order in which these words were entered by the user or if they are separated by other words does not influence the triggering of the rule.

Example:
The Expression “buy house” triggers a response even if the user entered “I am looking for a house to buy.”

Definition of an AI rule:
It is a set of keywords and Expressions corresponding to an Intent that triggers a response from the chatbot.


The chatbot does not understand a sentence in its entirety but it will recognize some words. It is by comparing these words with those configured in the rules of its AI that it will be able to associate the right answer.

The selection of the right Keywords and Expressions is therefore essential for a successful AI.

You need to look for the differentiating elements between rules and translate the intentions you want your chatbot to understand into relevant keywords.

Example:
Let’s imagine a chatbot specialized in real estate sales(and only sales).


We want the chatbot to react to the desire to buy a house, which we differentiate from the desire to buy an apartment. There are two intentions: the type of transaction (purchase) and the type of property (house).


Since it’s a sales-focused chatbot, we don’t need to detect the type of transaction. It is the type of property that is the differentiator.


Therefore, it is enough to make rules on synonyms of house and apartment.

“Buying House” Rule:

“Buying Apartment” Rule

Always type in the key words with the correct spelling. Botnation’s algorithms will handle typos, accents or plurals. However, when the user makes too many mistakes in the same word, the chatbot may not be able to understand and will return to the Default Response Sequence. The misunderstood words will then appear in the recommendation algorithm. Only these errors should be added to your AI rules. We will deal with this point later.

Example:
the keyword “purchase” will be recognized even when it is written:

  • purrchase
  • purchases
  • murchase

But it will not be understood if it is written:

  • purrshase
  • porchose

Don’t worry about case either, whether your keywords and expressions are in upper or lower case the algorithm will treat them the same.

Use an Expression only if one of the words is common to another rule.

Note that the AI triggers a rule only if all the words of an Expression are contained in the user’s sentence. So the more words in the Expression, the less likely the user is to use that combination.

Expressions of more than 2 words should be limited to strict necessity.

Do not include pronouns, conjunctions, prepositions, definite articles or contractions: the, the, his, the, I, you, he, she, to, from, and, or, nor, or, etc.

Example:
Now let’s imagine that our real estate chatbot also wants to offer property sales. The chatbot must react to the intention to buy or sell a house or an apartment. That’s four possibilities, so we need to create four AI rules.

And when you think about it in terms of keywords, you realize that there are going to be words common to several rules. You can’t use isolated keywords, you have to make associations to differentiate the intentions.

First, we need to imagine the different ways in which users might express purchase intent:

  • I want to buy …
  • I am considering the purchase of ….
  • I would like to be the owner of …

And for the sales intention we have :

  • I want to sell …
  • I am considering the sale of …
  • I sell…

When we isolate the differentiating elements we get :

  • for purchase: buy, purchase, owner
  • for sale: sell, sell, sell.

For the type of property we have :

  • synonyms for house: villa, residence
  • synonyms for apartment: studio.

This gives the following rules:

“Buying House” Rule

“Buying Apartment” Rule

“Selling House” Rule

“Selling Apartment” Rule

If in the same rule you have an isolated keyword that is found in an Expression, this Expression is useless because the rule is triggered anyway as soon as this keyword appears in the sentence.

Example:

If we have the following rule:

“Bank Account Problems” Rule

Since we have “bank” as an isolated keyword, the rule is triggered when the user enters any phrase containing this word. This includes sentences containing ” bankproblem”, ” bankaccount” and ” bankforbidden” and therefore makes these Expressions useless.

So in the end we have:

“Bank Account Problems” Rule

Between two conflicting rules, the AI will choose the one that has been used the most statistically and not the one that is potentially the most relevant (see the paragraph on Machine Learning).

AI optimization also involves using the fewest possible rules with the fewest possible Keywords and Expressions. The less there is, the more responsive the chatbot will be to find the right answer.


2.2 Negative Expressions

Definition:
It is an Expression that excludes a keyword. Simply add a minus sign in front of the keyword you want to exclude.

Example:
“maison -marseille” is triggered when the entered sentence contains the keyword “maison” AND does not contain the word “marseille

This is a useful feature when you want to distinguish between two rules that might overlap.

Example:
In our real estate chatbot, we want to distinguish between requests for houses with pools and those without.

So we make two rules:

“House without swimming pool” Rule

“House with swimming pool” Rule

The problem here is that “house” is common to both rules. The AI will not know what to choose between the two rules at first. Then with our learning algorithms the AI will choose the most popular answer of the two (see paragraph on Machine Learning). But then you have a 50/50 chance of giving the wrong answer.

One way to solve this problem is to use a Negative Expression by excluding “pool” from the first rule.

“House without swimming pool” Rule

“House with swimming pool” Rule

When using a Negative Expression, it must be active on all keywords and phrases in a rule.

Example:
For our no-pool rule:


“House without swimming pool” Rule

We see here one of the limits of the Negative Expressions because if we have two rules with several keywords, we have to make all the combinations. Furthermore, our algorithms will not apply to negative keywords (for response time optimization purposes) and do not detect input errors on them.

Example:
For our rules on homes with and without pools:


“House without swimming pool” Rule

“House with swimming pool” Rule

We will see later on a more efficient way on Botnation to solve this kind of rule conflict with Priority Keywords.

Negative Expressions are therefore reserved for chatbots whose AI is not very complex.


2.3 Priority Keywords (or Outgoing Keywords)

Definition:
It is a Keyword that will trigger a rule, even if the phrase entered by the user contains other keywords that could have triggered other rules. To activate this option on a Keyword, it is preceded by a _ (underscore).

Example:
If we take the example above and want a rule to be triggered when the Keyword “House” is used alone in a sentence but another rule is triggered when the user enters “I am looking for a house with a pool”.

So we have the same problem as before but by putting “pool” in Priority Keyword, the solution is really much simpler.

“House without swimming pool” Rule

“House with swimming pool” Rule

Unfortunately, Priority Keywords have their limits because they do not work on Expressions. And in a complex chatbot, rather than creating a set of complicated rules, one would prefer to use Botnation’s advanced AI features, i.e. Contexts and Tunnels.

Online help:
Exit keywords in NLP


<< Previous part Table of contents Next part >>

Download the guide in PDF format


➜ Discover Botnation and launch your chatbot easily!