Adding custom data to an NLU model

Intents and entities

Intent is a goal or purpose that the user expresses in their utterance, it represents an action that the user wants to perform. The same intent can be expressed by the user in many different ways, or phrases. These phrases are called utterances.

Entity represents values collected from the user in a conversation, so this is basically a keyword that you want to extract from the user’s utterance to resolve their query.

When you are creating a model with a specific domain, the out-of-the-box intents and entities are loaded automatically, if available. Your model will use these elements to understand the natural language input and direct the conversation flow.

The out-of-the-box data are labeled as RB-XP and cannot be deleted.

If needed, you can disable selected out-of-the-box intents by switching off the toggle as shown below.

Adding custom data

Besides using the existing resources, you may also want to extend your model’s understanding with your custom intents and entities. Also, if you are creating a custom domain model, you will have to add your own data from scratch. Either way, you can find the step-by-step instructions below.

Uploading custom data

You can upload your Custom Training Data using TXT, CSV, or TSV file format making sure the encoding is UTF-8 and that the line breaks are Unix/Linux style. Below in the text you can find an example on how the file’s content should look like.

To upload your custom data, proceed as follows:

1. Navigate to NLU → NLU Models tab.

2. Select a model and click on it. The model drill-down page opens.

3. Click the Upload icon.

4. In the opened dialog box, upload a file in TXT, CSV, or TSV by dropping it directly or clicking Browse Files to select one.

Below you can see an example of the training set structure:

I don't need my account anymore;Close.Account
i have a question;Inquire
When is the new model coming out;Inquire.Availability
I want to change my username.;Update.Account
My credit card was declined. Can you rerun it;Trouble.Payment

5. Click Upload. The dataset is uploaded, and the intents and utterances are now added to your model.

Adding intents

The intent is the goal of the user's request to the application that describes what the end-user wants. For example:

Airline Virtual Assistant: Welcome to ABC Airlines. What can I do for you today?
User: I'd like to get a ticket from New York City to Jamaica for next Friday.

In the example above, the user wants to buy a ticket, so the application will analyze the request and select an intent from the list of intents. In this case, it would be something like Flight_Ticket-Purchase.

To add an additional intent to your model, proceed as follows:

1. Navigate to NLU → NLU Models section.

2. Select a model and click on it. The model drill-down page opens.

3. Go to the Intents tab and click the Plus icon.

4. The following dialog box opens. Enter an intent name.

The intent name must start with a letter and contain no spaces. It can consist only of letters, dots, hyphens, or underscores. Symbols must be followed by a letter.

5. Click Create to confirm. The intent has been added.

The created intent is highlighted in red. The red Exclamation sign next to the Utterances list informs you that no or not enough utterances have been added to the intent.
Models with intents that are highlighted in red cannot be trained! Add utterances to your intent to be able to train your mode.

• To edit an intent’s name, click the Pencil icon.

• To delete an intent, click the Delete icon.

You can only delete custom intents. The out-of-the-box intents cannot be deleted or edited.

Adding utterances

The same intent can be expressed by the user in many different ways, or phrases. These phrases are called utterances.

You need to add at least five training utterances for each intent. Ideally, ten or more utterances should be added. The more utterances you add, the better the result will be.

To add utterances to an intent, follow the steps below:

1. Select an intent and click on it.
   

   

    

2. Enter an utterance into the highlighted input field.

3. Press <Enter> on your keyboard to add the utterance to the corresponding intent. The Exclamation sign next to Utterances will change its color from red to yellow depending on their number:

   • Red: Less than five utterances have been added

   • Yellow: Less than ten utterances have been added.

As soon as ten or more utterances are added, the exclamation sign will disappear and the intent will not be highlighted anymore.

To delete an utterance, hover over it and click the Delete icon.

Adding entities

Entities allow an application to understand the meaning of natural language text and extract relevant information for further processing.

An entity is a specific piece of information or data within a sentence that represents a real-world object, such as a person, place, thing, event, action, concept, and so on. It’s a label we use to shelter words or small phrases under the same umbrella. For example, green, orange, blue, purple, yellow, red are colors. Visa, MasterCard, American Express are card networks.

Custom domain models do not support entities.

To add an entity, follow the steps below:

1. Navigate to NLU → NLU Models section.

2. Select a model and click on it. The model drill-down page opens.

3. Go to the Entities tab and click the Plus icon.

4. The following dialog box opens. Enter an entity name.

The entity name must start with a letter and contain no spaces. It can consist only of letters.

5. Click Create to confirm. The entity has been added. The added entities are labeled as CUSTOM.

Adding values and alternatives

Sometimes, we can use different words for the same meaning. This means that several words under the same entity can have the same value. For example, cat, kitten, dog, canine, puppy, hen, duck, chicken are animals. As you can see below, dog, canine and puppy are different alternatives of the same value.

For each entity, you can add multiple values and multiple alternatives for each value. All alternatives (global and context ones) are mapped to the reference value that you’ve defined. If you do not want to declare a reference value and you just want the entity to be triggered based on your alternatives, just leave it blank.

To add a reference value and global alternatives to the entity, follow the steps below:

1. Select an entity and click on it.

2. Click the + Add Reference value and Alternatives button.

3. Enter the reference value.

4. Enter a global alternative and press <Enter> on the keyboard. You can add multiple alternatives to one value.

For alternatives, you can use single words or short phrases.

5. When you are finished, press <Enter> on the keyboard again. The value and alternatives are added.

To delete a reference value, hover over it and click the Delete icon.

To delete an alternative, double-click it and then click the Delete icon.

You can only delete your custom values and alternatives. The out-of-the-box elements cannot be deleted.

Adding contexts

Context makes a conversation more informal and human-like. It adds specific details to the dialog that help clarify the meaning.

For example, “credit” is a generic term that means debt. However, if the question is “Is it a debit or a credit card?”, then the meaning of the word “credit” is a “card type”. In the same context, if a user says “the second”, this would again mean a card type. However, without context, this is just an ordinal number.

To add a context, proceed as described below.

1. Click the Contexts button.

2. Click the Plus icon.

3. The dialog box opens. Enter the context name and tag. The context name must start with a letter and contain no spaces. It can consist only of letters, dots, hyphens, or underscores.

4. Confirm by clicking the Create button. The context has been added.

You can add a custom NLU tag in the corresponding text box in your miniApp. In this case, if a user uses one of the alternative words added under the context that has this tag, these words will get the related value.

Adding context alternatives

Context alternatives are words or short phrases that a customer may use under a specific context. These context alternatives are mapped to the declared reference value or just trigger the entity if no reference value exists, ONLY when the selected context is applicable.

To add a context alternative, proceed as described below:

1. Select a value and click the Arrow button.

2. Click the Add Context Alternatives button.

3. Add a context alternative and press <Enter> on the keyboard. You can add multiple alternatives to one value.

For context alternatives, you can use single words, short phrases, or regular expressions.

4. Select a context from the drop-down list of previously created contexts.

5. The context alternative has been added. To add another context alternative, click the Plus icon.

To delete a context alternatives block, click the Delete button.

Deleting contexts

To delete a context, you should use the search function to first locate where this context is being used. You can search for a context in the Entities list or in the Values list per entity.

Before deleting a context, you should delete all context alternatives that are linked to it or map them to a different context.

To find and deleted a context, proceed as described below:

1. Navigate to the Entities tab and click the Search icon.

2. Enter the context name into the search field and press <Enter> on the keyboard. You will see all the entities that use this context.

3. Select an entity and use the search function in the Values list for the entity. The search result is highlighted.

4. Click the Delete icon to delete the context alternatives linked to the context you searched for. Alternatively, you can map them to a different context from the drop-down list of previously created contexts.