.svg){kind=logo}
A special permission is required for this feature. Contact Omilia Support Team for details.
OCP NLU Service provides the option to apply custom Intent Selection Policy logic for NLU models. This logic will be applied to annotation responses, in order to combine results from both Rule-Based and Machine Learning and eventually point out which the winning Intent is between the two technologies.
To access the Intent Selection Policy settings:
-
From the OCP® main menu, navigate to NLU > NLU Models.
-
Select the NLU model for which you want to configure the policy.
-
Click the Settings tab.
-
Select the Intent Selection Policy sub-tab.
Here you can upload your Intent Selection Policy file.
Structure
The Intent Selection Policy can be provided to models only as an XML file.
Example for reference:
<IntentFusion type="rb">
<FusionPolicy priority="3" selectSource="rb">
<FusionRuleGroup logicalOperator="AND">
<FusionRule source="dm" item="SystemAction" attribute="type" value="Disambiguation"/>
<FusionRule source="rb" attribute="name" value="Appointment"/>
</FusionRuleGroup>
</FusionPolicy>
<FusionPolicy priority="2" selectSource="ml">
<FusionRuleGroup logicalOperator="AND">
<FusionRule source="ml" attribute="confidenceThreshold" value="0.65"/>
</FusionRuleGroup>
</FusionPolicy>
</IntentFusion>
For the example above there are 3 outcomes:
-
The policy with priority 2 will be applicable if the Intent coming from the ML-NLU has a confidence value greater than 0.65.
-
The policy with priority 3 will be applicable if the included System Action was of type Disambiguation and the Intent coming from the Rule-Based Technology has name "Appointment".
-
In case both policies are applicable, the one with the highest priority will be selected.
Intent Fusion
The Intent Fusion declaration (<IntentFusion>) consists of the following attributes:
-
type: Type of the policy. Valid values: “rb”.
It contains an array of Fusion Policies.
Fusion Policy
The Fusion Policy (<FusionPolicy>) represents a policy candidate to be selected.
It consists of the following attributes:
-
priority: Priority is a whole (>=0) number, representing the weight of the policy. The higher its value is, the more priority the policy can take, amongst other candidates. In the event that there are policies with the same priority, the first one in occurrence is selected. -
selectSource: The source of the intent that will be selected if the policy is applied.
Valid values: "rb", "ml".
Fusion Rule Group
A Fusion Rule Group (<FusionGroupRule>) is a set of conditions needed to be met in order for a policy to be applied. It is a logical operation between a set of rules and group rules.
It consists of the following attributes:
-
logicalOperator: Logical operator represents the logical operation to be applied, between the contents of the group. Valid values: "AND" and "OR".
A Fusion Rule Group can contain a set of Fusion Rules and other nested Fusion Rule Groups.
Fusion Rule
Fusion Rule (<FusionRule>) is a rule to be checked if applicable.
It consists of the following attributes:
-
source: Source is the origin the rule applies to.
Valid values: "ml", "rb", "dm". -
item: Item is the item the rule applies to.
Valid values:-
for "rb" and "ml" sources: "Intent"
-
for "dm" source: "SystemAction", "Intent", "Target", "Locale"
-
-
attribute: Attribute is the attribute following the item of the rule.
Valid values:-
name, subtype, type, fieldToElicit if the item is SystemAction
-
name, status if the item is Target or Locale
-
name, status, confidenceThreshold if the item is Intent
-
-
value: Value is the actual value of the attribute of an item. -
negate: Negate dictates if the result of the rule conditions should be inverted.
Annotations
The NLU Service fuses intents from both Rule-Based (RB) and Machine Learning (ML) results based on the active custom Intent Selection Policy.
If no custom policy is uploaded, the service applies a default behavior:
-
It selects ML-NLU intents that exceed a confidence threshold of
>0.65. -
Otherwise, it selects the RB-NLU intents.
-
Additionally, all NLU models inherit a built-in policy named
OutOfScopeDefaultto handle utterances where the ML intent is dropped.
Uploading a custom Intent Selection Policy XML file completely replaces this default behavior, including the OutOfScopeDefault policy. The logic defined in your custom XML will be used instead.
The matching of results is performed by pairing in the order received from each service. For instance, the first result that contains RB-NLU intents is matched with the first result that contains ML-NLU intents.
Results that do not have a pair, are handled separately. For example, an ML-NLU intent will be applied, if the provided Intent Selection Policy allows it. If there is no Intent Selection Policy, the intent will be dropped, in case it has low confidence (⇐0.65).
The fused result consists of the selected intents and the entities from the RB-NLU result (if a matching result exists).
Below you can find the examples:
Result with Rule-Based (RB) Intents
{
"utterance": "i want my checking bank balance and make a payment",
"intents": [
{
"name": "Account-Balance",
"source": "rb",
"confidence": 0,
"score": 9,
"constraint": {
"matched_selection_entities": [
{
"features": [],
"name": "Product",
"value": "bankAccount"
},
{
"features": [],
"name": "Element",
"value": "balance"
},
{
"features": [],
"name": "MetaAction",
"value": ""
},
{
"features": [],
"name": "Modifier",
"value": ""
}
],
"name": "001",
"reference": {
"name": "Balance-Inquiry"
},
"selection_entities": [
{
"features": [],
"name": "Product",
"value": "bankAccount"
}
]
}
},
{
"name": "Payment-Make_Payment",
"source": "rb",
"confidence": 0,
"score": 6.75,
"constraint": {
"matched_selection_entities": [
{
"features": [
{
"name": "mood",
"value": "volition"
}
],
"name": "Transaction",
"value": "pay"
},
{
"features": [],
"name": "Action",
"value": ""
}
],
"name": "001",
"selection_entities": [
{
"features": [
{
"name": "mood",
"value": "volition"
}
],
"name": "Transaction",
"value": "pay,payOff"
},
{
"features": [],
"name": "Action",
"value": "!stop"
}
]
}
}
],
"nbest_intents": [],
"entities": {
"Bank": {
"entity_instances": [
{
"begin": 16,
"confidence": 1,
"convered_text": "bank",
"end": 20,
"entity_features": {},
"value": "null"
}
],
"multi_value": false,
"name": "Bank",
"public": false,
"sensitive": false,
"stock": false
},
"BankAccount": {
"entity_instances": [
{
"begin": 7,
"confidence": 1,
"convered_text": "checking bank",
"end": 20,
"entity_features": {
"BankAccountType": "checking"
},
"value": "null"
}
],
"multi_value": false,
"name": "BankAccount",
"public": false,
"sensitive": false,
"stock": false
},
"BankAccountType": {
"entity_instances": [
{
"begin": 7,
"confidence": 1,
"convered_text": "checking",
"end": 15,
"entity_features": {},
"value": "checking"
}
],
"multi_value": false,
"name": "BankAccountType",
"public": false,
"sensitive": false,
"stock": false
},
"Element": {
"entity_instances": [
{
"begin": 21,
"confidence": 1,
"convered_text": "balance",
"end": 28,
"entity_features": {},
"value": "balance"
}
],
"multi_value": false,
"name": "Element",
"public": false,
"sensitive": false,
"stock": false
},
"Product": {
"entity_instances": [
{
"begin": 7,
"confidence": 1,
"convered_text": "checking bank",
"end": 20,
"entity_features": {},
"value": "bankAccount"
}
],
"multi_value": false,
"name": "Product",
"public": false,
"sensitive": false,
"stock": false
},
"Transaction": {
"entity_instances": [
{
"begin": 33,
"confidence": 1,
"convered_text": "make a payment",
"end": 47,
"entity_features": {
"mood": "volition",
"pos": "noun",
"tense": "present"
},
"value": "pay"
}
],
"multi_value": false,
"name": "Transaction",
"public": false,
"sensitive": false,
"stock": false
}
}
}
Result with Machine Learning (ML) Intents
{
"utterance": "i want my checking bank balance and make a payment",
"intents": [
{
"name": "BankAccount-Balance",
"source": "ml",
"confidence": 0.91614836,
"score": 0,
"constraint": null
}
],
"nbest_intents": [],
"entities": {}
}
Fused Result
Select first ML-NLU Intent (>0.65 confidence) and then RB-NLU intent with no pairing. No Intent Selection Policy provided.
{
"utterance": "i want my checking bank balance and make a payment",
"intents": [
{
"name": "BankAccount-Balance",
"source": "ml",
"confidence": 0.91614836,
"score": 0,
"constraint": null
},
{
"name": "Payment-Make_Payment",
"source": "rb",
"confidence": 0,
"score": 6.75,
"constraint": {
"matched_selection_entities": [
{
"features": [
{
"name": "mood",
"value": "volition"
}
],
"name": "Transaction",
"value": "pay"
},
{
"features": [],
"name": "Action",
"value": ""
}
],
"name": "001",
"selection_entities": [
{
"features": [
{
"name": "mood",
"value": "volition"
}
],
"name": "Transaction",
"value": "pay,payOff"
},
{
"features": [],
"name": "Action",
"value": "!stop"
}
]
}
}
],
"nbest_intents": [],
"entities": {
"Bank": {
"entity_instances": [
{
"begin": 16,
"confidence": 1,
"convered_text": "bank",
"end": 20,
"entity_features": {},
"value": "null"
}
],
"multi_value": false,
"name": "Bank",
"public": false,
"sensitive": false,
"stock": false
},
"BankAccount": {
"entity_instances": [
{
"begin": 7,
"confidence": 1,
"convered_text": "checking bank",
"end": 20,
"entity_features": {
"BankAccountType": "checking"
},
"value": "null"
}
],
"multi_value": false,
"name": "BankAccount",
"public": false,
"sensitive": false,
"stock": false
},
"BankAccountType": {
"entity_instances": [
{
"begin": 7,
"confidence": 1,
"convered_text": "checking",
"end": 15,
"entity_features": {},
"value": "checking"
}
],
"multi_value": false,
"name": "BankAccountType",
"public": false,
"sensitive": false,
"stock": false
},
"Element": {
"entity_instances": [
{
"begin": 21,
"confidence": 1,
"convered_text": "balance",
"end": 28,
"entity_features": {},
"value": "balance"
}
],
"multi_value": false,
"name": "Element",
"public": false,
"sensitive": false,
"stock": false
},
"Product": {
"entity_instances": [
{
"begin": 7,
"confidence": 1,
"convered_text": "checking bank",
"end": 20,
"entity_features": {},
"value": "bankAccount"
}
],
"multi_value": false,
"name": "Product",
"public": false,
"sensitive": false,
"stock": false
},
"Transaction": {
"entity_instances": [
{
"begin": 33,
"confidence": 1,
"convered_text": "make a payment",
"end": 47,
"entity_features": {
"mood": "volition",
"pos": "noun",
"tense": "present"
},
"value": "pay"
}
],
"multi_value": false,
"name": "Transaction",
"public": false,
"sensitive": false,
"stock": false
}
}
}