Testing Studio Structure

diamant

The property is mandatory and must be configured

app

string

{group_name}@SND_{app_name}

For example if the group name is abc_uat and the application name is ABC-Root the property value will be abc_uat@SND_ABC_Root

application

Information related to the application under test

miniapp_id

string

{accountId}.{name}.{type}.{diamant_app_name}.{group_name}

ID of the miniApp application/flow that we want to test. This information can be found under Console →miniApps → General, and is available for copying.

assertions

This section contains all the available assertions. Depending on the testing level, other assertion strategies can be configured

prompt

boolean

true/false

The prompt of the dialog step

end_type

boolean

true/false

The end type of the dialog

dialog_events

boolean

true/false

Events of the dialog

transfer_line

boolean

true/false

Transfer line of the dialog

destination_uri

boolean

true/false

Destination URI of the dialog

intent

boolean

true/false

Intent of the dialog step

grammar

boolean

true/false

Grammar of the dialog step

annotation

boolean

false

Annotation of the dialog step

interpretation

boolean

false

Interpretation of the dialog step

action_name

boolean

true/false

Action name of the dialog step

target_name

boolean

true/false

Target name of the dialog step

outbound_attached_data

boolean

true/false

Attached data of the dialog step

prompt_threshold

integer

0 to 1
default set to 0.8

The lowest limit for the similarity of prompts

id *

Test case unique identifier

description *

Test case description

group

Named tag to distinguish test cases. Used for filtering execution of test cases.

golden_dialog *

Referenced dialog.json ID number

simulation_data

Simulation data. The following simulation data categories are available:

• initial: contains application field-value pairs that you want to send upon dialog initiation (e.g. to simulate the value of Ani, Dnis or other fields right when the dialog starts).

• bio_data: contains voice biometrics field-value pairs that you want to simulate (e.g. BioScore or BioSearch)

These nodes should in turn contain field-value pairs that simulate the output parameters of that action.

Let’s simulate the following case:

• Ani gets the value 124 and Dnis gets the value 12345 on dialog start.

Ani, Dnis, balance and card are the parameters defined either by the application in test.

Here’s the YML syntax:

simulation_data:
  initial:
    Ani: 124
    Dnis: 12345

This parameter is mandated supporting service. Testing Studio does not filter or operate on any simulation data. They are just optionally sent to supporting service when a dialog starts in order to facilitate testing needs.

Include only type and values you want to use in the simulation data, otherwise the test will fail. For example, if you don’t want to send Dnis, don’t include it at all.

Make sure that you use the correct case for the fields you are simulating. For example Ani for the origin number and Dnis for the destination number.

tries *

How many times a failed test case is to be executed (0_to_20, default: 1)

A test case might fail due to various reasons. A common reason is that a dialog flow in an Omilia application is not deterministic, thus the flow might have a different dialog step order every time it is executed.

For this reason, a test case is executed multiple times (as determined by tries value) and considered failed only if assertions failed every time. If the test case passed at least once, it is considered as successful.

utterances

Actual utterances of the dialog to be tested. If omitted,the tool will use the utterances from the dialog.json file. Utterances consist of the following fields:

• utterance: The text representation of the utterance. This field is mandatory.

• modality: Whether this utterance will be sent as a chat, DTMF or with ASR support. Possible values are text, dtmf and speech respectively.

If modality is not present, the test case modality will be used.

The number of utterances in the test case should not exceed the number of actual steps of the referenced dialog, otherwise the test case will fail.

dialog_events_rules

A set of rules by which the Testing Studio will assert or ignore specific dialog events. It may comprise the assert or ignore nodes:

• Intent% means that the golden dialog must contain the same dialog events starting with Intent as the ones in the actual running case.

  • A successful scenario example:

Golden: OQ_Welcome, Intent-Balance, HUP
Actual: OQ_Welcome, AgentRequest, Intent-Balance, TRANSFER

• A failed scenario example:

Golden: OQ_Welcome, Intent-Card_Activate, HUP
Actual: OQ_Welcome, AgentRequest, Intent-Balance, TRANSFER

• Ani=% means that all dialog events starting with Ani can be ignored. The rest of the dialog events must all be the same.

  • A successful scenario example:

Golden: Ani=12345, OQ_Welcome, Intent-Balance, HUP
Actual: Ani=99999, OQ_Welcome, Intent-Balance, HUP

• A failed scenario example:

Golden: Ani=12345, OQ_Welcome, Intent-Balance, HUP
Actual: Ani=99999, OQ_Welcome, AgentRequest, Intent-Balance, HUP

YML syntax:

• Assert node

dialog_events_rules:
  assert:
    exp:
      - Intent%
      - OQ_Welcome
    operator: and

• Ignore node

dialog_events_rules:
  ignore:
    exp:
    - Ani:%
    operator: or

Since the Assert node only compares the declared expressions and implicitly ignores all the other ones, defining both type of nodes under dialog_events_rules is redundant. The Ignore node will be ignored.

session_channel

Represents the channel on which Testing Studio will start the dialog for the specific Test Case.

Possible values: web-chat, chat, ivr, miniapps.

handle_delays

An option to let Testing Studio handle delays (such as ‘firstWait’, ‘repeadedWait’) The default value is true

false