Testing Studio+™ Structure

id *

Test case unique identifier

description *

Test case description

group

Named tag to distinguish test cases. Used for filtering execution of test cases.

golden_dialog *

Referenced dialog.json ID number

handle_delays

boolean field that indicates if you want to let Testing Studio+ skip delays (for example, such us miniApp WebService delays)

If not set, handle_delays will be true.

simulation_data

Simulation data. The following simulation data categories are available:

• initial: contains application field-value pairs that you want to send upon dialog initiation (e.g. to simulate the value of Ani, Dnis or other fields right when the dialog starts).

• bio_data: contains voice biometrics field-value pairs that you want to simulate (e.g. BioScore or BioSearch)

These nodes should in turn contain field-value pairs that simulate the output parameters of that action.

Let’s simulate the following case:

• Ani gets the value 124 and Dnis gets the value 12345 on dialog start.

Ani, Dnis, balance and card are the parameters defined either by the application in test.

Here’s the YML syntax:

simulation_data:
  initial:
    Ani: 124
    Dnis: 12345

This parameter is mandated supporting service. Testing Studio+ does not filter or operate on any simulation data. They are just optionally sent to supporting service when a dialog starts in order to facilitate testing needs.

Include only type and values you want to use in the simulation data, otherwise the test will fail. For example, if you don’t want to send Dnis, don’t include it at all.

Make sure that you use the correct case for the fields you are simulating. For example, Ani for the origin number and Dnis for the destination number.

overwrite_simulation_data

If true, the simulation data specified in the test case file will overwrite any data set in the global properties file. This means that the test will run using only the simulation data defined within the test case file, ignoring those specified in the global properties file.

For example, if global properties include Ani=12 and a test case specifies Dnis=34, the test would run using only Dnis=34.

If false or not set, the simulation data from both sources will be merged.

Using the same example above, the test will now run using both Ani=12 and Dnis=34.

tries *

How many times a failed test case is to be executed (0_to_20, default: 1)

A test case might fail due to various reasons. A common reason is that a dialog flow in an Omilia application is not deterministic, thus the flow might have a different dialog step order every time it is executed.

For this reason, a test case is executed multiple times (as determined by tries value) and considered failed only if assertions failed every time. If the test case passed at least once, it is considered as successful.

utterances

Actual utterances of the dialog to be tested. If omitted, the tool will use the utterances from the dialog.json file. Utterances consist of the following fields:

• utterance: The text representation of the utterance. This field is mandatory.

• modality: Whether this utterance will be sent as a chat, DTMF or with ASR support. Possible values are text, dtmf and speech respectively.

• locale:
  This field indicates on what language the (voice) run will take place. The default value is “en-US”. Possible values: “en-US”,“fr-CA”, “es-US”, “de-DE”.

If modality is not present, the test case modality will be used.

The number of utterances in the test case should not exceed the number of actual steps of the referenced dialog, otherwise the test case will fail.

dialog_events_rules

These rules work only if the dialog_events assertion is enabled

A set of rules by which the Testing Studio+ will assert or ignore specific dialog events. It may comprise the assert or ignore nodes:

• when using the assert, then only what you have put under the assert node will be validated. Any other dialog event will not be validated.

• (same goes for ignore), when using ignore, it will check all the dialog events, except for those declared under the ignore node.

YML syntax:

• Assert node

dialog_events_rules:
  assert:
    exp:
      - Intent%
      - OQ_Welcome
    operator: and

Intent% means that the golden dialog must contain the same dialog events starting with Intent as the ones in the actual running case.

• A successful scenario example:

Golden: OQ_Welcome, Intent-Balance, HUP
Actual: OQ_Welcome, AgentRequest, Intent-Balance, TRANSFER

• A failed scenario example:

Golden: OQ_Welcome, Intent-Card_Activate, HUP
Actual: OQ_Welcome, AgentRequest, Intent-Balance, TRANSFER

• Ignore node

dialog_events_rules:
  ignore:
    exp:
    - Ani:%
    operator: or

Ani=% means that all dialog events starting with Ani can be ignored. The rest of the dialog events must all be the same.

• A successful scenario example:

Golden: Ani=12345, OQ_Welcome, Intent-Balance, HUP
Actual: Ani=99999, OQ_Welcome, Intent-Balance, HUP

• A failed scenario example:

Golden: Ani=12345, OQ_Welcome, Intent-Balance, HUP
Actual: Ani=99999, OQ_Welcome, AgentRequest, Intent-Balance, HUP

Since the Assert node only compares the declared expressions and implicitly ignores all the other ones, defining both type of nodes under dialog_events_rules is redundant. The Ignore node will be ignored.

dialog_kvps_rules

These rules work only if the dialog_kvps assertion is enabled.

A set of rules by which the Testing Studio+ will assert or ignore specific key-value pairs. It may comprise the assert or ignore nodes:

• when using the assert, then only what you have put under the assert node will be validated. Any other key-value pairs will not be validated.

• (same goes for ignore), when using ignore, it will check all the key-value pairs, except for those declared under the ignore node.

YML syntax:

• Assert node

dialog_kvps_rules:
  assert:
    exp:
      - Locale%
      - Region
    operator: and

• Ignore node

dialog_kvps_rules:
  ignore:
    exp:
      - Timezone%
    operator: and

Since the Assert node only compares the declared expressions and implicitly ignores all the other ones, defining both type of nodes under dialog_kvps_rules is redundant. The Ignore node will be ignored.

session_channel

Represents the channel on which Testing Studio+ will start the dialog for the specific Test Case

Possible values: web-chat, chat, ivr, miniapps.

If not set, session_channel will be chat.

handle_delays

An option to let Testing Studio+ handle delays (such as miniApp Webservice delays) The default value is true

false

locale

This field indicates on what language the (voice) run will take place. The default value is “en-US”.

Possible values: “en-US”, “fr-CA”, “es-US”, “de-DE”.