Test Suites Tab

Introduction

The Test Suites tab is your central workspace for organizing, configuring, and running tests in Testing Studio. From this tab, you can build and run test suites - collections of test case files grouped for execution - track results, and manage underlying test files.

The Test Suites tab now has two views: Test suites and Source files.

• Test suites view: The main working view for browsing, running, and managing suites and test cases.

• Source files view: The legacy view, showing the raw file tree and underlying YAML/JSON files. Use this only when you need to inspect or hand-edit source files directly.

Test suites view

The Test suites view is the primary workspace for managing and running test suites. Use it to browse suites, review test results, configure suite settings, and create, edit, or delete test cases and test suites.

Navigation

You will find the navigation information below to easily move through the test suites interface and locate the information you need.

1. The left panel lists all Suites in the project, with a suite count and a + New suite button.

2. The main panel shows the selected suite's summary: pass percentage, passed/failed/unavailable counts, total test cases, a progress bar, and the Last run timestamp.

3. Filter tabs (All / Passed / Failed) narrow the test case list below.

4. Each test case row shows its status (PASSED/FAILED), a Run button, a conversation preview, and a View golden link.

In a test case, expand the conversation to view the dialog and its associated intent shown below.

Create a new test suite

To create a new test suite, follow these steps:

1. In the left panel, click + New suite.

2. Enter a suite name (letters, numbers, and underscores only) and confirm.

2. The new suite appears in the SUITES list.

Rename a test suite

1. In the SUITES list, click the Options menu next to the suite you want to rename.

2. Select Rename.

3. In the dialog box that opens, update the Suite name field.

4. Click Save.

Configure test suite settings

Suite settings are the Global properties presented in a human-readable interface.

1. Select a test suite and click Suite settings in the suite header.

2. Update settings as needed:

• Channel & Behavior

  • Session channel: Specifies the communication channel used to run the test session, such as voice or chat.

  • Modality: Defines the interaction mode used by the test, such as voice, text, or a combination of supported input and output modes.

  • Tries on failure: Number of retry attempts if a turn fails before the test case is marked failed. Supported values: 0-20.

  • Handle delays: Controls whether the test skips waits such as WebService calls.
    Options: Default (inherits suite setting), On, Off.

• Simulation data

  • INITIAL: Key/value fields representing initial simulation parameters passed into the test session (for example, testMode, envMode). Click + Add field to add a new key/value pair.

  • BIO DATA: Key/value fields representing simulated biometric data used during the test. Click + Add field to add a new key/value pair.

3. Click Save.

Delete a test suite

1. In the SUITES list, open the Options menu for the suite you want to delete.

2. Select Delete.

3. Confirm by typing CONFIRM DELETE.

3. Click Delete to finalize.

Create a new test case

1. Select the suite you want to add the test case to and click + New test case.

2. In the dialog box that opens, fill in the test case details as described below:

• Details

  • Name: A short identifier for the test case. For example, activate_Card.

• Description: A text summary of what the test case checks (for example, Activate new card - happy path). Optional but recommended for readability.

• Conversation: Add one or more turns representing what the user says. Click + Add turn to add a new turn. Then enter an utterance and select type from the dropdown (text, touch, dtmf).

• Channel & Behavior

  • Locale: Select the language/locale for the test case. The Default option inherits the suite's setting.

  • Tries on failure: Number of retry attempts allowed if a turn fails before the test case is marked failed. Supported values: 0-20.

  • Handle delays: Controls whether the test skips waits such as WebService calls.
    Options: Default (inherits suite setting), On, Off.

  • Override global simulation data: Toggle to control whether this test case uses its own simulation data instead of the suite-level defaults. Options: Default, On, Off.
    Must be On for the Simulation data fields below to take effect.

• Simulation data

  • INITIAL: Key/value fields representing initial simulation parameters passed into the test session (for example, testMode, envMode). Click + Add field to add a new key/value pair.

  • BIO DATA: Key/value fields representing simulated biometric data used during the test. Click + Add field to add a new key/value pair.

3. Click Create test case to save the test data. The new test case appears in the test case list for that suite.

Edit a test case

To edit a test case, follow the steps below:

1. Find the test case you want to edit, click the Options menu, and select Edit.

2. In the dialog box that opens, update the data.

3. Click Save.

View a test case's golden dialog

1. Find a test case in the list and click View golden.

2. A side panel opens showing intent, locale, turns, the expected turns, and the expandable raw golden dialog JSON.

Goldenize a test case

Goldenizing updates the golden dialog with the output from the latest test run. Use this option when a conversational flow has intentionally changed or when you confirm that the latest execution output is correct.

To goldenize a test case:

1. Locate the test case in the test cases list.

2. Click the Options menu next to the test case and select Goldenize.

3. The golden dialog is updated and will be used as the new baseline for subsequent test runs.

Delete a test case

1. Find the test case, click its Options menu and select Delete.

2. Confirm by typing CONFIRM DELETE.

3. Click Delete to finalize.

Running tests

Run tests individually, in selected groups, or across an entire suite or project to validate your conversational flows and review their results.

Run a single test case

1. In the test cases list, find the test case you want to run.

2. Click Run in that test case’s row.

Run selected test cases

1. Check the boxes next to the test cases you want to include in the run.

2. Click Run selected.

Run selected test suites

1. Select the suite you want to run.

2. Click Run selected.

Run all suites in the project

1. Go to the Test Suites view.

2. Click Run all suites in the top-right corner.

Create a test file from Orchestrator

You can create test files directly from the chat session in Orchestrator, and they will automatically be added to the AutoGen_TestSuite folder in Testing Studio.

To create a test case in Orchestrator, follow the guidelines outlined below:

1. Navigate to Orchestrator and select a dialog application you want to test.

2. Open the Orchestrator Chat and click the three icon menu button and then select Create Test Case:

3. In the dialog box that opens, fill in the fields as described below:

• Name Test Case: Enter a unique name for the test case you are creating.

• Select Project: Choose a Testing Studio project where you would like to add the test case.

• Select Test Suite: Select a test suite folder to add the test cases. If none is selected, the generated test cases will be added automatically to the AutoGen_TestSuite folder in Testing Studio.

• Include Simulation Data: If marked, simulation data will be added.

4. Click Create Test Case. The generation will start and the test case will automatically be added to the AutoGen_TestSuite folder in Testing Studio.

   

Now, you can view the content of the created test file, edit, create, and delete test files as necessary.

Create a test file from Monitor

You can create test files directly from Monitor, and they will automatically be added to the AutoGen_TestSuite folder in Testing Studio.

To create a test case in Monitor, follow the guidelines outlined below:

1. Navigate to Insights → Monitor and select a dialog you want to test.

2. In the Dialog Review section, click the Options menu and select Create Test Case.

3. In the dialog box that opens, fill in the fields as described below:

• Name Test Case: Enter a unique name for the test case you are creating.

• Select Project: Choose a Testing Studio project where you would like to add the test case.

• Select Test Suite: Select a test suite folder to add the test cases. If none is selected, the generated test cases will be added automatically to the AutoGen_TestSuite folder in Testing Studio.

• Include Simulation Data: If marked, simulation data will be added.

4. Confirm by clicking Create Test Case.

Now, you can view the content of the created test file, edit, create, and delete test files as necessary.

Source files view

The Source files view displays the raw file tree and underlying YAML/JSON files. It is the legacy interface and is currently available alongside the new interface, so you can use either one during the transition.

Upload a Test Suite

You can upload your collection of test case files as a ZIP file from your computer to Testing Studio.

To upload a Test Suite, follow the steps below:

1. In the Test Suites tab, click the Options menu next to the Root Directory and select Upload.

2. Drag and drop the ZIP file onto the drop zone, or click Browse Files and select your file.

3. Click Upload to confirm. When the upload starts, you will see this pop-up notification.

4. As soon as the upload is over, you will see this message in the right upper corner.

5. The uploaded Test Suite will appear in the list.

View a test file content

You can view the content of a test file directly within the Test Suites tab: simply select the file you want to view and click on it to display the content.

Edit a test file

You can edit a test file directly in Testing Studio. To do so, follow the instructions below:

1. In the Test Suites tab, select the file you want to edit and click it to open.

2. Click Edit file to start editing. Note that only one user can edit a file at a time.

3. Click Save Changes to apply your edits, or Cancel → Discard to discard them.

Create a test file

To create a new test file, proceed as described below:

1. In the Test Suites tab, click the Options menu next to the folder where you want to create the file, then click Create File.

2. A new empty unnamed.json file will be added to the selected folder.

3. You can enter a new file name at this step or Rename your test file later.

4. Click the created test file to open it. You can now edit this file as described in the Edit a test file section.

   

Create a folder

To create a new folder, proceed as described below:

1. In the Test Suites tab, click the Options menu next to the folder that will contain it, then click Create Folder.

   

2. A new empty unnamed folder will appear.

   

3. You can enter a new folder name at this step or Rename your folder later.

Download a test file

To download a test file, follow the guidelines below:

1. Locate the test file you want to download.

2. Click the Options menu and select Download.

3. The selected file will be downloaded.

Rename a test file/folder

To rename a test file or a folder, follow the guidelines below:

1. Locate the test file or folder you want to rename.

2. Click the Options icon and select Rename.

3. Enter a new file or folder name and press <Enter>. The file/folder name will change accordingly.

Delete a test file/folder

To delete a test file or a folder, click the Options menu and select Delete.

Download a Test Suite

You can download your collection of test case files from Testing Studio to your computer.

To download a Test Suite, follow the steps below:

1. In the Test Suites tab, click the Options menu next to the Root Directory and select Download.

   

2. The Test Suite will be downloaded as a ZIP file to your computer.