Overview

The Monitor tab in Insights allows the user to drill down into any conversation occurring within an OCP application. This involves Orchestrator application dialogs, flows, and miniApp invocations originating from calls or web chats. To put it simply, the Monitor tab is the portal to understanding every single low detail information about a system-user interaction. As such, the Monitor tab is divided into three separate components: Search (top of the page), Dialog List (center left column), and Dialog Review (center right column). In the next section, you can find detailed explanations for each of them.

To access OCP® Monitor go to Insights Monitor tab in the side panel.

Monitor Components

  • Search component

  • Dialog list

  • Dialog Review

Monitor components overview

Search component

Click on the funnel filter icon  to expand and view the search fields area.

Monitor filter search area

There are multiple search criteria that can be used to fetch specific sessions. In particular:

Filter Name

Description

Datetime

You can select the specific time period you are interested in, by indicating start/end dates and start/end times. Time corresponds to local time zone.

Group

The resource group that miniApps or Flows belong to.

Name

The application name; can be multiple.

Type

The application type.

ANI

The identification of the caller’s phone number that initiated the dialog session (number of calls and …​ for chat).

Dialog Group ID

Like a dialog ID but for a Flow application that consists of multiple miniApps with their own specific dialog IDs. A Dialog Group ID search returns results based on the OCP structured hierarchy:

  1. Session, an external key that enables multiple OCP conversations to be reported on together.

  2. Conversation, a single Conversational AI session hosted within OCP.

  3. Self service, a defined unit of service also defined as Flow within the Conversational AI application (for example, Authentication, or Card Balance, or Transaction History).

  4. Task, a single action taken (such as a web service API query, gathering an alphanumeric capture, classifying a caller’s intent).

Channel

The session channel of the user such as voice or web-chat.

Result

Select to view only success or fail results.

Fail Reason

Select to view sessions with specific error that ended the session, as set up in miniApps or Orchestrator for a flow.

CISR

Select a CISR value, as declared for specific reviewed utterances

Keywords in Events:

Events that happen within a dialog session. Can be used to query dialogs based on custom loggings or other identifiers.

Utterance

The caller input, such as specific keywords or phrases.

Custom Tag

Select a Custom Tag value, as declared for specific reviewed utterances

Key-Value Pairs

Search for custom loggings as configured in Orchestrator

Turn on the show Live Toggle button, to get new session data every 15 seconds and monitor ongoing calls.

Searching by Dialog ID cancels all other selected filters.

Example search values

Search example 1

Search example 2

On the top right, the funnel-like icon shows whether you have filters in place.

  • Grey Funnel filter icon: No filters selected

  • Funnel filter icon with blue dot: Filters are selected (Click the View button first to apply the filters)

For accurate information on the presence of filters, every time you add or remove filters, you have to click View or Count, in order for the filter status to be refreshed.

Click the Count button to check the number of results of the selected filters before viewing them, to decide if the search criteria is the desirable one.

Dialog list

The Dialog list presents all the dialog events that meet the criteria of the search section. Every column contains information regarding the call. Click on a dialog to check its information in the Dialog Review area.

Field

Description

TIME

Shows the time when the dialog started.

ANI

The caller’s phone number if available.

CHANNEL

  • This call icon means it’s the call session channel.

  • This chat icon means it’s the web chat session channel

DURATION

Shows the length of the call

STEPS

Shows the number of steps in the session. Everything except no_input is counted, which means you get the total number of system prompts that were used and the user utterances, as well as one step added per the call ending (either hup, meaning hang-up, or transfer to an agent).

ENDING

  • This hung-up icon means the user hung up,

  • This call transfer icon  means the user was transferred to an agent

Click the Next button to move to the next page.

Dialog Review

The Dialog Review contains information about the selected dialog.

Field

Description

Dialog Group ID

Press the Copy icon to copy the Dialog Group ID.

Dialog ID

Press the Copy icon to copy the Dialog ID.

DateTime

The date and time the dialog took place.

miniApp Name

The name of the miniApp used for the selected dialog.

miniApp bot icon

Indicates the prompt generated by the miniApp.

User utterance icon

Indicates the utterance of the caller.

Numbering on the right

Represent the step number.

Click on the Show events button to get all the events - system and/or custom loggings from the application.

Click the Log File button to open the DiaManT log file of the selected dialog.

Click the JSON file icon to get a JSON file with all the dialog session info.

Click the TXT file icon to get an RTF file with all the dialog session info.

When it comes to the dialog itself, you can click the arrow pointing down to expand the intent interpretation view (when expanded, the arrow will be pointing up).

Whenever the utterances are recognized (aren’t no_input), click the Play icon to listen to the audio of the user’s responses to the questions and check whether the system captured them correctly.

If there is sensitive data in the utterance, the details are not shown, and the message ***sensitive content protected*** appears instead. However IntentConfidence and NLU Interpretation information are presented.

Sensitive info content

You can tag an utterance as correct or false, acceptance or rejection, or as a misinterpretation. To do so, click the # symbol and select the corresponding options in the pop-up window.

Utterance review

Utterance Review available options

Option

Description

CISR

  • correctAcc: The system correctly accepted the caller input, which means that either the correct intent was extracted, or the expected input was correctly identified (for example, the caller answering yes or no when asked to confirm payment).

  • correctRej: The system correctly rejected something that is not within the application’s scope (for example, the customer utterance: I could call back if this is a bad time.)

  • misinterpretation: Accepted with wrong understanding. A caller said I want the balance on card and the system understood I want the balance on my loan. The system understood the utterance as related to the application but it misinterpreted it by assigning the wrong intent.

  • falseAcc: The system mistakenly confused something out of the application’s scope with something within scope.

  • falseRej: Mistakenly rejected an in scope caller input (for example I want to know my balance) is not understood and is considered by the system to be irrelevant.

Custom Field

Add a keyword that could help categorize the utterance.