Overview

OCP® Reporting is an out-of-the-box reporting and analytics platform available for all conversations through OCP® applications. These involve Orchestrator Application dialogs, Flows, and miniApps innovations originating from calls or web chats. 

OCP Reporting® provides rich, graphical dashboards with key performance metrics in order to pinpoint opportunities for continuously improving the Virtual Assistant and gaining a deeper insight into what customers are saying. The reports are unsupervised as they come directly from the system without any analysis from a human.

To access OCP® Reporting, go to Insights → Reporting in the side panel.

Reporting Components 

The OCP® Reporting is divided into four main components: 

Reports

Reports offer near real-time metrics showing necessary information about the current status of your miniApp. They are created from a specific selection of existing batch metrics, targeted to ensure the user is informed about the current state of their system, without being overwhelmed with information.

To produce near real-time reports on top of the underlying data, the DWH infrastructure is used. Reporting metrics are broken down into 5-minute intervals and updated near real time, with an additional watermark of 5 minutes. This means any data delay (due to external services) can be captured even with up to 5 minutes delay. These metrics will be publicly available also through the Metrics API.

The reports put emphasis on near real-time reporting. While more analytical historical metrics can be found in Conversational Insights.

Sessions

The Sessions report shows the number of total sessions the application received.

Sessions refer to either the number of unique IVR calls or chat sessions (depending on the application), no matter how many miniApps, or Orchestrator Flows were triggered. For the session counter to increase, at least one miniApp must be triggered within OCP®.

Dialogs

The Dialogs report is applicable for Orchestrator applications and refers to the number of dialogs per Orchestrator application. A dialog is a unique conversation (call or chat) within an OCP® application.

The following metrics are available:

  • Completed: Number of completed dialogs. Includes both terminated and transferred dialogs, as well as edge cases, when the dialog end is unknown. 

  • Transferred: Total number of dialogs that got transferred to a live agent. 

  • Terminated: Total number of dialogs that got terminated by the user or the system. 

miniApp Invocations 

The miniApp Invocations report is applicable for OCP® miniApp applications. Each counter is increased after the compilation of each miniApp task. 

The following metrics are available:

  • Total: The number of total invocations (tasks) originating from miniApps. This includes successful miniApps, miniApps that ended with system errors, as well as miniApps that ended with any other failed reason like user termination or invalid input.

  • Successful: The number of total successful miniApps invocations. 

  • System Errors: The number of invocations ended with a system error, critical error, or timeout error only. 

miniApp NoInputs/NoMatches 

The miniApp NoInput/NoMatches report displays a total number of No Matches and No Inputs per miniApp. 

No Matches and No Inputs are applicable for those miniApps only that expect a user input. 

  • No Matches: Total number of steps with no match. A single miniApp may have multiple steps with no matches. This is a per-step metric. 

  • No Inputs: Total number of steps with no input. A single miniApp may have multiple steps with no inputs. This is a per-step metric. 

Search Filters

The search filter allows you to easily target specific applications or miniApps within OCP®.

  1. To open the search filters, click the Filters button.

2. Fill in the search filter fields.

  • Group: The OCP group that the metrics refer to. Only one group may be selected each time.

  • Name: The name of the miniApp for the miniApp Invocations report or Orchestrator application for the Dialogs report.

  • Type: The type of the miniApp shown in the reports. For Orchestrator applications, the type is always Flow. Not applicable for Session reports.

  • Channel: The channel of the user system interaction (for example, Voice or Chat). Applicable for all reports. 

  • Language: The selected language of the user system interaction (for example, English US).

  • Test Flag: Select true for fetching only test-flagged calls (or char sessions). To exclude test-flagged calls, select false. By default, all calls are reported. Applicable for all reports. 

  • Time: Select the time range visible in reports.

3. Click the View button.

Not all filters are applicable for all reports!
For example, Session reports may be selected by Language. However, this filter is not applicable for miniApp Invocations report, as the same session may reach multiple OCP® applications.

Advanced Conversational Insights

More detailed advanced metrics for the full application history may be found in OCP Conversational Insights®.

Metrics API

Insights Report Metrics, with corresponding filters, are also available through the Metrics API.

Downloading a report

You can download each report locally as a CSV file by clicking the Download button next to the graph.

Selecting metrics

You can exclude / include certain metrics from the report by checking / unchecking the checkbox next to each metric name as shown below.

Viewing metrics details

  1. Hover over the graph to see the metric’s time.

2. Click on a dot with a specific time.

3. Click the Show Actual Dialogs button. You will be redirected to OCP® Monitor to review the dialog IDs corresponding to the reported metric.