Optimise IQ

Currently Optimise IQ is only available in AWS US1 Multitenant Production, as part of a controlled release. Users must be explicitly added to view the new feature and provide feedback and thus improve the overall user experience. 

Optimise IQ marks the next era of analytics within the OCP® ecosystem. This new capability is designed to empower OCP® users to continuously improve their conversational applications by leveraging insights directly from within the OCP® environment. While performance metrics and KPIs are already available, the current process of analyzing data, identifying improvement opportunities, and implementing changes is highly manual and time-consuming. Optimise IQ addresses this gap by providing a set of interactive, graphical reports that highlight potential enhancements—such as updating existing intents or introducing new ones—based on real user interactions and application performance. These insights will be directly linked to actionable steps within other OCP components, creating a seamless and integrated workflow. Fully embedded in the OCP® Console, Optimise IQ offers a unified and data-driven approach to application optimization throughout the development lifecycle.

Overview

Optimise IQ allows users to view aggregated and processed results that failed to be recognised completely by the virtual agent, during an open question or a disambiguation question. 

Utterance Categories

Those are grouped under two major categories:

• Unmatched Utterances - Cases where the system failed to extract an Intent, either due to no-match or out of existing domain and Intent list context. 

• Low Confidence - Cases where the system extracted an Intent with very low confidence due to either ML-NLU or ASR scores.
  The numbers indicate both the impact of no matches as well as the unique records identified.

  

• Top Utterances - These are the top utterances that could not be related to an intent. User utterances are sorted based on frequency, or alphabetical.

  

• Top Proposed Intents - The proposed  intents can derive from existing identified intents, or new intents may be “SUGGESTED” as potential application enhancement. Top proposed Intents are sorted based on frequency, in order to assess impact, or alphabetical if the user wishes to explore.

  

Optimise Data

By selecting Optimise Data  the OCP user may decide one of the following:

• Review All, and navigate through extensive filtering.

• Download Data into CSV format for further offline processing and analysis.

• Send to NLU and generate a new NLU application for review before production. Existing intents will receive additional utterances, and suggested new intents will also be added to the NLU application.

To be able to select Send to NLU, you must first select an NLU Application in the Filters menu.

• Send to Pathfinder as part of an existing Pathfinder project enhancement (Not available yet)

Send to NLU 

In order for NLU to receive data an NLU Application should be selected first from the Filters menu.

The OCP user may also select one or multiple intents for NLU enhancement, controlling the business decision for the update. 

The filter selection is shared one last time for review prior to sending to NLU.

The following pop up will appear. 

When navigating to the NLU menu the latest NLU model it should be available for review. 

This model will be an enhancement of the existing model, with all the selected additions of corpus based on the production data input and thus closing the development circle. 

Optimise IQ navigation guide

Date

Users may navigate up to 2 months of historical data, assessing the impact of each potential intent expansion.  Select a specific date range or any of the prefixed ones for example, Last month.

Review 

When selecting the Review button in any category (Unmatched utterances, Low Confidence, Top Utterances or Top proposed Intents) the users may review all the corresponding utterances and navigate easily through data related to their selection. 

Search

Search functionality allows the user to type in the token they are interested in exploring. For example all the utterances that have the word “covid” within.

Filtering 

By clicking the Filter button you can narrow your search for data that can be optimized using the available filters.

For users with multiple or complex applications filters can specify specific target areas. 

• OCP Group: Example ABC_Bank_Dev, ABC_Bank_PROD etc

• NLU Application: Mandatory for NLU Actions

• Issue Type: All,  Unmatched, Low Confidence.

• Proposed Intent: potential intent expansion

• Min Frequency: Returns utterance with the desired level of impact to the application.

• Application Name: miniApp Name

• Application Type: miniApp Type

• Channel: Chat, IVR etc

• Locale: Language of the application for example English, French etc. 

Only English supported for V1. (e.g en-US, en-GB etc)

Secure Data 

Optimise IQ is available through OCP Insights UI, and securely accessible through Metrics API IAM integration. 

Data is processed, cleaned and aggregated. Only non-sensitive steps (Open Questions) are selected for application optimisation. Furthermore any potential sensitive identifiers, provided by user mistake, are stripped out as part of pre-processing, removing any sensitivity data or identification indicators.

Controlled Release Functionality 

Supported MiniApps Types

• Insurance

• Telco

• Energy

• Universal 

• Pandemic

• CarRetail

• Banking

• Banking 2

Supported Languages

• English

Supported Intent Suggestions:

• No Matches (supported)

• Low Confidence (supported only in NLU driven cases) 

Roadmap 

Optimise IQ will expand with further capabilities and PathFinder integration, to fully automate the application improvement lifecycle while enabling users ability to make data driven decisions.