Exports API
Data Compliance
Omilia OCP® Data Warehouse and Export API comply with General Data Protection Regulation (GDPR) requirements. For any data type exported from OCP, through Export API, the OCP users are responsible to ensure that their exported data are handled accordingly and meet necessary regulations.
Overview
Exports API is organized around HTTP REST. Our API allows you to request CDRs as exports for specific time range. The maximum date range supported is two days. The smallest requested date range is one hour (for example, 2023-01-12T12:00:00.000Z to 2023-01-12T13:00:00.000Z).
Minutes/seconds/milliseconds will not change the export (example, 2023-01-12T12:00:00.000Z to 2023-01-12T13:00:00.000Z is equal to 2023-01-12T12:10:00.000Z to 2023-01-12T13:20:00.000Z).
Each group can run one export at a time. After the successful creation of the export, the exported data is stored for one day, after this period the data is deleted from the API and the status of the export is updated to expired.
We use built-in HTTP authentication to validate users based on groups/roles stored in IAM.
Sequence Diagram
.png?inst-v=c16d1f0c-7331-444c-9451-2b5e6bdd9d07)
Exports access token creation
The access token is a random string generated by IAM and exports-api will use to validate the user against IAM server and check user’s whitelist of groups.
Whitelisted groups should be added as attributes assigned to the user or to a group which the user belongs to.
Access token creation
After creating the user, the client should obtain access token from IAM deployment via its API.
cURL Example Request
curl --location --request POST 'https://us1-m.ocp.ai/auth/realms/master/protocol/openid-connect/token' \
--header 'Content-Type: application/x-www-form-urlencoded' \
--data-urlencode 'client_id=ocp' \
--data-urlencode 'grant_type=password' \
--data-urlencode 'username=<your username>' \
--data-urlencode 'password=<your password>'
Response
A successful request returns the HTTP 200 status code and a JSON response body that contains the access token to use with exports-api.
HTTP Code Responses
Omilia's HTTP response codes can be found on the HTTP Code Responses page.
API Reference
{{baseUrl}} depends on the user's environment.
For us1 it is us1-a.ocp.ai
Create Export
Request
POST {{baseUrl}}/exports-api/v1/groups/{group}/jobs
{
"from_date": "2023-01-12T13:00:00.000Z",
"to_date": "2023-01-12T14:00:00.000Z",
"types": [
"Dialog Data"
],
"export_name": "Jan-12-2023-1300-Export"
}
Path Parameters
Property | Type | Required | Description |
---|---|---|---|
| string | Yes | The group (organization) the export requested belong to. |
Body Parameters
Property | Type | Required | Description |
---|---|---|---|
| string | Yes | Name of specific export (^[a-zA-Z0-9-][a-zA-Z0-9-\\s]+$). The export name is selected by the exporter. |
| DateTime(UTC) | Yes | Timestamp before |
| DateTime(UTC) | Yes |
|
| array(string) | Yes | The types of data needed for export
|
Response
A successful request returns the HTTP 200 status code and a JSON response body that shows the export id created (UUID) and a description field (String) with success.
Example
{
"description": "Task Submitted",
"export_id": "73bd903e-e5c5-4b87-a9a9-6a35e45af48f"
}
List Exports
Request
GET {{baseUrl}}/exports-api/v1/groups/{group}/jobs
Path Parameters
Property | Type | Required | Description |
---|---|---|---|
| string | Yes | The group (organization) that the list export refers to. |
Query Parameters
Property | Type | Required | Description |
---|---|---|---|
| Integer | No | Integer > 0 , default value = 1 |
| Integer | No | 100 >= Integer > 0 , default value = 50 |
| string | No | The types of data needed for export SUBMITTED, PROCESSING, RUNNING, READY, EXPIRED (Each one of these can be given independently) |
Response
A successful request returns the HTTP 200 status code and a JSON response body that shows a list of created exports from the specific group.
Example
{
"pagination": {
"pages": 1,
"page_number": 1,
"page_size": 2,
"total_results": 2
},
"exports": [
{
"name": "KT_dw_qa_2_AwsDev",
"status": "EXPIRED",
"group": "dw_qa_2_AwsDev",
"export_id": "3342d2f8-2279-43a3-b4e9-082ceed19a68",
"time_submitted": "2023-01-13T09:54:58.192Z",
"from_date": "2023-01-13T05:00:00Z",
"finished_date": "2023-01-13T09:55:50.57Z",
"to_date": "2023-01-13T06:00:00Z",
"types": [
"Dialog Step",
"Dialog Start",
"Dialog End"
]
},
{
"name": "tests",
"status": "EXPIRED",
"group": "dw_qa_2_AwsDev",
"export_id": "b52d8a2c-358f-486d-a464-f89dbf2eaa9b",
"time_submitted": "2023-01-05T12:19:54.061727Z",
"from_date": "2022-12-05T00:00:00Z",
"finished_date": "2023-01-08T18:00:01.784466Z",
"to_date": "2022-12-05T23:00:00Z",
"types": [
"Dialog Step"
]
}
]
}
GET Export Metadata
Request
GET {{baseUrl}}/exports-api/v1/groups/{group}/jobs/{export_id}
Path Parameters
Property | Type | Required | Description |
---|---|---|---|
| string | Yes | The group (organization) that the list export refers to. |
| UUID string | Yes | A valid UUID string referencing a created export. |
Response
A successful request returns the HTTP 200 status code and a JSON response body that shows metadata of the specific export.
Example
{
"export_id": "3fa85f64-5717-4562-b3fc-2c963f66afa6",
"num_of_files": 0,
"size_of_export": 0
}
List Export Data
Request
GET {{baseUrl}}/exports-api/v1/groups/{group}/jobs/{export_id}/data/compressed
Path Parameters
Property | Type | Required | Description |
---|---|---|---|
| string | Yes | The group (organization) that the list export refers to. |
| UUID string | Yes | A valid UUID string referencing a created export. |
Query Parameters
Property | Type | Required | Description |
---|---|---|---|
| Integer | No | Integer > 0, default value 1 |
| Integer | No | 50 >= Integer > 0, default value 20 |
To acquire the data from an export containing several files (information on the number can be found in the "GET Export Metadata" endpoint response body, or the response headers of the "LIST response data" endpoint), there are several approaches.
For example:
For an export containing 160 files, you can either access the specific endpoint 8 times, utilizing the default page size of 20 and incrementing the page number by 1 with each request (ranging from 1 to 8), or you can access the endpoint 4 times, using a page size of 50 and advancing sequentially from page numbers 1 to 4.
Essentially, what this means, is that the export process is paginated and consequently, it involves making multiple requests.
Response
A successful request returns the HTTP 200 status code with pagination HTTP headers and initiates a streaming response that when finished, a zip file is created and downloaded.
The HTTP headers you will be dealing with involve the following:
Pagination-pages
: This refers to the total number of accessible pages.Pagination-page_number
: Indicates the specific page you're getting.Pagination-page_size
: This represents the quantity of files downloaded with this request.
Example
.png?inst-v=c16d1f0c-7331-444c-9451-2b5e6bdd9d07)
Exports Supported
Currently the API supports only CDR exports. The CDRs have 3 different types: dialog_start
, dialog_step
, dialog_end
. For more information regarding CDRs please read the CDR Models Schema page.
Rate Limiting
Description
Exports-api has a rate limiting mechanism configured by its configuration file which applies a threshold of requests per group for a specific time window. The algorithm used is a token-bucket applied over a fixed window of time. This means that for a specific window a certain amount of tokens are produced for each group and they’re consumed as requests are landing on each endpoint. One limit per group applies for all endpoints. If the limit is exceeded, the API will respond with an error response of Status Code 429
(Too many requests). The API will again allow requests for this group after the bucket will refill with tokens in the next fixed request window. The status of the buckets are kept in a memory cache. In order to prevent the cache from growing indefinitely, there is a Time To Live (TTL) for each entry. Entries will be evicted if the TTL of the entry is exceeded (The TTL value configured should be more than the request window).
After eviction, a new cache entry will be created if a new request arrives.