Analytics export

The Analytics Data Export provides a way to access analytics data (aggregated data) collected through the Didomi platform at once, for CMP. Analytics Data Export is a premium feature.

Destinations

Logging and monitoring

Didomi offers the ability to retrieve information and monitor the status of your Analytics export. Read more on logging here.

Format

Data exported at the destination follows the Apache Hive naming conventions for partitioning and exposes 2 partitions:

Example of a path to the exported data: /data/export-id={batch export ID}/date={YYYY-MM-DD}/

CMP Hourly Metrics

Structure

Didomi exports CMP hourly metrics data in a /cmp_hourly_metrics folder and in files formatted as newline-delimited JSON (one line = one JSON object) and compressed as GZIP. We use the file extension json.gz.

Example: /data/export-id={export-id}/date={YYYY-MM-DD}/cmp_hourly_metrics

For every partition, the exported data can be split up into multiple files. Make sure that you read every .json.gz file in the partition folder to get all the records belonging to that export.

File names are subject to change over time so make sure that your read files from their extension (.json.gz) and not from a specific filename or format.

Additionally, a file named _SUCCESS is created once the export is complete. Do not read files from a partition until the _SUCCESS file is created as you might read partial records otherwise.

Content

The files exported by Didomi contain CMP hourly metrics for the period covered by the export. Every line of the files contains a self-sufficient JSON record (newline-delimited JSON) that includes CMP hourly metrics.

You should first split the files on new lines and then parse every line as an individual JSON record. It is a standard encoding for data processing tools like Hive, Spark, Presto, etc. and should be straightforward to import in your existing tools.

The schema of the objects exported is as follows:

Example of an exported file:

{"window_start":"2022-01-24T08:00:00.000Z","apikey":"7685b6f7-3062-491b-ba50-207f440951dc","domain":"io.didomi.sampleuitest","sdk_type":"sdk-ctv","device_type":"television","device_os":"Android TV","experiment_id":"","experiment_group":"","deployment_id":"6ExpWCHh","consents_asked":5.0,"consents_given":0.0,"consents_given_positive_other":0.0,"consents_given_denied_other":0.0,"consents_given_positive_scroll":0.0,"consents_given_positive_navigate":0.0,"consents_given_positive_click":0.0,"consents_given_denied_click":0.0,"consents_given_positive_external":0.0,"consents_given_denied_external":0.0,"consents_given_positive_webview":0.0,"consents_given_denied_webview":0.0,"consents_given_empty":0.0,"consents_given_invalid":0.0,"pageviews":0.0,"pageviews_with_positive_consent":0.0,"pageviews_with_denied_consent":0.0,"pageviews_with_empty_consent":0.0,"pageviews_with_invalid_consent":0.0}
{"window_start":"2022-01-24T14:00:00.000Z","apikey":"7685b6f7-3062-491b-ba50-207f440951dc","domain":"sandbox.app.domain","sdk_type":"sdk-web","device_type":"desktop","device_os":"Mac","experiment_id":"","experiment_group":"","deployment_id":"RDkUQWNK","consents_asked":0.0,"consents_given":0.0,"consents_given_positive_other":0.0,"consents_given_denied_other":0.0,"consents_given_positive_scroll":0.0,"consents_given_positive_navigate":0.0,"consents_given_positive_click":0.0,"consents_given_denied_click":0.0,"consents_given_positive_external":0.0,"consents_given_denied_external":0.0,"consents_given_positive_webview":0.0,"consents_given_denied_webview":0.0,"consents_given_empty":0.0,"consents_given_invalid":0.0,"pageviews":1.0,"pageviews_with_positive_consent":0.0,"pageviews_with_denied_consent":0.0,"pageviews_with_empty_consent":1.0,"pageviews_with_invalid_consent":0.0}
{"window_start":"2022-01-24T15:00:00.000Z","apikey":"7685b6f7-3062-491b-ba50-207f440951dc","domain":"localhost:8086","sdk_type":"sdk-web","device_type":"desktop","device_os":"Windows","experiment_id":"","experiment_group":"","deployment_id":"","consents_asked":0.0,"consents_given":0.0,"consents_given_positive_other":0.0,"consents_given_denied_other":0.0,"consents_given_positive_scroll":0.0,"consents_given_positive_navigate":0.0,"consents_given_positive_click":0.0,"consents_given_denied_click":0.0,"consents_given_positive_external":0.0,"consents_given_denied_external":0.0,"consents_given_positive_webview":0.0,"consents_given_denied_webview":0.0,"consents_given_empty":0.0,"consents_given_invalid":0.0,"pageviews":20.0,"pageviews_with_positive_consent":20.0,"pageviews_with_denied_consent":0.0,"pageviews_with_empty_consent":0.0,"pageviews_with_invalid_consent":0.0}
{"window_start":"2022-01-24T08:00:00.000Z","apikey":"7685b6f7-3062-491b-ba50-207f440951dc","domain":"review-fix-pcp-12-df4ht3--pmp-didomi.netlify.app","sdk_type":"sdk-web","device_type":"desktop","device_os":"Mac","experiment_id":"","experiment_group":"","deployment_id":"","consents_asked":0.0,"consents_given":0.0,"consents_given_positive_other":0.0,"consents_given_denied_other":0.0,"consents_given_positive_scroll":0.0,"consents_given_positive_navigate":0.0,"consents_given_positive_click":0.0,"consents_given_denied_click":0.0,"consents_given_positive_external":0.0,"consents_given_denied_external":0.0,"consents_given_positive_webview":0.0,"consents_given_denied_webview":0.0,"consents_given_empty":0.0,"consents_given_invalid":0.0,"pageviews":1.0,"pageviews_with_positive_consent":0.0,"pageviews_with_denied_consent":0.0,"pageviews_with_empty_consent":1.0,"pageviews_with_invalid_consent":0.0}
{"window_start":"2022-01-24T09:00:00.000Z","apikey":"7685b6f7-3062-491b-ba50-207f440951dc","domain":"review-fix-pcp-12-df4ht3--pmp-didomi.netlify.app","sdk_type":"sdk-web","device_type":"desktop","device_os":"Mac","experiment_id":"","experiment_group":"","deployment_id":"","consents_asked":0.0,"consents_given":0.0,"consents_given_positive_other":0.0,"consents_given_denied_other":0.0,"consents_given_positive_scroll":0.0,"consents_given_positive_navigate":0.0,"consents_given_positive_click":0.0,"consents_given_denied_click":0.0,"consents_given_positive_external":0.0,"consents_given_denied_external":0.0,"consents_given_positive_webview":0.0,"consents_given_denied_webview":0.0,"consents_given_empty":0.0,"consents_given_invalid":0.0,"pageviews":16.0,"pageviews_with_positive_consent":0.0,"pageviews_with_denied_consent":0.0,"pageviews_with_empty_consent":16.0,"pageviews_with_invalid_consent":0.0}
{"window_start":"2022-01-24T10:00:00.000Z","apikey":"7685b6f7-3062-491b-ba50-207f440951dc","domain":"review-fix-pcp-12-df4ht3--pmp-didomi.netlify.app","sdk_type":"sdk-web","device_type":"desktop","device_os":"Mac","experiment_id":"","experiment_group":"","deployment_id":"","consents_asked":0.0,"consents_given":0.0,"consents_given_positive_other":0.0,"consents_given_denied_other":0.0,"consents_given_positive_scroll":0.0,"consents_given_positive_navigate":0.0,"consents_given_positive_click":0.0,"consents_given_denied_click":0.0,"consents_given_positive_external":0.0,"consents_given_denied_external":0.0,"consents_given_positive_webview":0.0,"consents_given_denied_webview":0.0,"consents_given_empty":0.0,"consents_given_invalid":0.0,"pageviews":1.0,"pageviews_with_positive_consent":0.0,"pageviews_with_denied_consent":0.0,"pageviews_with_empty_consent":1.0,"pageviews_with_invalid_consent":0.0}
{"window_start":"2022-01-24T11:00:00.000Z","apikey":"7685b6f7-3062-491b-ba50-207f440951dc","domain":"review-feat-updat-lh42zm--pmp-didomi.netlify.app","sdk_type":"sdk-web","device_type":"desktop","device_os":"Mac","experiment_id":"","experiment_group":"","deployment_id":"","consents_asked":0.0,"consents_given":0.0,"consents_given_positive_other":0.0,"consents_given_denied_other":0.0,"consents_given_positive_scroll":0.0,"consents_given_positive_navigate":0.0,"consents_given_positive_click":0.0,"consents_given_denied_click":0.0,"consents_given_positive_external":0.0,"consents_given_denied_external":0.0,"consents_given_positive_webview":0.0,"consents_given_denied_webview":0.0,"consents_given_empty":0.0,"consents_given_invalid":0.0,"pageviews":2.0,"pageviews_with_positive_consent":0.0,"pageviews_with_denied_consent":0.0,"pageviews_with_empty_consent":2.0,"pageviews_with_invalid_consent":0.0}
{"window_start":"2022-01-24T13:00:00.000Z","apikey":"7685b6f7-3062-491b-ba50-207f440951dc","domain":"review-feat-updat-lh42zm--pmp-didomi.netlify.app","sdk_type":"sdk-web","device_type":"desktop","device_os":"Mac","experiment_id":"","experiment_group":"","deployment_id":"","consents_asked":0.0,"consents_given":0.0,"consents_given_positive_other":0.0,"consents_given_denied_other":0.0,"consents_given_positive_scroll":0.0,"consents_given_positive_navigate":0.0,"consents_given_positive_click":0.0,"consents_given_denied_click":0.0,"consents_given_positive_external":0.0,"consents_given_denied_external":0.0,"consents_given_positive_webview":0.0,"consents_given_denied_webview":0.0,"consents_given_empty":0.0,"consents_given_invalid":0.0,"pageviews":3.0,"pageviews_with_positive_consent":0.0,"pageviews_with_denied_consent":0.0,"pageviews_with_empty_consent":3.0,"pageviews_with_invalid_consent":0.0}
{"window_start":"2022-01-24T14:00:00.000Z","apikey":"7685b6f7-3062-491b-ba50-207f440951dc","domain":"review-fix-pcp-12-df4ht3--pmp-didomi.netlify.app","sdk_type":"sdk-web","device_type":"desktop","device_os":"Mac","experiment_id":"","experiment_group":"","deployment_id":"","consents_asked":0.0,"consents_given":0.0,"consents_given_positive_other":0.0,"consents_given_denied_other":0.0,"consents_given_positive_scroll":0.0,"consents_given_positive_navigate":0.0,"consents_given_positive_click":0.0,"consents_given_denied_click":0.0,"consents_given_positive_external":0.0,"consents_given_denied_external":0.0,"consents_given_positive_webview":0.0,"consents_given_denied_webview":0.0,"consents_given_empty":0.0,"consents_given_invalid":0.0,"pageviews":1.0,"pageviews_with_positive_consent":0.0,"pageviews_with_denied_consent":0.0,"pageviews_with_empty_consent":1.0,"pageviews_with_invalid_consent":0.0}
{"window_start":"2022-01-24T17:00:00.000Z","apikey":"7685b6f7-3062-491b-ba50-207f440951dc","domain":"review-feat-updat-lh42zm--pmp-didomi.netlify.app","sdk_type":"sdk-web","device_type":"desktop","device_os":"Mac","experiment_id":"","experiment_group":"","deployment_id":"","consents_asked":0.0,"consents_given":0.0,"consents_given_positive_other":0.0,"consents_given_denied_other":0.0,"consents_given_positive_scroll":0.0,"consents_given_positive_navigate":0.0,"consents_given_positive_click":0.0,"consents_given_denied_click":0.0,"consents_given_positive_external":0.0,"consents_given_denied_external":0.0,"consents_given_positive_webview":0.0,"consents_given_denied_webview":0.0,"consents_given_empty":0.0,"consents_given_invalid":0.0,"pageviews":1.0,"pageviews_with_positive_consent":0.0,"pageviews_with_denied_consent":0.0,"pageviews_with_empty_consent":1.0,"pageviews_with_invalid_consent":0.0}

Configuration

Please read our dedicated tutorial on configuring an analytics batch export

Last updated