Analytics export

The Analytics Data Export provides a way to access analytics data (aggregated data) collected through the Didomi platform at once, for CMP. Analytics Data Export is a premium feature.

Destinations

Logging and monitoring

Didomi offers the ability to retrieve information and monitor the status of your Analytics export. Read more on logging here.

Format

Data exported at the destination follows the Apache Hive naming conventions for partitioning and exposes 2 partitions:

Partition nameDescription

export-id

Unique ID for the configured batch export. This allows you to distinguish files exported between multiple configured exports.

date

The date/hour when the export happened (YYYY-MM-DD format).

Example of a path to the exported data: /data/export-id={batch export ID}/date={YYYY-MM-DD}/

CMP Hourly Metrics

Structure

Didomi exports CMP hourly metrics data in a /cmp_hourly_metrics folder and in files formatted as newline-delimited JSON (one line = one JSON object) and compressed as GZIP. We use the file extension json.gz.

Example: /data/export-id={export-id}/date={YYYY-MM-DD}/cmp_hourly_metrics

For every partition, the exported data can be split up into multiple files. Make sure that you read every .json.gz file in the partition folder to get all the records belonging to that export.

File names are subject to change over time so make sure that your read files from their extension (.json.gz) and not from a specific filename or format.

Additionally, a file named _SUCCESS is created once the export is complete. Do not read files from a partition until the _SUCCESS file is created as you might read partial records otherwise.

Content

The files exported by Didomi contain CMP hourly metrics for the period covered by the export. Every line of the files contains a self-sufficient JSON record (newline-delimited JSON) that includes CMP hourly metrics.

You should first split the files on new lines and then parse every line as an individual JSON record. It is a standard encoding for data processing tools like Hive, Spark, Presto, etc. and should be straightforward to import in your existing tools.

The schema of the objects exported is as follows:

Column NameExampleData TypeDescription

window_start

06/07/21 21:00

datetime

Day and hour of the data collected (YYYY-MM-DD-HH)

apikey

e23a01f6-a508-4e71-8f50-c1a9cae7c0d0

Text

API key of the organization

device_os

iOS

Text

OS info are regrouped under the following families: Windows, GNU/Linux, Mac, Android, iOS, Android TV, tvOS, Fire TV and other.

device_type

desktop

Text

Type of device: desktop, smartphone, phablet, tablet, television and other.

deployment_id

Text

Version ID of the PUBLISHED notice. Named production_config_id of deployment_id in the API.

domain

www.journaldugolf.fr

Text

subdomain or app ID where the consent is collected

experiment_group

control or test

Text

Only filled when a AB test is running. One test is the test group the other one is the control group

experiment_id

Text

Only filled when a AB test is running ID of the test. Defined by the client

sdk_type

sdk-web

Text

Type of SDK: sdk-mobile, sdk-app, sdk-ctv or sdk-amp

country

FR

Text

ID of the user country

browser_family

Chrome

Text

Browser info are regrouped under the following list of browser families: Safari, Firefox, Chrome, Samsung Browser, Facebook, Microsoft Edge, Internet Explorer and Didomi SDK (is used for the native app)

region

CA

Text

ID of the user state (for US and Canada only). Value is empty when the user country is not US nor Canada

consents_asked

21

integer

Number of consent asked (from consent.asked event - sampled)

consents_given

21

integer

Number of consent given (from consent.given event - not sampled)

consents_given_denied_click

21

integer

Number of denied consent given (from consent.given event - not sampled) - click

consents_given_denied_external

21

integer

Number of denied consent given (from consent.given event - not sampled) - external

consents_given_denied_other

21

integer

Number of denied consent given (from consent.given event - not sampled) - do not fall into the other categories

consents_given_denied_webview

21

integer

Number of denied consent given (from consent.given event - not sampled) - webview

consents_given_empty

21

integer

Number of consent given (from consent.given event - not sampled)

consents_given_invalid

21

integer

Number of invalid consent given (from consent.given event - not sampled)

consents_given_positive_click

21

integer

Number of positive consent given (from consent.given event - not sampled) - click

consents_given_positive_external

21

integer

Number of positive consent given (from consent.given event - not sampled) - external

consents_given_positive_navigate

21

integer

Number of positive consent given (from consent.given event - not sampled) - navigate

consents_given_positive_other

21

integer

Number of positive consent given (from consent.given event - not sampled) - do not fall into the other categories

consents_given_positive_scroll

21

integer

Number of positive consent given (from consent.given event - not sampled) - scroll

consents_given_positive_webview

21

integer

Number of positive consent given (from consent.given event - not sampled) - webview

pageviews

21

integer

Number of pageviews/app sessions (from pageview events - sampled)

pageviews_with_denied_consent

21

integer

Number of denied consent pageviews/app sessions (from pageview events - sampled)

pageviews_with_empty_consent

21

integer

Number of empty consent pageviews/app sessions (from pageview events - sampled)

pageviews_with_invalid_consent

21

integer

Number of invalid consent pageviews/app sessions (from pageview events - sampled)

pageviews_with_positive_consent

21

integer

Number of positive consent pageviews/app sessions (from pageview events - sampled)

median_lead_time

not supported anymore

Example of an exported file:

{"window_start":"2022-01-24T08:00:00.000Z","apikey":"7685b6f7-3062-491b-ba50-207f440951dc","domain":"io.didomi.sampleuitest","sdk_type":"sdk-ctv","device_type":"television","device_os":"Android TV","experiment_id":"","experiment_group":"","deployment_id":"6ExpWCHh","consents_asked":5.0,"consents_given":0.0,"consents_given_positive_other":0.0,"consents_given_denied_other":0.0,"consents_given_positive_scroll":0.0,"consents_given_positive_navigate":0.0,"consents_given_positive_click":0.0,"consents_given_denied_click":0.0,"consents_given_positive_external":0.0,"consents_given_denied_external":0.0,"consents_given_positive_webview":0.0,"consents_given_denied_webview":0.0,"consents_given_empty":0.0,"consents_given_invalid":0.0,"pageviews":0.0,"pageviews_with_positive_consent":0.0,"pageviews_with_denied_consent":0.0,"pageviews_with_empty_consent":0.0,"pageviews_with_invalid_consent":0.0}
{"window_start":"2022-01-24T14:00:00.000Z","apikey":"7685b6f7-3062-491b-ba50-207f440951dc","domain":"sandbox.app.domain","sdk_type":"sdk-web","device_type":"desktop","device_os":"Mac","experiment_id":"","experiment_group":"","deployment_id":"RDkUQWNK","consents_asked":0.0,"consents_given":0.0,"consents_given_positive_other":0.0,"consents_given_denied_other":0.0,"consents_given_positive_scroll":0.0,"consents_given_positive_navigate":0.0,"consents_given_positive_click":0.0,"consents_given_denied_click":0.0,"consents_given_positive_external":0.0,"consents_given_denied_external":0.0,"consents_given_positive_webview":0.0,"consents_given_denied_webview":0.0,"consents_given_empty":0.0,"consents_given_invalid":0.0,"pageviews":1.0,"pageviews_with_positive_consent":0.0,"pageviews_with_denied_consent":0.0,"pageviews_with_empty_consent":1.0,"pageviews_with_invalid_consent":0.0}
{"window_start":"2022-01-24T15:00:00.000Z","apikey":"7685b6f7-3062-491b-ba50-207f440951dc","domain":"localhost:8086","sdk_type":"sdk-web","device_type":"desktop","device_os":"Windows","experiment_id":"","experiment_group":"","deployment_id":"","consents_asked":0.0,"consents_given":0.0,"consents_given_positive_other":0.0,"consents_given_denied_other":0.0,"consents_given_positive_scroll":0.0,"consents_given_positive_navigate":0.0,"consents_given_positive_click":0.0,"consents_given_denied_click":0.0,"consents_given_positive_external":0.0,"consents_given_denied_external":0.0,"consents_given_positive_webview":0.0,"consents_given_denied_webview":0.0,"consents_given_empty":0.0,"consents_given_invalid":0.0,"pageviews":20.0,"pageviews_with_positive_consent":20.0,"pageviews_with_denied_consent":0.0,"pageviews_with_empty_consent":0.0,"pageviews_with_invalid_consent":0.0}
{"window_start":"2022-01-24T08:00:00.000Z","apikey":"7685b6f7-3062-491b-ba50-207f440951dc","domain":"review-fix-pcp-12-df4ht3--pmp-didomi.netlify.app","sdk_type":"sdk-web","device_type":"desktop","device_os":"Mac","experiment_id":"","experiment_group":"","deployment_id":"","consents_asked":0.0,"consents_given":0.0,"consents_given_positive_other":0.0,"consents_given_denied_other":0.0,"consents_given_positive_scroll":0.0,"consents_given_positive_navigate":0.0,"consents_given_positive_click":0.0,"consents_given_denied_click":0.0,"consents_given_positive_external":0.0,"consents_given_denied_external":0.0,"consents_given_positive_webview":0.0,"consents_given_denied_webview":0.0,"consents_given_empty":0.0,"consents_given_invalid":0.0,"pageviews":1.0,"pageviews_with_positive_consent":0.0,"pageviews_with_denied_consent":0.0,"pageviews_with_empty_consent":1.0,"pageviews_with_invalid_consent":0.0}
{"window_start":"2022-01-24T09:00:00.000Z","apikey":"7685b6f7-3062-491b-ba50-207f440951dc","domain":"review-fix-pcp-12-df4ht3--pmp-didomi.netlify.app","sdk_type":"sdk-web","device_type":"desktop","device_os":"Mac","experiment_id":"","experiment_group":"","deployment_id":"","consents_asked":0.0,"consents_given":0.0,"consents_given_positive_other":0.0,"consents_given_denied_other":0.0,"consents_given_positive_scroll":0.0,"consents_given_positive_navigate":0.0,"consents_given_positive_click":0.0,"consents_given_denied_click":0.0,"consents_given_positive_external":0.0,"consents_given_denied_external":0.0,"consents_given_positive_webview":0.0,"consents_given_denied_webview":0.0,"consents_given_empty":0.0,"consents_given_invalid":0.0,"pageviews":16.0,"pageviews_with_positive_consent":0.0,"pageviews_with_denied_consent":0.0,"pageviews_with_empty_consent":16.0,"pageviews_with_invalid_consent":0.0}
{"window_start":"2022-01-24T10:00:00.000Z","apikey":"7685b6f7-3062-491b-ba50-207f440951dc","domain":"review-fix-pcp-12-df4ht3--pmp-didomi.netlify.app","sdk_type":"sdk-web","device_type":"desktop","device_os":"Mac","experiment_id":"","experiment_group":"","deployment_id":"","consents_asked":0.0,"consents_given":0.0,"consents_given_positive_other":0.0,"consents_given_denied_other":0.0,"consents_given_positive_scroll":0.0,"consents_given_positive_navigate":0.0,"consents_given_positive_click":0.0,"consents_given_denied_click":0.0,"consents_given_positive_external":0.0,"consents_given_denied_external":0.0,"consents_given_positive_webview":0.0,"consents_given_denied_webview":0.0,"consents_given_empty":0.0,"consents_given_invalid":0.0,"pageviews":1.0,"pageviews_with_positive_consent":0.0,"pageviews_with_denied_consent":0.0,"pageviews_with_empty_consent":1.0,"pageviews_with_invalid_consent":0.0}
{"window_start":"2022-01-24T11:00:00.000Z","apikey":"7685b6f7-3062-491b-ba50-207f440951dc","domain":"review-feat-updat-lh42zm--pmp-didomi.netlify.app","sdk_type":"sdk-web","device_type":"desktop","device_os":"Mac","experiment_id":"","experiment_group":"","deployment_id":"","consents_asked":0.0,"consents_given":0.0,"consents_given_positive_other":0.0,"consents_given_denied_other":0.0,"consents_given_positive_scroll":0.0,"consents_given_positive_navigate":0.0,"consents_given_positive_click":0.0,"consents_given_denied_click":0.0,"consents_given_positive_external":0.0,"consents_given_denied_external":0.0,"consents_given_positive_webview":0.0,"consents_given_denied_webview":0.0,"consents_given_empty":0.0,"consents_given_invalid":0.0,"pageviews":2.0,"pageviews_with_positive_consent":0.0,"pageviews_with_denied_consent":0.0,"pageviews_with_empty_consent":2.0,"pageviews_with_invalid_consent":0.0}
{"window_start":"2022-01-24T13:00:00.000Z","apikey":"7685b6f7-3062-491b-ba50-207f440951dc","domain":"review-feat-updat-lh42zm--pmp-didomi.netlify.app","sdk_type":"sdk-web","device_type":"desktop","device_os":"Mac","experiment_id":"","experiment_group":"","deployment_id":"","consents_asked":0.0,"consents_given":0.0,"consents_given_positive_other":0.0,"consents_given_denied_other":0.0,"consents_given_positive_scroll":0.0,"consents_given_positive_navigate":0.0,"consents_given_positive_click":0.0,"consents_given_denied_click":0.0,"consents_given_positive_external":0.0,"consents_given_denied_external":0.0,"consents_given_positive_webview":0.0,"consents_given_denied_webview":0.0,"consents_given_empty":0.0,"consents_given_invalid":0.0,"pageviews":3.0,"pageviews_with_positive_consent":0.0,"pageviews_with_denied_consent":0.0,"pageviews_with_empty_consent":3.0,"pageviews_with_invalid_consent":0.0}
{"window_start":"2022-01-24T14:00:00.000Z","apikey":"7685b6f7-3062-491b-ba50-207f440951dc","domain":"review-fix-pcp-12-df4ht3--pmp-didomi.netlify.app","sdk_type":"sdk-web","device_type":"desktop","device_os":"Mac","experiment_id":"","experiment_group":"","deployment_id":"","consents_asked":0.0,"consents_given":0.0,"consents_given_positive_other":0.0,"consents_given_denied_other":0.0,"consents_given_positive_scroll":0.0,"consents_given_positive_navigate":0.0,"consents_given_positive_click":0.0,"consents_given_denied_click":0.0,"consents_given_positive_external":0.0,"consents_given_denied_external":0.0,"consents_given_positive_webview":0.0,"consents_given_denied_webview":0.0,"consents_given_empty":0.0,"consents_given_invalid":0.0,"pageviews":1.0,"pageviews_with_positive_consent":0.0,"pageviews_with_denied_consent":0.0,"pageviews_with_empty_consent":1.0,"pageviews_with_invalid_consent":0.0}
{"window_start":"2022-01-24T17:00:00.000Z","apikey":"7685b6f7-3062-491b-ba50-207f440951dc","domain":"review-feat-updat-lh42zm--pmp-didomi.netlify.app","sdk_type":"sdk-web","device_type":"desktop","device_os":"Mac","experiment_id":"","experiment_group":"","deployment_id":"","consents_asked":0.0,"consents_given":0.0,"consents_given_positive_other":0.0,"consents_given_denied_other":0.0,"consents_given_positive_scroll":0.0,"consents_given_positive_navigate":0.0,"consents_given_positive_click":0.0,"consents_given_denied_click":0.0,"consents_given_positive_external":0.0,"consents_given_denied_external":0.0,"consents_given_positive_webview":0.0,"consents_given_denied_webview":0.0,"consents_given_empty":0.0,"consents_given_invalid":0.0,"pageviews":1.0,"pageviews_with_positive_consent":0.0,"pageviews_with_denied_consent":0.0,"pageviews_with_empty_consent":1.0,"pageviews_with_invalid_consent":0.0}

Configuration

Please read our dedicated tutorial on configuring an analytics batch export

PreviousBatch importNextDestinations

Last updated