Analytics export

Search…
Analytics export
The Analytics Data Export provides a way to access analytics data (aggregated data) collected through the Didomi platform at once, for CMP. Analytics Data Export is a premium feature.
Destinations
​AWS S3 Bucket​
​GCP Storage Bucket​
Logging and monitoring
Didomi offers the ability to retrieve information and monitor the status of your Analytics export. Read more on logging here.
Format
Data exported at the destination follows the Apache Hive naming conventions for partitioning and exposes 2 partitions:
Partition name
Description
export-id
Unique ID for the configured batch export. This allows you to distinguish files exported between multiple configured exports.
date
The date/hour when the export happened (YYYY-MM-DD format).
Example of a path to the exported data: /data/export-id={batch export ID}/date={YYYY-MM-DD}/
CMP Hourly Metrics
Structure
Didomi exports CMP hourly metrics data in a /cmp_hourly_metrics folder and in files formatted as newline-delimited JSON (one line = one JSON object) and compressed as GZIP. We use the file extention json.gz.
Example: /data/export-id={export-id}/date={YYYY-MM-DD}/cmp_hourly_metrics
For every partition, the exported data can be split up into multiple files. Make sure that you read every .json.gz file in the partition folder to get all the records belonging to that export.
File names are subject to change over time so make sure that your read files from their extension (.json.gz) and not from a specific filename or format.
Additionally, a file named _SUCCESS is created once the export is complete. Do not read files from a partition until the _SUCCESS file is created as you might read partial records otherwise.
Content
The files exported by Didomi contain CMP hourly metrics for the period covered by the export. Every line of the files contains a self-sufficient JSON record (newline-delimited JSON) that includes CMP hourly metrics.
You should first split the files on new lines and then parse every line as an individual JSON record. It is a standard encoding for data processing tools like Hive, Spark, Presto, etc. and should be straightforward to import in your existing tools.
The schema of the objects exported is as follows:
Column Name
Example
Data Type
Description
window_start
06/07/21 21:00
datetime
Day and hour of the data collected (YYYY-MM-DD-HH)
apikey
e23a01f6-a508-4e71-8f50-c1a9cae7c0d0
Text
API key of the organization
device_os
iOS
Text
OS info are regrouped under the following families: Windows, GNU/Linux, Mac, Android, iOS, Android TV, tvOS, Fire TV and other.
device_type
desktop
Text
Type of device: desktop, smartphone, phablet, tablet, television and other.
​deployment_id​
​
Text
​Version ID of the PUBLISHED notice. Named production_config_id of deployment_id in the API.​
domain
​www.journaldugolf.fr​
Text
subdomain or app ID where the consent is collected
​experiment_group​
control or test
Text
Only filled when a AB test is running. One test is the test group the other one is the control group
​experiment_id​
​
Text
Only filled when a AB test is running ID of the test. Defined by the client
sdk_type
sdk-web
Text
Type of SDK: sdk-mobile, sdk-app, sdk-ctv or sdk-amp
consents_asked
21
integer
Number of consent asked (from consent.asked event - sampled)
consents_given
21
integer
Number of consent given (from consent.given event - not sampled)
consents_given_denied_click
21
integer
Number of denied consent given (from consent.given event - not sampled) - click
consents_given_denied_external
21
integer
Number of denied consent given (from consent.given event - not sampled) - external
consents_given_denied_other
21
integer
Number of denied consent given (from consent.given event - not sampled) - do not fall into the other categories
consents_given_denied_webview
21
integer
Number of denied consent given (from consent.given event - not sampled) - webview
consents_given_empty
21
integer
Number of consent given (from consent.given event - not sampled)
consents_given_invalid
21
integer
Number of invalid consent given (from consent.given event - not sampled)
consents_given_positive_click
21
integer
Number of positive consent given (from consent.given event - not sampled) - click
consents_given_positive_external
21
integer
Number of positive consent given (from consent.given event - not sampled) - external
consents_given_positive_navigate
21
integer
Number of positive consent given (from consent.given event - not sampled) - navigate
consents_given_positive_other
21
integer
Number of positive consent given (from consent.given event - not sampled) - do not fall into the other categories
consents_given_positive_scroll
21
integer
Number of positive consent given (from consent.given event - not sampled) - scroll
consents_given_positive_webview
21
integer
Number of positive consent given (from consent.given event - not sampled) - webview
pageviews
21
integer
Number of pageviews/app sessions (from pageview events - sampled)
pageviews_with_denied_consent
21
integer
Number of denied consent pageviews/app sessions (from pageview events - sampled)
pageviews_with_empty_consent
21
integer
Number of empty consent pageviews/app sessions (from pageview events - sampled)
pageviews_with_invalid_consent
21
integer
Number of invalid consent pageviews/app sessions (from pageview events - sampled)
pageviews_with_positive_consent
21
integer
Number of positive consent pageviews/app sessions (from pageview events - sampled)
median_lead_time
not supported anymore
​
​
Example of an exported file:
{"window_start":"2022-01-24T08:00:00.000Z","apikey":"7685b6f7-3062-491b-ba50-207f440951dc","domain":"io.didomi.sampleuitest","sdk_type":"sdk-ctv","device_type":"television","device_os":"Android TV","experiment_id":"","experiment_group":"","deployment_id":"6ExpWCHh","consents_asked":5.0,"consents_given":0.0,"consents_given_positive_other":0.0,"consents_given_denied_other":0.0,"consents_given_positive_scroll":0.0,"consents_given_positive_navigate":0.0,"consents_given_positive_click":0.0,"consents_given_denied_click":0.0,"consents_given_positive_external":0.0,"consents_given_denied_external":0.0,"consents_given_positive_webview":0.0,"consents_given_denied_webview":0.0,"consents_given_empty":0.0,"consents_given_invalid":0.0,"pageviews":0.0,"pageviews_with_positive_consent":0.0,"pageviews_with_denied_consent":0.0,"pageviews_with_empty_consent":0.0,"pageviews_with_invalid_consent":0.0}
{"window_start":"2022-01-24T14:00:00.000Z","apikey":"7685b6f7-3062-491b-ba50-207f440951dc","domain":"sandbox.app.domain","sdk_type":"sdk-web","device_type":"desktop","device_os":"Mac","experiment_id":"","experiment_group":"","deployment_id":"RDkUQWNK","consents_asked":0.0,"consents_given":0.0,"consents_given_positive_other":0.0,"consents_given_denied_other":0.0,"consents_given_positive_scroll":0.0,"consents_given_positive_navigate":0.0,"consents_given_positive_click":0.0,"consents_given_denied_click":0.0,"consents_given_positive_external":0.0,"consents_given_denied_external":0.0,"consents_given_positive_webview":0.0,"consents_given_denied_webview":0.0,"consents_given_empty":0.0,"consents_given_invalid":0.0,"pageviews":1.0,"pageviews_with_positive_consent":0.0,"pageviews_with_denied_consent":0.0,"pageviews_with_empty_consent":1.0,"pageviews_with_invalid_consent":0.0}
{"window_start":"2022-01-24T15:00:00.000Z","apikey":"7685b6f7-3062-491b-ba50-207f440951dc","domain":"localhost:8086","sdk_type":"sdk-web","device_type":"desktop","device_os":"Windows","experiment_id":"","experiment_group":"","deployment_id":"","consents_asked":0.0,"consents_given":0.0,"consents_given_positive_other":0.0,"consents_given_denied_other":0.0,"consents_given_positive_scroll":0.0,"consents_given_positive_navigate":0.0,"consents_given_positive_click":0.0,"consents_given_denied_click":0.0,"consents_given_positive_external":0.0,"consents_given_denied_external":0.0,"consents_given_positive_webview":0.0,"consents_given_denied_webview":0.0,"consents_given_empty":0.0,"consents_given_invalid":0.0,"pageviews":20.0,"pageviews_with_positive_consent":20.0,"pageviews_with_denied_consent":0.0,"pageviews_with_empty_consent":0.0,"pageviews_with_invalid_consent":0.0}
{"window_start":"2022-01-24T08:00:00.000Z","apikey":"7685b6f7-3062-491b-ba50-207f440951dc","domain":"review-fix-pcp-12-df4ht3--pmp-didomi.netlify.app","sdk_type":"sdk-web","device_type":"desktop","device_os":"Mac","experiment_id":"","experiment_group":"","deployment_id":"","consents_asked":0.0,"consents_given":0.0,"consents_given_positive_other":0.0,"consents_given_denied_other":0.0,"consents_given_positive_scroll":0.0,"consents_given_positive_navigate":0.0,"consents_given_positive_click":0.0,"consents_given_denied_click":0.0,"consents_given_positive_external":0.0,"consents_given_denied_external":0.0,"consents_given_positive_webview":0.0,"consents_given_denied_webview":0.0,"consents_given_empty":0.0,"consents_given_invalid":0.0,"pageviews":1.0,"pageviews_with_positive_consent":0.0,"pageviews_with_denied_consent":0.0,"pageviews_with_empty_consent":1.0,"pageviews_with_invalid_consent":0.0}
{"window_start":"2022-01-24T09:00:00.000Z","apikey":"7685b6f7-3062-491b-ba50-207f440951dc","domain":"review-fix-pcp-12-df4ht3--pmp-didomi.netlify.app","sdk_type":"sdk-web","device_type":"desktop","device_os":"Mac","experiment_id":"","experiment_group":"","deployment_id":"","consents_asked":0.0,"consents_given":0.0,"consents_given_positive_other":0.0,"consents_given_denied_other":0.0,"consents_given_positive_scroll":0.0,"consents_given_positive_navigate":0.0,"consents_given_positive_click":0.0,"consents_given_denied_click":0.0,"consents_given_positive_external":0.0,"consents_given_denied_external":0.0,"consents_given_positive_webview":0.0,"consents_given_denied_webview":0.0,"consents_given_empty":0.0,"consents_given_invalid":0.0,"pageviews":16.0,"pageviews_with_positive_consent":0.0,"pageviews_with_denied_consent":0.0,"pageviews_with_empty_consent":16.0,"pageviews_with_invalid_consent":0.0}
{"window_start":"2022-01-24T10:00:00.000Z","apikey":"7685b6f7-3062-491b-ba50-207f440951dc","domain":"review-fix-pcp-12-df4ht3--pmp-didomi.netlify.app","sdk_type":"sdk-web","device_type":"desktop","device_os":"Mac","experiment_id":"","experiment_group":"","deployment_id":"","consents_asked":0.0,"consents_given":0.0,"consents_given_positive_other":0.0,"consents_given_denied_other":0.0,"consents_given_positive_scroll":0.0,"consents_given_positive_navigate":0.0,"consents_given_positive_click":0.0,"consents_given_denied_click":0.0,"consents_given_positive_external":0.0,"consents_given_denied_external":0.0,"consents_given_positive_webview":0.0,"consents_given_denied_webview":0.0,"consents_given_empty":0.0,"consents_given_invalid":0.0,"pageviews":1.0,"pageviews_with_positive_consent":0.0,"pageviews_with_denied_consent":0.0,"pageviews_with_empty_consent":1.0,"pageviews_with_invalid_consent":0.0}
{"window_start":"2022-01-24T11:00:00.000Z","apikey":"7685b6f7-3062-491b-ba50-207f440951dc","domain":"review-feat-updat-lh42zm--pmp-didomi.netlify.app","sdk_type":"sdk-web","device_type":"desktop","device_os":"Mac","experiment_id":"","experiment_group":"","deployment_id":"","consents_asked":0.0,"consents_given":0.0,"consents_given_positive_other":0.0,"consents_given_denied_other":0.0,"consents_given_positive_scroll":0.0,"consents_given_positive_navigate":0.0,"consents_given_positive_click":0.0,"consents_given_denied_click":0.0,"consents_given_positive_external":0.0,"consents_given_denied_external":0.0,"consents_given_positive_webview":0.0,"consents_given_denied_webview":0.0,"consents_given_empty":0.0,"consents_given_invalid":0.0,"pageviews":2.0,"pageviews_with_positive_consent":0.0,"pageviews_with_denied_consent":0.0,"pageviews_with_empty_consent":2.0,"pageviews_with_invalid_consent":0.0}
{"window_start":"2022-01-24T13:00:00.000Z","apikey":"7685b6f7-3062-491b-ba50-207f440951dc","domain":"review-feat-updat-lh42zm--pmp-didomi.netlify.app","sdk_type":"sdk-web","device_type":"desktop","device_os":"Mac","experiment_id":"","experiment_group":"","deployment_id":"","consents_asked":0.0,"consents_given":0.0,"consents_given_positive_other":0.0,"consents_given_denied_other":0.0,"consents_given_positive_scroll":0.0,"consents_given_positive_navigate":0.0,"consents_given_positive_click":0.0,"consents_given_denied_click":0.0,"consents_given_positive_external":0.0,"consents_given_denied_external":0.0,"consents_given_positive_webview":0.0,"consents_given_denied_webview":0.0,"consents_given_empty":0.0,"consents_given_invalid":0.0,"pageviews":3.0,"pageviews_with_positive_consent":0.0,"pageviews_with_denied_consent":0.0,"pageviews_with_empty_consent":3.0,"pageviews_with_invalid_consent":0.0}
{"window_start":"2022-01-24T14:00:00.000Z","apikey":"7685b6f7-3062-491b-ba50-207f440951dc","domain":"review-fix-pcp-12-df4ht3--pmp-didomi.netlify.app","sdk_type":"sdk-web","device_type":"desktop","device_os":"Mac","experiment_id":"","experiment_group":"","deployment_id":"","consents_asked":0.0,"consents_given":0.0,"consents_given_positive_other":0.0,"consents_given_denied_other":0.0,"consents_given_positive_scroll":0.0,"consents_given_positive_navigate":0.0,"consents_given_positive_click":0.0,"consents_given_denied_click":0.0,"consents_given_positive_external":0.0,"consents_given_denied_external":0.0,"consents_given_positive_webview":0.0,"consents_given_denied_webview":0.0,"consents_given_empty":0.0,"consents_given_invalid":0.0,"pageviews":1.0,"pageviews_with_positive_consent":0.0,"pageviews_with_denied_consent":0.0,"pageviews_with_empty_consent":1.0,"pageviews_with_invalid_consent":0.0}
{"window_start":"2022-01-24T17:00:00.000Z","apikey":"7685b6f7-3062-491b-ba50-207f440951dc","domain":"review-feat-updat-lh42zm--pmp-didomi.netlify.app","sdk_type":"sdk-web","device_type":"desktop","device_os":"Mac","experiment_id":"","experiment_group":"","deployment_id":"","consents_asked":0.0,"consents_given":0.0,"consents_given_positive_other":0.0,"consents_given_denied_other":0.0,"consents_given_positive_scroll":0.0,"consents_given_positive_navigate":0.0,"consents_given_positive_click":0.0,"consents_given_denied_click":0.0,"consents_given_positive_external":0.0,"consents_given_denied_external":0.0,"consents_given_positive_webview":0.0,"consents_given_denied_webview":0.0,"consents_given_empty":0.0,"consents_given_invalid":0.0,"pageviews":1.0,"pageviews_with_positive_consent":0.0,"pageviews_with_denied_consent":0.0,"pageviews_with_empty_consent":1.0,"pageviews_with_invalid_consent":0.0}
Configuration
Please read our dedicated tutorial on configuring an analytics batch export
Batch import
Destinations
Last modified 3mo ago
Copy link
On this page
Destinations
Logging and monitoring
Format
CMP Hourly Metrics