Data File Specifications

Data File Specifications

Regardless of the choices made on transfer type and file type above, Lotame requires three files each day for us to process your segments into the Spherical platform successfully. 

 Segment Membership File

This is the main data file with a static filename of segmentmembership.json.gz. This file is GZIP encoded to be space efficient and minimize transfer timings and associated transfer costs.

Each row in the file is a JSON object for an ID. The ID should be unique in the file. The JSON object must be on a single row without line breaks. Details of the JSON object are in the appropriate use-case document on our Overview page. 

FILE SIZE NOTICE

If you expect at any time you may deliver files larger than 5GB GZIP'd, please let your account manager know and follow the Splitting Large Files section below.

Done File

The Done file has a filename of YYYYMMDD.done (the .done extension is case-sensitive). This zero-length file should be created when all processing on your end is complete and ready for Spherical to import the data. Processing on our end will poll for this file before importing the Segment Membership file.

This ZIP file at the top of the document has examples of each file for your reference when sending behaviors to Spherical.

Checksum File (Optional)

This file has a static filename of segmentmembership.json.gz.md5. This plain-text file only contains the MD5 checksum of the Segment Membership file discussed above. An example terminal command to create the file is:

md5sum segmentmembership.json.gz > segmentmembership.json.gz.md5

This file is optional to send for extra validation that your segmentmembership.json.gz file was successfully transferred. If used, Lotame will validate the MD5 checksum against the segmentmembership.json.gz file before processing. If you choose to use it, please inform your account manager so we can enable this extra processing.

Splitting Large Files

Only use this functionality if you ever expect to need to deliver a segmentmembership.json.gz in a single day that would be greater than 5GB after GZIP.

This scenario is needed when your deliveries of the segmentmembership.json.gz file may be greater than 5GB after GZIP. In this scenario, Lotame asks that you split your files so each is smaller than 5GB when GZIP'd. To support this, you need to adjust the file naming by adding a numeric identifier to the names of the JSON and MD5 files, as shown below. You should also notify your Technical Account Manager of the change so Lotame can enable our processing to look for the different file names.

• segmentmembership.001.json.gz & segmentmembership.001.json.gz.md5
• segmentmembership.002.json.gz & segmentmembership.002.json.gz.md5
• ...
• segmentmembership.###.json.gz & segmentmembership.###.json.gz.md5

There is still only one YYYYMMDD.done file to upload once you have completed all JSON and MD5 file uploads.

Note: This is meant to handle a handful of ~5GB files if you need to send more than 5GB per day. Do not send a lot of small files, instead batch them up to larger files to streamline your data loading.