DocumentationExperience PlatformDestinations Guide

Batch configuration batch-configuration

Last update: Tue Nov 07 2023 00:00:00 GMT+0000 (Coordinated Universal Time)

CREATED FOR:

  • Developer
  • User
  • Admin
  • Leader

Use the batch configuration options in Destination SDK to allow users to customize the exported file names and to configure the export schedule according to their preference.

When you create file-based destinations through Destination SDK, you can configure default file naming and export schedules, or you can give users the option to configure these settings from the Platform UI. For instance, you can configure behaviors such as:

  • Including specific information in the file name, such as audience IDs, destination IDs, or custom information.
  • Allowing users to customize the file naming from the Platform UI.
  • Configure file exports to occur at set time intervals.
  • Define what file naming and export schedule customization options the users can see in the Platform UI.

Batch configuration settings are part of the destination configuration for file-based destinations.

To understand where this component fits into an integration created with Destination SDK, see the diagram in the configuration options documentation or see the guide on how to use Destination SDK to configure a file-based destination.

You can configure the file naming and export schedule settings via the /authoring/destinations endpoint. See the following API reference pages for detailed API call examples where you can configure the components shown in this page.

This article describes all the supported batch configuration options that you can use for your destination, and shows what customers will see in the Platform UI.

IMPORTANT
All parameter names and values supported by Destination SDK are case sensitive. To avoid case sensitivity errors, please use the parameters names and values exactly as shown in the documentation.

Supported integration types supported-integration-types

Refer to the table below for details on which types of integrations support the functionality described on this page.

Integration type
Supports functionality
Real-time (streaming) integrations
No
File-based (batch) integrations
Yes

Supported parameters supported-parameters

The values that you set up here are surfaced in the Schedule audience export step of the file-based destinations activation workflow.

"batchConfig":{
   "allowMandatoryFieldSelection":true,
   "allowDedupeKeyFieldSelection":true,
   "defaultExportMode":"DAILY_FULL_EXPORT",
   "allowedExportMode":[
      "DAILY_FULL_EXPORT",
      "FIRST_FULL_THEN_INCREMENTAL"
   ],
   "allowedScheduleFrequency":[
      "DAILY",
      "EVERY_3_HOURS",
      "EVERY_6_HOURS",
      "EVERY_8_HOURS",
      "EVERY_12_HOURS",
      "ONCE"
   ],
   "defaultFrequency":"DAILY",
   "defaultStartTime":"00:00",
   "filenameConfig":{
         "allowedFilenameAppendOptions":[
            "SEGMENT_NAME",
            "DESTINATION_INSTANCE_ID",
            "DESTINATION_INSTANCE_NAME",
            "ORGANIZATION_NAME",
            "SANDBOX_NAME",
            "DATETIME",
            "CUSTOM_TEXT"
         ],
         "defaultFilenameAppendOptions":[
            "DATETIME"
         ],
         "defaultFilename":"%DESTINATION%_%SEGMENT_ID%"
      },
   "segmentGroupingEnabled": true
   }
Parameter
Type
Description
allowMandatoryFieldSelection
Boolean
Set to true to allow customers to specify which profile attributes are mandatory. Default value is false. See Mandatory attributes for more information.
allowDedupeKeyFieldSelection
Boolean
Set to true to allow customers to specify deduplication keys. Default value is false. See Deduplication keys for more information.
defaultExportMode
Enum

Defines the default file export mode. Supported values:

  • DAILY_FULL_EXPORT
  • FIRST_FULL_THEN_INCREMENTAL

Default value is DAILY_FULL_EXPORT. See the batch activation documentation for details about file exports scheduling.

allowedExportModes
List

Defines the file export modes available to customers. Supported values:

  • DAILY_FULL_EXPORT
  • FIRST_FULL_THEN_INCREMENTAL
allowedScheduleFrequency
List

Defines the file export frequency available to customers. Supported values:

  • ONCE
  • EVERY_3_HOURS
  • EVERY_6_HOURS
  • EVERY_8_HOURS
  • EVERY_12_HOURS
  • DAILY
defaultFrequency
Enum

Defines the default file export frequency.Supported values:

  • ONCE
  • EVERY_3_HOURS
  • EVERY_6_HOURS
  • EVERY_8_HOURS
  • EVERY_12_HOURS
  • DAILY

Default value is DAILY.

defaultStartTime
String
Defines the default start time for the file export. Uses 24-hour file format. Default value is “00:00”.
filenameConfig.allowedFilenameAppendOptions
String

Required. List of available file name macros for users to choose from. This determines which items are appended to exported file names (audience ID, organization name, date and time of export, and others). When setting defaultFilename, make sure to avoid duplicating macros.

Supported values:

  • DESTINATION
  • SEGMENT_ID
  • SEGMENT_NAME
  • DESTINATION_INSTANCE_ID
  • DESTINATION_INSTANCE_NAME
  • ORGANIZATION_NAME
  • SANDBOX_NAME
  • DATETIME
  • CUSTOM_TEXT

Regardless of the order in which you define the macros, the Experience Platform UI will always display them in the order presented here.

If defaultFilename is empty, the allowedFilenameAppendOptions list must contain at least one macro.

filenameConfig.defaultFilenameAppendOptions
String
Required. Pre-selected default file name macros that users can uncheck.

The macros in this list are a subset of the ones defined in allowedFilenameAppendOptions.
filenameConfig.defaultFilename
String
Optional. Defines the default file name macros for the exported files. These cannot be overwritten by users.

Any macro defined by allowedFilenameAppendOptions will be appended after the defaultFilename macros.

If defaultFilename is empty, you must define at least one macro in allowedFilenameAppendOptions.
segmentGroupingEnabled
Boolean

Defines whether the activated audiences should be exported in a single file or multiple files, based on audience merge policy. Supported values:

  • true: exports one file per merge policy.
  • false: exports one file per audience, regardless of the merge policy. This is the default behavior. You can achieve the same result by ommitting this paramter entirely.

File name configuration file-name-configuration

Use file name configuration macros to define what the exported file names should include. The macros in the table below describe elements found in the UI in the file name configuration screen.

TIP
As a best practice, you should always include the SEGMENT_ID macro in your exported file names. Segment IDs are unique, so including them in the file name is the best way to ensure that file names are unique as well.
Macro
UI label
Description
Example
DESTINATION
Destination
Destination name in the UI.
Amazon S3
SEGMENT_ID
Segment ID
Unique, Platform-generated audience ID
ce5c5482-2813-4a80-99bc-57113f6acde2
SEGMENT_NAME
Segment Name
User-defined audience name
VIP subscriber
DESTINATION_INSTANCE_ID
Destination ID
Unique, Platform-generated ID of the destination instance
7b891e5f-025a-4f0d-9e73-1919e71da3b0
DESTINATION_INSTANCE_NAME
Destination Name
User-defined name of the destination instance.
My 2022 Advertising Destination
ORGANIZATION_NAME
Organization Name
Name of the customer organization in Adobe Experience Platform.
My Organization Name
SANDBOX_NAME
Sandbox Name
Name of the sandbox used by the customer.
prod
DATETIME / TIMESTAMP
Date and time

DATETIME and TIMESTAMP both define when the file was generated, but in different formats.

  • DATETIME uses the following format: YYYYMMDD_HHMMSS.
  • TIMESTAMP uses the 10-digit Unix format.

DATETIME and TIMESTAMP are mutually exclusive, and cannot be used simultaneously.

  • DATETIME: 20220509_210543
  • TIMESTAMP: 1652131584
CUSTOM_TEXT
Custom text
User-defined custom text to be included in the file name. Cannot be used in defaultFilename.
My_Custom_Text
TIMESTAMP
Date and time
10-digit timestamp of the time when the file was generated, in Unix format.
1652131584
MERGE_POLICY_ID
Merge Policy ID
The ID of the merge policy used to generate the exported audience. Use this macro when you are grouping exported audiences in files, based on merge policy. Use this macro together with segmentGroupingEnabled:true.
e8591fdb-2873-4b12-b63e-15275b1c1439
MERGE_POLICY_NAME
Merge Policy Name
The name of the merge policy used to generate the exported audience. Use this macro when you are grouping exported audiences in files, based on merge policy. Use this macro together with segmentGroupingEnabled:true.
My Custom Merge Policy

File name configuration example

The configuration example below shows the correspondence between the configuration used in the API call and the options shown in the UI.

"filenameConfig":{
   "allowedFilenameAppendOptions":[
      "CUSTOM_TEXT",
      "SEGMENT_ID",
      "DATETIME"
   ],
   "defaultFilenameAppendOptions":[
      "SEGMENT_ID",
      "DATETIME"
   ],
   "defaultFilename": "%DESTINATION%"
}

UI image showing the file name configuration screen with preselected macros

Next steps next-steps

After reading this article, you should have a better understanding of how you can configure file naming and export schedules for your file-based destinations.

To learn more about the other destination components, see the following articles:

recommendation-more-help
7f4d1967-bf93-4dba-9789-bb6b505339d6