Data Prep overview

Data Prep allows data engineers to map, transform, and validate data to and from Experience Data Model (XDM). Data Prep appears as a “Map” step in the Data Ingestion processes, including CSV Ingestion workflow. Data engineers can use Data Prep to perform the following data manipulation during ingestion:

Data Prep also applies several intrinsic data validations to ensure that the data integrity is maintained as it is ingested. Where possible, Data Prep automatically maps the incoming data schemas to XDM. Data engineers can change, correct, and delete the suggested mappings and replace them with the mappings as appropriate.

Mapping

A mapping is an association of an input attribute or calculated field to one XDM attribute. A single attribute can be mapped to multiple XDM attributes by creating individual mappings.

To learn more about the different mapping functions, please read the mapping functions guide.

Calculated fields

Calculated fields allow for values to be created based on the attributes in the input schema. These values can then be assigned to attributes in the target schema and be provided a name and description to allow for easier reference. Calculated fields have a maximum length of 4096 characters.

To learn more about calculated fields, please read the calculated fields guide.

Escape special characters escape-special-characters

You can escape special characters in a field by using ${...}. However, JSON files that contain fields with a period (.) are not supported by this mechanism. When interacting with hierarchies, if a child attribute has a period (.), you must use a backslash (\) to escape special characters. For example, address is an object that contains the attribute street.name, this can then be referred to as address.street\.name instead of address.street.name.

Mapping set

A set of mappings that transform one schema to another are collectively known as a mapping set. A single mapping set is created as part of each data flow. A mapping set is an integral part of the data flows and is created, edited, and monitored as part of the data flows.

To learn more about mapping sets, including how to use the fields within a mapping set, please read the mapping set guide. To learn how to create a mapping set and use other API calls related to mapping sets, please read the mapping set section in the developer guide.

Data format handling

Data Prep can robustly handle different formats of data ingested into Platform. To learn more about how Data Prep handles different data types, please read the data format handling overview.

Send partial row updates using Data Prep

Streaming upserts in Data Prep allows you to send partial row updates to Profile Service data while also creating and establishing new identity links with a single API request. To learn more about how to stream upserts in Data Prep, see the document on sending partial row updates.

Attribute-based access control in Data Prep

Attribute-based access control in Adobe Experience Platform allows administrators to control access to specific objects and/or capabilities based on attributes.

Attribute-based access control ensures that you can map only the attributes that you have access to. Attributes that you do not have access to can not be used in pass-through mappings and calculated fields. As such, if you do not have access to a required field, then you cannot successfully save a mapping. Furthermore, you cannot map objects or object arrays if you do not have access to any one of the child attributes. However, you can map other elements within the object or object array individually.

See the attribute-based access control overview for more information.

Next steps

This document covered the basics on Data Prep in Adobe Experience Platform. To learn more about different mapping functions, please read the mapping functions guide. To learn more about how Data Prep handles different data types, please read the data format handling guide. To learn how to use the Data Prep API, please read the Data Prep developer guide.