Handling data formats with Data Prep
Data Prep can robustly handle different formats of data ingested into Adobe Experience Platform. This document outlines how different data formats are treated with Data Prep.
Booleans booleans
If the source type is a string and the target type is a boolean, Data Prep can automatically parse the value and convert the source value to a boolean.
The values y
, yes
, Y
, YES
, on
, ON
, true
, and TRUE
are automatically be parsed to be true
.
The values n
, N
, no
, NO
, off
, OFF
, false
, and FALSE
are automatically be parsed to be false
.
Dates dates
Data Prep supports date functions, both as strings and as datetime objects.
Date function format
The date function converts strings and datetime objects to become an ISO 8601 formatted ZonedDateTime object.
Format
date({DATE}, {FORMAT}, {DEFAULT_DATE})
{DATE}
{FORMAT}
{DEFAULT_DATE}
For example, the expression date(orderDate, "yyyy-MM-dd")
will convert an orderDate
value of “December 31st, 2020” into a datetime value of “2020-12-31”.
Date function conversions
When string fields from incoming data are mapped to date fields in schemas using Experience Data Model (XDM), the date format should be explicitly mentioned. If not explicitly mentioned, Data Prep will attempt to convert the input data by matching it to the following formats. Once a matching format is found, it will stop evaluating any subsequent formats.
"yyyy-MM-dd HH:mm:ssZ",
"yyyy-MM-dd HH:mm:ss.SSSZ",
"yyyy-MM-dd HH:mm:ss.SSS",
"yyyy-MM-dd'T'HH:mm:ss.SSSX",
"yyyy-MM-dd'T'HH:mm:ss'Z'",
"yyyy-MM-dd",
"yyyy/MM/dd",
"yyyy.MM.dd",
"yyyy-MMM-dd",
"yyyyMMdd",
"MM-dd-yyyy",
"MMddyyyy",
"M/dd/yyyy",
"dd.M.yyyy",
"M/dd/yyyy hh:mm:ss a",
"dd.M.yyyy hh:mm:ss a",
"dd.MMM.yyyy",
"dd-MMM-yyyy"
Date/time format strings format
The following table shows which pattern letters are defined for format strings. Please note that the letters are case sensitive.
Maps maps
Maps are currently not supported in Data Prep.