Create a Marketo Engage source connection and dataflow in the UI
This tutorial provides steps for creating a Marketo Engage (hereinafter referred to as “Marketo”) source connector in the UI to bring B2B data into Adobe Experience Platform.
Getting started
This tutorial requires a working understanding of the following components of Adobe Experience Platform:
- B2B namespaces and schema auto-generation utility: The B2B namespaces and schema auto-generation utility allows you to use Postman to auto-generate values for your B2B namespaces and schemas. You must complete your B2B namespaces and schemas first, before creating a Marketo source connection and dataflow.
- Sources: Experience Platform allows data to be ingested from various sources while providing you with the ability to structure, label, and enhance incoming data using Platform services.
- Experience Data Model (XDM): The standardized framework by which Experience Platform organizes customer experience data.
- Create and edit schemas in the UI: Learn how to create and edit schemas in the UI.
- Identity namespaces: Identity namespaces are a component of Identity Service that serve as indicators of the context to which an identity relates. A fully qualified identity includes an ID value and a namespace.
- Real-Time Customer Profile: Provides a unified, real-time consumer profile based on aggregated data from multiple sources.
- Sandboxes: Experience Platform provides virtual sandboxes which partition a single Platform instance into separate virtual environments to help develop and evolve digital experience applications.
Gather required credentials
In order to access your Marketo account on Platform, you must provide the following values:
munchkinIdclientIdclientSecretFor more information on acquiring these values, refer to the Marketo authentication guide.
Once you have gathered your required credentials, you can follow the steps in the next section.
Connect your Marketo account
In the Platform UI, select Sources from the left navigation bar to access the Sources workspace. The Catalog screen displays a variety of sources with which you can create an account.
You can select the appropriate category from the catalog on the left-hand side of your screen. Alternatively, you can find the specific source you wish to work with using the search bar.
Under the Adobe applications category, select Marketo Engage. Then, select Add data to create a new Marketo dataflow.
The Connect Marketo Engage account page appears. On this page, you can either use a new account or access an existing account.
Existing account
To create a dataflow with an existing account, select Existing account and then select the Marketo account you want to use. Select Next to proceed.
New account
If you are creating a new account, select New account. On the input form that appears, provide an account name, an optional description, and your Marketo authentication credentials. When finished, select Connect to source and then allow some time for the new connection to establish.
Select a dataset
After creating your Marketo account, the next step provides an interface for you to explore Marketo datasets.
The left half of the interface is a directory browser, displaying the 10 Marketo datasets. A fully-functioning Marketo source connection requires the ingestion of the nine different datasets. If you are also using the Marketo account-based marketing (ABM) feature, then you must also create a 10th dataflow to ingest the Named Accounts dataset.
Select the dataset you wish to ingest first, then select Next.
Provide dataflow details provide-dataflow-details
The Dataflow detail page allows you to select whether you want to use an existing dataset or a new dataset. During this process, you can also configure settings for Profile dataset, Error diagnostics, Partial ingestion, and Alerts.
To ingest data into an existing dataset, select Existing dataset. You can either retrieve an existing dataset using the Advanced search option or by scrolling through the list of existing datasets in the dropdown menu. Once you have selected a dataset, provide a name and a description for your dataflow.
To ingest into a new dataset, select New dataset and then provide an output dataset name and an optional description. Next, select a schema to map to using the Advanced search option or by scrolling through the list of existing schemas in the dropdown menu. Once you have selected a schema, provide a name and a description for your dataflow.
Enable Profile and error diagnostics
Next, select the Profile dataset toggle to enable your dataset for Profile. This allows you to create a holistic view of an entity’s attributes and behaviors. Data from all Profile-enabled datasets will be included in Profile and changes are applied when you save your dataflow.
Error diagnostics enables detailed error message generation for any erroneous records that occur in your dataflow, while Partial ingestion allows you to ingest data containing errors, up to a certain threshold that you manually define. See the partial batch ingestion overview for more information.
Enable alerts
You can enable alerts to receive notifications on the status of your dataflow. Select an alert from the list to subscribe to receive notifications on the status of your dataflow. For more information on alerts, see the guide on subscribing to sources alerts using the UI.
When you are finished providing details to your dataflow, select Next.
Skip unclaimed accounts when ingesting companies data
When creating a dataflow to ingest data from the companies dataset, you can configure Exclude unclaimed accounts to either exclude or include unclaimed accounts from ingestion.
When individuals fill out a form, Marketo creates a phantom account record based on the Company Name that contains no other data. For new dataflows, the toggle to exclude unclaimed accounts is enabled by default. For existing dataflows, you can enable or disable the feature, with changes applying to newly ingested data and not existing data.
Map your Marketo dataset source fields to target XDM fields
The Mapping step appears, providing you with an interface to map the source fields from your source schema to their appropriate target XDM fields in the target schema.
Each Marketo dataset has its own specific mapping rules to follow. See the following for more information on how to map Marketo datasets to XDM:
Based on your needs, you can choose to map fields directly, or use data prep functions to transform source data to derive computed or calculated values. For comprehensive steps on using the mapping interface, see the Data Prep UI guide.
Once your mapping sets are ready, select Next and allow for a few moments for the new dataflow to be created.
Review your dataflow
The Review step appears, allowing you to review your new dataflow before it is created. Details are grouped within the following categories:
- Connection: Shows the source type, the relevant path of the chosen source entity, and the amount of columns within that source entity.
- Assign dataset & map fields: Shows which dataset the source data is being ingested into, including the schema that the dataset adheres to.
Once you have reviewed your dataflow, select Save & ingest and allow some time for the dataflow to be created.
Monitor your dataflow
Once your dataflow has been created, you can monitor the data that is being ingested through it to see information on ingestion rates, success, and errors. For more information on how to monitor dataflows, see the tutorial on monitoring dataflows in the UI.
Delete your attributes
Custom attributes in datasets cannot be retroactively hidden or removed. If you want to hide or remove a custom attribute from an existing dataset, then you must create a new dataset without this custom attribute, a new XDM schema, and configure a new dataflow for the new dataset that you create. You must also disable or delete the original dataflow that consists of the dataset with the custom attribute you want to hide or remove.
Delete your dataflow
You can delete dataflows that are no longer necessary or were incorrectly created using the Delete function available in the Dataflows workspace. For more information on how to delete dataflows, see the tutorial on deleting dataflows in the UI.
Next steps
By following this tutorial, you have successfully created a dataflow to bring in Marketo data. Incoming data can now be used by downstream Platform services such as Real-Time Customer Profile and Data Science Workspace. See the following documents for more details:
Appendix appendix
The following sections provide additional guidelines that you may follow when using the Marketo source.
Error messages in the UI error-messages
The following error messages are displayed in the UI when Platform detects issues with your setup:
Munchkin ID is not mapped to the appropriate organization
Authentication will be denied if your Munchkin ID is not mapped to the Platform organization that you are using. Configure the mapping between your Munchkin ID and your organization using the Marketo interface.
Primary identity is missing
A dataflow will fail to save and ingest if a primary identity is missing. Ensure that a primary identity exists within your XDM schema, before attempting to configure a dataflow.
