Query Service packaging

This document outlines the different types of packaging and query execution capabilities available in Query Service.

Adobe Experience Platform Query Service can be divided into two capabilities based on the query patterns that can be executed:

  • Ad hoc queries are SQL queries used to explore ingested datasets for verification, validation, experimentation, and so on. These queries do not write data back into the Platform data lake.
  • Batch queries are SQL queries used to perform the post-ingestion processing of ingested datasets. These queries clean, shape, manipulate, and enrich data, the results of which are written back to the Platform data lake. These queries can be scheduled, managed, and monitored as batch jobs.

Query Service capabilities are packaged with the following products and add-ons:

  • Platform-based applications (Adobe Real-Time Customer Data Platform, Adobe Customer Journey Analytics, and Adobe Journey Optimizer): Query Service access to execute ad hoc queries is provided from the outset with every variation and tier of Platform-based applications.
  • Data Distiller (add-on package that can be purchased with Adobe Real-Time CDP, Customer Journey Analytics and Adobe Journey Optimizer): Query Service access to execute batch queries is provided with Data Distiller.

Entitlements entitlements

The following table outlines the key Query Service entitlements based on how they are packaged:

Query Service Entitlement
Packaged with Platform-based applications
Packaged with Data Distiller
Query Pattern Supported
Ad hoc queries only
Batch query
Use Case Supported
  • Exploration​
  • Data Discovery​
  • Data Validation
  • Experimentation
  • Cleaning
  • Shaping
  • Manipulating
  • Enriching
Semantics Supported
  • SELECT queries
  • CTAS and ITAS queries
Maximum Execution Time
10 minutes
24 hours
License Metric

Query User Concurrency:

  • 1 concurrent user (Real-Time CDP, Adobe Journey Optimizer)​
  • 5 concurrent users (Customer Journey Analytics)​

Query Concurrency:

  • 1 concurrent running query (all applications)​

An additional ad hoc query users pack add-on can be purchased to increase your authorized ad hoc query entitlement.

  • +5 additional concurrent users per pack
  • +1 additional concurrent running query per pack

Compute Hours:

  • Variable (scoped based on your application entitlement)

Compute Hours is a measure of the amount of time taken by the Query Service engine to read, process, and write data back into the data lake when a batch query is executed.
With the Data Distiller SKU, you also get an additional user and query concurrency, which can be used towards the execution of ad hoc queries. The Data Distiller SKU includes:

  • +5 additional concurrent users
  • +1 additional concurrent running query
Accelerated query and reporting usage
No
Yes - Concurrent accelerated queries allow you to read data from the accelerated store and display within your dashboards. A dedicated entitlement for storing reporting models and datasets in the accelerated store is also provided.
Data lake storage capacity
Your total storage entitlement is dependant on your platform-based applications licenses. For example, Real-Time CDP, AJO, CJA, and so on.
Yes - An additional storage entitlement is provided to persist your raw and derived datasets for Data Distiller use cases beyond a seven-day data expiration date.
Your data lake storage capacity is measured in terabytes (TB) and depends on the quantity of Compute hours that you have bought. Please check the product description for more details.
Data export allowance
Your total export entitlement is dependant on your platform-based applications licenses. For example, Real-Time CDP, AJO, CJA, and so on.
Yes - An additional export entitlement is provided to allow for the export of derived datasets created using Data Distiller.
Your annual data export allowance is measured in terabytes (TB) and depends on the quantity of Compute hours that you have bought. Please check product description for more details.
Query Execution Interface
  • Query Service UI
  • Third-party client UI
  • PostgresSQL client UI
  • Query Service UI
  • Third-party client UI
  • PostgresSQL client UI
  • REST APIs
Query Results Returned Via
Client UI
Derived dataset stored in data lake
Result Limit
  • Query Service UI - The number of output rows can be configured with a UI setting to between 50-500 rows.
  • Third-party clients - 50,000
  • PostgresSQL client - 50,000
  • Query Service UI - The number of output rows can be configured with a UI setting to between 50-500 rows.
    Note: CTAS and ITAS queries only generate success messages as the query output is stored in derived datasets.
  • Third-party clients (no upper limit to rows)
  • PostgresSQL client (no upper limit to rows)
  • REST APIs (no upper limit to rows)
Read Dataset Capacity
Yes
Yes
Write Dataset Capacity
No
Yes
Scheduling Capacity
No
Yes
Monitoring Capacity
Yes
Yes
Query Alert Setup Capacity
No
Yes

Access control access-control

Access control for Experience Platform is administered through the Adobe Admin Console where product profiles link users with permissions and sandboxes. See the access control overview for more information.

In order to use Query Service, the Manage Queries permission must be enabled within Admin Console. This permission allows users to execute ad hoc and batch queries. Detailed instructions for requesting access to the product profile Manage Queries permission have been outlined in the manage permissions for a product profile and manage users for a product profile documents.

After purchasing the Data Distiller add-on, the Write Dataset permission must be granted. This permission allows Data Distiller users to execute batch queries.

The following table outlines the effects of the Manage Queries permission:

Permission
Function
Manage Queries (without write data permission)
Provides access to execute ad hoc queries
Manage Queries (with write data permission)
Provides access to execute batch queries

Sandbox support sandbox-support

Sandboxes are virtual partitions within a single instance of Experience Platform. Each Platform instance supports multiple production and non-production sandboxes, each maintaining its own library of Platform resources. Non-production sandboxes allow you to test features, run experiments, and make custom configurations without impacting your production sandboxes. For more information on sandboxes, see the sandboxes overview. All Query Service entitlements are shared across all sandboxes.

Next steps

By reading this document, you should have a better understanding of the different packaging types and query execution capabilities available in Query Service. To learn more about Query Service, such as well-known industry use cases, read the use case documentation. For more general information, visit the Query Service overview.

recommendation-more-help
ccf2b369-4031-483f-af63-a93b5ae5e3fb