Documentation Experience Platform Query Service Guide

Query Editor UI guide

Last update: Wed Jan 31 2024 00:00:00 GMT+0000 (Coordinated Universal Time)

Topics:
Queries

CREATED FOR:

Developer
User
Admin
Leader

Query Editor is an interactive tool provided by Adobe Experience Platform Query Service, which allows you to write, validate, and run queries for customer experience data within the Experience Platform user interface. Query Editor supports developing queries for analysis and data exploration, and allows you to run interactive queries for development purposes as well as non-interactive queries to populate datasets in Experience Platform.

For more information about the concepts and features of Query Service, see the Query Service overview. To learn more about how to navigate the Query Service user interface on Platform, see the Query Service UI overview.

NOTE

Certain Query Service functionality is not provided by the legacy version of the Query Editor. The screenshots used in this document are taken using the enhanced version of the Query Editor unless otherwise stated. See the section on the enhanced Query Editor for more details.

Getting started getting-started

Query Editor provides flexible execution of queries by connecting to Query Service, and queries only run while this connection is active.

Accessing Query Editor accessing-query-editor

In the Experience Platform UI, select Queries in the left navigation menu to open the Query Service workspace. Next, to start writing queries, select Create Query at the top right of the screen. This link is available from any of the pages in the Query Service workspace.

The Queries workspace overview tab with Create query highlighted.

Connecting to Query Service connecting-to-query-service

The Query Editor takes a few seconds to initialize and connect to Query Service when it is opened. The console tells you when it is connected, as shown below. If you attempt to run a query before the editor has connected, it delays execution until the connection is complete.

The console output of the Query Editor upon initial connection.

How queries are run from Query Editor run-a-query

Queries executed from Query Editor run interactively which means that if you close the browser or navigate away, the query is cancelled. The same is true for queries made to generate datasets from query outputs.

The Enhanced edition of the Query Editor allows you to write more than one query in the Query Editor and execute all queries sequentially. See the section on executing multiple sequential queries for more information.

Query authoring using Query Editor query-authoring

Using Query Editor, you can write, execute, and save queries for customer experience data. All queries executed, or saved in Query Editor are available to all users in your organization with access to Query Service.

IMPORTANT

The legacy editor will be retired on 01-April-2024, and will no longer be available for use.

Enhanced Query Editor toggle enhanced-editor-toggle

A UI toggle allows you to switch between the legacy and enhanced version of the Query Editor. The legacy version is enabled by default, although the enhanced version provides better accessibility and multi-theming support. Enable the enhanced version to access to the Query Editor settings.

The Query Editor with the enhanced Query Editor toggle highlighted.

Activating the toggle switches the editor to light theme and improves the legibility of your syntax. A settings icon also appears above the Query Editor input field that incorporates the auto-complete toggle. From the settings icon, you can enable dark theme or disable/enable auto-complete.

TIP

With the enhanced Query Editor, you can Disable syntax auto complete while authoring a query without losing your progress. Typically, if you disable the auto-complete feature while editing, all changes to the query are lost.

To enable dark or light themes, select the settings icon ( ) followed by the option in the dropdown menu that appears.

The Query Editor with the settings icon and Enable dark theme dropdown menu option highlighted.

Execute multiple sequential queries execute-multiple-sequential-queries

The Enhanced edition of the Query Editor allows you to write more than one query in the Query Editor and execute all queries in a sequential manner.

The execution of multiple queries in a sequence each generate a log entry. However, only the results of the first query display in the Query Editor console. Check the query log if you need to troubleshoot or confirm the queries that were executed. See the query logs documentation for more information.

NOTE

If a CTAS query is executed after the first query in the Query Editor, a table is still created however there is no output on the Query Editor console.

Execute selected query execute-selected-query

If you have written multiple queries but need to execute only one query, you can highlight your chosen query and select the
Run selected query icon. This icon is disabled by default until you select query syntax within the editor.

The Query Editor with the Run selected query icon highlighted.

Result count result-count

The Query Editor has a maximum 50,000 row output. You can choose the number of rows are displayed at one time in the Query Editor console. To change the number of rows displayed in the console, select the Result count dropdown and select from the 50, 100, 150, 300, and 500 options.

The Query Editor with the Result count dropdown highlighted.

Writing queries writing-queries

Query Editor is organized to make writing queries as easy as possible. The screenshot below shows how the editor appears in the UI, with the SQL entry field and Play highlighted.

The Query Editor with the SQL input field and Play highlighted.

To minimize your development time, you are recommended to develop your queries with limits on the number of rows returned. For example, SELECT fields FROM table WHERE conditions LIMIT number_of_rows. After you have verified that your query produces the expected output, remove the limits and run the query with CREATE TABLE tablename AS SELECT to generate a dataset with the output.

Writing tools in Query Editor writing-tools

Automatic syntax highlighting: Makes reading and organizing SQL easier.

An SQL statement in the Query Editor demonstrating syntax colour highlighting.

SQL keyword auto-complete: Start typing your query then use the arrow keys to navigate to the desired term and press Enter.

A few characters of SQL with the auto complete dropdown menu providing options from the Query Editor.

Table and field auto-complete: Start typing the table name you want to SELECT from, then use the arrow keys to navigate to the table you are looking for, and press Enter. Once a table is selected, autocomplete recognizes fields in that table.

The Query Editor input displaying drop down table name suggestions.

Format text format-text

The Format text feature makes your query more readable by adding standardized syntax styling. Select Format text to standardize all the text within the Query Editor.

NOTE

The Format text feature does not work with anonymous blocks. To learn how to chain one or more SQL statements sequentially, see the anonymous block documentation.

The Query Editor with Format text and the SQL statements highlighted.

Copy SQL copy-sql

Select the copy icon to copy SQL form the Query Editor to your clipboard. This copy feature is available for both query templates and newly created queries in the Query Editor.

The Queries workspace with an example query template with the copy icon highlighted.

Auto-complete UI configurational toggle auto-complete

The Query Editor automatically suggests potential SQL keywords along with table or column details for the query as you write it. The auto-complete feature is enabled by default and can be disabled or enabled at any point by selecting the Syntax auto-complete toggle to the top right of the Query Editor.

The auto-complete configuration setting is per user and remembered for the consecutive logins for that user.

NOTE

The syntax autocomplete toggle is only available for the legacy version of the Query Editor.

Query Editor with the syntax auto-complete toggle highlighted.

Disabling this feature stops several metadata commands from being processed and providing recommendations that typically benefit the speed of the author when editing queries.

When you use the toggle to enable the auto-complete feature, recommended suggestions for table and column names as well as SQL keywords become available after a short pause. A success message in the console beneath the Query Editor indicates that the feature is active.

If you disable the auto-complete feature, a page refresh is required for the feature to take effect. A confirmation dialog appears with three options when you disable the Syntax auto-complete toggle :

Cancel
Save changes and refresh
Refresh without saving changes

IMPORTANT

If you are writing or editing a query when disabling this feature, you must save any changes to your query before refreshing the page or all of your progress will be lost.

The confirmation dialog to disable the auto-complete feature.

To disable the auto-complete feature, select the appropriate confirmation option.

Error detection error-detection

Query Editor automatically validates a query as you write it, providing generic SQL validation and specific execution validation. If a red underline appears below the query (as shown in the image below), it represents an error within the query.

The Query Editor input displaying SQL underlined in red to indicate an error.

When errors are detected, you can view the specific error messages by hovering over the SQL code.

A dialog with an error message.

Query details query-details

To view a query in the Query Editor, select any saved template from the Templates tab. The query details panel provides more information and tools to manage the selected query. It also shows useful metadata such as the last time that the query was modified and who modified it, if applicable.

NOTE

The View schedule, Add schedule and Delete query options are only available after the query has been saved as a template. The Add schedule option takes you directly to the schedule builder from the Query Editor. The View schedule option takes you directly to the schedule inventory for that query. See the query schedules documentation to learn how to create query schedules in the UI.

The Query Editor with the query details panel highlighted.

From the details panel you can generate an output dataset directly from the UI, delete or name the displayed query, view the query run schedule, and add the query to a schedule.

To generate an output dataset, select Run as CTAS. The Enter output dataset details dialog appears. Enter a name and description, then select Run as CTAS. The new dataset is displayed in the Datasets Browse tab. See the view datasets documentation to learn more about available datasets for your organization.

NOTE

The Run as CTAS option is only available if the query has not been scheduled.

The Enter output dataset details dialog.

After you execute the Run as CTAS action, a confirmation message pops up to notify you of the successful action. This popup message contains a link that provides a convenient way to navigate to the query logs workspace. See the query logs documentation for more information on query logs.

Saving queries saving-queries

The Query Editor provides a save function that allows you to save a query and work on it later. To save a query, select Save in the top-right corner of Query Editor. Before a query can be saved, a name must be provided for the query using the Query Details panel.

NOTE

Queries named and saved in using the Query Editor are available as templates within the Query dashboard Templates tab. See the templates documentation for more information.

When you save a query in the Query Editor, a confirmation message pops up to notify you of the successful action. This popup message contains a link that provides a convenient way to navigate to the queries scheduling workspace. See the schedule queries documentation to learn how to run queries on a custom cadence.

Scheduled queries scheduled-queries

Queries that have been saved as a template can be scheduled from the Query Editor. Scheduling queries allows you to automate query runs on a custom cadence. You can schedule queries based on frequency, date, and time, and also choose an output dataset for your results if necessary. Query schedules can also be disabled or deleted through the UI.

Schedules are set in the Query Editor. When using the Query Editor, you can only add a schedule to a query that has already been created, saved, and run. The same limitation does not apply to the Query Service API:

See the query schedules documentation to learn how to create query schedules in the UI. Alternatively, to learn how to add schedules using the API, read the scheduled queries endpoint guide.

Any scheduled queries are added to the list in the Scheduled queries tab. From that workspace you can monitor the status of all scheduled query jobs through the UI. On the Scheduled queries tab, you can find important information about your query runs and subscribe to alerts. The available information includes the status, schedule details, and error messages/codes if a run failed. See the Monitor scheduled queries document for more information.

How to find previous queries previous-queries

All queries executed from Query Editor are captured in the Log table. You can use the search functionality in the Log tab to find query executions. Saved queries are listed in the Templates tab.

If a query was scheduled, then the Scheduled Queries tab provides improved visibility through the UI for those query jobs. See the query monitoring documentation for more information.

NOTE

Queries that are not executed are not saved by the Log. In order for the query to be available in Query Service, it must be run or saved in Query Editor.

Executing queries using Query Editor executing-queries

To run a query in Query Editor, you can enter SQL in the editor or load a previous query from the Log or Templates tab, and select Play. The status of query execution is displayed in the Console tab below, and output data is shown in the Results tab.

Console console

The console provides information on the status and operation of Query Service. The console displays the connection status to Query Service, query operations being executed, and any error messages that result from those queries.

The Console tab of the Query Editor console.

NOTE

The console only shows errors that resulted from the execution of a query. It does not show the query validation errors that occur before a query is executed.

Query results query-results

After a query has been completed, the results are displayed in the Results tab, next to the Console tab. This view shows the tabular output of your query, displaying between 50 and 500 rows of results depending on your chosen result count. This view allows you to verify that your query produces the expected output. To generate a dataset with your query, remove limits on rows returned, and run the query with CREATE TABLE tablename AS SELECT to generate a dataset with the output. See the generating datasets tutorial for instructions on how to generate a dataset from query results in Query Editor.

The Results tab of the Query Editor console displaying the results of a query run.

Run queries with Query Service tutorial video query-tutorial-video

The following video shows how to run queries in the Adobe Experience Platform interface and in a PSQL client. The video also demonstrates the use of individual properties in an XDM object, Adobe-defined functions, and how to use CREATE TABLE AS SELECT (CTAS) queries.

https://video.tv.adobe.com/v/29796?quality=12&learn=on

Transcript

Adobe Experience Platform use Query Service From the Adobe Experience Platform UI, let’s navigate to queries. From the query environment, let’s click the create query button. You are now seeing a command line like user interface which allows you to create platform directly by making use of SQL statements. For instance, by typing show tables and clicking the execute button, I’m seeing all platform datasets as a response. One of these datasets is the website interactions EMEA dataset. Let’s create that dataset now.

Let’s start with some basic queries. For instance, I’m interested in knowing from what location customers of La Boutique were visiting product pages. This query will give me some insight in that. Please note the limit of ten. This is for performance reasons. When executing queries, it’s best to limit your responses to just a couple of lines while you’re testing and developing your queries. You can now see a response from Platform containing location information of Belgian cities. This response is a flattened object, and not a single value. The place_context.geo XDM object contains four attributes: schema.latitude, schema.longitude, country, and city. When such an object is declared as a column, it will return the entire object as a string. In order to query individual properties of an XDM object, we have to use the dot notation. To find out more about the structure of an XDM schema, let’s go to schemas in the Platform UI, and open the EMEA website interaction data schema. You can see the place context object alongside many other. By opening the place context object, and going a little deeper, you can see the geo objects, and in there, for instance, country code. The way to get to this property from the query service UI is by specifying this path. The same concept applies also to the schema.latitude property. Let’s go back to the Query Service UI, and optimize the query that we entered a couple of minutes ago by specifying the specific properties of the place context object. And the result is now a lot more readable and understandable.

Next, I’m interested in knowing the amount of products used per day for the last ten days. So I’m selecting the dates and making a count of the number of products to events. Next, I’m interested in having a visitor count by the hour for July 17.

I’m also interested in knowing the amount of visitors on July 16 per country and city.

As an analyst, I also need to visualize the most popular products. In this case, I want to see the five most popular products with the Lisette dress being the clear winner there. So far, we’ve only used the Query Service UI inside Adobe Experience Platform to execute these queries, but we can also execute the same queries from an external environment by using PSQL to achieve the same outcome. PSQL is a command line application which requires me to authenticate first. The credentials to authenticate to query service from PSQL can be found in the queries and credentials window. By clicking this icon, I’m copying the full PSQL commands to my clipboard.

Let’s open a terminal window, paste the PSQL commands, and hit enter. I’m now authenticated, and can execute queries here as well.

With the Lisette dress being so popular I’d like to have an overview of the individual EC IDs of the visitors that came to see this product. This query is showing me the first ten EC IDs. I’m also very much interested about the activity funnel on my website. Products used don’t pay money for La Boutique, purchases do. So how does the flow from a product view to an add to cart and purchase look like. The result of this query shows the fallout in the funnel. So far, we’ve been using basic queries to query experience event data. Let’s go a bit deeper now, and use Adobe defined functions. SQL wasn’t built for experience analysis, so Adobe decide to create a number of Adobe defined functions to make it possible to use SQL to better understand experiences. The first two Adobe defined functions we’ll use are the Sessionization function and the Next Previous function. The Sessionization function, it produces the visit groupings found with Adobe analytics. It performs a similar time-based grouping, but now has customizable parameters. And Next and Previous help you to understand how customers navigate your site.

For this specific query, I’d like to know what people did on my website before reaching the El Silencio product page as the third page in a session. The result is a table that shows a number of journeys that occurred and how those journeys look like. You can see that the third page visit is always El Silencio, and you can also see the two steps that happened before that, and the step that happened after that. Journeys are session based with the session timeout being set to thirty minutes here. With Sessionization, a number of standard fields are made available. Let’s explore these fields. As you can see, the session structure contains the fields is new, times, time difference, number, and depth. Let’s go back to the query service UI, and continue with another Sessionization example which shows me the visitors’ sessions and page views per day.

As a final query, I’d like to do advanced call center analysis. In this example, were joining three datasets in Platform: experience events data, call center interactions which are sent by the call center to Platform, and we are now combining those datasets with loyalty data. The output gives me a dataset with EC IDs and CRM IDs, location information, feedback from the call center including the topic of the call and the customer feeling, and also loyalty information. This is a really interesting combination of data, and as an analyst I’d like to provide La Boutique management with a visualization that shows this information. To do that, I’m going to use the CTAS functionality available with Query Service. CTAS stands for create table as select, and allows me to write the output of this query back to a dataset in Platform. By going back to the query UI, in the log menu option, I’m seeing an overview of all the queries that have been made. The first one in this list is my last query. By clicking the query, I’m seeing the query overview menu pop up, and I can view the full query that was sent to Query Service. I now want to output the result of this query into a dataset in Platform. To do that I need to click the output dataset button. I’m naming my query call center analysis, and by clicking run query, the process has now started through the HDP API of query service. This process takes a couple of minutes, and once finished will have a new dataset in Platform. By going to datasets in Platform, I can open the call center analysis datasets, and preview the results. By making use of Query Service, analysts can now explore and consume data from Adobe Experience Platform in near real time from any environment, and perform flexible home channel queries right at the heart of Adobe Experience Platform. Analysts can send their insights back to Adobe Experience Platform for activation, and grants can now maximize the usefulness of that data to create and deliver exceptional experiences. With that, you should now be able to set up and use Query Service.

Next steps

Now that you know what features are available in Query Editor and how to navigate the application, you can start authoring your own queries directly in Platform. For more information about running SQL queries against datasets in Data Lake, see the guide on running queries.

recommendation-more-help

ccf2b369-4031-483f-af63-a93b5ae5e3fb