Set up local Dispatcher Tools set-up-local-dispatcher-tools

Adobe Experience Manager (AEM)'s Dispatcher is an Apache HTTP Web server module that provides a security and performance layer between the CDN and AEM Publish tier. Dispatcher is an integral part of the overall Experience Manager architecture and should be part of local development setup.

The AEM as a Cloud Service SDK includes the recommended Dispatcher Tools version that facilitates configuring validating, and simulating Dispatcher locally. Dispatcher Tools is composed of:

Note that ~ is used as shorthand for the User’s Directory. In Windows, this is the equivalent of %HOMEPATH%.

Prerequisites

Download the Dispatcher Tools (as part of the AEM SDK)

The AEM as a Cloud Service SDK, or AEM SDK, contains the Dispatcher Tools used to run Apache HTTP Web server with the Dispatcher module locally for development, and the compatible QuickStart Jar.

If the AEM as a Cloud Service SDK has already been downloaded to setup the local AEM runtime, it does not need to be redownloaded.

Extract the Dispatcher Tools from the AEM SDK zip

The version of Dispatcher Tools is different from that of the AEM SDK. Ensure that the version of Dispatcher Tools is provided via the AEM SDK version matching the AEM as a Cloud Service version.

$ chmod a+x aem-sdk-dispatcher-tools-x.x.x-unix.sh
$ ./aem-sdk-dispatcher-tools-x.x.x-unix.sh

$ chmod a+x aem-sdk-dispatcher-tools-x.x.x-unix.sh
$ ./aem-sdk-dispatcher-tools-x.x.x-unix.sh

All commands issued below assume that the current working directory contains the expanding Dispatcher Tools contents.

This video uses macOS for illustrative purposes. The equivalent Windows/Linux commands can be used to achieve similar results.

Understand the Dispatcher configuration files

The Dispatcher Tools provides a set of Apache HTTP Web server and Dispatcher configuration files that define behavior for all environments, including local development.

These files are intended to be copied into an Experience Manager Maven project to the dispatcher/src folder, if they do not exist in the Experience Manager Maven project.

A complete description of the configuration files is available in the unpacked Dispatcher Tools as dispatcher-sdk-x.x.x/docs/Config.html.

Validate configurations

Optionally, the Dispatcher and Apache Web server configurations (via httpd -t) can be validated using the validate script (not to be confused with the validator executable). The validate script provides a convenient way of running the three phases of the validator.

$ ./bin/validate.sh ./src

$ bin\validate src

$ ./bin/validate.sh ./src

Run Dispatcher locally

AEM Dispatcher is run locally using Docker against the src Dispatcher and Apache Web server configuration files.

$ ./bin/docker_run_hot_reload.sh <src-folder> <aem-publish-host>:<aem-publish-port> <dispatcher-port>

$ bin\docker_run <src-folder> <aem-publish-host>:<aem-publish-port> <dispatcher-port>

$ ./bin/docker_run_hot_reload.sh <src-folder> <aem-publish-host>:<aem-publish-port> <dispatcher-port>

The <aem-publish-host> can be set to host.docker.internal, a special DNS name Docker provides in the container that resolves to the host machine’s IP. If the host.docker.internal does not resolve, please see the troubleshooting section below.

For example to start the Dispatcher Docker container using the default configuration files provided by the Dispatcher Tools:

Start Dispatcher Docker container providing the path to the Dispatcher configuration src folder:

$ ./bin/docker_run_hot_reload.sh ./src host.docker.internal:4503 8080

$ bin\docker_run src host.docker.internal:4503 8080

$ ./bin/docker_run_hot_reload.sh ./src host.docker.internal:4503 8080

The AEM as a Cloud Service SDK 's Publish Service, running locally on port 4503 is available through Dispatcher at http://localhost:8080.

To run Dispatcher Tools against an Experience Manager project’s Dispatcher configuration, point to your project’s dispatcher/src folder.

$ ./bin/docker_run_hot_reload.sh ~/code/my-project/dispatcher/src host.docker.internal:4503 8080

$ bin\docker_run <User Directory>/code/my-project/dispatcher/src host.docker.internal:4503 8080

$ ./bin/docker_run_hot_reload.sh ~/code/my-project/dispatcher/src host.docker.internal:4503 8080

Dispatcher Tools logs

Dispatcher logs are helpful during local development to understand if and why HTTP Requests are blocked. Log level can be set by prefixing the execution of docker_run with environment parameters.

Dispatcher Tools logs are emitted to the standard out when docker_run is run.

Useful parameters for debugging Dispatcher include:

One or many parameters, can be passed to docker_run

$ DISP_LOG_LEVEL=Debug REWRITE_LOG_LEVEL=Debug ./bin/docker_run_hot_reload.sh ~/code/my-project/dispatcher/src host.docker.internal:4503 8080

$ DISP_LOG_LEVEL=Debug REWRITE_LOG_LEVEL=Debug bin\docker_run <User Directory>/code/my-project/dispatcher/src host.docker.internal:4503 8080

$ DISP_LOG_LEVEL=Debug REWRITE_LOG_LEVEL=Debug ./bin/docker_run_hot_reload.sh ~/code/my-project/dispatcher/src host.docker.internal:4503 8080

Log file access

Apache web server and AEM Dispatcher logs can be directly accessed in the Docker container:

When to update Dispatcher Tools dispatcher-tools-version

Dispatcher Tools versions increment less frequently than the Experience Manager, and thus Dispatcher Tools require fewer updates in the local development environment.

The recommended Dispatcher Tools version is that which is bundled with the AEM as a Cloud Service SDK that matches the Experience Manager as a Cloud Service version. The version of AEM as a Cloud Service can be found via Cloud Manager.

Note that Dispatcher Tools version does not match the Experience Manager version.

How to update the baseline set of Apache and Dispatcher configurations

The baseline set of Apache and Dispatcher configuration is enhanced regularly and released with the AEM as a Cloud Service SDK version. It is best practice to incorporate the baseline configuration enhancements into your AEM project and avoid local validation and Cloud Manager pipeline failures. Update them using the update_maven.sh script from the .../dispatcher-sdk-x.x.x/bin folder.

This video uses macOS for illustrative purposes. The equivalent Windows/Linux commands can be used to achieve similar results.

Let’s assume you created an AEM project in the past using AEM Project Archetype, the baseline Apache and Dispatcher configurations were current. Using these baseline configurations your project-specific configurations were created by reusing, and copying the files like *.vhost, *.conf, *.farm and *.any from the dispatcher/src/conf.d and dispatcher/src/conf.dispatcher.d folders. Your local Dispatcher validation and Cloud Manager pipelines were working fine.

Meanwhile, baseline Apache and Dispatcher configurations were enhanced for various reasons like new features, security fixes and optimization. They are released via a newer version of Dispatcher Tools as part of the AEM as a Cloud Service release.

Now, when validating your project-specific Dispatcher configurations against the latest Dispatcher Tools version they start failing. To resolve this, the baseline configurations need to be updated by using below steps:

$ ./bin/validate.sh ${YOUR-AEM-PROJECT}/dispatcher/src

...
checking 'conf.dispatcher.d/renders/default_renders.any' immutability (if present)
checking existing 'conf.dispatcher.d/renders/default_renders.any' for changes
checking 'conf.dispatcher.d/virtualhosts/default_virtualhosts.any' immutability (if present)
checking existing 'conf.dispatcher.d/virtualhosts/default_virtualhosts.any' for changes
no immutable file has been changed - check is SUCCESSFUL
Phase 3 finished

Troubleshooting

docker_run results in ‘Waiting until host.docker.internal is available’ message troubleshooting-host-docker-internal

The host.docker.internal is a hostname provided to the Docker contain that resolves to the host. Per docs.docker.com (macOS, Windows):

From Docker 18.03 onwards the recommendation is to connect to the special DNS name host.docker.internal, which resolves to the internal IP address used by the host

When bin/docker_run src host.docker.internal:4503 8080 results in the message Waiting until host.docker.internal is available, then:

Example error

$ docker_run src host.docker.internal:4503 8080

Running script /docker_entrypoint.d/10-check-environment.sh
Running script /docker_entrypoint.d/20-create-docroots.sh
Running script /docker_entrypoint.d/30-wait-for-backend.sh
Waiting until host.docker.internal is available

Additional Resources