How and when to use Apache Sling Dynamic Include

AEM sites contain mostly static data which is generally cached by the Dispatcher. However, there are occasions when pages might have a component with dynamic data, which results in the entire page being non-cacheable. Apache Sling Dynamic Include (SDI) is the way to go here. With SDI enabled, the entire page will be cached with some placeholder(include) for components with dynamic data; the placeholder will be replaced with the actual data when accessing the page. This article walks you through technical details for a few SDI use cases with caching at the Dispatcher level.

Let’s start by setting up SDI; we can achieve this by the code using the below instructions; alternatively, SDI can be downloaded and installed from the Sling Dynamic Include bundle.

SDI Set Up Instructions

Sling Dynamic Include must be installed in AEM from the code by embedding it as a part of your bundle.

1. Add SDI dependency in the parent pom.xml

2. Add the SDI dependency in the ui.apps pom.xml. And, embed it as a package for deployment.

<dependencies>
    .....
    <!-- Apache Sling Dynamic Include -->
    <dependency>
      <groupId>org.apache.sling</groupId>
      <artifactId>org.apache.sling.dynamic-include</artifactId>
    </dependency>
</dependencies>
....
<embeddeds>
    <!-- Apache Sling Dynamic Include -->
    <embedded>
        <groupId>org.apache.sling</groupId>
        <artifactId>org.apache.sling.dynamic-include</artifactId>
        <target>/apps/app-name/install</target>
    </embedded>
</embeddeds>

Dispatcher utilizes Apache Server Side Includes (SSI) to handle dynamic content for HTML documents. Following are the basic dispatcher guidelines for this,

Apache/Dispatcher Set Up Instructions

1. Add the module mod_include.so from httpd.conf or from the configuration file that loads the modules.
2. Update the publisher vhost file to process include directives.

<VirtualHost *:80>
        ServerName      publish
        .....
        
        DocumentRoot    /mnt/var/www/html
        <Directory />
                .......
                # Directives for the SSI working
                Options FollowSymLinks Includes
                AllowOverride None
                AddOutputFilter INCLUDES .html
                .......
        </Directory>
</VirtualHost>

Use Case One: Ignore the component from caching at dispatcher

To exclude the component with dynamic data from caching in the dispatcher, the AEM publish server must be configured for Sling Dynamic Include via the OSGi Configuration Factory.

<jcr:root xmlns:sling="http://sling.apache.org/jcr/sling/1.0" 
    xmlns:cq="http://www.day.com/jcr/cq/1.0"xmlns:jcr="http://www.jcp.org/jcr/1.0" 
    xmlns:nt="http://www.jcp.org/jcr/nt/1.0"
  jcr:primaryType="sling:OsgiConfig"
  include-filter.config.enabled="{Boolean}true"
  include-filter.config.path="/content/site-name"
  include-filter.config.resource-types="[app-name/components/dynamic-component]"
  include-filter.config.include-type="SSI"
  include-filter.config.add_comment="{Boolean}true"
  include-filter.config.selector="nocache"
  include-filter.config.ttl=""
  include-filter.config.required_header="Server-Agent=Communique-Dispatcher"
  include-filter.config.ignoreUrlParams="[]"
  include-filter.config.rewrite="{Boolean}true"
  include-filter.config.appendSuffix="{Boolean}false"
/>

This configuration ensures the component with the sling:resourceType as app-name/components/dynamic-component in the site hierarchy /content/site-name is not cached at the dispatcher level by setting the include type as SSI, adding a debug comment:

<!-SDI include (path: %s, resourceType: %s)-->

on the page HTML when the component is replaced. A nocache selector will be used to get actual content from the publisher by making the HTTP request for the component from the dispatcher. The dispatcher needs to be updated to disable caching for the nocache selector.

Apache/Dispatcher Set Up Instructions

Disable the caching for nocache selector in the dispatcher farm.

/0100  { /glob "*.nocache.html*" /type "deny" }

The following is variations in markup which the component is requested,

1. HTML markup rendered in the browser

<!-- SDI include (path: /content/site-name/en/_jcr_content/par/dynamic-component.nocache.html, resourceType: app-name/components/dynamic-component) -->
<div class="dynamic-component">
    ...
    .....
</div>

2. HTML markup cached in the dispatcher

<!-- SDI include (path: /content/site-name/en/_jcr_content/par/dynamic-component.nocache.html, resourceType: app-name/components/dynamic-component) -->
<!--#include virtual="/content/site-name/en/_jcr_content/par/dynamic-component.nocache.html" -->

This #include tag on the cached page on the dispatcher, instructs the dispatcher to check for the cached component resource /content/site-name/en/_jcr_content/par/dynamic-component.nocache.html. With the dispatcher farm configured not to cache the nocache selector, this component resource would not exist in the cache, so the dispatcher requests data from the publisher for this component resource whenever the cached page is accessed.

Use Case Two: Cache the component at dispatcher for a small period

To cache the component with dynamic data for a certain period, SDI has to be configured with TTL and a custom selector on the publisher instance via the OSGi Configuration Factory.

<jcr:root xmlns:sling="http://sling.apache.org/jcr/sling/1.0" 
    xmlns:cq="http://www.day.com/jcr/cq/1.0"xmlns:jcr="http://www.jcp.org/jcr/1.0" 
    xmlns:nt="http://www.jcp.org/jcr/nt/1.0"
  jcr:primaryType="sling:OsgiConfig"
  include-filter.config.enabled="{Boolean}true"
  include-filter.config.path="/content/site-name"
  include-filter.config.resource-types="[app-name/components/dynamic-but-cached-component]"
  include-filter.config.include-type="SSI"
  include-filter.config.add_comment="{Boolean}true"
  include-filter.config.selector="dynamicttl"
  include-filter.config.ttl="3600"
  include-filter.config.required_header="Server-Agent=Communique-Dispatcher"
  include-filter.config.ignoreUrlParams="[]"
  include-filter.config.rewrite="{Boolean}true"
  include-filter.config.appendSuffix="{Boolean}false"
/>

The above configuration ensures the component with the sling:resourceType as app-name/components/dynamic-but-cached-component in the site hierarchy /content/site-name to be being cached at dispatcher level for 3600 seconds, adding a debug comment:

<!-- SDI include (path: %s, resourceType: %s) -->

on the page HTML when the component is replaced. A dynamicttl selector will be used to get actual content from the publisher by making the HTTP request for the component from the dispatcher only if the cached version of the previous component request isn’t available or the cached version is expired. The dispatcher has to be updated to allow the dynamicttl selector.

Apache/Dispatcher Set Up Instructions

TTL feature is being utilized, it needs to be enabled in the dispatcher farm.

/enableTTL "1"

The following is variations in markup which the component is requested,

1. HTML markup rendered in the browser

<!-- SDI include (path: /content/site-name/en/_jcr_content/par/dynamic-but-cached-component.dynamicttl.html, resourceType: app-name/components/dynamic-but-cached-component) -->
<div class="dynamic-but-cached-component">
    ...
    .....
</div>

2. HTML markup cached in the dispatcher

<!-- SDI include (path: /content/site-name/en/_jcr_content/par/dynamic-but-cached-component.dynamicttl.html, resourceType: app-name/components/dynamic-but-cache-component) -->
<!--#include virtual="/content/site-name/en/_jcr_content/par/dynamic-but-cached-component.dynamicttl.html" -->

In this use case, #include tag on the cached page on the dispatcher, instructs the dispatcher to check for the cached component resource /content/site-name/en/_jcr_content/par/dynamic-component.dynamicttl.html. With the dispatcher farm configuration with TTL enabled, the dispatcher checks whether a cached version of this component resource exists within the stipulated lifetime span; if this exists then the dispatcher responds with this cached version, else the dispatcher would request data from the AEM publish server.

3. The content hierarchy in the dispatcher should look like below, with the cached version of the component dynamic-but-cached-component.dynamicttl.html.ttl

├── en
│   ├── _jcr_content
│   │   ├── par
│   │   │   ├── dynamic-but-cached-component.dynamicttl.html
│   │   │   ├── dynamic-but-cached-component.dynamicttl.html.h
│   │   │   ├── dynamic-but-cached-component.dynamicttl.html.ttl
├── en.html
├── en.html.h

Common Errors and Solutions

1. When the dispatcher fails to get component content from the publisher with our custom selector, we need to confirm the dispatcher filter has been configured to allow the custom selector. Sample error,

"GET /content/site-name/en.html" - hit [publishfarm/-] 0ms
"GET /content/site-name/en/_jcr_content/par/dynamic-but-cached-component.dynamicttl.html" - blocked [publishfarm/-] 0ms

2. Occasionally, the dispatcher rewrite rule interjects the component content request with the custom selector and creates a 404 error. This can be solved by adding a rewrite rule targeting the custom selector as below,

RewriteRule ^/content/site-name/(.*)\.dynamicttl.html -  [L]

To conclude, Sling Dynamic Include is a great way to utilize caching mechanisms even with dynamic content. This article is a good starter for anyone planning to use or understand Sling Dynamic Include for AEM implementation.