Easily Coordinate External Deployments with Cloud Manager and Adobe I/O Events: A Step-by-Step Guide
If you’re an Adobe Managed Services (AMS) customer, you likely depend on a series of complex applications that span both Adobe Experience Manager (AEM) and other systems to get your jobs done. In some cases, integrations between these services may be loosely coupled, with deployments being done on completely separate cycles. But in many cases, the applications are tightly coupled and need to be deployed in a coordinated fashion. This is made possible using Adobe Cloud Manager with Adobe I/O Events capability, and in this post we’ll break exactly how to do this.
If you don’t know, Adobe Cloud Manager allows Adobe Managed Services (AMS) customers to easily and efficiently deploy their custom applications to their managed Adobe Experience Manager environments. Cloud Manager uses industry-standard tools like git for version control and Apache Maven for the build process along with AMS’s consistent Command & Control infrastructure. Combined with I/O Runtime, it’s a powerful tool that keeps it easy to expedite the delivery of customizations or updates to AEM workflows without compromising performance or security.
For the purpose of this post, we’re going to use Adobe I/O Runtime, Adobe’s serverless platform, as a specific example of an external system. But the patterns described here can be applied universally with some platform-specific changes of course.
The overall architecture of AEM and Adobe I/O Runtime
The overall architecture of the application we’ll consider consists of three AEM environments: dev, stage, and production, each consisting of author, publish, and dispatcher services. Within AEM, there is a component that invokes an Adobe I/O Runtime action. In the dev environment, this component should always invoke the latest version of the action code; in other words, every time a deployment to dev is done, the action should be updated with the latest code. In stage and production, the action needs to be versioned so that, for example, version 2019.807.72445 of the application code invokes version 2019.807.72445 of the action.
Application project structure
The application project structure, stored in the Cloud Manager-provided git repository, is a standard AEM project structure with the addition of a new top-level directory named runtime-actions. This directory contains the Adobe I/O runtime action code.
Within the runtime-actions directory, there is a sub-directory for each action to be deployed into I/O Runtime. Each of these directories contain a package.json file and at least an index.js file. For example, if you had a single action, named echo, the structure would be:
The component that invokes the Runtime action is a standard OSGi component which has a configurable endpointUrl property. As described above, this value needs to vary in order to access a different endpoint in dev, stage, and production, so a runmode-specific configuration is applied to do this:
In the config.dev directory, the configuration is very simple:
In both the config.stage and config.prod directories, the configuration file uses build-time interpolation to add the version information. This uses the ARTIFACTS_VERSION environment variable set automatically by Cloud Manager.
When the Maven build process is run by Cloud Manager, this ARTIFACTS_VERSION environment variable will be populated with a unique, continually increasing version. The same version is available through the Cloud Manager API and can be used to create a unique action name in I/O Runtime, as we will see below.
For this variable to be interpolated at build time, you need to ensure that the maven-resources-plugin is configured correctly. An example of such a configuration is:
Additionally, if you are using the filevault-maven-plugin you will need to force the plugin to use the processed resources:
<!-- more configuration here -->
Mapping Cloud Manager events to external deployment activities
In order to achieve a coordinated deployment, there are three events from Cloud Manager that are relevant:
- When a dev deployment starts, update the “dev” action to the latest code.
- When a stage deployment starts, create a new versioned action.
- Prior to the production deployment beginning, ensure that the versioned action exists.
The first two events are roughly similar; the only distinction is whether they update an existing action or create a new action. The last event is a bit more complex. From the perspective of the event provider (Cloud Manager in this case), Adobe I/O Events are ‘fire and forget.’ Cloud Manager does not wait for a response from an event handler. Since we want to ensure that the versioned action is present before deploying to production, it is necessary to use Cloud Manager’s existing ‘approval’ capability to wait for an external signal before proceeding with the production deployment.
Deploying to Adobe I/O Runtime from Adobe I/O Runtime
There’s a number of ways to slice this, but my preference is to have two separate actions: one that will handle the events, and one responsible for the build and deployment of the runtime actions, as well as checking that all of the actions that were expected to be created actually were. For simplicity, we’ll call that second action the ‘deployer’ action. The event handler can process the event from Cloud Manager and then decide to invoke the deployer action with the right parameters. The reason for this separation is that, as my integration needs grow, I may want to segment out other actions that get invoked on these same events or different events.
First, let’s look at the deployer action. This action is going to be configured with three variables:
- The Git Repository URL
- The Username
- The Password
The parameters to be passed in from the event handler action are the Git ref (the branch or tag) and the version of the action to be created/updated. For the ‘checking’ mode, a third parameter is used to indicate this.
In pseudo-code, the deployer action should do the following when in normal mode:
- Clone the git repository and checkout the right branch or tag. We can use https://isomorphic-git.org for this.
- Get a list of the existing Adobe I/O Runtime actions.
- In the git checkout, iterate through the runtime-actions directory and for each subdirectory:
a. Run npm install
b. Run npm run build
c. Create a zip file of the subdirectory. We can use archiver for this.
d. Generate an action name by combining the directory name and the version parameter.
e. If the action already exists (which would generally be the case in dev), update it. Otherwise, create it.
In check mode, the steps are:
- Clone the git repository and checkout the right branch or tag.
- Get a list of the existing Adobe I/O Runtime actions.
- In the git checkout, iterate through the runtime-actions directory and for each subdirectory.
a. Confirm that the expected action exists in the list of Adobe I/O Runtime actions with the expected version.
- If all the actions are present, the action should return a positive result. Otherwise, it should return a negative result.
Processing Cloud Manager Adobe I/O Events
The other action is the one that will receive events from Cloud Manager via Adobe I/O. The basic structure of this action is similar to the one built in the Cloud Manager API Tutorial, but since the event handler is done as an Adobe I/O Runtime action, we don’t need to worry about the signature verification, as Adobe I/O handles this automatically.
In pseudo-code, the event handler action needs to do the following:
- If the event is a ‘started’ event for a pipeline step:
a. Retrieve the step state representation.
b. Retrieve the execution representation. We need this to get the branch and artifact version properties.
c. If the step being started is a ‘deploy’ step for a ‘dev’ environment, get the branch from the execution’s build step and invoke the deployer action with the branch as the ref parameter and ‘dev’ as the version parameter.
d. If the step being started is a ‘deploy’ step for a ‘stage’ environment, get the artifacts version from the execution and invoke the deployer action with the artifacts version as both the ref and the version parameter.
2. If the event is a ‘waiting’ event for a pipeline step:
a. Retrieve the step state representation.
b. Retrieve the execution representation. Again, we need this to get the artifact version property.
c. If the step waiting is the ‘approval’ step, invoke the deployer action with the artifacts version as the ref and version parameter and pass the check parameter.
i. If the result shows that not all actions are present, execute a PUT request against the cancel endpoint for the approval step.
ii. If the result shows that all actions are present, execute a PUT request against the advance endpoint for the approval step.
Tying it all together
The event handler and deployer actions can be a single, configurable, project, deployed to I/O Runtime using wskdeploy.
Currently in I/O Console, actions deployed using wskdeploy cannot be used directly as event handlers in Adobe I/O events, but it is easy enough to work around, just create a sequence action with no namespace. For example, if the event handler action is named cmruntime/event-handler, you can do: wsk action create cmdeployer — sequence cmruntime/event-handler.
Then, in the Adobe I/O console, add an Event Registration pointing to the event handler runtime action and subscribed to the Pipeline Execution Step Started and Pipeline Execution Step Waiting events.
With this in place, you can add your I/O Runtime Actions code to your Cloud Manager git repository and start your Cloud Manager pipeline.
Cloud Manager and I/O Runtime: We’re just getting started
Of course, this is just a starting point. Your project requirements may involve different requirements and different integration targets. But I hope this has provided a good starting point to start building this type of integrated solution.
By combining Cloud Manager CI/CD pipelines, Adobe I/O Events, and a serverless platform like Adobe I/O Runtime, a wide variety of integrations are possible with minimal glue code.
If you’d like to see the full code used in this project, you can check out https://github.com/adobe/adobe-cloudmanager-io-runtime-deployer.