Maintaining Day 0 Compatibility For 23 Confluence Apps on the Atlassian Marketplace
A journey starting from compatibility updates lagging behind for over a week in some cases to having perfect day 0 compatible versions of our apps.
Background:
Atlassian has active development and release cycles for its server and data center hosted Confluence platforms. From the beginning of 2022 up to July, a total of 31 versions have been released averaging over 4 releases each month¹. Confluence releases follow the semantic versioning conventions of major.minor.patch for the latest versions. Additionally, certain patches get backported to LTS versions for critical bug fixes.
ServiceRocket lists 14 Confluence server and 9 Confluence data center apps on the Atlassian marketplace. With such a broad portfolio of app listings, keeping up with all the changes in each Confluence release for each individual app became a cumbersome task that consumed more time than we were able to afford. This in turn caused some of our apps to occasionally block our customers from upgrading due to not having support for the latest versions of Confluence.
To complicate matters further, Atlassian marketplace APIs try to help developers by automatically upgrading (auto-bumping) the compatibility ranges for apps when the release is a patch. On the other hand, if the release is a major or minor version, the apps’ compatibility ranges are not automatically changed. What this means for us can be summarized in this table:
Early Efforts:
We realized early on that the current approach was not ideal for several reasons including:
- Not scalable for the number of apps we need to support. Takes too much developer time for tedious tasks limiting the time developers have for innovation in other areas.
- For patch Confluence releases, autobump means that occasionally, incompatible versions would be listed as compatible without running any tests.
- For major/minor Confluence releases, customers won’t be able to upgrade Confluence until tests are manually triggered and then compatibility is manually bumped on the marketplace for each of our apps.
Since the manual approach was in no way scalable enough, we began looking for alternative approaches to automate this repetitive process. Fortunately, Atlassian publishes an Early Access Programme (EAP) feed for upcoming Confluence releases. Using Jenkins declarative pipelines, we built an automation with a cron scheduler to test all of our apps with the latest EAP build of Confluence once a week.
Pseudocode
This EAP compatibility check has 2 stages:
1. Build Pipelines
At the defined schedule, the latest available EAP build is polled. Then, build jobs are triggered for each of our apps running the automated tests against the polled Confluence version.
2. Run Compatibility Check
Each app build and automated tests would run under this stage.
At the end of each job, a Workplace group announcement notifying the team of the outcome would be posted.
Despite the fact that this automation had better prepared us for upcoming compatibility issues by working as an early warning system, the actual General Availability (GA) release response was still fully manual and required developer time to drive.
Solution:
Phase 1: Build “Upgrader” and “Downgrader” Pipelines
To handle GA releases, 2 different scenarios need to be considered depending on the type of the release. As summarized in the table above, an “upgrader” and a “downgrader” pipeline were developed for the major/minor and patch releases respectively.
Pseudocode
Diagram
Phase 2: Build in-house Poller
The Upgrader and Downgrader pipelines made it easy to perform the necessary changes in bulk for apps on the marketplace as well as on the codebases. However, they still needed to be triggered manually and developers still had to actively monitor release announcements.
Therefore, the next logical step was to have the pipelines trigger automatically when a release is detected, with no developer intervention. We decided to build our own poller to achieve fine control and better security for our build server.
A file is stored on Jenkins with the current latest version of Confluence. Then, each workday hour, the poller job would poll the Confluence GA feed and trigger the upgrader or downgrader builds if a new release is detected. Finally, the new latest release version would be archived for comparison with the next runs.
Pseudocode
Sample Runs:
To show the system at work end-to-end, the first screenshot shows the EAP release candidate build of Confluence 7.15.0 on Sunday as scheduled, notifying that one of our apps called “Scaffolding” is passing the build.
Then, a few days later, the GA version of 7.15.0 was released and automatically detected by our poller which also triggers the “Upgrader” pipelines for all apps.
The Scaffolding build was triggered and after passing the build, the compatibility of the app listing was upgraded on the marketplace.
Final Thoughts:
Building automations is an investment that is almost always worth making. Teams can sometimes fall into the trap of normalizing inefficient processes. In our case, Confluence releases used to be a significant pain point for us as the associated compatibility work wasted considerable amounts of time that could have been spent on product roadmap items. Thankfully, we identified this inefficiency in our processes and managed to incrementally introduce improvements in the form of the phases described above. Currently, next-to-zero developer time is spent on compatibility for regular Confluence releases.
