sfpowerscripts (now sfp and a standalone CLI), one of the first Salesforce DX CLI plugins, introduced the concept of pooling scratch org four years ago. The idea was to make just-in-time environments accessible to everyone by having a pool of scratch orgs prepared with all necessary components.
Diéffrei Quadros, one of the contributors to #flxbl, has written extensively on the concept of scratch org pools. Please find the links in the footnote.
Numerous projects were able to use scratch orgs and pools, but many older orgs needed help to keep up. This was mainly due to the significant work required to make the repository deployable, often delaying the adoption of modular and source-driven development in the organisation. Salesforce teams often believe you can only utilise packaging effectively if you conduct scratch org-based development.
Many organisations are shifting towards source-driven development, packaging, and DevOps-centric processes. To facilitate this transition and help organisations reach the goal of source-code provisioned environments, a new approach was developed — Sandbox pools. These pools act as a supportive tool to aid in the transition process.
Just in time ‘Review’ Environments
Creating a “dedicated” environment for validating/reviewing changes just in time is a significant improvement. Projects using just-in-time scratch orgs for CI have discovered issues much earlier and can test changes in an isolated environment, which removes queues and inconsistency in long-running CI sandboxes.
For the customers on Hyperforce, Quick Clone Sandboxes, which are way faster, still take around 2–3 hours to provision. We needed an automated process to create these environments beforehand so we could immediately assign one to a pull request.
Additionally, the same mechanism is used for review environments, available until they expire (more on it later), providing a safe space to test changes and receive feedback from end users without risking corruption. If the environment is corrupted, we can get a new one.
We will look into how the sandboxes are created and provisioned and the lifecycle in a subsequent post. In this post, we look into the lessons learnt from pooling developer sandboxes based on cloning a ‘snapshot’ for each active branch.
Lesson 1: Managing Sandbox Limitations
We faced a challenge while managing sandboxes — specifically, the restriction that developer sandboxes can only be deleted and recreated once every 24 hours. This meant we had to make multiple attempts to ensure that an adequate number of sandbox licenses were available to match a project’s development pace.
After analysing the demands of a few projects, we estimate that you need at least three times as many sandboxes as there are developers on the team (accounting for one sandbox per developer and approximately two per work item count), which essentially mimics the ‘Active/Daily’ scratch org limits.
Lesson 2: Sandbox Expiry and Rotation
We introduced a system to maintain the freshness and relevance of sandboxes. Sandboxes are assigned to pull requests or issues and expire 24 hours later to ensure that environments are consistently updated.
However, the nature of development workflows means that reviews and testing might take longer than a day. To accommodate this, a one-time extension is allowed, capped at 48 hours, to ensure that longer review cycles do not compromise sandbox integrity.
Lesson 3: Continuous Integration Challenges
One of the main difficulties we faced was ensuring that sandboxes stayed in sync with the active branch, especially in an environment where commits were made frequently. To overcome this, we needed a strong CI process that deployed individual changes and integrated recent updates from the main branch into the sandboxes.
This helped to maintain consistency and reduce integration-related problems. Therefore, you may encounter issues if you deploy only the changes made in a pull request.
Lesson 4: Organising by Domains
Limiting the synchronisation of review sandboxes to impacted domains is essential to optimise sandbox updates in a high-frequency team. Otherwise, a significant amount of time will be spent updating the review environments. #flxbl provides domain-related concepts such as ‘release config’ that can come to the rescue.
With this approach, we can focus sandbox updates on specific domains affected by a change, reducing deployment overhead. It also ensures that sandboxes remain aligned with targeted areas of development.
Any updates beyond your domain need to be manually updated, or you will receive them once the upstream changes are deployed to the source snapshot sandbox.
Lesson 5: Educational and Process Adjustments
In our developer channels, we often receive queries asking about unexpected changes in their work or failed test cases after clicking the update button. This confusion arose during the transition to sandbox pooling, which required effort to educate and adjust processes for developers. They had to understand the interconnected nature of their work within the broader project ecosystem and recognise the impact of trunk changes on their sandbox environments.
Integrating sandbox pooling into #flxbl practices has significantly assisted organisations in modernising their Salesforce delivery journey.
How about integrating both scratch orgs and sandboxes into the same workflow? Stay tuned!
