Chained Queueables in Salesforce

Some years ago, thinking about asynchronous processing in Salesforce was sort of painful. You needed to either use Scheduled Apex or @future annotated methods and if you did any of those, you know how limited they were.

To summarize the points I think are more horrible about those two friends are the Scheduled Apex’s rather rigid structure and slow performance, and for Future methods the worst part was they could not be chained by calling a future method inside another one.

The Queueable interface brought to the Apex ecosystem a lightweight alternative for asynchronous processing without the drags of both Schedulable Apex and @future-annotated methods, and it definitely works like a charm. You simply implement an interface with a one and only method called execute and you’re all set for some serious asynchronous programming, and you can even chain one after the other. How cool is that?

However… life is not so easy. Apex only supports chaining only 1 queueable from another, and the same applies to putting them in the queue from a Scheduled Apex context such as a Batchable class.

So what happens if you want to fire, say, two queueables from another one, of from a Batch Process? Yup, you guessed right: Ka-boom.

The ChainedQueueable Class

So, I thought about a solution for said limitations for a little while, and came up with a simple idea that can make our lives so much easier when working with complex asynchronous scenarios.

While we can’t bypass Salesforce’s limitation of not calling two Queueables (or more) from another one, we can definitely save us the complexity of running everything in just one queueable or the (even) worse alternative of making different queueable classes depend on each other so one can call the next one after it finishes.

However, that last sentence gave me the idea of what could actually work: What if we define a sort of manager class that knows of all queueables I want to execute, and tells each one what to do after they end? That would definitely avoid all limitations while reducing dependency between classes to a minimum.

Here’s a pseudo-code of how would that work:

Manager m = new Manager();
m.add(queueable1);
m.add(queueable2);
.
.
.
m.add(queueableN);
while (m.hasNext()) {
    executeAndWaitToFinish(m.getNext()); 
} 

While the above sounds overall simple, it’s not so much to do this in Apex, since we don’t have a reliable and efficient way to stop execution until a queueable has finished in order to execute the next one. The only one capable of doing so is the Queueable itself.

So, I thought it might be a good idea to create an abstract implementation of the Queueable interface (let’s call it “ChainedQueuable”) that knows about that manager class I was talking about and can ask it for the next job to execute when it has finished its own work, we don’t need to make all Queueable implementations depend on each other, nor worry about what’s actually next because we know it’s a ChainedQueueable and can work with that.

The Chain itself, or manager, would need to hold a list of all jobs pending to be executed and should have methods for adding to the chain and start execution (aka. “runNext”). It should also be able to know when there’s nothing left to process and stop at once.

Did I lose a lot of people so far? Then let the code speak loud and clear: here’s the final implementation:

/**
 * Implements a chain of Queueable jobs.
 */
public class QueueableJobChain {
    
    // Here we store the currently 
    private List<ChainedQueuable> chain;
public QueueableJobChain() {
  this.chain = new List<ChainedQueuable>(); 
 }
/**
     * Adds a queueable to the beginning of the chain.
     */ 
 public void chainFirst(ChainedQueuable cq) {
  this.chain.add(0, cq);
 }
/** 
     * Adds a queueable to the end of the chain. 
     */ 
 public void chainLast(ChainedQueuable cq) {
  this.chain.add(cq);
 }
/** 
     * Checks if there are more items in the chain left to be run
     */ 
 public Boolean hasNext() {
  return this.chain.size() > 0;
 }
/**
     * Runs the first queueable in the list.
     */ 
 public void runNext() {
  if (this.hasNext()) {
   ChainedQueuable cq = this.chain.remove(0);
   cq.setChain(this);
   System.enqueueJob(cq);
  }
 }
}
/** 
* This is meant to be the super-class for any Queueable job class we want to participate in a chain.
*/ 
public abstract class ChainedQueuable implements Queueable {
        // Reference to the chain this queueable belongs to, because we can't allow callbacks to the parent chain
        // for when this class's execute method ends. 
        protected QueueableJobChain chain;
        
        /**
         * The constructor receives a chain as parameter. 
         */ 
        public ChainedQueuable(QueueableJobChain chain) {
            this.chain = chain;
        }
        
        /**
         * Executes the next queueable in the chain, if this queueable is in a chain. 
         */ 
        protected void executeNext() {
            if (this.isChained()) {
                this.chain.runNext();
            }
        }
        
        /**
         * Sets the job's chain.
         */ 
        public void setChain(QueueableJobChain chain) {
            this.chain = chain;
        }
        
        /**
         * execute method from the Queueable interface. You're meant to override this in any sub-class.
         */ 
        public virtual void execute(QueueableContext context) {
            this.executeNext();
        }
        
        /**
         * Tells us if the job is in a chain or not. 
         */ 
        protected boolean isChained() {
            return this.chain != null;
        }
 }

Using the QueueableJobChain

Using that implementation is quite easier than you might think. Let’s put a simple example:

I have to implement two Queueables, A and B, and chain them using that QueueableJobChain class in order to have them execute from another queueable called C.

First, we need to make sure A and B extend ChainedQueueable:

public with sharing class A extends ChainedQueuable {
    public A(QueueableJobChain chain) {
        super(chain);
    }
    public override void execute(QueueableContext context) {
         /* execute your stuff here */
         
         super.execute(context); // runs the next queueable in the chain.
    }
}
public with sharing class B extends ChainedQueuable {
    public B(QueueableJobChain chain) {
        super(chain);
    }
    public override void execute(QueueableContext context) {
         /* execute your stuff here */
         
         super.execute(context); // runs the next queueable in the chain.
    }
}

And now, here’s the code from C:

public with sharing class C implements Queueable {
    public void execute(QueueableContext context) {
         /* Do whatever C needs to do */
         QueueableJobChain chain = new QueueableJobChain();
         chain.chainFirst(new A(chain));
         chain.chainFirst(new B(chain));
         chain.runNext(); // runs B, then runs A.
    }
}

Simple enough? I think so. We’ve reduced the amount of extra work needed and we’re making sure we add only 1 job to the system’s Queue every time.

Now that you mention it…

Yeah, I know adding a dependency from the ChainedQueueable class to the QueueableJobChain is less than ideal but Apex doesn’t allow callbacks or any way of waiting except for polling, so I believe this solution solves a few cons of the Queueable interface without losing features.

So, what do you think? Any ideas on how to make it better? I’d love to read your comments!

Cheers, and hope you’ve enjoyed reading!