Apex Queueable and easy chaining

Queueables are great, but there is only one slight inconvenience with them that you can’t enqueue more than one job from Queueable.

So let us have a scenario:

• We have Cool_Object__c
• Inside its trigger handler, we invoke 2 long-running operations as part of some integration and use Queueable for each one of them.

In this case, we’ll have problems running DML operations on this SObject from another Qeueuable.

Failing example

Source code can be found in related Github.

A detailed example of the problem

(Feel free to go to the next section if you don’t need a detailed explanation on the issue).

So we have our CoolObjectHandler where we enqueue 2 Queueables

// CoolObjectHandler::handle();
switch on Trigger.operationType {
    when AFTER_INSERT {
        System.enqueueJob(new Queueable1(Trigger.new));
        System.enqueueJob(new Queueable2(Trigger.new)); // Limit Exception when invoked from Queueable context
    }
}

and run DMLs fine directly in the Synchronous context.

// insertObjectImmediate.apex
insert new Cool_Object__c(Name = 'Test');

Would work fine since both Queuables are invoking directly in the transaction and we’ll have the following execution tree.

Queuables invoked from the transaction (no Exception)

If we try to execute the following code:

// insertObjectQeueuable.apex
System.enqueueJob(new SuperIntegrationQueueable());

With SuperIntegrationQueueable being

// SuperIntegrationQueueable::execute(QueueableContext ctx)
Cool_Object__c coolObject = new Cool_Object__c(
        Name = 'Queueable for ages'
);
insert coolObject;
System.enqueueJob(new SyncChangesQueueable(coolObject));

Then we’ll have following execution tree:

Queuables are invoked from another Queueable (Queueable Limit Exception)

And when the platform try to execute

System.enqueueJob(new Queueable2(Trigger.new));

We will receive:

FATAL_ERROR|System.LimitException: Too many queueable jobs added to the queue: 2
Class.CoolObjectHandler.integration2: line 19, column 1
Class.CoolObjectHandler.handle: line 10, column 1
Trigger.CoolObjectTrigger: line 2, column 1

Proposed Solution

To solve this problem, I came with an idea of storing List Queueable in the state of another Queueable object and enqueue that Object at the end of the transaction.

Now, let’s go step by step :)

1. We create a class that stores List<Queueable> and it is enqueued at the end of the transaction (QueueableChain.cls).
2. We create a helper method that would add Queueables to this List<Queueable> instead of invoking System.enqueueJob() (QueueableWrapper.enqueue(…)).
3. We create another helper class that will implement Queueable and will be our workhorse from now on. We need to have control over our flow and make sure that System.enqueueJob() is executed on our QueueableChain only once at the end of the transaction (QueuableWrapper.cls).

Step 1. QueueableChain

// QueuableWrapper.cls
private class QueueableChain implements Queueable {
    private List<QueueableWrapper> chain = new List<QueueableWrapper>();
    public void execute(QueueableContext ctx) {
        if (this.chain.isEmpty()) {
            return;
        }
        QueueableWrapper current = this.chain.remove(0);
        if (!this.chain.isEmpty()) {
            for (QueueableWrapper queueable : this.chain) {
                enqueue(queueable);
            }
        }
        current.execute(ctx);
    }
    public void add(QueueableWrapper queueable) {
        if (queueable == null) {
            return;
        }
        this.chain.add(queueable);
    }
    public void commitQueue() {
        System.enqueueJob(this);
    }
}
private static final QueueableChain currentChain = new QueueableChain();

• We have our QueueableChain class and currentChain static variable to store state during the transaction.
• We enqueue an instance of that class behind the scenes.
• When execute() is invoked we execute the first Queuable in the chain and enqueue all the other to preserve state from previous transactions.
• All Queueables must extend QueueableWrapper because QueueableChain is a private class and it is enqueued from at the end of QueueableWrapper execution.

2. Helper method for enqueuing

That one is pretty simple

// QueuableWrapper.cls
public static void enqueue(QueueableWrapper queueable) {
    currentChain.add(queueable);
    if (!System.isQueueable()) {
        System.enqueueJob(queueable);
    }
}

We still want to use System.enqueueJob during our transaction, because we can and because I don’t know how we can commitQueue only once at the end of the transaction)

3. Helper class to manage execution flow

public abstract class QueueableWrapper implements Queueable {
    abstract void work();
    public void execute(QueueableContext ctx) {
        this.work();
        currentChain.commitQueue();
    }
// previously referenced code
}

So we introduce abstract method work, which will store our business logic.

In the execute method we invoke our work and afterward commit our QueueableChain by finally invoking System.enqueueJob().

If inside QueueableWrapper::work() we only use QueueableWrapper.enqueue() instead of System.enqueueJob() then all Queueables will be added to currentChain and after that, we will execute System.enqueueJob() only once at the end of the transaction.

How to transfer to QueueableChain

1. Start flow by flow
2. Replace all System.enqeueueJob() with QueueableWrapper.enqueue
3. Rewrite Queueable classes. a) Replace implements Queueable with extends QueueableWrapper. b) replace execute(QueueableContext ctx) with work()
4. and you are good to go.

This is my first article, so I would greatly appreciate any tips and comments for improvement. You can also submit your PullRequest to my Github repository.

References:

ytiq/apex-queueable-wrapper

Queueable Apex | Apex Developer Guide | Salesforce Developers

Photo by JJ Ying on Unsplash