Workflows4s Goes Effect-Polymorphic

Whether I Like It Or Not

Workflows4s is now effect-polymorphic! With the recently released version 0.6.0 you can run your workflows on cats-effect’s IO, ZIO's Task, plain Future or Try — the library no longer assumes a specific effect runtime. The main user-facing change is:

object MyWorkflowCtx extends WorkflowContext {
  type Effect[T] = cats.effect.IO[T] // or Future[T], Try[T], etc.
  ...
}

And everything works as it used to! (Mostly)

Abstract over effect types #59 was one of the oldest issues in the project and something I had at the back of my head before I even decided how to name this project. And now it’s finally solved. The rest of this article is about why we did it and why we did it in a particular way.

Why bother?

Because community. I really want Business4s to be welcoming to all parties without picking favorites, even if I have mine. But convergence if effect systems isn’t happening on its own, and in the meantime, hardcoding IO puts up a wall between Workflows4s and people who'd otherwise use it. I'd like to keep all those potential users and that alone justified at least exploring the design space.

There are concrete technical wins too. Lukasz laid out the motivation in issue #59:

1. Ox / direct-style friction — bridging Loom-based effects through IO.blocking / unsafeRunSync fights the underlying model and adds context-switching overhead just to satisfy the IO signature.
2. ZIO and lost fiber context — zio-interop-cats works, but crossing the runtime boundary loses things people might rely on (FiberRef, MDC, tracing). It also drags in a heavy dependency tree purely for the bridge.

There were specific use cases too: Marcin told me he intends to run Workflows4s on AWS Lambda in as lightweight way as possible — IO plus its runtime was a hefty tax there.

The technical arguments alone wouldn’t have been enough — they’re real but mostly manageable through cats-effect interops, with acceptable tradeoffs. Combined with the community motivation though, this had to be solved.

How it happened

It was quite an endavour and here’s the timeline:

1. April 2025 — Issue #59 created.
2. December 2025 — Lukasz championed the effort, analysed the problem and produced both PoC and first implementation.
3. March 2026 — I talked with Marcin and a few others about Workflows4s during the Scalar conference in Warsaw.
4. April 2026 — Final push, this time against the lessons learned.

Lukasz’s involvement was critical here — I would probably have never acted on this idea without an external push. He not only pushed for it but also implemented almost the whole thing, with a couple of PoCs along the way. Unfortunately the one-shot approach didn’t work: the size of the PR prevented any sensible code review by both me and CodeRabbit.

That led me to start over on top of the lessons learned. I’ve split the work into 15 separate, individually compiling PRs (#227-#241), each carefully reviewed. They totalled to 3306 insertions / 2580 deletions across 200 files.

Encoding the effect type

Now let’s talk a bit about the technical side of the story.

I’ve considered 3 primary ways to incorporate the effect type. To compare them fairly, it helps to know where we start: workflows4s already extracts Event and State from a WorkflowContext. It would be nice if whatever we do with Effect fit that same pattern.

Type parameter — F[_] on WIO

The most straightforward approach: add F[_] next to Ctx and thread it through everything.

// simplified
sealed trait WIO[F[_], ...]
case class RunIO[F[_], ...](run: In => F[Event], ...) extends WIO[F, ...]

This is what Lukasz’s PoC used and it’s also what powered PRs 1–9 of the rewrite. It works, it’s reliable and the compiler is happy. But the price is one more type parameter to carry around.

Type member of WorkflowContext - match-type extraction

WIO had a WorkflowContext container from the very beginning. We already used it to fix two types shared across the whole workflow definition: Event and State. Putting Effect next to them feels like the natural move:

trait WorkflowContext {
  type Effect[_]
  type Event
  type State
}

But then we need to extract it from Ctx <: WorkflowContext

object WorkflowContext {
  type AuxEff[F[_]] = WorkflowContext { type Effect[T] = F[T] }
  // Workaround for lack of general type projections
  type GetEff[Ctx <: WorkflowContext] = [A] =>> Ctx match {
    case AuxEff[f] => f[A]
  }
}
case class RunIO[Ctx <: WorkflowContext, In, Out, Evt](
    f: In => WorkflowContext.GetEff[Ctx][Evt],
    // ...
)

This is the same pattern we already use for extracting Event and State. Unfortunately it doesn’t work for higher-kinded types:

The match type contains an illegal case:
    case WorkflowContext.AuxEff[f] => f[A]
The pattern contains an unaccounted type parameter `f`.

Match types in Scala 3 can’t bind a higher-kinded pattern variable in this position.

This has been reported to the compiler team as #25968. The trick that unblocks it came from Michal: declare AuxEff as an abstract type whose lower and upper bounds are the same refinement.

type AuxEff[F[_]] >: WorkflowContext { type Effect[T] = F[T] }
                  <: WorkflowContext { type Effect[T] = F[T] }

Because AuxEff is now an abstract alias rather than a transparent refinement, the match-type machinery treats f as a regular type variable and the reduction goes through. The actual code in workflows4s reads:

type AuxEff[_F[_]] >: WorkflowContext { type Effect[T] = _F[T] }
                   <: WorkflowContext { type Effect[T] = _F[T] }

type Effect[T <: WorkflowContext] = [A] =>> T match {
  case AuxEff[f] => f[A]
}

It works, but only just. The compiler reduces the match type unreliably enough that we had to add a few escape hatches around it — implicit conversions and a LiftWorkflowEffect typeclass that can witness the lift between an effect and its WCEffect[Ctx] form. Fortunately those workarounds shouldn’t surface in user code.

Type member of WorkflowContext - path-dependent types with modularity improvements

The third option is to drop type parameters on WIO entirely and lean on path-dependent types - have a WIO literally know which WorkflowContext value it belongs to. I've explored this in #187 and the early experiments were promising:

trait WorkflowContext {
  type Event
  type State
  type Effect[T]
}

sealed trait WIO[-In, +Err](tracked val ctx: WorkflowContext) {
  type Out <: ctx.State
}
object WIO {
  abstract class RunIO[-In, +Err, Evt](tracked override val ctx: WorkflowContext) extends WIO[In, Err](ctx) {
    def buildIO: In => ctx.Effect[Evt]
    // ...
  }
}

This requires Scala 3’s experimental modularity improvements — tracked lets a constructor parameter's value participate in the instance's path-dependent types, plus a few related features. I’ve hit one compiler bug with it (scala/scala3#25164) but it’s fixed now.

I paused this direction for one main reason: I heard there are no plans to support the modularity-improvements experimental flag in IntelliJ’s Scala plugin. That basically kills the idea for now if we aim for general adoption.

The final approach

The first option — F[_] on WIO - is the most straightforward, and I almost shipped with it. Only after most of the work was done did I switch to approach 2.

First reason: API shapes how we think about the problem. When defining a workflow, the effect type is a complete implementation detail and should be the least of our concerns. Repeating it on the most core data structure of the whole library is the opposite of that mindset.

That being said, it could be hidden from the user through context-scoped alias:

trait WorkflowContext {
  type Effect[_]
  type WIO[-In, +Err, +Out] = _root_.WIO[Effect, In, Err, Out, Ctx]
}

…but this doesn’t solve the inconsistency: Event and State come out of WorkflowContext via the type system, while Effect would come out of it via convention. And consistency is not the last issue. This encoding also opens the possibility of having, at least in theory, two WIOs from the same Ctx but with different Fs — and that’s a liberty we don’t need.

To sum it up: the simple encoding goes strongly against conceptual integrity which is my primary design principle these days. Hence I went with the type member.

I still keep the door open to the path-dependent approach if support for it — both in the compiler and in the ecosystem — matures enough. It feels more principled and less workaround-ish than what we ended up with.

Layered architecture

Now let’s see what it really means for Workflows4s to become effect-polymorphic. The library has a few layers, each responsible for different things:

1. WIO - the core data type defining the workflow, plus a few helper classes that operate on it (ActiveWorkflow and the evaluators).
2. WorkflowRuntime - a particular way to run a workflow. Handles persistence and similar concerns; produces a WorkflowInstance from a WIO.
3. WorkflowInstanceEngine - the customization entry point for runtimes. Adds things like logging, metrics, registration, and modifies how the workflow proceeds.
4. Edge components — KnockerUpper, WorkflowRegistry. Interfaces pluggable into the engine but not mandatory. They tend to use specialized approaches with significant tradeoffs (e.g. waking up workflows via Quartz).

Each layer was affected differently:

WIO was barely impacted. Only a handful of nodes (RunIO, retry handlers, checkpoints) had to switch from IO to the abstract WCEffect[Ctx]. The bulk of the AST didn't care.

WorkflowRuntime was minimally affected as it was already parametric over effect type. Some implementations changed but the interface didn’t.

WorkflowInstanceEngine, on the other hand, went through a significant redesign. While its goal stayed the same, it's now parametrized by WorkflowContext at the type level.

// It went from 

trait Engine {   
  def doStuff[Ctx <: WorkflowContext](...) 
}

// To

trait Engine[F[_], Ctx <: WorkflowContext] {   |
  def liftWCEffect: [A] => WCEffect[Ctx][A] => F[A]   
  def doStuff(...): F[...] 
}

This means it’s less generic and more powerful at the same time. The change was forced by a new responsibility: the engine is now the component that bridges WCEffect[Ctx] (effect used within WIO) with the runtime's F[_] (what the user wants to run in). When those two coincide, the lift is derived automatically; when they don't (e.g. workflow uses IO, Pekko runtime demands Future), the user can engine.mapK(...). There's a fuller treatment in the Effect Types docs page.

Edge interfaces — KnockerUpper and Scheduler interfaces were generalized to work with any F[_] rather than hardcoded IO.

Other interesting bits

MonadThrow is all you need

The constraint on the effect type in the core of the library is now cats.MonadThrow — we need to sequence operations and manage errors, and that's it. Luckily, we don't need any concurrency primitives, which is great for portability.

One caveat: the default basic engine needs to suspend clock reads. It does this via F.unit.map(_ => clock.instant()), which should be good enough for most use cases. Users who need stricter and more principled semantics can swap the engine.

We could go further and introduce a custom typeclass to drop the dependency on cats-core entirely. That would also let us use by-name parameters and implement a valid-but-unlawful Id instance. For now it’s left as a future improvement.

Cross-effect ForEach and Embedded

When we embedd a workflow within a workflow, the outer one’s effect doesn’t have to match the inner’s. This is theoretically useful and supported but I still see embedding as a an edge case — most users should need only a single WorkflowContext per workflow.

Smaller wins

• In-memory runtimes were redone. They now do roughly the same thing but split by audience: InMemorySynchronizedRuntime works for any MonadThrow[F] and uses Java primitives for synchronization, while InMemoryConcurrentRuntime (in workflows4s-cats-effect) requires Async[F] and uses Ref / Semaphore.
• Many implementations were generalized while staying in the cats-effect realm — things that used to be hardcoded to IO now work with F: Async, broadening their usefulness within that ecosystem.
• A test matrix was developed to verify different runtimes against different effect types. We tried to cover as much as possible, from the obvious (“Try on Java primitives", "IO on cats-effect runtime") to the more exotic ("ZIO running on Pekko"). It’s not yet the case but I’d love to see also Kyo and Ox there.
• Pekko runtime is Future-native — before we were forced to go through IO even if the underlying operations were using Future, now it’s no longer the case.

Summary

And here we are, Workflows4s no longer hardcodes any effect runtime and even though it comes with a hefty price (for both users and maintainers) I honestly believe the change is net-positive.

You can find more detailed examples in the docs. Migration should be relatively painless: workflow definitions are source-compatible, and the changes elsewhere (engine wiring, runtime instantiation) are mechanical. A detailed migration guide lives in this PR comment.

Many people contributed — Lukasz most of all, plus everyone who chimed in on the issue thread, Michal P. for the match-type trick, Marcin and others at Scalar for the conversations that pushed me to actually do this. I'm grateful to all of them and really appreciate the contirbutions.

This was a big change, and I’m happy it landed. Hopefully, it brings even more users to Workflows4s and Business4s community.