(In)Validating Library Design. DomainDocs4s: DDD, Reflection & Classpath Scanning.

7 min readOct 20, 2024

Last week I got an idea.

This post goes through the process of transforming that idea to a draft of library design and then checking its technical feasibility. You can see it as an open and very informal ADR.

Living, Code-Driven, Domain-Oriented Documentation

So what do we want to build exactly? A way to build domain documentation, e.g. Ubiquitous Language glossary, by taking symbols (classes, methods, etc.) from code and attaching some text to it.

And what exactly we might want to annotate?

Packages
Classes, traits and objects
Fields, methods, type declarations

So that we could write code like that:

@domainDoc("Named bucket of assets")
enum Account(val name: String) {
  ...
}

@domainDoc("Movement of positive amount between two accounts")
case class Movement(currency: Currency, amount: BigDecimal @@ Positive, from: Account, to: Account)

@domainDoc("Collection of movements")
trait Ledger {
  def movements: Seq[Movement]  
  
  @domainDoc("State of the accounts produced by summing all the movements")
  def balances: Map[Account, Map[Currency, BigDecimal]
}

@domainDoc("Ledger keeping track of user assets")
class UserLedger(userId: UserId) extends Ledger

@domainDoc("Ledger keeping track of company's asset")
object OperatorLedger extends Ledger

From which we could produce different documentation formats, e.g.

Hierarchical glossary of terms (in e.g. Markdown)
Diagram showing relationships between entities (similar to UML class diagram but with no technical details)
Detailed doc on all possible Accounts (enum values)

Splitting The Complexity

Let’s try to split the problem into smaller parts and discuss them one by one. There are three main things needed to make such a library happen:

Finding all the interesting symbols
Encoding the documentation
Extracting the documentation

At this point we will ignore the subject of producing a usable output as this is not technically challenging.

Finding All The Symbols

There are quite a lot of different ways to find symbols interesting for us. Those ways come with various tradeoffs that we will discuss later, for now let us just list and briefly describe them.

Scala Compile-Time Reflection — We can use TASTy files generated during compilation and TASTy Inspector to get access to almost all the information available in scala code.
Classpath Scanning — If we limit ourselves to JVM, we can scan the JVM classpath using either an unmaintained but still functional ronmamo/reflections library or very fast classgraph project.
SemanticDB —We can instruct sbt to build for us the SemanticDB, a set of .proto files containing most of the semantic information about our code.

Encoding The Information

In the initial example we used an annotation to attach a documentation string to a symbol. This is a way to do this but not the only one. Here is a more comprehensive list of the options:

Scala Annotation

case class domainDoc(doc: String) extends scala.annotation.Annotation 

@domainDoc("My documentation")
class MyThing()

Java Annotation

@Target({ElementType.TYPE, ElementType.LOCAL_VARIABLE, ElementType.FIELD, ElementType.PACKAGE, ElementType.METHOD})
@Retention(RetentionPolicy.RUNTIME)
public @interface domainDoc {
    String doc();
}

@domainDoc(doc = "My documentation")
class MyThing()

Inheritable Trait

trait DomainDoc {
  def domainDoc: String
}

class MyThing
object MyThing extends DomainDoc {
  def doc = "My documentation"
}

ScalaDoc Section

/**
 * @domainDoc my documentation
 */
class MyThing

All those come with different tradeoffs that we will investigate later on.

Extracting The Information

Once the information is encoded, we need a way to extract it. Not all the options listed below can handle all the encodings, but let’s ignore this for the moment. Those are the options:

Scala Compile-Time Reflection — Similarly as with listing the symbols, we can use TASTy files and TASTy inspector to access the code almost directly.
Scala Runtime Reflection — Unfortunately, Scala 3 doesn’t come with rich-enough runtime reflection out of the box as scala-reflect was never migrated to Scala 3. Luckily, we have some libraries that try to expose a similar API: gzoller/scala-reflection and izumi-reflect.
Java Reflection — Once again, if we limit ourselves to Java-compatible runtime and constructs, we can use Java reflection directly.

Final Touchdown

Now that we have an understanding of possible options, let’s try to analyse and compare them.

Before going further, let’s explain a few caveats marked in the table above.

Scala annotations can’t be attached to packages (or package objects) since… always. Check scala/bug#3115 for details.
Sadly, I didn’t manage to use either of the community-maintained options. izumi-reflectfocuses primarily on types and have almost no support for symbols. gzoller/scala-reflect gives access to annotations, and it has internal tests where extracting annotation values should be possible, but I wasn’t able to reproduce this in my examples. All things considered, I’m a bit hesitant to explore those options further, especially in the presence of alternatives.
Java's ability to annotate packages is not well-known, so if you’re curious, here is an example.
Attaching trait to a type declaration could be possible through union type, but it feels like a hack.

First Decision: Encoding

On the surface, the clear winner is ScalaDoc as the only one that can be attached to everything. In practice, I won’t consider it as a primary candidate. Why? I believe comments don’t feel as first-class citizens and it's much easier to forget about adding or updating them. It’s not the API I want to expose but for psychological reasons, not technical ones.

Then the choice is between Java and Scala annotations, and it's not a hard one. While Scala annotations can’t support packages, they come with much more natural API, they are more cross-platform compatible and have better support in scala tooling. Java annotations, on the other hand, are much easier to access (we will talk about that a bit later) but can’t support type declarations, which, in my opinion, are a crucial element of domain modelling in Scala.

When it comes to attaching docs to packages, we have two options: don’t support it at all or have an alternative mechanism for attaching it. Both options seem viable, and we don’t need to commit to any of them at this point.

Hence Scala annotations become our primary candidate for an encoding mechanism.

Second Decision: Extraction

We have little choice here, as Scala annotations are not available through Java reflection and so we are stuck with Scala reflection. And if we want to be Scala 3 compatible, we are limited to compile-time reflection based on TASTy.

To be honest, at this point I was considering abandoning the whole idea. While implementing the library seems technically possible, the code required to scan raw tasty files, find the annotations, extract constant parameters and instantiate annotations is not the code I want to write. Part of getting old is knowing one’s limitations. I’m good at writing high-level domain code, and I have very little interest in writing macros or diving into compiler internals. I could do it probably, but I just don’t want to.

If there were no alternatives, I would either abandon the idea or re-evaluate Java annotations and Java reflection. The problem with the latter is that it’s another compromise I don’t want to make. I’d rather not build anything at all rather than build a half-baked crippled imitation of what it could be. Relying on Java constructs would work probably but with significant limitations.

Luckily, I found tasty-query! It handles all tasty interactions and exposes a high-level API similar to SemanticDB. It even allows you to get constant annotation arguments. And that’s exactly what I needed! To make it even better, it’s a well-maintained project owned by ScalaCenter, which gives good long-term guarantees.

Third Decision: Listing Symbols

Actually, there is almost no decision to take here. tasty-query exposes API very similar to classgraph and it solves both problems for us.

Theoretically, there is one more question to ask here: do we want to gather information only from our project or from the whole classpath, including dependencies. I can easily imagine a use case where some internal library contains common domain models and attaches some docs to them. Luckily tasty-query allows one to handle both approaches.

Final Decision

Now we know we can build this library and how. Unfortunately, those are not the right questions to ask. The right question is: should we? And that one is harder to answer, especially if we are considering it as a usable project, not a throw away experiment. Let’s ask ourselves a few helper questions:

Can it benefit enough people to justify creation and maintenance cost?
Does it solve a popular enough problem?
Is the problem important and costly enough to justify adding a dependency to solve it?
Can we expose a good enough API?
Can we solve the problem in a generic enough way?

Sadly, I’m still not convinced we can answer yes to all of these. I’m not convinced enough people care (or can be convinced to care) about domain-driven design. I’m not convinced enough people care about documenting it. I’m not convinced we can produce universal-enough formats from the gathered information. But I’m also not convinced we can’t!

And you can help. If you have an opinion, feedback or a suggestion, please come to Business4s Discord and speak up! If there is enough interest, I will definitely try to build it. Otherwise I will stay satisfied with the fun I had when doing this analysis.

Code written for the above evaluation can be found here.

Business4s Blog