Designing a Macro System That Respects Grammar: Lessons from Rust

Macros are among the most powerful tools in a programming language designer’s toolbox. They allow programmers to express repetitive patterns, create domain-specific abstractions, and sometimes even extend the language itself. But with great power comes the potential for great confusion — especially when macros break the grammar of the host language or ignore the type system.

In this post, we’ll explore what it means to create a grammar-respecting macro system. We’ll use Rust’s macro system as a case study: it gets many things right, but also reveals limitations that hint at deeper design questions — particularly the desire for type information during macro expansion.

What Does It Mean to “Respect Grammar”?

When we say a macro system should “respect the grammar,” we mean that:

• Macros should produce code that is grammatically valid.
• Macros should integrate seamlessly with the parser.
• Macros should follow the syntactic expectations of the language.

This is more than syntactic sugar. A macro that fits cleanly into the grammar reduces surprises for users and helps tooling — like IDEs, linters, and formatters — work reliably.

Historically, some macro systems (like C’s preprocessor) simply operate on tokens or text without awareness of grammar. This can lead to confusing errors and unmaintainable code. Modern macro systems try to do better.

Rust Macros: A Tale of Two Systems

Rust offers two distinct macro systems, each with different strengths and weaknesses when it comes to grammar awareness.

1. macro_rules!: Declarative Macros

The older macro_rules! system in Rust is pattern-based and token tree–oriented. It allows you to write macros that match specific patterns of tokens and substitute them into output fragments.

What’s good:

• It enforces local grammar by matching against defined syntactic fragments (like expr, ident, ty, etc.).
• It works with the parser, so syntax errors are relatively localized and clear.
• It supports hygienic macros — identifiers introduced inside the macro don’t accidentally collide with identifiers outside.

What’s not so good:

• It lacks deep integration with the full grammar. Matching happens at the level of token trees, not full parse trees.
• There’s no semantic (type) information during expansion.
• You often need hacky workarounds to write flexible macros, like manually parsing comma-separated lists or nesting invocations.

For example, you can match an expr fragment, but you can’t introspect what kind of expression it is—or whether it’s well-typed.

macro_rules! my_print {
   ($val:expr) => { println!("The value is: {}", $val); };
}

This macro will happily accept an ill-typed expression like my_print!(3 + "oops");, and the error only surfaces after macro expansion. Worse, you can’t tailor the expansion based on the type of $val.

2. Procedural Macros: Powerful but Unstructured

Rust’s procedural macros (also called “proc macros”) operate on the abstract syntax tree (AST). They are much more powerful:

What’s good:

• They work on structured syntax trees.
• They can generate arbitrarily complex code.
• They allow full programmatic control in Rust itself.

What’s not so good:

• They operate outside the normal parsing pipeline.
• They cannot access type information (due to the phase separation in Rust’s compilation).
• Tooling often struggles with them — proc macros can’t be expanded in many IDEs.

Because proc macros run before type checking, you again can’t make decisions based on type. For example, a derive macro can’t emit different code depending on whether a struct contains numeric or string fields unless you manually annotate them.

The Dream: Type-Aware Macros

There’s a natural desire for macros to “know more” — especially to make decisions based on type information. For instance:

• Can we write a macro that only expands if the input expression is of a certain type?
• Can we simplify DSLs by having macros that reflect over types and generate code accordingly?
• Can macro errors report semantic problems before the type checker complains?

This would require integrating macro expansion with type checking, or at least allowing some limited feedback loop. But this is tricky: type checking depends on the structure of the program, which may depend on macros, which might themselves want to depend on types. You can quickly fall into circularity.

Some advanced languages (like Template Haskell, or staged languages) embrace multi-phase compilation, where macros can run in stages, gradually gaining access to richer semantic information. Rust has so far avoided this to preserve determinism and toolability — but the tension remains.

Toward Saner Macro Systems

Here are some principles that a grammar-respecting macro system could follow, inspired by the best (and worst) of Rust:

1. Integrate with the grammar. Macro expansion should produce syntactically valid code that fits into the language’s existing parse trees — not token spaghetti.
2. Expose semantic constraints. If full type awareness isn’t possible, consider supporting lightweight annotations, like traits or marker types, that macros can inspect.
3. Support staged execution. Allow macros to execute in phases — early macros for syntax sugar, later macros that can observe or react to type information.
4. Preserve hygiene and locality. Macro-generated names should not leak or collide with user code, and error messages should be readable.
5. Make tooling a first-class concern. Macro systems that break IDEs and formatters do more harm than good. Tooling should understand and, ideally, expand macros.

Conclusion

Macros don’t have to be wild beasts that trample grammar and confuse humans. With the right design, they can be powerful yet principled extensions of the language.

Rust gives us a glimpse of both what’s possible and what’s missing: macro_rules! shows how to keep macros grammar-aware and hygienic, while procedural macros open the door to powerful code generation at the cost of losing type visibility and integration.

The next frontier? Macro systems that are type-aware, grammar-respecting, and tool-friendly. It’s a tall order — but one worth striving for if we want abstraction mechanisms that feel like natural parts of the language, not hacks on top of it.