Generating Angular HTML Templates through Razor Fluent APIs

… But WHY?

Building and maintaining component templates in large and complex Angular applications tends to become difficult very quickly. Using an example, I will show you how we solved this issue using template generation with Razor.

When confronted with this idea, the default reaction of most frontend developers will be: “Why would I even want to generate HTML templates for Angular when I could just write the HTML myself?”. There is nothing inherently wrong with writing all the HTML by hand. So, what problem is this trying to solve? Why did I have to go and make things so complicated? Let me tell you because there are actually a couple of pretty good reasons to choose this route.

Even moderately experienced full-stack and frontend developers have at some point run into the scaling issues that are somewhat inherent to HTML-based applications.

• HTML is declarative and as a natural consequence usually involves a lot of duplication because things that should look the same need the same markup.
• The more complex the application is, the harder it is to keep the HTML readable. Many different attributes and classes, all with descriptive names tend to generate a bit of a tag soup.
• Keeping the client-side validation in sync with the server-side validation is tedious and error-prone and unfortunately equally necessary.
• Every new cross-cutting concern that needs to be represented in some form in the HTML code multiplies the previous issues.
  These scaling issues can be mitigated with reusable HTML components, be it native or with the help of frameworks like React and Angular, but that mitigation can only go so far. And some of the problems, like the many attributes, are exacerbated through such frameworks.

A good example is worth more than a thousand words, so let’s look at one. The example is of course incomplete and simplified, but it should still help to give a rough idea about what is going on.

Photo by Hair Spies on Unsplash

First, we have a Razor Template file that might contain something like this:

It is a Razor template, but it doesn’t actually contain any HTML! It uses a domain-specific fluent API to model the UI in a readable and controllable form. And besides a bit of “Razor noise”, this should give everybody who reads this code a pretty good idea of what UI this represents.

• It contains a form that allows the user to change some values
• In the form, there is something called a “block” which is an application-specific visual concept that mostly consists of a title and some content.
• In the block, there is a generic container and a form group which represents a pair of label and control that belong together
• Then we have a label as a “4 column” wide element (Bootstrap grid) and the actual input control which is a text input, also “4 columns” wide

It looks relatively straightforward, but there is a lot of “magic” and implicit business requirements behind all of this. The resulting HTML markup looks significantly more complicated

There is a lot going on in this HTML markup that was completely abstracted away by the Razor template. Everything in this HTML snippet is relevant and necessary for the application to work correctly, even the things that look unnecessary. And this is only a very basic example! Without the abstraction provided by this templating mechanism, every developer working on this application would have to know every little detail about the many different “aspects” of the underlying components, directives and configuration, including how they all interact with each other. This is exactly the kind of problem that normally results in a big bowl of copy-pasta with a huge side-order of ¯\_(ツ)_/¯.

The Problem

This approach is certainly not recommended for small or medium-sized projects. Setting all of it up correctly takes quite a bit of effort that is difficult to justify for smaller projects. The project for which we introduced this is big. Really big. For most of the issues that we face in this project, there would be alternative solutions, when considered separately. However, when all of those problems show up together and the solution needs to scale to create a maintainable application built by a constantly changing team of local and offshore developers, things get a bit more sticky.

We will not go much into the specifics in this article, but the application in question has some very interesting requirements:

• Migrated Desktop Application
  It is essentially a re-engineering of a fairly old desktop application. As such, some compromises had to be made when compared to how one would normally develop this type of web application. If you don’t know what exactly that means, then you should count yourself lucky — desktop-style applications on the web can be quite challenging.
• Highly dynamic
  The application consists of separate “modules”, all of which can be opened in parallel — multiple times — and must maintain their state. Inside every “module” is a series of screens that usually contain complex forms or data grids or both.
• Massive in scale
  There are at this point roughly 1500 different components (not counting library components), 250 different grids, 350 forms, thousands of form fields, etc. And the application isn’t even done yet.
• Diverse/inconsistent by design
  There are only very few common patterns across all the different modules. Every module is targeting a specific business case with very specific requirements. The end result of this is: that almost every configuration option that exists is used in at least a handful of places.
• Maintained by a large, heterogeneous team
  Most of the developers are full-stack developers with a .NET C# background. And as you would expect, the experience in frontend development for newcomers joining the project is usually quite limited. There are also many junior developers with very little practical experience and the effort of first teaching them proper web application development and then introducing them to a project that isn’t really something that could be considered a “normal web application” would be enormous.

Now, let’s have a look at some of the most important aspects of the overarching problem and how the Templating DSL helps mitigate them.

Validation and Binding to Server DTOs

One of the problems that we were most aware of in the beginning is that of keeping client-side validation and server-side validation aligned with each other. Everything that is validated on the client MUST also be validated on the server — we are assuming that everybody knows that by now. Validation errors that are generated on the server side should generally be as close to the client-side implementation as possible, down to the field names. We took some inspiration from the ASP.NET (pre-Core) forms helper methods and decided that the best way to maintain consistency would be to declare as much as possible on the server side, directly on the C# DTO that builds the basis for the UI and then use something similar to the Razor forms model binding mechanism to extract the metadata from the DTOs and attach it to the generated markup.

This doesn’t only work pretty well for validation, but also the translation of field labels and linking server-side validation errors up with their corresponding UI elements. And all of that without tightly coupling the C# DTOs to the actual UI representation.

In addition to the validation aspect, there is also the concern of automated UI tests using browser automation. The sub-team that handles this is also using C# to write their tests and through the use of the binding metadata, we can help decorate the HTML with appropriate testing attributes that make it much easier to find and interact with a specific form field by again using the C# DTO structure. From the example at the beginning of the article: <app-textbox ... data-at-id="MyEntityDto-Code">

The “data-at-id” is derived from the root model DTO of that particular component and is used to identify the field in automated tests, making these tests much more resilient to minor UI changes and less brittle than they would otherwise ever be.

At the core of this is a custom HtmlHelper extension method called Bind

The BindingExpression is an abstract representation of the concept of a "binding" that can easily be mocked in unit tests without the need for all of the Razor "cruft" that the HTML helper brings along.

Working with Highly Configurable Library Components

As mentioned above, the application contains many different grids which is one of the reasons why we decided early on to go with Kendo UI for Angular components — it was deemed the most mature implementation at that time. And Kendo is great because it is very, very configurable. But like all very configurable things, it needs to be… configured. Since there are virtually no defaults that would work for even a significant subset of all grids, that means every grid needs quite a lot of configuration.

It is not at all uncommon in this application that the complete HTML markup for a single grid (formatted to be readable) is between 500 and 1000 lines of HTML! This number is so big because every column needs to be configured separately based on its data type, its desired visual representation, its searching and filtering configuration and also its export configuration. In addition to the native Kendo configuration options, there are also various custom interconnected directives to make subtle or substantial tweaks to the normal behavior. If you multiply that by hundreds of grids, you will inevitably arrive at a state of massive cognitive overload.

Using the DTO binding mechanism and an abstraction of the domain concepts, we can reduce the number of lines that a developer has to write down to less than 100 — which looks something like this

Of course, there are other approaches to achieve a similar level of encapsulation, but so far we have not seen one that would work equally well with a similar number of different use cases.

Type-Safe Fluent APIs as a Tool for Scaling

The last major point to drive home here is that types are good and type safety and proper IDE support for contextual code completion (i.e. IntelliSense) is one of the best tools to help reduce the number of potential error sources in large-scale projects. The internals of the fluent Templating API have been built to make it as easy as possible to do the right thing and as difficult as possible to accidentally pass the wrong type of argument. An “Angular expression” has its own type and so does an “Angular interpolation”. Angular input-, output- and two-way-bindings have their own types as well. And so do CSS classes.

This doesn’t mean that developers don’t need to know what the HTML templates look like or how Angular templates generally work. It only means that they can focus on delivering business value by relying on this abstraction layer for their day-to-day work. And when they need to add new functionality to core components, then they can drop down one level of abstraction and focus on that.

Implementation

In the Razor templates, we create controls through a control factory which is nothing more than a series of factory methods and the control factory is exposed to Razor as an extension method of IHtmlHelper<TModel> that helps to capture some contextual information from the HTML helper when needed. Since many different controls share some of the same functionality, the controls can make use of reusable behaviors in a composite pattern and they can also use mixins to create consistent APIs without the need for a "God base control class". More on mixins will follow later. This high-level API is supported by a low-level API for building a DOM structure and rendering that DOM structure to proper HTML. This lower level API is essentially "Microsoft.AspNetCore.Mvc.Rendering.TagBuilder on steroids" and deals with topics like Angular binding types and HTML attributes so that the rendered HTML looks like what developers would write by hand.

The Core Abstractions

All the core abstractions exist to serve the composability of the templating API. Small and isolated pieces which can be combined and re-combined with very little effort to create more complex controls. That is what keeps the code maintainable and flexible even at extreme scales. So, let’s dive a bit deeper into those essential concepts.

Capturing Child Content
In the end, it’s still HTML and we have to be able to add arbitrary content as the children of some controls. This should work regardless of how exactly the content itself is defined, which can be quite tricky because of the way that Razor works: Normal markup and partials are directly written to “the output stream” like in the following example

Whatever happens between BeginForm and the dispose call at the end is rendered directly to the output which means that it cannot influence the already written "<from>" tag in any way.

This is where the fluent control API fundamentally differs from the assumptions that the Razor engine makes: We want to be able to add a content block to a control in such a way that we are able to delay the writing of that content so that the control can decide where to render which block of content. This was a deliberate design decision on our part because it just makes it much easier to reason about the code when it doesn’t mainly consist of side effects — yes, the immediate writing of markup to a “global” stream in many situations becomes an undesired side-effect. This also makes all of the controls much easier to unit-test!

The solution requires a bit of a workaround, but that workaround has been very stable, even across .NET Framework, .NET Core and now .NET and is easy to maintain when compared to use that we get out of this abstraction.

As you can see in the basic example at the start of the article, we use this concept a lot.

For controls that need multiple different content blocks, we use Templated Razor Delegates in combination with fluent configuration methods which are a bit more awkward to use but provide some additional flexibility.

A PartialTemplate represents an arbitrary markup fragment that can be added to any control that has a "slot" for it. Here, we use it to customize the content of a tab-strip tab title. And then for the content of the tab we also use a partial template, but this time it is a template that is automatically created from SomeOtherControl .

“Mixins” based on Interfaces and Extension Methods for Code Reuse

A lot of the “magic” of this templating approach comes from clever use of overloads and well-defined semantic types — as opposed to using strings everywhere. But one problem that this raises is that of code reuse. It must be said at this point that using inheritance as a means of code reuse is generally a very bad idea since it always leads to “bloated base class syndrome”. On the other hand, having interfaces that must support multiple different overloads is also less than ideal since it leads to lots of code duplication. The happy middle ground for the high-level control APIs is to use minimalist interfaces and add the “syntactic sugar” on top of that in the form of extension methods.

Most of the controls are implemented based on a composite pattern using behaviors. The composite interface just needs one method that allows adding behaviors that are executed on the control when it is rendered.

In many cases, the actual mixin then doesn’t even need to specify additional interface methods which means adding a mixin to a specific control is as simple as adding the interface to the list of implemented interfaces. As an example, we can see how the “IClass” mixin is implemented. It deals with CSS classes and so the word Class in the code below refers to CSS class names and not to C# classes.

Note that interface default implementations would also work to solve that problem, but:

• Default interface implementations did not exist yet when we started this implementation.
• Even now, we consider extension methods to be the better solution for this since the additional overloads are usually really nothing more than syntactic sugar for dealing with type conversions and they should not be part of the interface itself.

Rendering the HTML Files

Now that we have some idea of how the HTML is built from the ground up, we can come to the last and very important part: How to combine the Razor templates with Angular to build a working application.

We have found that the easiest way to “hook into” the Razor engine with ASP.NET Core is to simply follow the normal web application startup process but instead of listening to incoming HTTP requests, we use reflection to find all of the (compiled) templates, instantiate and render them out to files. Instead of the final call to builder.Run() to start the server, we just run our custom code if the web application is started with the "--render" argument.

This means that when we make changes to the *.cshtml templates, we first have to build the web application and start the web application in render mode before we can start the Angular build. This is sometimes a bit inconvenient, but with some development tooling and shell scripting, this is easy enough to manage.

By allowing the normal application configuration process to run, we end up with a fully configured IServiceProvider that we can then use to instantiate the IRazorViewEngine and everything else that we need. The overall process for rendering consists of these steps

• Get the Razor view engine instance
• Find all compiled razor views through the ApplicationPartManager
• Determine the target path for the output file
• Invoke the Razor engine’s FindView to get a view engine result
• Render the view from the view engine result