Building Angular apps at scale

This is a preview of a talk I’ll give on Monday at the San Francisco Angular meetup. If you’re in the bay area you can come by and ask questions: https://www.meetup.com/Angular-SF/events/242290184/

Please note this describes an EXPERIMENTAL, UNRELEASED feature in Angular. It is not a stable API. Don’t depend on it, as there may be breaking changes.

I hand-crafted these SVGs from source, so it’s bespoke.

One of the things that happens when your project is successful is that it grows. It grows in the number of engineers, and that only compounds the other growth: source lines of code.

One of the things that happens when your project has 100k or 1m SLOC is that it takes longer and longer to build the whole thing. That brings your productivity to a crawl, along with all those other engineers that wrote so much code.

This is a problem with an obvious solution: split the project up into smaller projects, each of which is quicker to build. This is typically done in frontend projects by having multiple “packages” (in the npm sense), either in one monorepo or across several repos. This does make each package quicker to build, but now you have a release engineering task any time you change package A and want to see whether package B that depends on it still works. It can be clunky (actually release package A to some local repository any time you change it), or use a trick like npm link, or a whole workflow tool like Lerna.

We haven’t quite solved the problem though. First, an individual application should probably stay in a single package, so that you can quickly iterate and test on it. As that application grows, it gets slower to build. Also, the inconvenience of building across package boundaries means you don’t test between them as often. And finally, web developers are accustomed to instant edit-refresh workflow when their source code is directly interpreted in a browser, and we don’t want to slow this down. Even a smallish package can take too long as we add tooling.

So I put forth the requirement: even as an application grows, most source edits should only take 1–2s to appear in the browser.

We’ll need a pretty clever build system to pull this off!

This is not new!

As with many things in web development, we can look to the past to find solutions to problems that are novel within the frontend world. It turns out a build tool from long ago already handled very large projects, and keeps build times low with incrementalism: only re-build the parts that changed. That tool is Make (and clones like Jake).

Make is cool because you give it declarative instructions: to produce C, you must take source A and run it through tool B. Given a complete graph, Make can determine based on the changed sources, which tools need to be re-run to keep all the outputs up-to-date. This is way better than most JavaScript build tools, where you procedurally instruct it to run tool X on source Y every time, and so the build always starts from scratch.

You might object: Make is for old clunky C++ code! I never ./configure.sh; make; make install anymore because I have a docker image with the server code already installed! Don’t bore me with an article about make.

I won’t bore you — make is indeed clunky in the declarative language you express the steps it should run. So at Google about 10 years ago, we wrote Bazel, a build tool with the same graph of actions and ability to run an incremental build, but with better semantics for declaring the build.

I haven’t used Jake, but as I understand it, you just get the declarative syntax (in JavaScript) for defining the build graph. With Bazel we get much more, because there are already plug-ins that provide “rules” that can contain most of the recipe for running compilers, giving you much shorter files.

Do Angular apps have a problem?

Large Angular applications developed with Angular CLI can take 30s to refresh in the browser for development — and this doesn’t even include running Angular’s template compiler (ngc) to discover mistakes in template expressions. Production mode builds are slower and take a ton of memory. Can we achieve the 1–2s development turnaround I think we deserve, and ALSO run ngc on each change to catch mistakes? Let’s try Bazel!

We definitely need to split our project into packages, like the Lerna workflow I talked about before — but we’re going to be much more fine-grained. Instead of a package being 1:1 with something you deploy, we’ll have a Bazel package for each subdirectory in our application. This is good because even as the project grows large, we try to keep only a “reasonable” number of files in a single directory. This means the time for a typical build is a constant multiplier of the typical package size, not linear in the size of the project.

Introducing Bazel for Angular

I just landed a new package in the Angular repo: @angular/bazel. This re-hosts Angular’s ngc compiler on top of Bazel’s knowledge of the build graph and which actions to perform as sources change. This is still Alpha quality, and not even released yet as I write this.

To use it, you need to install bazel (just brew install bazel if you’re on Mac). Then tell Bazel what is the root of your workspace, in a file called WORKSPACE. We’ll list the tools we want to use with Bazel:

With this file in place, we can call our first Bazel command. We need to lay out the node_modules directory in the project, but to keep our build hermetic, we want to do this using the same version of node and npm/yarn that our CI and co-workers use. So we run

bazel run @yarn//:yarn

A quick explanation of the syntax here: @yarn is the name of the repository that was installed when we ran the node_repositories rule. The // means the root package of that repository. The colon refers to a rule in that root package, which is the yarn executable. The entire last argument is called a Label.

Now the setup is finished. To configure our build, create a file like BUILD.bazel. This just declares an ng_module rule which roughly corresponds to the @NgModule in our source file.

Running that rule will just compile the sources in this one package, using ngc:

bazel build src

But first, it will compile the hello-world component in isolation, because it’s listed in the dependencies of src. And if you check the BUILD.bazel file for hello-world, you’ll see it will need to run the Sass compiler first to get the CSS input to the Angular compiler. Now if we make a change, we’ll only re-run the compilers on the needed target(s).

Note: At Google, we use a tool that can create Bazel build configuration files directly from our Angular source files. Until we open-source that tool, you’ll have the chore of writing these BUILD.bazel files.

By default, Bazel creates several symlinks in your workspace which reference test logs, compilation outputs, and other generated artifacts. You can find the .js files for the Angular app in bazel-bin. In my repos I configure Bazel not to write these symlinks, since I don’t look at the outputs very often, and I’d rather not teach tools to ignore them. Run bazel info to find the outputs if you don’t have the symlinks.

A complete example app is at https://github.com/alexeagle/angular-bazel-example

Questions

What about serving the app, testing, bundling for production?
We can continue to use Webpack and Karma for these. Example coming soon.

Does it work with Angular CLI?
Not yet, but Victor Savkin is working on that in a branch of the angular-cli project. More news soon.

Is the Angular team abandoning technology X?
No, this is still experimental, we have made no commitment to this. We’ll try it out with large enterprises first. If the technology matures, it might make sense to think about using Bazel by default in the Angular CLI, and what other tooling it would replace. Since Angular CLI abstracts the build system, it’s possible for us to make this switch without affecting users.

Can I use this yet?
Consider this Alpha software, and please do not depend on it for anything important. Breaking changes are still possible.