Abstract Syntax Trees

To understand Abstract Syntax Trees, allow me to give you a rundown of how our loyal and modest browser compiles JavaScript.

In the past, traditional compilers in desktop applications went through hoops to generate intended native code.

Lexer → parser → Translator → Interpreter

There was a Lexer, a Parser, a Translator and an Interpreter; Lexer converted source code into tokens which were then passed to the Parser. Parser transformed the tokens into Abstract Syntax Trees, a representation of code in a tree structure and sent them to Translator. The translator converted the syntax tree to bytecode which finally the Interpreter altered it to machine code. These steps do not scale well performance wise, especially machine code generation through an interpreter on a loosely typed dynamic language, JavaScript.

Just In Time compilers were the answer. By compiling source code when required utilizing various optimization strategies, application performance skyrocketed.

In the browser land, the source is converted into an Abstract Syntax Tree by the Lexical Analyzer and Parser. The resulting tree is similar to the DOM tree we are familiar with. Once the JS engine receives the AST, it uses it to generate machine code and begins executing instructions.

Our topic of interest here is the AST.

So, Why are AST’s important? Since AST’s are an agnostic and generalized representation of source code, it can be used for code Analysis; such as syntax analysis, extracting documentation, type checking, API conformity, and Transformation; such as transpilation, code generation, formatting, and refactoring. These features aren’t new to us; linters and code editors have long leveraged the power of AST to offer developers productivity and sanity.

Specification — The ESTree Spec

Let’s take a look at how esprima, a standard-compliant ECMAScript parser generates AST for a constant declaration.

The constant declaration is transformed into an AST with nodes decorated with meta data.

Now that we have the source code represented as a tree, we can traverse the tree and make desired changes. There are also helpers for traversal. With estraverse and esprima, you can assess, extract or transform the code to another form. These are exciting times, I agree.

As next steps, let’s visualize the AST.

Originally published at javascriptstore.com on October 2, 2017.