How to write a template compiler in JavaScript

Converting EJS-like templates to functions

Templating is a huge part of web services. It allows developers to express values from inside their markup (HTML) and have them show up in the output. Templates are valuable because, servers can respond with data based upon the request itself, instead of just using static HTML files.

EJS stands for Embedded JavaScript, and is kind of like PHP-style templates in JavaScript. An EJS evaluation, surrounded in <% %>, allows for any JavaScript expression, but only evaluates it. An EJS expression, surrounded in <%= %>, is also JavaScript, and prints out the evaluated result.

Here is an example of EJS templates in use:

<ul>
<% for(var i=0; i<supplies.length; i++) {%>
 <li><%= supplies[i] %></li>
<% } %>
</ul>

We want to turn any given EJS template into a function, so we can use it to output HTML from our server. How can this be done?

Writing the compiler

I’ve chosen EJS because, since it is already JavaScript, it’s very easy to compile.

First, define the compiler function

function compile(template){

RegExp

If you want a refresher on regular expressions, you can do a Google search on them or use the MDN resources. Regular expressions are helpful when compiling templates as they can capture certain parts of the string to insert back into a new string. You can use this property to wrap certain parts of the text. We’ll be using three different RegExps in the function.

  // evaluations can only be on one line
  var evalExpr = /\<\%\=(.+?)\%\>/g;
 
  // expressions can be on any number of lines
  var expr = /\<\%([\s\S]+?)\%\>/g;
 
  // empty echos
  var empty = /echo\(\”\”\);/g;

The first RegExp matches evaluation expressions, and is used in our function to replace the evaluations with echo calls (will talk about echo later).

  template = template
    // replace all evaluations with echos or their contents
    .replace(evalExpr, ‘`); \n echo( $1 ); \n echo(`’)

The second one matches normal expressions, and is used in our function to surround the expression with closing parenthesis and opening echo calls

    // replace <% in expressions with `");`
    // and %> in expressions with `echo("`
    .replace(expr, '`); \n $1 \n echo(`');

The third RegExp will is used to remove excess empty echo calls from the template like so

  // wrap the whole thing in an echo
  template = 'echo(`' + template + '`);';
  // remove empty echos
  template = template
    .replace(empty, "");

Parser

Now comes the fun part: defining the parsing function. I define it with a template string because they can cross newlines

  var script = 
`(function parse(data){
  
  // stores the parsed template
  var output = "";
  
  // appends HTML to the parsed template
  function echo(html){
    output += html;
  }
  
  // contains echos, etc
  ${ template }
  
  return output;
})`;

The function is wrapped in a pair of parenthesis because then it can be passed directly to eval. As you know, template is the textual representation of our compiled template. It gets inserted into the script using another handy property of template strings, interpolation.

Note: the parsing function is returned from the compiler in text form. eval or writing it to a file and requireing it are necessary to use the parser.

Entire function

So our whole compiling function looks like this

function compile(template){
  
  // regex for evaluations
  // matches anything between <%= and %> except line breaks
  // evaluations can only be on one line
  var evalExpr = /\<\%\=(.+?)\%\>/g;
  
  // regex for expressions
  var expr = /\<\%([\s\S]+?)\%\>/g;
  
  // regex for empty echos
  var empty = /echo\(\"\"\);/g;
  
  template = template
    // replace all evaluations with echos or their contents
    .replace(evalExpr, '`); \n echo( $1 ); \n echo(`')
    
    // replace <% in expressions with `");`
    // and %> in expressions with `echo("`
    .replace(expr, '`); \n $1 \n echo(`');
  
  // wrap the whole thing in an echo
  template = 'echo(`' + template + '`);';
  
  // remove empty echos
  template = template
    .replace(empty, "");
  
  // stores the JavaScript text to be written to be returned
  var script = 
`(function parse(data){
  
  // stores the parsed output
  var output = "";
  
  // appends HTML to the parsed template
  function echo(html){
    output += html;
  }
  
  // contains echos, etc
  ${ template }
  
  return output;
})`;
    
  return script;
}

Cool right? It takes a template like this one

<ul>
<% for(var i=0; i<data.supplies.length; i++) {%>
 <li><%= data.supplies[i] %></li>
<% } %>
</ul>

And returns a parsing function in text form like this

(function parse(data){
  
  // stores the parsed output
  var output = "";
  
  // appends HTML to the parsed template
  function echo(html){
    output += html;
  }
  
  // contains echos, etc
  echo(`<ul>`); 
  for(var i=0; i<data.supplies.length; i++) { 
 echo(`<li>`); 
 echo(  data.supplies[i]  ); 
 echo(`</li>`); 
  }  
 echo(`</ul>`);
  
  return output;
})

It can then be require’d or eval’d and used by passing it data.

Here is a JSFiddle showing that this actually works

Restrictions

This compilation is very simplistic. It has the absolute bare minimum of features. One restriction is that you must refer to every variable by property of the data object. This isn’t necessarily a breaking issue, but the inability to use as object as a namespace can be annoying.

Fixing this problem is very complicated. Since there is no way of defining variables with programmatically created named, you’d have to find all of the unique variable references in the template by inspecting the AST tree of the parser. Then, you’d have to insert definitions of those variables into the top of the parser function.

Another restriction this has is that it will throw an error if a variable is referred to that is not defined, or will print undefined into the output. For instance, if data.supplies is undefined in the above example, the parsing will throw

TypeError: Cannot read property '0' of undefined

To some, this may be a feature. But for users, it could be bad if a template failed to parse. It’s better to send output based on incomplete data than no output at all. To fix this, you’d have to set default empty values for different data types, and recursively set default values for every referenced variable in the template.

I hope you enjoyed this little dive into template compilation in JavaScript. Stay tuned for more interesting JavaScript tips and lessons!