Create a Custom Web Editor Using TypeScript, React, ANTLR, and Monaco Editor

Part 1: Build a web editor with syntax colorization

Amazzal El-habib
Jan 6 · 9 min read
Image for post
Image for post

Have you ever wondered how web editors like Visual Studio (Online), CodeSandbox, or Snack work? Or have you wanted to make a custom web or desktop editor and didn’t know how to start?

In this article, I’m going to explain how web editors work, and we'll create one for a custom language.

The language we are going to build the editor for is simple. It declares a list of TODOs and then applies to them some predefined instructions. I’ll call this language TodoLang. Here are some examples of those instructions:

We simply add some TODOs using the following:

ADD TODO "TODO_TEXT";

Or we can complete a TODO using COMPLETE TODO “todo_text” so that the output of interpreting this code can tell us about the remaining TODOs and the ones we have done so far. This is a simple language I’ve invented for the purpose of this article. It may seem useless, but it has everything I need to cover in this article.

We are going to make the editor support the following features:

  • Auto formatting
  • Auto completion
  • Syntax highlighting
  • Syntax and semantic validation

Note: The editor will only support one code or file editing at once. It will not support multiple file or code editing.

Here are some semantics I’ll be using for semantic validation of TodoLang code:

  • If a TODO is defined using the ADD TODO instruction, we can re-add it.
  • The COMPLETE instruction should not be applied in a TODO that has not been declared using ADD TODO.

I’ll get back to these semantic rules later in this article.

Before we dig deep into the code, let’s start first with a general architecture of a web editor or any editor in general.

Image for post
Image for post
App architecture. Image credit: author

As we can see from the above schema, in general, there are two threads in any editor. One is responsible for UI stuff, such as waiting for the user to type some code or do some actions. The other thread takes the changes the user made and does the heavy calculations, which includes code parsing and other compilation stuff.

For every change in the editor (it could be for every character the user typed or until the user stopped typing for 2 seconds), a message will be sent to the Language Service worker to do some actions. The worker itself will respond with a message containing the results. For example, when the user types some code and wants to format the code ( clicks Shift + Alt + F), the worker will receive a message containing the action “Format” and the code to be formatted. This should happen asynchronously to have a good user experience.

Language service, on the other hand, is responsible for parsing the code, generating the Abstract syntax tree(AST), finding any possible syntax or lexical errors, using the AST to find any semantic errors, formatting the code, etc.

We can use a new, advanced way to handle the language service by using the LSP protocol, but in this example, the language service and the editor will be in the same process, which is the browser, without any back-end processing. If you want your language to be supported in other editors, such as VS Code, Sublime, or Eclipse, without reinventing the wheel, it’d be better to separate the language service and the worker. Implementing LSP will allow you to make plugins for other editors to support your language. Take a look at the LSP page to learn more.

The editor provides an interface that allows the user to type the code and make some actions. As the user types, the editor should consult a list of configurations for how it should highlight the code tokens (keywords, types, etc.). This could be done by the language service, but for our example, we will do that in the editor. We will see how to do that later.

Image for post
Image for post
Communicating with the web worker. Image credit: author

Monaco provides an API monaco.editor.createWebWorker to create a proxy web worker using built-in ES6 Proxies. Use getProxy method to get the proxy object (language service). To access any service in the language service worker, we will use this proxied object to call any method. All the methods will return a Promise object.

Check out Comlink, a tiny library developed by Google that makes working with web workers enjoyable using ES6 Proxies.

Without further ado, let’s start writing some code.

What Are We Going to Use?

For this project I’m going to use :

For UI

As defined on its website, “ANTLR (ANother Tool for Language Recognition) is a powerful parser generator for reading, processing, executing, or translating structured text or binary files. It’s widely used to build languages, tools, and frameworks. From a grammar, ANTLR generates a parser that can build and walk parse trees.” ANTLR supports a lot of languages as target, which means it can generate a parser in Java, C#, and other languages. For this project, I’m going to use ANTLR4TS, which is a Node.js version of ANTLR that can generate a lexer and a parser in TypeScript.

ANTLR uses a special syntax for declaring a language grammar which is typically placed in a *.g4 file. It allows you to define lexer and parser rules in a single combined grammar file. In this repository, you will find grammar files for a lot of well-known languages.

This grammar syntax uses a notation known as Backus normal form (BNF) to describe the syntax of languages.

Here is a simplified grammar of our TodoLang. It declares a root rule for TodoLang, todoExpressions, which holds the list of expressions. The expressions in TodoLang can be either an addExpression or completeExpression. The asterisk (*), as in regular expressions, means that the expression may occur zero or more times.

Each expression begins with a terminal keyword (add, todo, or complete) and has a string (“…”) identifying the TODO.

The Monaco Editor is the code editor that powers VS Code. It’s a JavaScript library that offers an API for syntax highlighting, auto-completion, etc.

TypeScript, webpack, webpack-dev-server, webpack-cli, HtmlWebpackPlugin, and ts-loader.

So let start by initiating the project.

Initiate a New TypeScript Project

For that let’s initiate our project:

npm init

Create a tsconfig.jsonfile with this minimum content:

Add a config file webpack.config.js for webpack:

Add dependencies for React and TypeScript:

npm add react react-domnpm add -D typescript @types/react @types/react-dom ts-loader html-webpack-plugin webpack webpack-cli webpack-dev-server

Create an src directory with your entry point index.tsand index.html that contains a div with an id container.

Here is the source code for this starter project.

Add Monaco Editor Component

If you are targeting an existing language like TypeScript, HTML, or Java, you don’t have to reinvent the wheel. Monaco Editor and Monaco Languages support most of those languages.

For our example, we are going to use a core version of Monaco Editor called monaco-editor-core.

Add the package:

npm add monaco-editor-core

We also need some loaders for CSS as Monaco uses them internally:

npm  add -D style-loader css-loader

Add these rules to the module property in webpack config:

{test: /\.css$/,use: ['style-loader', 'css-loader']}

Finally, add CSS to the resolved extensions:

extensions: ['.ts', '.tsx', '.js', '.jsx','.css']

Now we are ready to create the editor component. Create a React component (we will call it Editor), and return an element that has a ref attribute so we can take its reference to let Monaco API inject the editor inside it.

To create a Monaco editor, we need to call monaco.editor.create. It takes as arguments the DOM element in which Monaco will inject the editor, and some options for language id, the theme, etc. Check out the documentation for more details.

Add a file that will contain all the language configuration in src/todo-lang:

Add a component in src/components:

Editor component

We basically use a callback hook to get the reference of the div when mounted, so we can pass it to the create function.

Now you can add the editor component to your application, and add some styling if you want.

Register Our Language Using Monaco API

To make Monaco Editor support our defined language (e.g., when we created the editor, we specified the language ID), we need to register it using the API monaco.languages.register. Let’s create a file in src/todo-lang called setup. we also need to implement monaco.languages.onLanguage by giving it a callback that will be called when the language configuration is ready. (We will use this callback later to register our language providers for syntax highlighting, auto-completion, formatting, etc.):

Now call the set-up function from the entry point.

Add Web Workers for Monaco

So far, if you run the project and open it in the browser, you will get an error concerning the web worker:

Could not create web worker(s). Falling back to loading web worker code in main thread, which might cause UI freezes. Please see https://github.com/Microsoft/monaco-editor#faq
You must define a function MonacoEnvironment.getWorkerUrl or MonacoEnvironment.getWorker

Language services create web workers to compute heavy stuff outside of the UI thread. They cost hardly anything in terms of resource overhead, and you shouldn’t worry too much about them, as long as you get them to work (see above the cross-domain case). (Source) Check also this.

There is a web worker that Monaco Editor uses; I think it’s used for highlighting and performing other built-in actions. We will create another one that will handle our language service.

Let’s first tell webpack to bundle Monaco’s editor web worker. Add this line to the entry point:

entry: {app: './src/index.tsx',"editor.worker": 'monaco-editor-core/esm/vs/editor/editor.worker.js'},

Change the output to tell webpack to give a specific name to the web worker without the hash and use ‘self’ as global object as it’s required by Monaco. Here’s the webpack config file content so far :

As we can see from the above error, Monaco calls a method from global variable MonacoEnvironment called getWorkerUrl. Go to the set-up function and add the following:

This will tell Monaco where to find the worker. We’ll add our custom language service worker sooner.

Run the application, and you should see an editor that does not yet support any features:

Image for post
Image for post

Add Syntax Highlighting & Language Configuration

In this section, we will add some keywords highlighters.

Monaco Editor uses Monarch library, which allows us to create declarative syntax highlighters using JSON. Take a look at their documentation if you want to learn more about this syntax.

Here an example of Java configuration for syntax highlighting, code folding, etc.

Create a file in src/todo-lang called config.ts. We are going to configure the TodoLang highlighter and tokenizer using Monaco API: monaco.languages.setMonarchTokensProvider. It takes two parameters, the language ID, and the configuration of type IMonarchLanguage.

Here is the configuration for TodoLang:

We basically specify the CSS classes or token names for each type of keywords in TodoLang. For example, for keywords COMPLETE and ADD, we instructed Monaco to give them a class keyword and class type for type keywords TODO. We also instructed Monaco to colorize strings by giving them a CSS class of type string, predefined by Monaco. Keep in mind that you can override the theme and add new CSS classes by using defineTheme API and specify it when creating the editor or setting it using setTheme.

To tell Monaco to consider this configuration, go the set-up function, in the onLanguage callback, call monaco.languages.setMonarchTokensProvider, and give it the configuration as the second argument:

Run the app. the editor should now support syntax highlighting.

Image for post
Image for post

Here is the source code of the project so far: amazzalel-habib/TodoLangEditor.

In the next part of this article (check it out here) , I’ll cover the language service. I’ll use ANTLR to generate the TodoLang lexer and parser, and implement most features of the editor using the AST provided by the parser. Then we’ll see how to create a web worker to provide the language services with auto-completion.

Advice for programmers.

Medium is an open platform where 170 million readers come to find insightful and dynamic thinking. Here, expert and undiscovered voices alike dive into the heart of any topic and bring new ideas to the surface. Learn more

Follow the writers, publications, and topics that matter to you, and you’ll see them on your homepage and in your inbox. Explore

If you have a story to tell, knowledge to share, or a perspective to offer — welcome home. It’s easy and free to post your thinking on any topic. Write on Medium

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store