My way to VS Code extension
Backstory
Among the various tasks I was facing during my working day, the one I have struggled the most with is localization. Not because it’s a complicated task that requires additional skills to be completed, but because it forces you to do manual work with no automation at all. There was no tool which I could use to simplify the localization process in that time, and it made me feel like I was wasting my time with monotonous and repetitive manual actions. To understand why localization was hard for me, let’s briefly take a look at the project I’m working on and our approach for working with internationalization (i18n) in our team.
There is an article (in French) written by hugo mercier which provides some overview of the project structure. I would recommend to read it if you want to have comprehensive understanding of what we are working with. I’ll give a short project description in terms of localization here.
So, there is an Angular application for 4 countries, each one is additionally divided in desktop and mobile projects, which all in all comes up to 8 projects. We use a basic approach for internalization with storing localization in xlf files (xml formatted file), that means that one translation.xlf file is related to one project. Using separate translation.xlf files for desktop and mobile projects for one country based on the fact that usually text which suitable for desktop might be difficult to reflect for mobile.
One of the most frequent tasks with localization is adding new keys. Let’s assume that there is a page title:
@@PAGE_TITLE defines the key which will be seeking in the concerned translation.xlf file during the build process. Adding localization in our case means to open all translation[locale].xlf files (there are 8 of them) and adding the following code with localization specified in target tag:
In total, such straightforward action requires us to: find, open and edit 8 different localization files. When adding a new page or component, the amount of required additions increases and a simple task becomes boring and exhausting.
In the end, even though adding new translations is not that difficult, real “issues” pop up during refactoring or adjustment of business requirements. In such cases, localization work may not only contain adding new values but also:
- renaming keys (@@page_title becomes @@component_title)
- removing unnecessary localization keys in html templates and localization files
- reusing existing translations
All in all, the developer is charged with the responsibility to make sure that:
- all translations are correct and not mixed up (especially for various localizations for desktop and mobile for one country, it is important not to mix them up)
- redundant translations and keys are removed
The actions mentioned above require intensive use of text search in IDE, which slows down development pace in case of a big amount of files. All in all, this simple tasks require more time to be accomplished, productivity and attention decrease, which leads to bugs that’ll hopefully get caught by the QA. Moving tasks back to development is an unpleasant moment for a developer.
Since we use Visual Studio Code as our IDE, the next logical step was to look for a suitable extension to simplify localization work in Angular. Almost every issue I faced was previously resolved and all I had to do was to find and use already existing solutions or tools. However, in the case of i18n, I was out of luck. I searched for different extensions for VS Code, but could not find anything that could simplify my issues with localization. This was the moment I understood “if there is no suitable existing extension — I need to write it”.
Getting familiar with VS Code Extensions
I started exploring VS Code extensions not from documentation but from an already existing extension written by one of our developers Bilel Msekni. After exploring the code and debugging for a while, I obtained some knowledge and began to get a good grasp on how extensions work. After that, I started to read the documentation. VS Code provides a comprehensive and powerful API for adding various features necessary for your work.
At the beginning, I was interested in implementing two features which would improve my productivity when working on localization:
- validating i18n attributes in html files and showing warnings if there are missing translations in any related localization files
- having a quick overview of existing localizations for a given key, without opening xlf files
To be able to validate files, it’s necessary to have information about all the available keys and their translations (translation.xml files) and also where they are used (html files). The first thing that comes to mind is parsing files with regular expressions. Given the application size and the potential number of files that need to be parsed (these are hundreds of files), it immediately becomes clear that this is a resource-intensive process that will easily suspend the VS Code UI if it runs in the same thread. After a bit of documentation reading, I found what I need — Language Extension. A crucial feature of this extension type is the client-server architecture, in which the server is a separate Node process that can be used for heavy tasks. Also, Language Extension provides basic operations which are extremely important: mouse hovering, IntelliSense, Find All References, Go to Definition, etc. All these features are available out of the box; you just have to plug them to your implementation.
To understand how the client-server approach works, I investigated lsp-sample. The first thing I was interested in was understanding how the client connects to the server. I found the answer in the extension.ts file:
The example is very well commented, there is no reason to go through each line of the code. The only thing to mention is that the client is being used for communicating with the server. The server is located in the server.js file, in which a connection to the client is created:
In the given code, the connection object is used for exchanging messages with a client. The onInitialize function is called before initializing connection and can be used for additional configuration on startup.
It’s worth mentioning that there is a document manager (documents object) provided by the API. It contains different methods for manipulating opened files in VS Code workspace:
The methods provided by documents are informative and do exactly what they say. It is necessary to point out that the document implies the file description interface: name, path and file content in a string format. Often debugging someone else’s code is not the most exciting thing to do, but in that case it was not difficult and it didn’t take much time. At this stage, the gathered information was enough to start writing the extension.
Creating the extension
The starting point in writing the extension was looking deeper into Angular.json file and the way it can help with my task. Let’s take look closer at it (in fact, the file contains much more configuration but I left only the crucial ones):
As shown, all the projects defined in our application are listed in projects key:
- sourceRoot — source directory; turned out to be useful since it allows you to search for the necessary files in the file system
- tsConfig is a file with application settings. Among other settings, it lets you write patterns to exclude directories and files from the build process. They are really important for extensions, and can be used to determine which projects the particular template.html file belongs to. Using this information, it’s possible to make one-to-many relationships between template.html <-> translation.xlf
- i18nFile — localization file for particular project
Here is tsConfig file example:
The idea behind exclude is the same as with gitignore file: all files or directories that match at least one pattern are excluded from the project at the build stage.
All the projects are in a single repository and share the same code; each project is built independently and includes only the files related to it. It’s done for the sake of minimizing bundle sizes. By default, all the files in the repository are related to every project. To be able to determine which ones should be omitted during the build — tsConfig and exclude patterns are used. Each file’s path is validated against the exclude patterns. If there is no matching, then the file is included in the bundle.
This is why tsConfig is so important and is intensively used in my extension. It helps to set up the relation between the html file and its localization.
Since the Angular.json file stores data in JSON format, it is easy to read it and work as with a regular JSON object in JavaScript.
The main feature I lacked when working with localization is template validation against suitable translations. To do this, the template.html file (the one you are working with and which is open in VS Code) ought to be associated with one or more translation.xlf files. With such information, it’s possible to check whether the translation file contains all the keys declared in the template. Here is a small diagram of how localization files are searched for:
Having collected the information I needed — I proceeded directly to programming. When the client starts, there are three basic steps:
- find and parse the angular.json file;
- find and send information regarding localization files to the server;
- find the paths to all html files of the project and send them to the server:
It should be noted that localization and html file parsing takes place on the server side, while the client is responsible only for their search. This is due to the fact that the server has limited API for working with workspace and it is much easier to search files on the client side and send found file paths to the server for further processing. Notifications to the server are made by calling the client.sendNotification method.
Having written the basic client actions I shifted focus to the server and the first thing needed to do is catching events from the client:
So, having all the necessary information on the server, what’s left to do is parsing the html and xlf files. Which I did with RegExp. Parsing html templates implies finding attributes started with i18n prefix, its value is localization key I was looking for. Parsing localization files means finding all <trans-unit /> tags also with extracting the keys that will leverage as an identifier for searching translations, for instance:
The essence of parsing is to find @@ page_header in the template as well as its corresponding localization. The page_header key is used as an ID for joining; well, just like in SQL!
When files are parsed it’s time to proceed directly to the validation of html templates. Only files opened in the IDE will be validated which can be achieved with documents manager:
The onDidChangeContent event is triggered in two cases: when file is opened or edited; just what I needed.
What do we have in the end? Server knows about all keys and their translations and also is able to receive notification when opening or editing a file. It remains to add validation:
Even though the code looks messy, I’ve tried to keep it as clean as possible. Diagnostics object contains information (start and end indexes) where warning should be displayed as well as text to display. By using this.connection.sendDiagnostics collected information is being sent to the IDE to show the warnings:
If there is no localization — i18n attribute is underlined in green. On mouse hover, the tooltip with localization info is popping up:
The beginning is not so bad but it’s still not enough to simplify approach for managing localization; except the information that there is missing translation it’s necessary to know which translation.xlf file doesn’t have it. The question is how to display translations; eventually, I chose the easiest way: on hovering over i18n attribute available translations or an indication of their absence should be displayed. As I already mentioned VS Code provides a powerful API:
All you need to do is subscribe to the registerHoverProvider event of the languages object. Both are part of the API. This event is triggered when the user hovers the cursor over any word. Not only is the active document passed to the handler but also the position of the highlighted word (position is an index in the line). Having the active word give us a chance to check whether this is an i18n attribute. If so, we look for possible translations, create a response and send it to the IDE. In short, it looks like this:
In the IDE result looks promising — the popup contains actual localization values:
After such result I was so inspired that I added a bunch of features to the extension:
Go Toparticular translation file from the Hover PopupGo To Definitionfrom html template to relevant translation fileFind All Referencesfor showing translation unit usages acrosshtmltemplatesIntelliSenseRenameapplies changes for bothhtmltemplate and translation fileGenerate translation unit(-s)for a single or bulk translation units generating- Command for
removing translationsand all references to it
Example how these features work can be fond by this link.
I did a bit more than I expected from the start. Initially, it was important for me to cut down manual actions while working with localization. In sum I enjoyed not only development process but also the final result. I lost the horror of localization tasks that now cause me a lively interest, since this is an additional opportunity to test the extension, one more time :)
p.s. as an unexpected side effect of writing this VS Code extension, I decided to write another one for Google Chrome to save me from one another task which I struggle daily (yes, unfortunately, we have them enough).
