Internationalizing an Angular 8 app using Firebase and POEditor
When developing a web application, at some point someone might conclude that it would be nice to allow users to use your application in multiple languages. By changing an option in some options menu, or maybe by clicking a link, the site should load a different language and show the user its preferred language. On paper, this sounds pretty simple. However, as is most often the case in software development, beneath the surface are a lot of intricacies that can lead to headaches if not appropriately handled. This blogpost is the culmination of my headaches over the past year or so trying to integrate an Angular project with translations from POEditor.
The basic premise
Consider the case where you have an application that has to be translated. How can you make all of your markup translatable? Fortunately, Angular already has a solution for this, called Angular i18n. An in-depth guide about how this works can be found here, we will assume the reader has some knowledge about how this works.
After marking all text that has to be translated with i18n directives and running ng xi18n, we end up with an XLF file containing all translatable strings in our project, which looks something like this:
In order to translate this file, the Angular team suggests you make a copy of this file for each language you want to support and send the file to the person who will translate the terms for you. This might work when localization has to be added to a fully finalized application, but when a web app is still subjective to change, sending such files to all translators whenever textual changes happen would be too much of a hassle. We don’t like hassles. Therefore, we decided to look for a tool which could streamline this process for us. After some research, we ended up choosing POEditor.
POEditor is an online tool in which we can define our translatable strings and assign contributors to provide the translations for each language. To begin, we make a new account. For the sake of this tutorial, a free account is satisfactory. With this account, we can store up to 1000 strings, which is plenty for now. However, do note that if you plan to use this tool for a real project, this number is quickly exceeded, which means that you will have to pay for it. After we have registered an account, we can create our translation project. From the projects page, click the “Add new project” button, enter a name and description, and click on “Add”.
The first thing to do for a new project, is to select the languages in which you want your app to be translated. Simply click “Add language” and select one from the dropdown list to do this. After we are done, we can import our messages.xlf file in our project by clicking on the ‘Import Terms’ button. After selecting our messages.xliff file (note that ng xi18n produces a file called messages.xlf, which you have to manually change to messages.xliff for POEditor to accept the file), we have some options for importing. First, you can either choose to import just the terms, in which case the translations of all languages will be left blank, or you can choose to also use your terms as the translations for a specific language. For example, when your web app is written in English, we can import the terms to be the English ‘translations’ of the terms in our project. We also have the option to overwrite all existing translations (which should not be done lightly), and to mark new or obsolete terms with a certain label. After we hit import, we will be notified of how many strings are updated, as well as a list of all terms which have become obsolete after the import. This list can be used to delete these unused translations, but care should always be taken that the old translations are not still useful. I18n also offers support for pluralizations, which work without a problem with POEditor.
After importing the terms, translators should be notified and granted access to the project to allow them to do their work. Accounts can be given contributor privileges to a specific language, meaning that they can only change translations for that specific language. When the translations are done, we can export the translations of each language separately, in which case the name of the file will signify which language it belongs to. For example, exporting the French translations of our project will result in a file called messages.fr.xliff. These files can then be used to build separate versions of your website for each language you want to support (see later for a command how to do this).
Translating dynamic values
Up to this point, everything sounds simple enough. However, there are a few problems one might encounter when doing this. First off, we might define strings in the class of a component, which are then used in the template of the component through string interpolation. What happens when we try to translate this? Consider the two files below. An array called colors is defined in the class definition, which are then used in the template file to generate a table structure. There are also two buttons which allow us to change our language. Adding an i18n directive to the <td> element will not result in the color names being extracted by `ng xi18`, since this tool is only able to statically analyze components. If the term cannot be extracted from your project, it will also not be translated. Instead, we can define a custom pipe which can handle the translation for us.
The translate pipe accepts two arguments next to the translatable term: the language to translate to, and the translation table where the translation can be found. This pipe will take the current color, do some pre-processing on the input term and then hands it over to a custom TranslationService which will do the actual translation. We will be using Firebase Realtime Database to store our translations. In-depth tutorials about Firebase and how to set it up can be found elsewhere.
This TranslateService will do the actual work. For example, to translate ‘blue’ to French, this service selects the translations on the path translations/color/blue and then returns the French translation that is specified there, if it exists.
We now need a way to extract these translatable strings from our project to import them in POEditor. Unfortunately, as far as I know, there is not really a good way to do this automatically, so we will have to do this manually. However, we can implement functionality in our translation service which detects missing translations and notifies us of this. For example, records can be written to our database signaling when a certain term could not be translated. This also helps to find bugs when for example we made a spelling error when adding our colors to POEditor, causing the translation service to look for the color ‘blue’ while we have only defined the color ‘bleu’.
In order to keep some structure in this process, we will define a new POEditor project for each category of translations that we have in our project. For example, we might want to translate colors, countries and dog breeds, so we will define three new POEditor projects: Colors, Countries and Dog-Breeds, in which we add the terms that we want to translate manually. After the translators are done, we also need a way to extract these translations from POEditor into Firebase. Luckily, POEditor also has an API which we can use for this. Using this API, we can automatically loop over the POEditor projects and write translations to our database.
First off, we need to get an API key for our POEditor account. This can be found under Account Settings > API Access. Next, we need the ID of our Firebase project in order to write to it. Place both of these in a new file (do not commit them to Git), and import these in your script. Then, go to your Firebase project and go to Settings > Service Account and click on the button ‘Generate new private key’. Place the downloaded file in the same directory as your script file, and also keep it away from Git. Next, it is a good idea to define a list of blacklisted projects (consisting of {name, id} objects), which will not be read by the script, and add your main translations project to this list. Else, all of the terms in your main translation project will also be written to Firebase, which we do not want. Finally, we need to define how our HTTP requests will be handled. We will only use the API endpoints to list projects, languages per project and terms per language, so only these three have to be defined. This is shown in the code snippet below.
Now we can start writing our script. First, we need to setup everything for our script, and then select all projects in our POEditor projects list that are not blacklisted, and return a list of projectNames and project IDs.
Then, pass this list of projects to writeTranslationsPerProject, which looks like this:
This function calls constructTranslationsperLanguage() for each project, which prepares a JSON object of translations, and then writes it to our database. Afterwards, we automatically stop the process since it would otherwise keep running indefinitely.
Note the encodeAsFirebaseKey function. Object keys in Firebase realtime database are not allowed to contain certain characters like commas and full stops. In order to still allow terms to contain these characters, we can escape them as is done here. We could also have used something like encodeURI() or encodeURIComponent(), but since these functions do not encode full stops and encode more characters than are strictly necessary here, we have chosen to encode our strings manually. When the script is finished running, all translations should be written to your Firebase database.
Markup in translatable strings
Going back to the component in the last section, you might have noticed the following part:
<tr i18n><td>Original</td> <td>Translated</td> </tr>
If we run ng xi18n, this will result in the following entry in our messages.xlf file:
Angular is denoting here that the source string contains HTML markup, which is useful since we do not want to lose this information. However, we run into a problem when we try to import this file into POEditor:
As you can see, the information about the enclosing markup has been omitted. POEditor is a general purpose translation tool which simply does not support Angular’s xliff format as extensively as it could. This problem could be circumvented by splitting strings like this one in multiple smaller strings until no nested HTML tags are left. However, often the context in which a certain text appears matters for the translator, and splitting every piece of text in small substrings could mean that the translators will not be able to correctly judge what the context of a string is. We therefore need a better way to do this.
Luckily, this problem can be solved relatively easily by escaping the < and > of the <x> tags. When importing terms, POEditor ignores everything that is between <> characters, but will add everything without a problem if we escape these characters before importing the terms. This can be done using the following script:
Afterwards, we can import the resulting xliff file to obtain the following strings in POEditor:
Great stuff, our HTML is saved! When exporting the translations of the resulting POEditor project, we also need to unescape these same strings. This can be done with the following script:
This script reads the exported translations.xlf file for each language that is defined in the languages array, and rewrites the translation lines containing <x some-more-text &bt; to <x some-more-text>, after which the result is written back in the appropriate translation file.
Both scripts can be run by first compiling it using tsc path-to-file && node path-to-compiled-file. Afterwards, the resulting files can be used when building the app for different languages. For example, to build the French version of this demo app, we would run:
ng build — output-path=dist/fra — aot — configuration=config-alias — base-href /fra/ — i18n-file=src/i18n/messages.fra.xlf — i18n-format=xlf — i18n-locale=fra
If we then deploy to firebase, we can access the French version of our site by browsing to <name-of-app>.firebaseapp.com/fra.
Concluding remarks and looking ahead
Angular i18n and POEditor are powerful tools to help us internationalize an application, but there are some problems that should be known and mitigated before starting the internationalization of your Angular app. This week, Angular 9 was finally released. One of its key features is a significant overhaul of how internationalization works, using the new @angular/localize package. This package features great stuff like improved build times when building for multiple locales, as well as the possibility to extract strings from Typescript files for translation. This mitigates part of the problem we explored in “Translating dynamic values”, although the problem still exists when the strings are not present in your Typescript files but are for example read from a database. I18n seems to be more developer-friendly in Angular 9, exactly how painless it will be is a topic for a future blog post.
