[erlang_ls] A comparison with sourcer
erlang_ls is not by any means the first, the only or the best Erlang implementation of the Language Server Protocol (aka LSP). This is the first of a series of blog posts aiming at covering the other existing implementations, explaining differences and similarities with erlang_ls. Since things around erlang_ls will inevitably evolve, this post will tend to become outdated very quickly and it will require constant amendments. Also consider that erlang_ls is an extremely young project and that some of the functionalities discussed here are not fully implemented, yet. The intent of these posts is not to criticize other implementations, but to explain why a new implementation was created. As always in software design, there are no best solutions, but choices and trade-offs. The real goal of these posts is to trigger discussions, so that different implementations could learn from each other. Let’s begin with sourcer.
sourcer
To use the author’s own words, sourcer is the reincarnation of a 7 years old, battle-tested, Erlang plugin for Eclipse, named ErlIDE, to use Microsoft’s Language Server Protocol.
The project is split into two separate Erlang applications: lsp_server and sourcer. lsp_server is the actual implementation of the Language Server Protocol (aka LSP) and it supports both TCP and STDIO transports. The application consists of a single one_for_one supervisor with three child processes. One of the three children, named jsonrpc, is a TCP server responsible for encoding/decoding LSP messages and for dispatching these messages from/to the other two children: lsp_server and lsp_client, respectively the LSP server itself and a middleman towards the actual LSP client (i.e. the text editor). The sourcer application contains the port of the erlide libraries to the new format. Here the origin of the project is clearly visible in the structure of the code-base, in the references to erlide and the presence of a src2 folder, containing all the erlide code not yet ported to the new project. In reality, the project contains a third Erlang application, named erlang_ls (yes, what a fortunate name collision), but that is just a wrapper to create an Erlang escript to bootstrap the whole server, so we can safely ignore it here.
In sourcer, a separate worker process is spawn to handle each individual LSP method (e.g. textDocument/completion ortextDocument/hover). sourcer uses a mix of OTP patterns and basic, low-level, Erlang spawning and message passing, which the erlang_ls project tries to avoid, preferring to adopt OTP guidelines and patterns across the entire code-base.
sourcer defines the concept of a cancellable worker. Essentially, since every LSP method is executed within a separate process, this process can be killed at any time. Each worker processes can send partial results to a central server, which can return these partial results to the client upon cancellation. This is not the cases for the erlang_ls project, where processes can be killed, but no partial results are sent back to the client. The rationale behind this choice is that partial result could often be misleading to the end user. Please notice that request cancellation and partial results are optional features according to the LSP protocol, so both approaches are valid.
sourcer supports full-text synchronization, meaning that upon each change the entire content of the modified buffer is sent to the server. All contents for all buffers are stored into the state of a single process. On the contrary, erlang_ls has a dedicated supervisor for text synchronization purposes, where each opened buffer is modeled as a separate Erlang process with its own state, favouring decoupling.
An interesting idea in sourcer is the presence of a database which is populated by worker processes and queried by the LSP server. The database acts as a memoization tool and avoids re-calculating already calculated information.
sourcer implements a custom scanner and parser, which try to handle situations where, for example, an un-closed string would provoke the rest of an Erlang module to be un-parsable. It does so by artificially adding quotes and other potentially missing tokens. Having a temporarily un-parsable module is not considered an actual problem by the erlang_ls project, which prefers simplicity in this scenario and uses the default Erlang scanner and parser.
sourcer implements some very basic project support, mainly focusing on rebar3 conventions. erlang_ls tries to be agnostic of the building tool whilst providing workspace functionalities.
sourcer provides auto-formatting of Erlang modules using custom tools (i.e. the sourcer_indent module). erlang_ls prefers standard Erlang tools, such as the erl_tidy module. In the same way, sourcer prefers to run its own custom cross-reference analysis of Erlang modules, whilst erlang_ls prefers the standard xref tool.
