Understanding the Language Server Protocol
Behind the Screens of a programming language — part 1
This article provides a gentle introduction to essentials of the Language Server Protocol 3.17. The basic properties such as the messages and capabilities of the protocol is explained in brief in this article.
Background
Have you ever thought about coding without a code editor? I remember how I struggled to write a python program in a text editor back when I was getting started with programming. I would try to fix syntax and runtime errors and compile the program at least 10 times until it worked. Writing a program that just worked was the ultimate victory of the day for me. This was until I had switched to a proper IDE. After switching to the IDE, I felt that I was trying to write more efficient code rather than just the code that compiles and runs. Arguably, even skilled developers would struggle to develop a syntax error-free or bug-free program as fast without using a code editor or an IDE. The gist is, development tools have become an essential requirement for the developers. A code editor/IDE is analogous to a pair programmer, which takes care of all the repetitive behind-the-screen work, allowing the programmers to put much focus on the logical aspect of the program.
The evolution of code editors is amazing. The ancestor of code editors is a typewriter-like machine that was used to edit punch cards. Since this machine did not have a screen, everything was physically printed on paper to provide some sort of development experience. Since the invention of the punch card editor, the development experience has evolved drastically to the level of modern Code editors and cool AI pair programmers such as GitHub copilot.
Modern-day code editors
Modern-day code editors were initially designed to provide an integrated development environment for a specific programming language. Later on, code editors started to adopt the concept of extensibility. Today, some of them are made to be lightweight and extensible. Such editors are used to writing code in almost every language just by adding a specific extension or plugin. To accommodate this requirement, programming language providers had to develop extensions for their programming language. An extension provides various development support features for a particular programming language. Suppose there is a language called A. The general functionality of an extension in a particular code editor for A is depicted at a high level in the following image.
The tooling component often parses and compiles the source code being written by the users in the editor and provides a set of development support features based on the context.
As most of the editors had a distinct set of APIs of writing extensions and providing language features, programming language providers used to develop an extension for each editor that they support.
1. How many code editors are there?
2. What are the code editors that we are going to support?
3. Are we repeating the same logic in each extension?
4. Is not there a way to standardize all of this?
I assume these are possibly the questions they might have had when they were developing multiple extensions for multiple code editors repeating the same underlying logic.
The Language Server Protocol
Microsoft introduced the Language Server Protocol as a solution to the above problem. It standardizes the communication between language development support tools and code editors. It is analogous to a protocol like HTTP. There are two entities in LSP named Language Server(LS) and Language Server(LS) Client. LS implements a set of features defined by the LSP protocol land provides them as services/procedures for the LS Client. Using LSP, code editors and language tooling providers come to a consensus on providing a set of language smartness features such as code completion, go to definition, code-action, etc. With LSP, language support providers have to develop only a single language server that can be reused for any editor which supports LSP. The following image summarizes the functionality of the LSP.
As depicted in figure 2, LSP provides a consensus for the communication between Language Tools and code editors. Following is the overview of the main components of the protocol.
Language Server: A process that provides language smarts by communicating with development tools.
Language Client: A code editor/development tool or an extension that can communicate with a particular Language Server.
Protocol: The protocol is analogous to HTTP. It consists of a header part and a content part. The header part consists of two header fields named content length and content type. Content length is a number indicating the length of the content part in bytes. And the content type indicates the mime type of the content part which defaults to application/vscode-jsonrpc; charset=utf-8 which is used to encode the content part. Refer the base protocol for more details.
The content part uses JSON RPC to describe a message. There are three types of messages sent from client to server or vice versa.
- Request message: Request message from sender to receiver. Receiver must respond with a response.
2. Response message: Must be sent as a result for a particular request
3. Notification message: Notifications are basically events. Receiver must not send a response. Note that notification message does not have an id.
LS client and LS communicate upon certain actions or events. Those have been formalized into a set of documented features by the LSP. For example, providing auto-completion support is considered the auto-completion feature. Providing code-action support is another feature. The ability to provide support for a particular feature/event/action is known as a capability. Both the LS and LS Client have a set of capabilities. The following image summarizes those capabilities.
Mandatory Capabilities
Certain capabilities should be implemented by the Language Client. The Language Server Client must have the following capabilities implemented under Text Document Sync.
didOpen: Notification sent from client to server to signal of the opening of a new document
didChange: Notification sent client to server to signal a change in a document
didClose: Notification sent from client to server to signal the closure of a document.
The server should implement either all of them or none. Therefore, the server can opt not to have any of the capabilities.
Initialize Request and Initialize Response
At the initialization of the language server-client connection, client and server have to announce their capabilities. The client sends the initialization request with its capabilities to which the server responds with its capabilities. We call this the static capability registration. Following is an example of Ballerina Language Server and Ballerina VSCode LS client initialization.
The following is the response message sent by the language server for the initialization request message. Server announces its capabilities to the language client through the response.
With the reception of the response, the client sends a notification to the server confirming the successful initialization of the connection.
Capability Registration
As depicted in the initialize request from the client, some capabilities have a dynamic registration property(capability). These are the capabilities that can be registered at the client-side on demand. For instance, the hover capability under Text Document Capabilities is as follows.
If dynamic registration is allowed, the server can opt-out of registering the capability statically at the initialize request demand the client to register it later(when required) dynamically. But the Language Server must not do both. If the Language Server does both, the same capability might get registered twice. The Language Server response above does not contain the hover capability under the Text Document Capabilities. That is, the server has decided to register the capability when required. It has not announced itself as a Text Document Hover provider. Therefore, the client won’t be registering the hover capability and the hover feature will not be available at the client-side unless it’s dynamically registered by the Language Server.
In the same way, capabilities can be unregistered at the client-side on demand. Please refer LSP specification for a detailed explanation.
What’s next
In this article, I have discussed the origin of the Language Server Protocol and the basics of the Language Server Protocol. In the next article, I will explain how to develop an LS client and a Language Server for a programing language. In the meantime, refer to my previous article on developing a VSCode extesion as a heads up for the next article.
Thank you for reading!
