Everything you need to know to start contributing to Erlang today!

"In real open source, you have the right to control your own destiny." — Linus Torvalds.

Hi folks! Ukrainian Erlanger is here 🤘! In this article, I want to talk about contributions to Erlang open-source code. And no — I'm not talking about some Erlang project — I'm talking about contributions that go directly into Erlang/OTP itself.
Perhaps it is worth noting at this point that I did not initially think that this article would turn out to be so large. In fact, it is divided into two main sections, each one with its own subsections. So, if you don't want the story behind it and you just want to learn the commands you need to use to build and test Erlang/OTP, jump straight to the "Build & install Erlang Open Source code" section. Otherwise, let me tell you a story that might inspire you…

On the 8th of October of 2021, we celebrated the anniversary of Open Source Erlang — he turned 23 years old 🎂, as reported by Francesco Cesarini on Twitter. Erlang has gone through many changes and trials, and, as a result, it has grown to an incredible scale, thanks in large part to its open-source contributors. But regardless of its development to this day, Erlang still requires more love and more contributions: your love and your contributions.

I don't know about you, but I have found myself thinking that I want to be useful both for the Erlang language and for the Erlang community from the first days of acquaintance with the language. I could summarize the main obstacles for me in these two questions:

• What exactly is needed for Erlang?
• How exactly can I contribute to Erlang by myself?

Those questions are also an obstacle for most developers, who have brilliant ideas, but since they don't know where to start, they keep these ideas to themselves. I will describe how I figured out how I could safely plant my contribution. I hope this information will benefit other developers who want to contribute to Erlang.

So, let's rock 🤘!

What exactly is needed for Erlang?

First of all, you need to detect areas where you can help, and you should be able to find existing ideas or issues. This is necessary because the Erlang Core Team already discussed, accepted, or rejected some ideas. Therefore, it's generally not worth pursuing them any further. To collect that information, you need to refer to several sources:

• Your brain and imagination
• erlangforums.com
• Erlang issues in Github
• The Erlang Community

Your brain and imagination

Of course, you can take the first ideas straight up from your brain and your imagination. Assuming that you already have experience with some languages and/or with Erlang, it won't be hard for you to invent incredible things and integrations that did not exist in the Erlang language yet. Also, let's not forget about improvements: A vast number of functions that we use every day in our projects, as a result, turn into utilities and then into libraries. If widely used enough, they can become a part of the Erlang standard library. Also, don't forget existing features, supported protocols, tools, common and unit tests, logging, and many other points in Erlang where there is a need for more love that you can provide if you want, of course.

erlangforums.com

As I mentioned in previous articles, many excellent ideas are constantly discussed with the Erlang Core Team and the Erlang community. To find them, you should at the very least turn your attention to this topic in the forums: What do you think should be in OTP?.
On it, folks are actively discussing the future development of Erlang, and you can find there some outstanding and renowned developers from different organizations and Erlang Core Team.
Another topic to keep an eye on is: What do you think should not be in OTP?. There is also a lively discussion about improving Erlang but in a slightly different direction on this topic. According to the community, the main Erlang repository needs to be cleaned up and enhanced by removing all unnecessary things that developers do not use regularly or can be taken out into separate libraries. The goal is to significantly speed up and make the language and its libraries easier to maintain in the future.

Erlang issues in Github

Yep, this is a great source to find existing issues in Erlang! As you might have already noticed, issues in GitHub are marked with labels. There are multiple ones, but the ones worth paying attention to are bug, enhancement, and help wanted. These labels clearly state what these tasks were created for and how the Erlang Core Team treats them. Of course, before taking on any task, you should make sure that no one else is working on it yet and you clearly understand what you need to do for implementation. If you have any questions or concerns, the Erlang community will help you solve them!

Erlang Community

The Erlang community is a fantastic group of folks where you can scoop up your ideas, make new friends and mentors, improve yourself as a developer, and much more. In the community, you will always be warmly welcomed, and you'll always find somebody willing to help you with any question you might have. Here, no one will judge you for bad ideas or unsuccessful attempts to improve something.

This is a small example of what happened to me:
Some time ago, I started thinking about creating a CLI for Mnesia (once I get a minimal version running, I'll do my best to describe this great tool, but that will be in a different article). I figured out I needed some help, even just to finalize my general idea around this project. So, I told Brujo about how small I felt my Erlang network was (i.e., I didn't know who could help me with this project), and I thanked him for investing his time mentoring me. He explained to me that "if you look further enough, you can always find somebody to help you."

This is true, you see: quite recently, I had no idea how to create plugins for rebar3, didn't know how to work with ASTs in Erlang, or how to work with yecc and leex, or how to build and test Erlang/OTP for local development. Many more things helped me expand my inner world of Erlang. Various and extraordinary meetings in the community, with people seemingly unrelated to each other, helped me find the right solution, the right way to implement certain things. As I said earlier, there are no bad ideas because each idea creates the beginning of the next one. Remember: it's better to have a stupid idea than not having ideas at all 😃 — feel free to share them, talk about them, try to bring them to life! You'll surely find somebody else willing to at least discuss them with you.

So, if you are not a part of the Erlang community yet, if you had not joined the Erlang Slack, or if you still have not joined erlangforums.com, you should do it ASAP! We are always glad to see new people, and we are ready to support you in any situation! Please do not be shy to join us! Present yourself, tell us what you think, what problems you are experiencing, what you would like to do, and we can take it from there!

How exactly can I contribute to Erlang by myself?

Some time ago, I created a pull request for Erlang/OTP. This PR was relatively small and included a fix for a rather old bug. Since I clearly understood what data we receive as input and what information we should produce at the output, I decided to reproduce the problem in a stand-alone file locally. Not the best approach. In my case, I would say that it was a somewhat dumb idea, but this dumb idea actually led me in the right direction.

After a successful test, I felt confident, so I decided to update the Erlang source code directly. I pushed my changes into my repository fork; I verified that GitHub Actions passed successfully, and then I plucked up the courage to open the PR.

Please note that at the very beginning, I had no idea how to locally build Erlang/OTP for development, run tests, debug, etc. In other words, I initially relied solely on GitHub Actions which run on my fork after pushing my changes.

The first comment in the pull request came from Björn Gustavsson:

I think that you’ll need to add a test case …

I didn't keep reading the comment beyond that since my brain was already collapsed after those few words 😅. This is exactly what I was afraid of from the very beginning. I did not know how to upgrade the tests locally and could rely solely on GitHub Actions — this meant that I had to wait for each pass of tests in CI and fix issues if they were found by it. This is an incredibly massive amount of time that I could not afford to spend…

So, then I went into the Erlang Slack 😃. After some conversation with Brujo, he told me to ask Lukas Larsson for help. I wrote to Lukas and asked him about mentoring me on this endeavor. He answered pretty quickly and agreed to help me. As it turned out, Lukas was already working on a new development guide. After our discussion and reading some additional documentation, it turned out that building and running the tests of Erlang/OTP is really simple, as you can see below.
Note that all of the examples are done on Linux Mint.

Build & install Erlang Open Source code

Here is how I went with it:

# Clone created fork
$ git clone https://github.com/vkatsuba/otp.git
# Go to "otp" folder
$ cd otp
# Add upstream of original Erlang source repository
$ git remote add upstream https://github.com/erlang/otp.git
# Create your working branch
$ git checkout -b my-develop-branch

To build Erlang source code for development, you need to be sure that you already have the following tools installed:

• GNU make
• C compiler — gcc or clang
• Perl 5
• ncurses, termcap, or termlib
• sed

Now you need to configure ERL_TOP and PATH variables:

$ export ERL_TOP=$PWD
$ export PATH=$ERL_TOP/bin:$PATH

And then you can run the test commands(this can take a while 😉):

$ ./otp_build setup -a --prefix=$PWD/tests_install
$ make install
% ./otp_build tests
$ export PATH=$PWD/tests_install/bin:$PATH

At this point, you can already run the Erlang source code, like this:

$ $ERL_TOP/tests_install/bin/erl

Good job! 🎉
Now you can run the latest version of the Erlang Open Source code on your computer.

Adding a new module and running Unit Tests

Let's try to create a simple Unit Test, then. For instance, I will create the module otp/lib/eunit/src/contribution_tests.erl:

%%% @doc Unit test for a contribution to Erlang Open Source code
-module(contribution_tests).
-include("eunit.hrl").
simple_test() ->
    ?assertEqual(2, 1 + 1).
simple_two_test() ->
    ?assertEqual(3, 1 + 2).

And add the path of the new module into /otp/lib/eunit/src/Makefile:

...
SOURCES= \
    contribution_tests.erl \
    eunit_striptests.erl  \
    ...
...

Then, let's try to build the source, boot up an Erlang shell and run our new Unit Tests:

$ make install
$ $ERL_TOP/tests_install/bin/erl
Erlang/OTP 25 [DEVELOPMENT] [erts-12.1.5] [source-32d401265d] [64-bit] [smp:8:8] [ds:8:8:10] [async-threads:1] [jit:ns]
Eshell V12.1.5  (abort with ^G)
1> eunit:test(contribution_tests).
  2 tests passed.
ok

Amazing! ✨
Now we know how to add new modules, build and run Unit Tests for them.

Git commit & push code changes

Let's take a look at the git status of our repo:

$ git status
...
Changed:      lib/eunit/src/Makefile
...
Added:
...
 erts/test/otp_SUITE_data/otp_version_tickets
 erts/test/otp_SUITE_data/otp_versions.table
 lib/compiler/test/bs_bincomp_r24_SUITE.erl
 lib/compiler/test/bs_construct_r24_SUITE.erl
 lib/compiler/test/bs_utf_r24_SUITE.erl
 lib/compiler/test/fun_r23_SUITE.erl
 lib/eunit/src/contribution_tests.erl
 tests_install/

This time we should add lib/eunit/src/Makefile and lib/eunit/src/contribution_tests.erl, commit and then push our changes!

But what if your change was not just the addition of a new module, but instead, you applied changes in many existing files? Then you can run the following command. You just need to pay attention to the following warning: this command will reset all added files, keeping only the changed ones.

$ git clean -xfdq
$ git status
$ git add .

Before creating a commit, I recommend reading the Erlang Wiki: Writing good commit messages. This wiki page provides helpful information about creating an outstanding commit and what precisely the Erlang Core team expects in commit messages and why.

Common Tests

Similarly to the previous part where we worked with eunit, let's try and add a new module, create Common Tests for it, build and run all this stuff.

Let’s add /otp/lib/dialyzer/src/contribution_example.erl:

%% Module for a new contribution to Erlang Open Source code
-module(contribution_example).
-export([plus/2]).
-spec plus(integer(), integer()) -> integer().
plus(X, Y) ->
    X + Y.

Just like in the previous section, we also need to update a Makefile but at this time, it is otp/lib/dialyzer/src/Makefile:

....
MODULES = \
    contribution_example \
    cerl_prettypr \
    ...
...

Then we need to add the new Common Tests for our new module. Let’s add new suite at otp/lib/dialyzer/test/contribution_example_SUITE.erl:

%% Module for Common Test for contribution Erlang Open Source code
-module(contribution_example_SUITE).
-include_lib("common_test/include/ct.hrl").
%% Test server specific exports
-export([all/0, groups/0,init_per_suite/1, end_per_suite/1,
  init_per_group/2,end_per_group/2]).
-export([init_per_testcase/2, end_per_testcase/2]).
%% Test cases must be exported.
-export([test_new_module/1]).
all() ->
    [test_new_module].
groups() ->
    [].
init_per_suite(Config) ->
    Config.
end_per_suite(_Config) ->
    ok.
init_per_group(_GroupName, Config) ->
    Config.
end_per_group(_GroupName, Config) ->
    Config.
init_per_testcase(_Case, Config) ->
    Config.
end_per_testcase(_Case, _Config) ->
    ok.
%%%
%%% Test cases start here.
%%%
test_new_module(doc) ->
    ["Test new module"];
test_new_module(Config) when is_list(Config) ->
    2 = contribution_example:plus(1, 1).

And of course, we need to add the path of this new common test suite to otp/lib/dialyzer/test/Makefile:

...
AUXILIARY_FILES=\
    ...
    erl_types_SUITE.erl \
    contribution_example_SUITE.erl \
    ...
...

You need to know something about common tests: To run them correctly, you need to install the ts framework. This framework will help run tests. To install ts framework and run the new Common Tests:

$ export ERL_TOP=$PWD
$ export PATH=$ERL_TOP/bin:$PATH
$ ./otp_build setup -a --prefix=$PWD/tests_install
$ make install
$ ./otp_build tests
$ export PATH=$PWD/tests_install/bin:$PATH
$ cd release/tests/test_server
$ $ERL_TOP/tests_install/bin/erl

The last command will start the Erlang shell. Now you need to install ts and run the test module:

Erlang/OTP 25 [DEVELOPMENT] [erts-12.1.5] [source-32d401265d] [64-bit] [smp:8:8] [ds:8:8:10] [async-threads:1] [jit:ns]
Eshell V12.1.5  (abort with ^G)
1> ts:install().
2> ts:run(dialyzer, contribution_example_SUITE, [batch]).

The result should look like this:

....
Common Test starting (cwd is /home/vk/otp/release/tests/dialyzer_test)
Common Test: Running make in test directories...
Recompile: contribution_example_SUITE
...
TEST INFO: 1 test(s), 1 case(s) in 1 suite(s)
Testing tests.dialyzer_test.contribution_example_SUITE: Starting test, 1 test cases
Testing tests.dialyzer_test.contribution_example_SUITE: TEST COMPLETE, 1 ok, 0 failed of 1 test cases
...
done

Cool! Our new Common Tests passed!

So, as you'd just seen, to run new common tests using the ts framework, you can do: ts:run(Application, CommonTest, [batch]).

I guess you have already noticed that it is not enough to add a module with logic and a module with tests. You also need to change Makefile in src and in the test folder. Perhaps it is also worth noting that Makefile is sometimes placed in some applications' root folders. For example, for eunit the Makefile is placed in lib/eunit/Makefile, but in the case of dialyzer, it’s in otp/lib/dialyzer/test/Makefile and otp/lib/dialyzer/src/Makefile. So be very careful and always look for Makefile both in the root folder and the subfolders.

Static Analysis

For static analysis, OTP includes the following make commands in its root folder:

$ make xmllint
$ make dialyzer
$ make format-check

And for building documentation…

$ make release_docs

Summary

Throughout the current article, we learned…

• where we can get ideas
• who can help us and where to find them
• what we should do if we are stuck in any step
• how to build and run our own version of Erlang
• how to add new modules into the Erlang/OTP code
• how to create and run Unit Tests and Common Tests
• how we can do static analysis of the Erlang/OTP code.

You are now ready to produce your first extraordinary contributions to the Erlang language itself with all this knowledge!

What are you waiting for?! Just do it! 😈

References

• Erlang/OTP
• Developing Erlang/OTP
• Building and Installing Erlang/OTP
• Erlang Wiki: Running tests
• Testing Erlang/OTP
• Contributing to Erlang/OTP
• Erlang Wiki: Writing good commit messages
• Erlang Slack
• erlangforums.com

Special Thanks!

• Thank you, Brujo, for your fantastic mentoring and responsiveness! 🚀
• Thank you, Lukas, for your excellent mentoring and your excellent guide for Erlang local building & testing and other outstanding documentation and also for your excellent communication skills!🤘
• Thank you, Björn Gustavsson, for your incredible responsiveness and everything you do for the Erlang community! 💪