Communicating between Elixir and Java using Jinterface

In this article, we will explore how to establish seamless communication between Elixir and Java by utilizing the powerful Jinterface library. By creating both an Elixir node and a Java node within a Docker container, we can demonstrate a practical scenario where the Java node requests specific metrics from the Elixir node, and the Elixir node responds with the requested data. This cross-language communication allows us to harness the strengths of both Elixir and Java.

We will try to keep things simple and not use any frameworks, high level abstractions such as GenServer, or any other libraries besides Jinterface. We will also not use any build tools such as Maven or Gradle. We will compile and run our code from the command line. But you’re free to use whatever works best for you!

Why would I want to use Elixir with Java?

Some of the reason why we would want to use Java when you already have a Elixir application are:

• You have a existing Java application that you want to integrate with your Elixir application
• You want to use a Java library that is not available in Elixir
• Performing some type of tasks in Java could be more efficient than in Elixir

What do I need to get started?

• Docker installed on your host machine
• Basic text editor

When all is set and ready let’s dive in and build this bridge between Elixir and Java!

How are we going to do all this?

Let’s fire up terminal and create new project directory called bridge and navigate to it.

mkdir bridge
cd bridge

We are going to create few directories, one for our Elixir application and one for Java.

mkdir -p java/src/java_node
mkdir elixir

Elixir part

We start of with our Elixir application by navigating to the elixir directory and creating a new file called main.ex.

cd elixir
touch main.ex

We then follow by writing a simple server that waits for incoming messages and responds back to them asynchronously.
To be more specific we will be listening for incoming message that either asks for total atoms or total processes running on the BEAM instance.
Once the message is received we will get the requested data from our local instance and send it back to the process that requested it. When this is done we simply increment our processes and atoms and wait for the next message by calling the loop function recursively.

bridge/elixir/main.ex

defmodule Main do
  # The main API function to start the server and register the process
  # with the name `handler`.
  def start() do
    Process.register(Process.spawn(__MODULE__, :loop, [], []), :handler)
  end

  # The loop function which waits for incoming messages, handles them
  # and recursivly calls itself again.
  def loop() do
    {msg, from} = receive do
      {:total_atoms, from} ->
        {{:atoms, :erlang.system_info(:atom_count)}, from};
      {:total_processes, from} ->
        {%{total_processes: :erlang.system_info(:process_count)}, from}
    end

    send_data(msg, from)
    increment()
    loop()
  end

  # Function to increment the atom count by spawning a new process.
  # this will be used for demonstration purposes to show how the data is updated.
  def increment() do
    Process.spawn(__MODULE__, :loop, [], [])
    String.to_atom(Integer.to_string(Enum.random(1..100000)))
  end

  # Respond to the client with the data.
  defp send_data(data, pid) do
    Process.send(pid, data, [])
  end
end

# Start the server.
Main.start()

That was it on the Elixir side, now let’s jump to Java side of the code!

Java part

Navigate to the java_node directory and create new file called Main.java.

cd ../
cd java/src/java_node
touch Main.java

This file will be our entry point to the Java application. We will create a new class called Main and add a main method to it.
The main method will be responsible for creating a new instance of the Main class which in turn initiates Collectable instances
and calls the start method on them which starts separate threads for each Collectable instance.
Don’t worry about the Collectable class yet, we will get to that in a minute.

bridge/java/src/java_node/Main.java

package java_node;

import com.ericsson.otp.erlang.*;
import java.io.IOException;

public class Main {
    private final Collectable collectables[];

    private Main() throws IOException {
        OtpNode localNode = new OtpNode("java_node", "cookie");

        collectables = new Collectable[] {
            new TotalAtomsCollectable(localNode),
            new TotalProcessesCollectable(localNode)
        };
    }

    public static void main(String[] args) throws IOException {
        Main main = new Main();

        for (Collectable c : main.collectables) {
            c.start();
        }

        // Wait indefinitely until a termination signal is received.
        synchronized (main) {
            try {
                main.wait();
            } catch (InterruptedException e) {
                Thread.currentThread().interrupt();
            }
        }
    }
}

I would like to point out some important things here.

The first import statement import com.ericsson.otp.erlang.*; is used to import the Jinterface library. Without this library we would have to implement the Erlang distribution protocol ourselves which would be a lot of work.

Inside our constructor the first statement OtpNode localNode = new OtpNode(“java”, “cookie”); creates a new node called java_node with the cookie value of cookie.
The cookie value is important because both nodes need to have the same cookie value in order to communicate with each other.

When writing code we should follow the DRY principle which stands for Don’t Repeat Yourself. For this purpose we will create a new abstract class called Collectable. This class will provide the basic functionality that all collectable sub classes will need to have. If in the future we decide to collect more metric we can simply create a new subclass that extends the Collectable and inherit the base functionality.

touch Collectable.java

bridge/java/src/java_node/Collectable.java

package java_node;

import com.ericsson.otp.erlang.*;
import java.io.IOException;

abstract public class Collectable extends Thread {
    protected static final String erlangProcess = "handler";
    protected static final String beamNode = "beam_node";
    protected OtpNode localNode;
    protected OtpMbox mbox;
    protected String collectableName;

    public Collectable(OtpNode localNode) {
        this.localNode = localNode;
        collectableName = this.getClass().getSimpleName();
        mbox = localNode.createMbox(collectableName);
    }

    abstract protected String messageKey();
    abstract protected void handleMessage(OtpErlangObject message);

    @Override
    public void run() {
        while (true) {
            try {
                sendMessage(mbox);
                receiveMessage();
            } catch (OtpErlangDecodeException | OtpErlangExit | IOException e) {
                e.printStackTrace();
            }

            try {
                Thread.sleep(5000); // Wait 5 seconds.
            } catch (InterruptedException e) {
                Thread.currentThread().interrupt();
            }
        }
    }

    private void sendMessage(OtpMbox mbox) throws IOException {
        OtpErlangObject elements[] = new OtpErlangObject[] {
            new OtpErlangAtom(messageKey()),
            mbox.self()
        };

        mbox.send(erlangProcess, beamNode, new OtpErlangTuple(elements));
    }

    private void receiveMessage() throws OtpErlangDecodeException, OtpErlangExit {
        OtpErlangObject message = mbox.receive(10000);
        handleMessage(message);
    }
}

This is the abstract class where most of the Java magic is happening. Let’s take a closer look at the code and break it down.

We can spot familiar import statement import com.ericsson.otp.erlang.*; which is used to import the Jinterface library.
Next we define the abstract class and extend the Thread class. This is done so that we can start a new thread for each Collectable instance
and collect metrics concurrently.

Inside the constructor we create a new mailbox for each Collectable instance. The mailbox is used to receive messages from the Elixir node and also send messages to the Elixir node. Each mailbox has a unique name which is the same as the name of the Collectable instance.

Next we define some abstract methods that the concrete classes will need to implement for their needs.

Jumping to the run method we can see that we have an infinite loop that will run until the thread is terminated. Inside the loop we send message to our Elixir node and ask the corresponding metric, then we wait for the response from the Elixir node and handle the response accordingly. After that we wait for 5 seconds and repeat the process again.

Below the run method we have two private methods sendMessage and receiveMessage. The sendMessage method is used to send a message to the Elixir node using the mailBox object created in the constructor. We construct the message that we want to send by creating an array of generic Otp objects. We populate the array with two elements, the first element is the message key which is used to identify the message on the Elixir side. This is going to be an atom with the value of the messageKey method. The second element is the mailbox object PID(Erlang process identifier). This is used by the Elixir node to send the response back to our Java node.

Moving on to our receiveMessage method we can see that we are waiting for 10 seconds for a message to arrive.
Once the message arrives we call the handleMessage method which is implemented by the concrete classes.

Uh, alot of stuff is going on there but stay with me we are half way done! :)

The collectable implementations

Lets create two new files inside our java_node directory

touch TotalAtomsCollectable.java
touch TotalProcessesCollectable.java

These are going to be much more straight forward!

bridge/java/src/java_node/TotalAtomsCollectable.java

package java_node;

import com.ericsson.otp.erlang.*;
import java.io.IOException;

public class TotalAtomsCollectable extends Collectable {
    public TotalAtomsCollectable(OtpNode localNode) {
        super(localNode);
    }

    @Override
    protected String messageKey() {
        return "total_atoms";
    }

    @Override
    protected void handleMessage(OtpErlangObject message) {
        OtpErlangTuple tuple = (OtpErlangTuple) message;
        OtpErlangAtom key = (OtpErlangAtom) tuple.elementAt(0);
        OtpErlangLong value = (OtpErlangLong) tuple.elementAt(1);

        System.out.println("Key " + key + " has value " + value);
    }
}

This will be the concrete class responsible for collecting metrics about total atoms. Inside the class we simply extend the Collectable which calls our implementation functions.
The messageKey function simply returns the key that will be used to map the message on the Elixir side.
The handleMessage function is unmarshalling the Elixir data structure and printing the values. In fact we could simply print the whole OtpErlangObject but I wanted to demonstrate how we can deconstruct the Elixir data structures on Java side.

Now let’s write code for the other implementation that will be responsible for collecting data about total processes running on the Elixir node.

bridge/java/src/java_node/TotalProcessesCollectable.java

package java_node;

import com.ericsson.otp.erlang.*;

public class TotalProcessesCollectable extends Collectable {
    public TotalProcessesCollectable(OtpNode localNode) {
        super(localNode);
    }

    @Override
    protected String messageKey() {
        return "total_processes";
    }

    @Override
    protected void handleMessage(OtpErlangObject message) {
        System.out.println("Erlang term in string representation: " + message.toString());
    }
}

This is should be self explanatory now, and we are not going to go into details here.

Good job we have done writing the Java code and can continue with the Docker setup and build process.

Docker part

Let’s navigate back to our project root bridge directory

cd ../../../

.. and create two files for our Docker setup

touch Dockerfile
touch docker-compose.yml

We start of with the bridge/Dockerfile

FROM openjdk:11-jdk

RUN apt-get update

# Install build tools
RUN apt-get install -y git build-essential libncurses5-dev

# Install Erlang
RUN git clone https://github.com/erlang/otp.git /usr/local/src/otp
WORKDIR /usr/local/src/otp
RUN git checkout maint-25
RUN ./otp_build autoconf
RUN ./configure --prefix=/usr/local/otp
RUN make -j$(nproc)
RUN make install
ENV PATH="/usr/local/otp/bin:${PATH}"

# Install Elixir
RUN git clone https://github.com/elixir-lang/elixir.git /usr/local/src/elixir
WORKDIR /usr/local/src/elixir
RUN git checkout v1.15
RUN make clean compile
ENV PATH="/usr/local/src/elixir/bin:${PATH}"

WORKDIR /app

In short this Dockerfile will be based on openjdk:11 image and will install Erlang and Elixir from source.

We could setup two containers one for java and other for elixir but for simplicity I decided to use one single container and compile Erlang and Elixir from the source. This way we will get the Jinterface library without Java build tools. The official Elixir docker image did not include Jinterface.

Next we have the file bridge/docker-compose.yml

version: "3.5"

services:
  bridge:
    build:
      context: .
    volumes:
      - ./:/app
    working_dir: /app
    tty: true

Inside it we define our container service and mount our code to the container.

Now it’s time to build :)

Execute in the terminal the build command (you can probably have a coffe meanwhile because it may take some time).

docker-compose build

Once finished we can start our container by executing

docker-compose up -d

Compiling and running the system

We arrived at the last step of the tutorial which is running the system. For this we need to SSH into the container, compile the code and run it.

First SSH into the container

docker exec -it bridge_bridge_1 bash

Now lets start the Elixir node, for this we need to navigate to the elixir directory

cd elixir
iex --sname beam_node --cookie cookie main.ex

With the command above the started Elixir node with a short name beam_node and cookie with the value cookie. These values are also used in our Java code.

Now open up new terminal window (don’t close the old one and keep the Elixir node running)

SSH into the container again in the new terminal window

docker exec -it bridge_bridge_1 bash

Next let’s compile our java code.

javac java/src/java_node/*.java -d java/out/ -classpath ".:/usr/local/otp/lib/erlang/lib/jinterface-1.13.2/priv/OtpErlang.jar"

Here we can see that we are telling the Java compiler where he can locate the jar file containing the Jinterface.

If you’re unable to locate the OtpErlang.jar check where is the Erlang installed by executing in the elixir shell `:code.lib_dir()` function which points to the Erlang installation directory. You should locate the Jinterface library there.

Now the last command which starts the Java node

java -classpath "java/out:/usr/local/otp/lib/erlang/lib/jinterface-1.13.2/priv/OtpErlang.jar" java_node.Main

Voila! When all good we should see the message passing between the two nodes every 5 seconds.

Key atoms has value 17787
Erlang term in string representation: #{total_processes => 64}
Key atoms has value 17794
Erlang term in string representation: #{total_processes => 66}

Source code

Source code for the article can be found in the Github https://github.com/alfetahe/bridge-example