Multi-language Flowex components

Anton Mishchuk
Apr 14, 2017 · 4 min read

In Flow-Based Programming (FBP) paradigm each component is an independent process — “black box”. Component only transforms input to output, and this transformation is independent of internal implementation. So in general components can be written in different languages and then linked together in a single network.

Ability to run alias code is very important for such new languages as Elixir. One can reuse tonnes of libraries from more mature languages and choose the specific language for the specific problem.

In this article, I’ll show examples of multi-language Flowex components. I will use Export library to run Ruby and Python and Porcelain library to run command-line applications. There is the multi_flowex project on GitHub, for whom interesting in details.

Erlang ports

Erlang port is an abstraction that provides a byte-oriented interface to an external program over STDIN and STDOUT by sending and receiving lists of bytes, including binaries. Ports are process specific, so only process created a port can communicate with the port and if this process terminates, so does the port. The external program resides in another OS process. By default, it reads from standard input (file descriptor 0) and writes to standard output (file descriptor 1). The external program is to terminate when the port is closed.

Erlport project is not in active development but demonstrates how one can connect Erlang to other programming language. Currently, the project supports only Ruby and Python. There is a wrapper for Elixir — Export.

Porcelain allows launching and communicating with external programs.


multi_flowex is a simple Flowex project with Ruby, Python and shell “pipes”. Each component says “Hello”. So passing an information packet (IP) with empty input return IP with the text: “Hello from Ruby, Hello from Python, Hello from shell”.

Let’s consider each component in details. (Read more about Flowex module pipes here)

Ruby pipe

defmodule RubyPipe do
defstruct [:data]

@ruby_dir Application.app_dir(:multi_flowex, "priv/ruby")
@main_file “main”
use Export.Ruby def init(opts \\ %{}) do
{:ok, ruby} = Ruby.start(ruby_lib: @ruby_dir)
Map.put(opts, :ruby, ruby)
def call(%{data: data}, %{ruby: ruby}) do
{:ok, result} =, push(data, "Hello from Ruby"), from_file: @main_file)
%{data: result}

The init function starts Ruby process and adds its pid to options. @ruby_dir specifies the path to ruby code and @main_file the file with simple Ruby code:

def push(array, el)
result = array.push(el)[:ok, result])

It just pushes element to array.

The code in the call function of the component calls the push method with initial data and “Hello from Ruby”.

Take a look at the test of the module to get the idea what each function returns.

defmodule RubyPipeSpec do
use ESpec

let :opts, do: described_module().init

it "has ruby process pid in opts" do
expect(opts().ruby).to be_pid()
it "pushes to data when .call" do
result = described_module().call(%{data: []}, opts())
expect( eq(["Hello from Ruby"])

Python pipe

The pipe with Python code looks similar:

defmodule PythonPipe do
defstruct [:data]
@python_dir Application.app_dir(:multi_flowex, "priv/python")
@main_file “main”
use Export.Python

def init(opts \\ %{}) do
{:ok, py} = Python.start(python_path: @python_dir)
Map.put(opts, :py, py)
def call(%{data: data}, %{py: py}) do
{"ok", result} =, append(data, "Hello from Python"), from_file: @main_file)
%{data: result}

And the Python code is:

def append(array, el):
return (‘ok’, array)

Shell pipe

A component which uses Porcelain just runs echo program and gets its output:

defmodule ShellPipe do
defstruct [:data]

def init(opts), do: opts

def call(%{data: data}, _opts) do
output =“echo ‘Hello from shell’”).out
result = data ++ [String.rstrip(output)]
%{data: result}

The pipeline

Let’s look at the pipeline itself.

defmodule MultiPipeline do
use Flowex.Pipeline
defstruct data: [] pipe RubyPipe, 3
pipe PythonPipe, 4
pipe ShellPipe, 2
pipe ElixirPipe

The pipeline starts 3 Ruby pipes, 4 Python and 2 Shell pipes. It’s just for demonstration purposes, we definitely don’t need such parallelism is not necessary for this “Hello, world!” example. One can check that after starting the pipeline 3 Ruby and 4 Python processes appears in the system. Porcelain starts `echo` process only on demand, when IP enters the component.

The last pipe is plain Elixir pipe which just joins a list into final result string:

defmodule ElixirPipe do
defstruct [:data]

def init(opts), do: opts
def call(%{data: data}, _opts) do
%{data: Enum.join(data, “, “)}

The spec below shows the pipeline in action:

defmodule MultiPipelineSpec do
use ESpec

let :pipeline, do: MultiPipeline.start()
it “returns hello” do
result =, %MultiPipeline{})
greetings = “Hello from Ruby, Hello from Python, Hello from shell”
expect( eq(greetings)

So one can see greetings from 3 different programs!


Elixir/Erlang ecosystem has a flexible and safe way of running external programs thus allowing to leverage the advantages of other programming languages. At the same time, Flowex makes it easy to encapsulate alias programs inside independent processes and organize communication between them.

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade