Plug and Play All Your Observable Streams With Cycle.js

A Thorough Walkthrough

The real tragedy would be if people forgot, you can have new ideas about programming models… If the next generation of programmers grows up never being exposed to these ideas, and they teach [only what they know] to the next generation, so the second generation then grows up thinking, “It’s all been figured out. We know what programming is. We know what we are doing.” they grow up with dogma. And once you grow up with dogma, it’s really hard to break out of it.
~ Bret Victor, The Future of Programming

Introduction

In 2014, front-end developer André Staltz set out to solve an imminent problem that developers are facing, especially in the Web world: dealing with large code bases and data flow. While even large corporations such as Facebook and Google and others have attempted this and brought forward popular frameworks, e.g., Angular and React, they seem to have missed the target. They have not solved the problem of how to achieve explicit data-flow graphs, nor efficiently dealing with the problem that new developers joining a large code base should be able to get a summarized overview of the architecture without spending multiple hours reading each file’s source code to build a mental model of the architecture.

Beyond astonishing and mind blowing, what Staltz achieved with just around 100 lines of code is a true paradigm changer. This changer is Cycle.js.

Overview

Cycle.js is an extension of the RxJS library more than a framework (though it will probably fall into that category). RxJS is the JavaScript version of ReactiveX, an API for asynchronous programming with observable streams. While RxJS solves the part of composing asynchronous and event-based programs by using observable sequences, Cycle.js does a unique thing by looping data flow between your application and the outside world. We will go into the details how this works later, but this makes explicit data-flow graphs possible; i.e., reasoning of dependencies between operations. Neither Angular nor React (nor any other that I am aware of) does this. This unique feature of Cycle.js has the benefit of making it far more easy to read and understand large code bases.


The New Functional and Reactive Programming Era

There is a debate whether reactive programming, when functional, can be called functional reactive programming (FRP), because it differs from the original specification by Conal Elliot. In this article, however, I will use the term functional reactive programming to mean something different than the original specification, but to be fair, I will shorten it fRP (with a small f).

The Future Is Functional and Concurrent

We come from a time of object-oriented programming (OOP). A time that is dominated by imperative actions: Do this! Do that! Set this value! Remove that object! That time is over. The future is functional. The future is concurrent. To eliminate any doubt of these claims, let’s take a look at Moore’s Law:

Moore’s law is the observation that, over the history of computing hardware, the number of transistors in a dense integrated circuit has doubled approximately every two years.
~ Wikipedia

A modern laptop has a billion transistors on it. The transistor density is 8.26 million transistors/mm². Nobody can keep a billion transistors busy by trying to get them all to do something at once. The only way you can do that is by dividing your work up. Thus, the future is functional and the future is concurrent.

In the old days, we were all concerned about state. Programming was about maintaining state. It was about class invariants. It was about hiding data. It was about object orientation. Object-oriented programmers attempt to classify things into incredible hierarchies. In fact, we were not doing objected-oriented programming; we were doing class-oriented programming by dividing the world up into static hierarchies. But programming is, in fact, about transformation of data. That is what functional programming (FP) is concerned about. It is the transformation that actually gets work done. In OOP, we are thinking about coupling data and functionality together to have data and methods all in a class. With FP, we separate the two out. We decouple them. And decoupling leads to more flexible designs. Besides transformation of data, we also need to think about transformation of code.

The world is dynamic, and flexibility and transformability of code is critical if we are to generate the next version of the Web. In practice, this means that the vast amount of present libraries, frameworks and APIs in the near future will become obsolete if they don’t transform their code to a functional approach. Economics 101 tells us that this will automatically weed out any that doesn’t have a demand strong enough, because transformation of code comes at a cost. However, the cost of not transforming code is, as mentioned, an inevitable quickly-approaching death. My position is that hubs in the open source community will drive most of these transformations with support from corporations and other interest groups.

The Turning Tide

Reactive programming is oriented around data flows and the propagation of change. Erik Meijer gave us Rx because he was induced by push-based systems. When you want to stay up to date about the state of the world, it is much better to push instead of to pull. I will use a recent example Erik gave at the GOTO Copenhagen 2015 conference to illustrate why.

In the image to the left, Erik wants to stay up to date about what Alice, Bob and Carol are doing. Imagine that Erik is an app on your phone and Alice, Bob and Carol are sensors or sources of information on your phone, e.g., UI, motion, and position. If Erik is going to pull, he first has to ask Alice, “Is there anything interesting I should know?” but Alice is a little boring and answers, “No. Nothing happened.” Then Erik asks Bob, “Bob? Bob? Bob?! Are you there?” But Bob doesn’t answer, and if Erik is not careful, he is blocked by continuously asking Bob until Bob answers. This is why you must do asynchrony and never do a blocking call. While Erik was busy pulling information from Alice and Bob, Carol had all kinds of exciting things going on, but Erik never had a chance to ask her. When Erik finally asks Carol, she says, “Sorry, you already missed all the action. It’s now old stuff.” Therefore, when you want to stay up to date with the world, you have to push; you cannot pull.

Another way to look at this is this simple and naive implementation of pull. You are building a Web page that you want to refresh, and a common way is to use setInterval() to pull from the server at some rate.

window.setInterval( function () {
//...pull from server...
}, ??? );

The question marks in the pseudo-code above are there because of the question: how often do you need to pull? The answer is: you don’t know. As we have seen with Alice, if you pull too fast, you waste resources because there is nothing interesting going on. With Bob, we have seen that if you pull at the wrong moment, you get blocked. And if you pull too slow, you may lose out on the action, as in the case of Carol. Thus, you cannot write code like this, because when you pull, you never get it right. Fortunately, JavaScript, with ES7 / ES 2016, now has async/await.

Here is the same example, but now the sources push. Again, the sources can be any source of information you can imagine, e.g., friends, stock ticks, or GPS. In this case, Alice is relaxed. Every once in a while something interesting happens, but not often. Bob is off line, but then suddenly he is back. You are not blocking because he will tell you when he is back. Erik doesn’t miss out on interesting stuff that Carol has to say, either. If Carol is pushing too fast, Erik can have some code that drops her messages or calms down the stream. This is why reactive is so important.

Push works, pull doesn’t.
~ Erik Meijer

Everything Is a Stream of Events

As we have seen in Erik Meijer’s examples above, everything is a stream of events. On the client side, this can be user mouse clicks, mouse moves, key presses, touches, swipes, etc. On the server side, it can be stock ticks, Web page updates, order fulfillments, etc. Seen from a program’s perspective, these streams of events are from sources in the outside world. It is the responsibility of the program to connect these observable streams and transform the data. To do this, the program must model the flow of data from these sources and build a sink, i.e., the output that is provided to the outside world, which will be explained in detail in the next section. It is here Cycle.js comes into play.


The Inner Workings of Cycle.js

The core of Cycle.js is incredibly small but highly efficient to meet the demand for modeling flow of data and dealing with the communication between the application and the outside world. With Cycle.js, your program is a closed world, but obviously, if nothing comes in or out of the program, it is useless. Cycle.js provides a simple and ingenious way to compose your program’s interaction with the outside world while maintaining a true separation of concerns. A Cycle.js program’s entry point is called main(). In this function, the program is enclosed. Everything the program does happens from within that function.

As noted above, if nothing can come in or out of the program, it is useless. The input to the program is in Cycle.js known as sources.

One of the unique points of Cycle.js is how the sources are injected to the program. Cycle.js comes with an on-switch simply called run(). This is one of the two APIs that exist in Cycle.js and is used to kick-start Cycle.js. run() takes two arguments: a function and an object. The function is the program’s entry point main. The object is the sources that main should know about.

As you can see, we still haven’t declared what the sources are. The sources in Cycle.js are provided as drivers. A driver is the connection point to the outside world. Drivers are where side effects take place, e.g., updating the DOM in the browser, or sending a message to a server, but they also provide input to the program and reads output from the program. Cycle.js comes with a couple of common drivers, for example, the DOM driver. The DOM driver uses virtual-dom under the hood to effectively handle DOM manipulation and DOM state. Drivers are usually made with a factory function which accepts some options. The factory function will then return the actual driver function configured with the options. In case of the DOM driver factory function, it takes a CSS selector as argument.

Now our program can access the DOM driver’s API for reading events that happen in the DOM. As you recall, everything is a stream of events, and because Cycle.js is an extension of RxJS, drivers provide these streams of events from the outside world.

As the avid reader will recall, Cycle.js is essentially about modeling flow of data. This is the early beginning to start visualizing this flow. We now have a flow of input event data from an HTML element streaming into our program, available for it to use and act upon. But wait a minute. How can we then listen for events on a nonexistent element? We haven’t told the DOM to render any HTML INPUT element. That is true, and we won’t. Telling is imperative, and imperative commands don’t belong in our closed-world program. Even if we did choose to make imperative statements, the DOM is the outside world, and it is the responsibility of the driver to handle the rendering. Thus, imperative commands should exist solely in drivers.

The way the driver receives information about what it should do is by the same means of a push-based system. The program pushes information to the driver in the same form of event streams that the driver pushes to the program. Cycle.js uses the term sinks for the streams that are pushed out of the program to the outside world, i.e., the drivers. We simply return the sinks from the main().

In Cycle.js, the sinks are, like the sources, an object. The sinks’ keys correspond with the driver names, i.e., the keys in sources, to enable Cycle.js to do the mapping.

This code will actually execute and display an INPUT range slider in the browser. However, it doesn’t do anything else. The alert reader will probably wonder how we can listen for input events on the INPUT element when it is declared later in the code. The answer lies hidden in the core of Cycle.js, and the following explanation should complete the understanding of the flow of data in Cycle.js.

When run(main, sources), the kick-starter, is called, Cycle.js creates a proxy sink for each driver, and then calls each driver passing it the proxy sink. The proxy sink is both a replay-able, observable stream and an observer in one, known in Rx as a ReplaySubject. It is configured to replay just the last element. The results of each driver invocation are stored in a sources object and then passed to main(sources). main proceeds to execute, now having access to the driver APIs. When main returns the sinks, Cycle.js creates a disposable subscription to which each sink, i.e., the returned observable streams from main, is added and subscribed to, using the proxy sink as observer.

In effect, Cycle.js circularly connects main with the drivers, creating a feedback loop of data. The driver source observables are streaming events (or pushing data) into main, and main’s sink observables are streaming events (or pushing data) out to the drivers, which then can produce side effects based on the data from the program.

We will finish the example in the Nested Dialogues and Model, View, Intent section. However, before you jump to that, it is advisable to read the next sections first to get a better grasp of how Cycle.js and its constituents work.

An Extension of RxJS

Cycle.js embraces functional reactive programming (fRP). Because programming is about transformation of data, and architecture is about flow of data, Cycle.js is really nothing more than functions (transformation) and observables (data flow). The observable streams as input to functions are called sources, and the observable streams as output from functions are called sinks. The observable streams that make the data flow are at the heart of Cycle.js; thus, Cycle.js is really just a clever extension of RxJS. Cycle.js, in its true essence, simply connects the data flow in a unidirectional cycle, a feedback loop, using drivers that are functions and can produce side effects and main() which is the actual program function.

Unlike frameworks like React and Angular, Cycle.js’s use of RxJS is not a last-minute patch. It is build around RxJS (though ports to other reactive libraries are quite possible). This is an enormous advantage and gives rise to the explicit data-flow graphs, which is unachievable with React and Angular.

Drivers For Side Effects

As described previously, drivers are the building blocks for handling side effects. Any effect that has interaction with calling functions or the outside world, i.e., outside main(), is considered a side effect, e.g., drawing a pixel on the screen, storing a value in a database, or sending a message to a server. When our program requires side effects, we use a driver. Cycle.js offers, at current time of writing, the following drivers:

Understanding the Driver

We will build a very simple driver that doesn’t really do much, but it will be easier to reason about the basic structure of drivers and fitting the terminology that were introduced earlier.

Using the above factory will only give us a no-op function, which is pretty useless, so let’s do something about that. Remember that a driver, just as our program, accepts data input and exits data output. The terminology is source and sink. The driver accepts a sink from main(), but from the drivers perspective, it is a source of data. Likewise, the output from the driver’s perspective is a sink, but a source from the program’s perspective.

It should be noted that it is not a requirement that the driver returns an observable stream as output. It could, as in the case of the DOM driver, be an object with methods, or nothing at all for that matter. Likewise, it is also not a requirement that the driver accepts a source. When a driver accepts a source as input, it is a sink driver. Yes, this sounds strange and can easily cause confusion, but remember that from the program’s perspective, we return a sink that the driver accepts. When a driver returns a sink as output, it is a source driver because the program gets it as source. Our driver is a source-and-sink driver. As a rule of thumb, if the driver provides data to the program, it is a source driver, and if the program provides data to the driver, it is a sink driver.

If we don’t require our driver to be configured with options, we can remove the outer factory function altogether. The factory function in above example is there to illustrate the common way of structuring drivers.

Here is how we can use it in our Cycle.js program:


Nested Dialogues and Model, View, Intent

Now that we covered the inner workings of Cycle.js, let’s finish the INPUT range slider program which we began earlier. As we do so, we will cover some new concepts.

We left our program displaying the INPUT range slider and reading the value on input event. Yet, we never used the value for anything. Now we will take the INPUT range value to dynamically display n amount of DIVs on the screen, corresponding with the slider value. To make the DIVs a little interesting, we will add some styling and display which number DIV it is.

If you adjust the slider now, you will see little dark-grey squares appearing or disappearing, depending on whether you increase or decrease the value. However, we want to see some numbers in those squares, but if you look at the code, there doesn’t seem to be any way we can pass numbers to the dynamicDIV. Also, our main is beginning to get cluttered, so let’s clean it up.

Our main is now much cleaner. The function DynamicDIVSlider is like a component, and what is interesting to notice is how it accepts sources and returns sinks; just like our main. In fact, this is beginning to become fractal.

In Cycle.js, components in the program that accept sources and return sinks are called dialogues, because they listen and respond. And because of their fractal nature, dialogues can be nested. This concept of Nested Dialogues was introduced by André Staltz in his blog post Unidirectional User Interface Architectures.

As you can see, a dialogue is just a function; just like our main is just a function, and our drivers are just functions. They all share this commonality of accepting sources and returning sinks. main is, in concept, a dialogue, i.e., our main dialogue; the root.

Before we get our numbers to show up in the squares as we require, we will extract the dynamicDIV as another dialogue and nest it in our DynamicDIVSlider dialogue.

We have now obtained a Nested Dialogue-approach, but there is more we need to do. Our DynamicDIV dialogue should accept properties in which the number to display should be held.

Moving the slider forward and back will now show squares with a number. Perhaps you already noticed, but with a slight of hand we changed a variable to be called view, which is now a function that takes some state as argument, in this case value.

Model, View and Intent

Slowly we are moving towards a separation that André Staltz has named Model, View, Intent or MVI in short form. This concept is part of the unidirectional data flow in Cycle.js. The Intent is a function, like almost everything else, whose purpose is to interpret the user’s intent into actions. If we look at our code, we can see that we in our DynamicDIVSlider reads the value of the slider by using the DOM driver. What we actually are doing is interpreting the user’s intent when the user moves the slider. In the light of this new concept, let’s refactor our code. Since we will only refactor the DynamicDIVSlider now, I will just show that function.

This might look like a step backwards, because our code actually got a bit more complicated. However, we can now see how the intent modifies the state of our dialogue. Instead of having state flying freely around in the dialogue, we use a Model to encapsulate it. Business logic resides in the model.

The model is also just, yes, you guessed it, a function. It takes the actions from the intent, often combined with other properties passed to the dialogue, and returns an observable stream of state. Let’s see how that works.

The final bit we are missing is our View. Right now it is awkwardly being returned in the sinks. Let’s refactor.

And here is the final refactored and compacted code.

A simple challenge is to modify the final code to let the DynamicDIVSlider accept an observable stream of properties through its sources, through which an initial value, min and max can be set. You could also try to figure out how to display the value of the slider. Have fun!

Final Words

It’s been a long ride, but we made it. We have learned how the future is functional and concurrent. We have learned how push-based systems are superior to pull-based systems. We have seen how everything is a stream of events. We have seen how reactive programming using observable streams solves the problem of state. We have learned how Cycle.js creates a feedback loop and how explicit flow of data are modeled to reason about. We have learned how side effects are handled by drivers in Cycle.js. We have learned about nested dialogues, and how model, view and intent fit in writing Cycle.js programs. We have learned to embrace functions and leave imperative and class-oriented code behind us. We have learned to use Cycle.js by making ourselves a simple driver and a simple program. So now that we can cycle, let’s turbocharge it and spin it real fast. Cycle.js is a Ferrari on steroids when you use it cleverly.


Thanks to everyone in the Cycle.js community who helped this article on its way. Thanks to Thom for inspiration. A special thanks to André Staltz and Tylor Steinberger.