For those who are already familiar with Cycle.js: I am suggesting a concept where components could be separated to work “in parallel”. I compared this system with electricity and power grid where consumers are independent but still sharing the same stream. If you are not interested in the whole story and you’re only wondering why the hell would anyone do that, please jump to the section “Why would you do that?”
I worked on many applications which at the time I would categorize as “large” but the first one that lived long enough to “grow out of proportions” was an online code editor. At first, I was happy the way it was designed, but then few more members joined the project. The fact that almost no feature could be implemented without first consulting me about it was flattering. My ego was pleased. But then it started to be obvious that I should be embarrassed, not flattered. It was, of course, not about my greatness or their incompetence, it was about the application architecture itself.
What went wrong?
In his (relatively old but still relevant) article, Addy Osmany asks a few questions when designing modular application:
1. How much of this architecture is instantly re-usable?
2. How much do modules depend on other modules in the system?
3. If specific parts of your application fail, can it still function?
4. How easily can you test individual modules?
Needless to say, our design failed in most of these points.
With these problems in mind, I was experimenting with various architectures like flux and later Redux, but then I came across reactive programming paradigm and Cycle.js framework. At first, I didn’t understand why is there so much “fuss” about some arrays changing over time. Ok, you can map it, filter it, split it... Big deal. But, shortly after, I realized it was actually a big deal and Cycle.js apps really started to feel like driving a car rather than riding a horse.
But even so, these questions still bugged me, and I couldn’t figure out how would I design large-scale application using Cycle.js. Because as we can deduce from those questions, for creating a large-scale applications we need loosely coupled modules.
Luckily, modules or “components” (more popular term used in their documentation) are possible to create in Cycle.js:
Cycle.js has components, but unlike other frameworks, every single Cycle.js app, no matter how complex, is a function that can be reused in a larger Cycle.js app.
But the problem is — they are usually not loosely coupled. The proposed method of using components is to create a parent-child relationship with single root function:
Sources and sinks are the interface between the application and the drivers, but they are also the interface between a child component and its parent.
The problem of this approach is that parent needs to know about the child and that it needs to “manually” provide streams for its children.
Picture yourself how you are providing food for your (or someone else's) children. Now multiply that by a factor of 100 and imagine that you are in charge of feeding more then a hundred of kids. I think that its nearly impossible to accomplish without some kind of system.
Take for example official TodoMVC application of Cycle.js. Just like you would expect it is composed of two components with parent-child relationship. Single todo item is represented with component Task and its parent TaskList is in charge of rendering all todo items. TaskList is a main component and it’s using three drivers:
DOM driver — for rendering components to the DOM
Storage driver — for communication with localstorage
History driver — for reacting to browser route changes
But what if later we decide to replace localstorage with PouchDB? Or we wish to implement cross-browser support with communication over websockets? Every time something changes, we need to “rewire” that in the main (parent) component — connecting the appropriate drivers and passing streams to its children.
Complexity of the TaskList component will only increase over time, and not only is this bad for code readability but it’s much harder to maintain it especially if multiple developers are working on the project.
Building a city with Cycle.js
Cycle.js is all about data flow and managing I/O which is similar to many real-world engineering problems, and that is probably the reason why applications written in this way are so natural to think about. Observables can be compared to pipes in hydraulic systems, currents in integrated circuits, radio signals in telecommunications or even to company services and markets revenues.
Interestingly, many of these real-world examples can be complex systems with a clear separation of concerns and modular design.
Take electricity for example. Suppose we are creating a light bulb as a Cycle.js component. The source of that component would be an electrical current, and its sinks are light and heat. Then we add a TV as another component — again source is an electrical current, and the sinks are light, sound and heat.
If we now scale this system, we may end up with a building connected to an electrical grid. Other buildings can use the same grid with electrical current generated by a power station. We can do this for water pumps, telecommunications, and other streams as well. Yes, we just used Cycle.js for building a city!
Could we then use this concept in our Cycle.js apps? What if we can create a similar system with multiple components independent from each other but still using the same streams in a way light bulb and TV are using a power grid?
Why would you do that?
Ok, but why do any of this? Isn’t this just unnecessary complication?
I agree that this may be overcomplicated for a TodoMVC example, and perhaps it should not be used for it. But suppose that TaskList is a much more complex component, and it is something you have worked on for a year. Suppose your colleague had created another one, just as complex.
One year later, you are working with another five people on the same project, and there are now about 20+ components in your app and all of them are sharing and producing some streams.
After your application scaled to a really big one, few interns are joining the project, and in order for them to test their features, they need to know about the main function and how to set it up. After you explain to them how to use it and where can they find sources for their own child components, they will finally go on and implement some features. And guess who is going to merge interns version of the main function with the current one? I think you know the answer.
If large amount of users are using your app, chances are you will be asked to create a customizable and pluggable application with a clear instructions and API for creating plugins. But how to create a platform where users can install and develop extensions if all components are tangled up together in one big main function? Just imagine the ecosystem of hundreds of extensions that you have to fit in to your app.
Do you still think wiring all of components in the similar way we did it for TodoMVC is a good idea? If your answer is yes, I can only encourage you to go on and try it your way. But I wouldn’t be surprised if about three years from now you will write a blog post about how its not such a great idea to start learning about large-scale application design after going through all of this and how the set of ideas and patterns that worked for you in medium size apps could not be applied to something larger.
Cycle.js Grid Driver
So what would happen if Cycle.js components are independent but could also use some important stream like electricity? Looking at our TodoMVC example, could we find something like electricity inside it? Is there some important stream others can use?
Well, what about actions? Actions from user click inside TaskList, actions from route changes in a browser, actions coming from websockets or from any piece of code that can generate them that we haven’t initially planned for?
Furthermore, what if we stop caring how data stream will come in our component and weather it is from localstorage, indexdDB table or something else?
What would happen to TaskList component?
Well, since it would be only concerned of rendering data stream it would no longer need to accept storage or history drivers and all the code related to this streams would be removed from it. For it to function, only two streams would be necessary: DOM and application state (data).
In order for this to work in Cycle.js, we need a way to share streams between components easily — that is to use something like “power grid”. A component should be able to use this grid as a “consumer” (read access), “generator” (write access) or both (read and write access).
That’s why I’ve created: Cycle Grid Driver.
TodoMVC implementation with a grid driver
Let’s now try to reimplement TodoMVC app with the grid driver.
Instead of using one component and connecting everything inside it, we can now extract some functionality from TaskList into separate components, one for creating application state and one for interaction with browser route changes. Our app would now consist of TaskList, State and History:
We have just created three loosely coupled components and what used to be bundled together is now functioning independently. Since TaskList is now used only for rendering state (received from grid) and sending actions back to the grid, adding new features or replacing some functionality like replacing storage or creating cross-browser support would not increase its complexity.
Is there any example of designing large apps like this?
This is actually an old idea introduced by Nicholas Zakas but it is, of course, adapted for using it with observables and Cycle.js.
With observables, architecture proposed by Nicholas becomes much simpler (there is no need for a event emitter). Grid plays the role of what he calls a “sandbox” and components generating streams for a grid would be the “application core”. So like the sandbox, grid is the only thing modules are aware of and basically limits what a module can do. If you separate streams that you realize could be used for other modules (and for other developers) you are essentially creating an API for a module in a form of streams.
If a module depends only on the sandbox (or the grid in this case) it doesn’t need any streams that would be some other module sinks. For any stream it may require, it must ask the sandbox (or “get” from the grid). Additionally, for creating a feature, there is no need to change the main function.
When there is a group of developers working together, it is really helpful to have this kind of restrictions. Having isolated “playgrounds” where all the toys are provided by “park managers”, kids (and sometimes adults) could play without bothering each other. Kids are of course interns, adults would be junior or senior developers and park managers are project leaders working on the grid and providing the toys.
Even tough in this example I’ve used component which is providing application state, I am not proposing that every Cycle.js apps should have one (like in Redux). Also, I am not against wiring parent-child streams directly (the same its wired here in TaskList) but if a child is not essential for a parent to function and could be reused or removed, I believe it should then be separated and use grid (or something similar) for its sources.