The Decentralized Collaboration Approach to Algorithmic Trading
How to crowdsource trading intelligence into Decentralized Trading Organizations to beat trading firms at their own game.
In the summer of 2017, a small group of fintech developers and pro-traders started building the ultimate trading automation platform, open-sourced it, developed a community around it, and began recruiting talent to form an algorithmic trading group. The article lays out the collaboration features built in the platform and explains how specialists in multiple disciplines may aggregate skills and material resources to launch Decentralized Trading Organizations.
The Superalgos Platform is free and open-source and is currently the #1 “trading bots” project on GitHub search results. It likely is the most powerful and flexible toolkit to design, test, and deploy sophisticated trading bots — or that’s what verified user reviews suggest.
It’s a technical platform oriented to algo traders and data scientists. That said, the visual environment paired with the native trading system design framework makes it accessible to non-developers too.
Trading bitcoin using the Superalgos Platform resembles entering a battlefield in a Mitsubishi MK-6 Amplified Mobility Platform — James Cameron’s vision of a futuristic armored exoskeleton ready for combat, as seen on Avatar (2009).
Indeed, the platform can unleash overwhelming power.
However, trading in and of itself is a very complex problem, and trading automation adds further complexity on top. An ambitious trading operation requires many different skill sets.
To start with, you need to be able to:
- come up with trading ideas;
- understand existing indicators or build your own technical studies;
- set up and run a data-mining operation;
- design strategies and trading systems;
- design execution algorithms;
- backtest and forward-test your creations;
- deploy your first few strategies.
Although the list requires many different types of skills — including both creative and technical — most hobbyist algo traders are ready to cope with the above playing solo.
However, what if you’re serious about trading? What if you want to go beyond the hobbyist approach and pursue true performance and consistency?
In that case, you will need to add a few more tasks to the list:
- design, test and deploy tens of strategies spanning multiple markets;
- run machine-learning algos to optimize parameters on each strategy;
- scale up the data-mining operation across a network of coordinated machines to monitor tens of markets processing data with as many indicators as your strategies need, in as many exchanges as required;
- set up a trading farm, with multiple machines running your trading algorithms in coordination with your data-mining facility;
- set up portfolio management bots feeding context info to trading bots so that they may make decisions factoring in the overall state of the account;
- do whatever it takes to keep everything up and running 24/7.
The Superalgos Platform is ready to do all of the above, but… are you? Take a minute to consider the following questions…
- Are you a seasoned trader capable of coming up with all the right trading ideas to test?
- Are you a data scientist ready to conduct a deep exploration of the technical analysis and quantitative trading spaces?
- Are you well versed in strategy-testing and capable of certifying you’re not overfitting the data sets?
- Are you a system administrator capable of designing the deployment of the technical infrastructure?
- Are you a Dev-ops engineer ready to keep the whole operation up and running with 24/7 availability and redundancy where needed?
- Are you a developer ready to debug whatever may fail at the application level?
If you answered no to any of the above, then you need to work with a team!
Think of it this way…
If you show up in the markets wearing your MK-6 AMP, you will certainly be able to take out the fools doing manual trading or playing with silly plug-and-play bots they got on some random web portal or online service. Those are going to war armed with stones and sticks — maybe a bow and arrow — so no wonder they’re an easy kill.
But what will happen when you run into Special Forces type operations run by trading firms?
Firms recruit dozens of Math and Computer Science PhD's coming right out of MIT to form elite commando teams. They too fit them with exoskeleton suits, and back them with heavy artillery, a navy, an air force, logistics support, a situation room full of battle-hardened generals, and piles of cash to throw at any issues that may emerge.
There’s nothing you can do against firms if you’re fighting solo!
Let me try a less dramatic football analogy…
Playing the markets solo is like taking a corner kick and running for the header — all by yourself. For Americans, it’s like throwing a forward pass and going for the catch — same guy.
That’s why we built Superalgos with frictionless collaboration as a leading design principle from the get-go. We were a group of friends with complementary skills, and we wanted to build our trading army!
It took us four years to build the platform that would allow just that, so it’d be silly to — as a user — not take advantage of the built-in collaboration features!
Standardization
Trading intelligence emerges from harnessing the interactions of multiple complex concepts: the information resulting from processing data, the models deriving from the information, the strategies that describe how to use the information in the models, the execution of those strategies, etc.
Standardizing the format of trading intelligence assets make assets portable. Portable assets are sharable. In Superalgos, everything is standardized.
Starting with:
- raw data extracted from exchanges as 1-minute OHLCV data sets;
- candles in all supported time frames;
- data sets corresponding to indicators’ processed data in all time frames;
- the definition of indicators in terms of dependencies, data products, the visualization of data over the charts, etc.;
- the definition of strategies and trading systems;
- the definition of portfolio manager bots and machine learning algorithms;
- the definition of networks of machines and each network node;
- the definition of all sorts of parameters, trading sessions, data tasks, etc.;
All of the above assets are generated by users in standardized formats. This means that those assets are portable, thus, may be deployed on any instance of the platform by anyone in a trading team.
This is the first step to enable collaborative work, allowing multiple parties to work on multiple assets, and share them with the rest of the team when and if necessary.
Granularity
Superalgos leverages a visual scripting environment to help break up complexity and visualize it in a digestible manner. The software infrastructure contributes to harnessing complex concepts and their interactions by breaking complexity into small units of information called nodes, arranging them into larger data structures called hierarchies.
A hierarchy is a data structure representing an overarching, top-level concept, resulting in a long chain of nodes, usually with many ramifications. The system manages different types of hierarchies, each of them with a specific focus.
For example, a trading system may contain multiple strategies, and each strategy features four stages, each with its definitions consisting of conditions, situations, formulas, and more.
Each of those definitions exists as a self-contained data structure.
This means that users can not only share trading systems but can exchange information at whatever level is appropriate: a formula, an execution algorithm, the definition of one of the orders, one of the stages of a strategy, or the whole strategy.
Communications
Standardization and granularity enable collaborative work because users playing the markets as a team can exchange every piece of trading intelligence they generate and use other people’s work within their premises.
As powerful as that sounds, the truth is that’s just the groundwork for what comes next: Superalgos Platform clients may interconnect over peer-to-peer networks to exchange information in a frictionless manner, directly from client to client.
The current implementation enables teams to deploy both open or closed p2p networks to broadcast and consume information as signals while running trading sessions.
Let’s unpack the above statement.
Teams deploy closed networks when they wish to exchange signals with a defined group of users that are invited into the group. Networks may also be open so that any user may plug into the network to broadcast or consume signals.
The information that may be broadcasted at this point is virtually all types of data structures managed by the system while running a trading session, with the same level of granularity described earlier. To keep network traffic manageable, signals don’t carry the actual data. Instead, they feature meta-data indicating what type of information the signal represents, where the data is stored, and how to access it.
The data may be distributed across multiple storage providers for redundancy and resilience. All messages are cryptographically signed and data is stored encrypted.
Modularity
In Superalgos, each of the functions involved in setting up and running a trading operation may function independently from each other, in multiple configurations.
For example, you may have a single machine running data tasks along with trading tasks and handle the whole deployment on a local machine. Or you may choose to have an instance of the platform controlling a large data-mining operation spanning multiple machines and even multiple networks, while another instance of the platform runs a trading farm consuming information produced elsewhere.
The flexible modularity of the platform allows having different team members, each with specific competencies and responsibilities. It also means that a trading team may scale up an operation by adding new recruits willing to contribute material resources, like compute or data storage facilities, as well as expertise or work.
No Trust Required
Superalgos is designed for trustless collaborations. The premise is that you should be able to partner with other users, even if you don’t know them that well, or fully trust them.
The core idea is that a trading team should collaborate to produce a collective trading intelligence and a common infrastructure, but still, each member should be able to trade without counterparty risks and make their own decisions.
The counterparty risk bit is solved because each team member may run their trading bots on their premises, feeding on the trading intelligence produced by the team in terms of strategies, data, and so on. Team members do not need to trade from a central account. They may all trade independently, controlling the funds, exchange keys, and — ultimately — the specific behavior of their bots.
For example, the trading team may set up a common data-mining infrastructure and a trading farm with the trading bots required to run each strategy. The shared setup embodies the trading intelligence of the group in the form of information, models, and strategies. This trading farm may then broadcast the signals of each bot to the team’s peer-to-peer network. Then, each team member may run their trading bots listening to the signals in the network, each under their accounts. Also, each member may decide if they wish to follow the signals to the letter or if they wish to introduce additional logic to customize the behavior of their bots.
Because incumbencies of team members may be clearly delimited, the intellectual property generated by core members of the team doesn’t need to be shared with all team members. For example, core members producing strategies may want to keep them secret whenever possible. That is perfectly possible as not everyone in the team needs to have access to the actual definitions of the strategies.
Of course, teams are also free to work in a trusted environment, sharing as much sensitive information or intellectual property as they wish.
Going Massive
Up until now, we covered how Superalgos enables frictionless collaboration. But the ultimate goal is to be able to handle groups of massive proportions, so as to maximize the potential of the collective intelligence.
Practical limits on how much a trading collaboration can grow may or not exist. It’s not hard to imagine tens of specialists working together on a trading team. Can hundreds of people actively contribute to a collective trading intelligence without stepping on each other’s toes? How about thousands? We don’t know yet, but we’ll find out!
Regardless of how many people may actively contribute there is another interesting avenue worth exploring. Picture a successful decentralized trading organization, as described earlier, with good and consistent performance metrics.
What would happen if the group decided to broadcast trading signals to a permissionless peer-to-peer network right after all team members placed their orders?
Take a few minutes to ponder that idea.
More Info
- Learn more about the Superalgos Project and building a Trading Intelligence Network.
- Download the platform directly from the repository.
- Join the community or ask questions on Telegram or Discord!
Join Coinmonks Telegram Channel and Youtube Channel learn about crypto trading and investing
