A Conversion Guide for Machinations and CadCAD
Motivation
In my conversations with professionals in the fields of token engineering, economic design, and game design, I have discovered that there are common gripes about the tools we use to model and reason about our complex systems. Machinations and CadCAD seem to be the common tools of choice in these communities. I am writing a series of articles to help teams understand which tool will suit their use case better and how to transition from one to another.
My intention is to also provide teams with a roadmap to ideate, communicate, and iterate their models quicker using a visual tool like Machinations before switching to CadCAD to access Python’s data science tooling, conduct deeper analysis and move towards building digital twins of their systems.
Tools of the trade
With the increasing prevalence of complexity in our socio technical systems, particularly those related to financial products and protocols, it is critical to assist teams in comprehensively understanding and balancing various variables as they design and launch their systems.
Complex systems like Web3 ecosystems or Game Economy design are notoriously difficult to reason about due to the density of interactions and interlinkages of variables, sensitivity to initial conditions and properties such as emergence. (See: https://complexityexplained.github.io/)
However, we can understand these systems better using models that we can build on tools like: Vensim, Stella Architect, Anylogic, netlogo, Machinations and CadCAD. Machinations and CadCAD seem to be the current tools of choice amongst Token Engineers for various reasons we will discuss ahead.
What is Machinations
Machinations is a powerful tool for designing and testing game mechanics, but its usefulness extends far beyond the world of gaming. At its core, Machinations is a flowchart-based modeling tool that enables us to represent complex adaptive systems and explore their behavior. It provides a way to design and simulate systems through which we can explore how different variables and parameters impact the behavior of a system, helping us to design more effective systems and make better decisions.
What is CadCAD
CadCAD is an open source Python-based simulation framework designed for modeling complex adaptive systems. The name “CadCAD” stands for “Complex Adaptive Dynamics Computer-Aided Design,” and it was developed with the goal of making it easier to model and simulate complex systems. One of the key advantages of CadCAD is its flexibility and extensibility. Since it is built on top of Python, you can use a wide range of Python libraries and tools to customize and extend the simulation framework to fit your needs. This makes CadCAD a powerful tool for a wide range of applications.
Pros and Cons of Each
Pros of Machinations:
- Intuitive and easy to use for non-technical users
- Provides a visual representation of the system components and their relationships, making it easy to understand and communicate the system’s behavior.
- Allows for rapid iteration and experimentation with different system configurations and parameters
- Easy intuitive features to perform Monte Carlo runs and other common simulation analysis techniques
Cons of Machinations:
- Component limit on lower tier plans which can limit you building larger models
- Rendering of each flow on screen seems to lag the browser for larger models.
- More appropriate for system dynamics models than agent based models given the exponential increase in components needed to model agents.
- Can be relatively difficult to integrate with other tools or systems
Pros of CadCAD:
- Open source and free!!
- Highly customizable and extensible
- Easy to integrate with APIs and other data sources
- Enables simulation and analysis of large-scale systems with many interacting components
- Natively supports a range of simulation techniques, including Monte Carlo simulations and parameter sweeps and can be easily coupled with data science tools.
Cons of CadCAD:
- Steep learning curve, especially for non-technical users who may not have experience with Python and modeling techniques
- Not as intuitive compared to visual tools like Machinations during the design and planning phase
- May require effort to customize and extend the models
How to decide which tool is the one for you?
(This will be broken down in-depth in a future blog)
- Team’s expertise in python and modeling
- How much time do you have to iterate and plan the system?
- What is the goal of the model?
- What modeling paradigms make the most sense for our goals?
My suggestion would be to build initial versions of model on Machinations and port the model to CadCAD when you hit the component limits
A practical guide to translate Machinations to CadCAD:
For simple models we can just use the Machinations model as our reference and follow the prescribed steps below. For larger more complicated models I will showcase how we can build a translation layer between Machinations and CadCAD later in these series of articles.
Inifinite runner model example:
Here’s a simple machinations example on modeling the dynamics of infinite runner games like Subway Surfer or Temple run. This example can be found here in the docs:
Link : https://machinations.io/docs/tutorials-examples/infinite-runner-game-loop/
I have converted the above machinations model into CadCAD you can follow along on this github link: https://github.com/rororowyourboat/Machinations_CadCAD_conversion_guide
Anatomy of a Machinations model
- Pools
- Sources/ Drains
- Converters
- Traders
- Gates
- State connections
- Resource connections
Anatomy of a CadCAD model
- State variables
- Policy functions
- State update functions
- Partial state update blocks
- Monte Carlo runs
- Parameter sweeps
So we need a way to map components and flows in our Machinations model to the CadCAD framework
Prescribed steps for simple models:
- List down all the Pools → these are your state variables and corresponding state update functions
- State connections + the numbers on top of your resource connections → logic in your policy functions
- All your Resource connections → Will be used in the state update logic
- delay functions → an if condition involving timestep in the corresponding policy function
- random gate → Logic in the corresponding policy function
- End condition → Conditional element in your policy functions
- Check if model is giving similar results
Some of the runs on CadCAD:
To test whether we converted the model successfully we can compare the results of our model with Machinations model by running 100 simulations on both and comparing the means for some of our variables:
Once we have the confidence that our cadCAD model is giving us the same results we can move on to analysis
Analysis
Parameter Sweeps + Monte Carlo Analysis:
We can run scenario analysis with different initial conditions and see how our metrics perform.
We can measure the mean timestep at which players crash under different difficulty levels
Machine Learning and Parameter Search Algorithms:
We can potentially use algorithms to automate the task of finding appropriate parameters to meet our goals
For example: Let’s say the game team has a shop in the game where certain items cost 50 coins. And they believe that the players should ideally have to play 10 games to earn these items to adequately incentivize them while keeping these items valuable.
To model this we can break the question down as: What should be the initial value of the crash chance parameter such that the median player gets 5 coins per game.
An algorithm like this can run a 100 simulations for each parameter, then find the median coins minted, if the result does not fit our goals it will change the parameters and test again until it finds the right parameter set.
Conclusion
In conclusion, Machinations and CadCAD are both powerful tools for designing and simulating complex adaptive systems. While Machinations is more intuitive and easier to use for non-technical users, CadCAD provides flexibility and extensibility to model systems with higher complexity, interconnections and components. The ability to convert models from Machinations to CadCAD can allow teams to quickly iterate and experiment with different system configurations and parameters on machinations and switch to Python’s rich ecosystem of libraries and tools to extend the model and conduct further analysis and simulations. By understanding the anatomy of both systems, teams can quickly and easily convert their models and explore the behavior of their systems in ways that were previously not possible.
Hope this helped :D Reach out if you have any suggestions or feedback.
I help web3 projects design and model their token economies. Feel free to reach out on Twitter: @RowRowRoUrBoat.
Big thanks to Pravar Joshi (@0xPravar) for helping me with the research and giving feedback to refine this post.
