Self-Assembling and Self-Organizing AI- The future of Machine Learning?
This can change the way we approach Deep Learning and Artificial Intelligence
With Transformers, Deep Learning, and Large Language Models like PaLM, GPT-3, and DALLE-2, it can be hard to find other kinds of Machine Learning to be impressed by. In this article, I will be sharing a variant of AI that has a lot of potential to shape the future of Machine Learning and Tech as a whole. In this article, I will be covering Self-Organizing AI, a unique and powerful idea. I will primarily be referring to the writeup, The Future of Artificial Intelligence is Self-Organizing and Self-Assembling by the amazing ML Researcher Sebastian Risi. He also has an amazing interview on Yannic Kilcher’s Youtube channel, which I would suggest listening to here.
Why Self-Organizing AI? What is the current problem with our Deep Learning Systems?
Before getting into an idea, it is helpful to understand the context surrounding the idea. What is the problem that your idea is solving, and where are the current solutions failing? This approach can help you develop a deeper appreciation for the concept and will make your learning process more fun. More details on this learning approach are in the article below-
So what is wrong with our current Deep Learning Systems? One of the biggest problems we see in our current systems is how fragile they can be. One research paper showed that we can fool State of The Art image classification networks just by changing just one pixel. Deep reinforcement learning agents (Gleave et al. 2019) completely break down if confronted with an unknown player strategy and even the vaunted Large Language Models are susceptible to delusion (caused by providing misleading text input).
Self Organizing Systems seek to solve this issue. As introduced in the writeup-
… combines ideas from deep learning with ideas from self-organization and collective systems. In this first post, we’ll look at some of the developed approaches and the domains they have been applied to, ranging from growing soft robots and Minecraft machines to self-assembling modular robots, and creating more resilient and adaptive reinforcement learning agents. The merger of these ideas could ultimately allow our AI systems to escape their current limitations such as being brittle, rigid, and not being able to deal with novel situations.
Google’s amazing Pathways system, which can be used to handle a variety of tasks on a single network using multi-modal learning, sparse activation, and training for multiple tasks (all powerful ideas), can possibly be used to overcome these problems. However, it has over half a trillion parameters. Sebastian presents a solution that is much more cost-effective.
Now that you’re aware of why self-organization can be useful, let’s get into what it is and how it works.
Self Organizing Systems
Risi’s work takes inspiration from nature (really interesting how so many major breakthroughs come from here). They were interested in how, “groups with millions or even trillions of elements can self-assemble into complex forms based only on local interactions and display, what is called, a collective type of intelligence. For example, ants can join to create bridges or rafts to navigate difficult terrain, termites can build nests several meters high without an externally imposed plan, and thousands of bees work together as an integrated whole to make accurate decisions on when to search for food or a new nest.”
What is fascinating about these large-scale organizations are done by following very simple rules, without a grand external blueprint. Contrast this with human-engineered solutions, where we have a top-down approach, and things are built from rigorously laid out plans. This can make organization easier but is inherently less flexible and thus more fragile to changing circumstances. This is one of the reasons that Deep Learning Networks can be fragile. They will train for the data they are presented with but will not be able to adapt to new kinds of problems. This is also why ensembles outperform singular networks (and why Random Forests are my favorite model).
Self-Organizing systems don’t face the same problems. “Self-organizing systems are made out of many components that are highly interconnected. The absence of any centralized control allows them to quickly adjust to new stimuli and changing environmental conditions. Additionally, because these collective intelligence systems are made of many simpler individuals, they have in-built redundancy with a high degree of resilience and robustness. Individuals in this collective system can fail, without the overall system breaking down.” Take a look at the video above, where a robot can regrow an entire lost limb to start moving. Contrast this with our Deep Learning networks, which break when we change one pixel.
Such an approach also leads to much better generalization when dealing with domains never encountered before. “We found that starting from completely random weights, evolved Hebbian rules enable an agent to navigate a dynamic 2D-pixel environment; likewise, the approach also allows a simulated 3D quadruped to learn how to walk while adapting to some morphological damage not seen during training and in the absence of any explicit reward or error signal in less than 100 timesteps.”
One of the very interesting ideas I found while looking into this idea was how these simple rules could encode very potentially complex behaviors. These complex behaviors would be the result of the many simple interactions between the millions of individual members. Using this kind of encoding could be amazing for all kinds of compression algorithms, and is something I will be looking into. If you know anything interesting, you know how to reach me.
Controlling the Learning of these methods
One of the biggest challenges of such an approach is actually directing the learning process in the direction you want. Remember, if something can explore a large search space, it will explore that. Which can be very expensive. Especially when we have a lot of organisms interacting. Predicting what outcome we will get from our system (before running it) is impossible.
While you can’t tell your system what to do, you can guide your AI towards certain outcomes using nudges (similar to Evolution or RL).
Guiding a swarm system can only be done as a shepherd would drive a herd: by applying force at crucial leverage points, and by subverting the natural tendencies of the system to new ends.
This is speculation on my part, but I think implementing something like the attention mechanism to your data might be useful. As I’ve covered here, attention gives vision networks ‘a global view’ of an image. Something like that might be useful in designing a nudge(r), that will guide the learning outcome towards a direction (like a sheepdog). I’d be interested in discussing this with any of you.
For growing towards specific target structures, researchers have found success integrating gradient-based approaches alongside the Cellular Automata -creating NCA, or Neural Cellular Automata. In NCAs a neural network learns local rules based on communicating with its local neighbors, updating its internal state. This allows us to avoid hardcoded CAs but it leads to our next problem.
Training Costs
Gradients are awesome. However, “using gradient descent-based approaches requires backpropagating the gradients through the whole sequence of developmental steps. Thus this process becomes increasingly infeasible in terms of memory requirements with an increase in developmental steps.” These costs can get out of control. Also-
NCA is only trained to grow a given structure and not to discover new structures
To work around this, one such alternative is, “searching for self-organized patterns in complex dynamical systems are more open-ended search methods such as quality diversity (QD) (Pugh et al. 2016) and intrinsically-motivated learning approaches (Baranes & Oudeyer 2013). In these methods, the idea is to not search for one particular solution (as is typical in machine learning) but instead try to find a maximally diverse set of outcomes.” This reminds me of how Evolutionary Algorithms work to maximize solutions for a fitness function. Google AI was able to create new ML algorithms using EAs.
Sebastian’s writeup also mentions possibly using RNN-based controllers. The challenge with these was that they would get stuck in local optima. Adding an element of randomness (randomly jumping to a new point) or even momentum can help tackle this issue. Furthermore, it has been shown that Sparsity can be used to reduce training costs by 8x while maintaining performance. This might be useful going forward.
I’m very excited about the future of this approach. This can have a lot of implications going forward, and I will definitely be covering them. If you’re a Machine Learning person, you should definitely connect with me across various platforms to not miss out. Links at the end of this article.
If you’re looking to get into ML, this article gives you a step-by-step plan to develop proficiency in Machine Learning. It uses FREE resources. For the best results, pair this with my newsletter, Tech Made Simple. More information below.
For Machine Learning a base in Software Engineering, Math, and Computer Science is crucial. It will help you conceptualize, build, and optimize your ML. My daily newsletter, Technology Made Simple covers topics in Algorithm Design, Math, Recent Events in Tech, Software Engineering, and much more to make you a better developer. I am currently running a 20% discount for a WHOLE YEAR, so make sure to check it out.
I created Technology Made Simple using new techniques discovered through tutoring multiple people into top tech firms. The newsletter is designed to help you succeed, saving you from hours wasted on the Leetcode grind. I have a 100% satisfaction policy, so you can try it out at no risk to you. You can read the FAQs and find out more here
Feel free to reach out if you have any interesting jobs/projects/ideas for me as well. Always happy to hear you out.
Reach out to me
Use the links below to check out my other content, learn more about tutoring, or just say hi. Check out the free Robinhood referral link. We both get a free stock (you don’t have to put any money), and there is no risk to you. So not using it is just losing free money.
Check out my other articles on Medium. : https://rb.gy/zn1aiu
My YouTube: https://rb.gy/88iwdd
Reach out to me on LinkedIn. Let’s connect: https://rb.gy/m5ok2y
My Instagram: https://rb.gy/gmvuy9
My Twitter: https://twitter.com/Machine01776819
If you’re looking to build a career in tech: https://codinginterviewsmadesimple.substack.com/
Get a free stock on Robinhood: https://join.robinhood.com/fnud75
