The Relationship between combinatorics, machine learning and artificial neural networks
Let’s first identify components Combinatorics to know how to be employed in ML and ANNs.
Combinatorics arise in many areas of pure mathematics, notably in algebra, probability theory, topology, and geometry, and combinatorics also has many applications in mathematical optimization, computer science, ergodic theory and statistical physics.
In the later twentieth century, however, powerful and general Theoretical methods were developed, making combinatorics into an independent branch of mathematics in its own right. One of the oldest and most accessible parts of combinatorics is graph theory, which also has numerous natural connections to other areas. Combinatorics is used frequently in computer science to obtain formulas and estimates in the analysis of algorithms.
Sub Fields of combinatorics:
1- Graph theory > related to information theory.
2- Combinatorics optimization > related to operations research, algorithm theory Machine Learning, Image Analysis and ANNs
3- Dynamical systems > related to graph dynamical system.
Dynamical systems + ML = RNNs
4- Design theory > connections to coding theory and geometric systems
5- Probabilistic combinatorics > related to statistical physics and analysis of algorithms in computer science, as well as classical probability, additive and probabilistic number theory, the area recently grew to become an independent field of combinatorics.
6- Algebraic combinatorics > related to group theory and representation theory
Machine Learning is closely related to (and often overlaps with) computational statistics, which also focuses on prediction-making through the use of computers. It has strong ties to mathematical optimization,
So Machine Learning needs >> advanced study in the field of mathematical optimization on discrete and combinatorics objects. and Optimization is started as a part of combinatorics and graph theory, but is now viewed as a branch of applied mathematics and computer science, related to operations research, algorithm theory and computational complexity theory and using tools from complex analysis and probability theory and also Combinatorics Optimization
Basically, combinatorics studies countable sets. Probability uses combinatorics to assign probability (value between 0 & 1) to events. Statistics takes sample and compare them to probability models.
Those fields of study have massive influence in many other fields. They are key in Machine Learning and Data Science in general.
In Fact, Real-world machine learning tasks frequently involve combinatorial structure. How model, infer or predict with graphs, matchings, hierarchies, informative subsets or other discrete structure underlying the data
>> ML + Combinatorics = Advance ML
Artificial neural networks is the most obvious two I can think of are feature selection and parameter optimization in feed-forward artificial neural networks.
In feature selection you’re trying to find an optimal combination of features to use in your dataset from a finite possible selection. Greedy algorithms, meta-heuristics and information gain filtering are all common approaches.
Back-propagation is an algorithm used in artificial neural networks to find a near-optimal set of weights/parameters. It’s incredibly effective.
There is a use App in this field >>> Integrated automated system for combinatorial data analysis and topological data analysis
Combinatorics optimization + neural networks + reinforcement learning (ML + control systems) = Neural Combinatorial Optimization with Reinforcement Learning
Reference:
https://vincentlauzon.com/tag/mathematics/
