The Surprisingly Effective Genetic Approach to Feature Selection
A deeper look at an underrated method
Genetic and evolutionary algorithms are often bashed as not being good enough to compete with the capabilities of neural networks, and for the most part, it’s true, which is why the industry seldom even considers these types of algorithms. They are too general whereas there are other specific solutions that are designed for specific problems, and require too much computing power. But there is one fascinating application of genetic algorithms to feature selection, an important part of machine learning.
We’ll explore the genetic/evolutionary model of thinking, how that approach can be applied to feature selection, and why it is effective, alongside diagrams and analogies.
In genetic algorithms, a population of candidate solutions, also known as individuals, creatures, or phenotypes, are evolved towards better solutions in an optimization problem. Each candidate has a set of properties that can be mutated and altered.
These properties can be represented as a binary string (a sequences of zeroes and ones), but there exist other encodings. In the case of feature selection, each individual represents one selection of features, and each ‘property’ represents one feature, which can be turned on or off (1…
