What separates us from AI, Part 3: Get Meta (recursive automation, not automated recursion)
In the first post of this series, I hinted that something (at level F4) was creating those F3 meta-learners. What is it?
It’s us! We are F4. In fact, we’re also F3, and F2, and F1. We’re a versatile lot — we can play any of these roles.
When we do arithmetic in our heads or recognize our friends in photos, we’re playing the role of F1. When we create an Excel spreadsheet or a python program that does some simple math, we’re playing the role of F2, creating a software F1. And when we implement an architecture and optimization routine for some model in Tensorflow or Pytorch, we’re playing the role of F3, creating a software F2.
The machine learning hierarchy was built by us, bottom-up, slowly replacing and automating a role that we ourselves were playing. We first started with F0 data, and implemented F1 ‘functions’ by writing software that took input and generated output.
The bulk of machine learning was taking this one level higher — instead of us writing the F1 functions by hand, we wrote F2 learners by hand and generated F1 functions through some optimization process. Then, in a few cases, instead of writing F2 learners by hand, we implemented F3 meta-learners by hand that acted on F2 learners, which would eventually generate F1 functions. In that final step, we’re playing the role of F4, birthing F3.
This didn’t happen overnight, and in no sense is that process ‘complete’. We still write F1 functions by hand, along with the higher operators. But this is one of key differences between humans and modern ML/AI — we’re capable of going higher and automating ourselves out of a role.
We’re capable, in other words, of meta-modeling, of going from F0 to F1, then F1 to F2, then F2 to F3, and so on. Let’s give this meta-operator a special name…FΩ.
FΩ actually isn’t a horizontal level of the hierarchy at all, like the others, but an operator that sits vertically within it. And the practice of machine learning is us implementing the FΩ operator over and over again (whether we go beyond the top layer, or search more broadly in an existing layer from the layer below it) to find interesting/useful generators.
And as far we we know, FΩ doesn’t exist in software, only within ourselves. There’s no software that churns out interesting ensemble methods…as far as we know. But human researchers, as a group, do that all day long.
So what’s the significance? What could FΩ have to do with efficient search?
Was this worth reading? If so, please clap below.
