Hardware🢂 Software🢂 Learnware
Looking Into the Economy and Infrastructure of a New Type of Software
“Neural networks … represent the beginning of a fundamental shift in how we write software” (Andrej Karpathy, Software 2.0, Medium)
One of my joys as a student in multidisciplinary studies was attending a class and having my knowledge of biology reinforced by the concepts I learned in my computer science lecture (or vice versa). This week I experienced a similar joy when I read Andrej Karpathy’s great post “Software 2.0,” where he analyzes the difference between the new type of software “Software 2.0” vs the old type “Software 1.0”. Karpathy reaches a very similar conclusion to concepts mentioned here, although his point-of-view is quite different.
In order to elaborate, let’s examine how the rise of this new type of software could affect the software industry and market.
Software vs Learnware
Here are some of the major differences between the old software and the new, adaptive software described in Karpathy’s post, which I will refer to as “Learnware.”. (I decided to call this new type of software “Learnware” because it’s based on learning systems):
Learnware affecting Software Market
You have probably noticed the ways in which the software market has started to change, but some of the changes are less apparent than others…
Collaborative Processing and Service Orientation
In software we are used to collaborating with each other via libraries and APIs (Application Programming Interfaces). Every interface’s tasks need to be well-defined. It’s usually very clear to the user what is done when calling the interface and how.
For example, imagine two restaurants that do not have menus:
At the first restaurant, aptly if curiously named, “Old Software Type,” the guests need to arrive with exact instructions for the meal they want. They must show up with a detailed recipe to ensure they receive the right food.
At the second, newer restaurant, called “Learnware,” the guests make more abstract requests like, “I’m sad, give me something that would make me happy” and the chef is able to generate an optimal meal on the spot.
It is clear that some of the guests of the first restaurant would prefer to save the hassle of writing a recipe and just cook at home, right ? That, however, that is not as possible for the guests at the second restaurant.
Such a baseline very often leads the API user to take an “I know best what I need so I’ll just code it myself” approach (which is usually the wrong thing to do, but that’s a different story). In Learnware, the “I’ll do it myself” approach makes even less sense, since by simply defining what the user needs (eg. “find a cat in the picture”), the developer is still not close to a solution that s/he can code by his or herself (unlike a clearly defined task, “draw a rectangle with corners x1,y1,x2,y2” in the old type of software).
Due to such profound differences, I’d expect more and more tools and platforms for collaborative processing to arrive in the near future.
Main affecting factors from table above: F, G, H
Learnware initiatives
We can describe most business activities that incorporate new software as “Learnware Initiatives”. These initiatives include new applications based on visual and speech recognition, visual generation, speech synthesis, robotics, gaming, translation, decision making, and more.
Main affecting factors from table above: A
Job Market
Clearly the presence of Learnware will change the job market drastically. No matter how companies brand the job of feeding knowledge into Learnware (programmer,data scientist, quant), as Learnware gets more common and simpler to train, the required skillset to work with this new software will evolve. I would expect that the future requirements for Learnware developer roles will be totally different from those required of the old type of software programmers.
Main affecting factors from table above: B, C, D
Techniques for Feeding Knowledge
As a software developer, you use languages like C++ , Java, or Python for programming knowledge into software. The same techniques are currently used for building and training Learnware. However, while regular software programming languages are designed to best describe, manage, and maintain various sets of instructions, the key to programming knowledge in Learnware is accurately describing an optimal data flow graph. Therefore, I’m not sure using the older type of programming languages is the optimal method for developing Learnware.
Main affecting factors from table above: C, D
Software Libraries for Building Learnware
We’ve already seen these newer software libraries around for several years: Tensorflow, Pytorch, Keras, Theano, MXNet…
Main affecting factors from table above: B, C, D
Specialized Hardware
In the old type of software, which had a large instruction set, the introduction of new hardware required adjustments to the compilation and code levels. In contrast, in Learnware, the usage of new hardware is much more transparent. The race for dedicated hardware for Learnware has been ongoing for several years , with NVidia in the forefront.
Main affecting factors from table above: B, E
Tools for Looking into the Black Box
We need tools for looking into Learnware Black Boxes. These tools are important for several reasons, but primarily for development purposes. We will probably be able to train our Learnware better if we understand how each Learnware reaches a decision. Another reason we need these tools is to encourage the acceptance of Learnware from a social point-of-view. For example, European member states are expected to adopt new legislation that governs how AI’s decisions can be challenged if they appear unfair or arbitrary. Early drafts of the General Data Protection Regulation (GDPR) enshrined what is called a “right to explanation” in law.
Main affecting factors from table above: H
Taking Part in the Revolution
I assume many readers are involved, in some way or another, in activities that fit into the sections above (and if you are not, you are missing out on a lot of fun!). We at Dopamine.ai are working intensively on solutions for collaborative processing. If you are interested in reading more or receiving updates on our project, please subscribe here.
