Neo takes the Red Pill: Part 3
Objective: To contrast problem solving in AI with traditional software.
Storytelling can stand in for experience to build your intuition. This simple story is intended to help the lay person understand how thinking about traditional software projects is different from thinking about AI projects.
This fictitious story is about a project at a bank. It is based on one of the most pervasive applications of AI — handwritten digit recognition in bank checks. As a fact, LeNet-5 neural network has been used to read 10–20% of all bank checks in US, and is based on a paper from way back in 1998.
Great Expectations
A long time ago, Mr. Smith, a top level executive at a big bank, learns about a software project being built at a competitor wherein their ATM machine could automagically read the handwritten digits on a bank check. Amazed at the cost savings and customer convenience enabled by the feature, Mr. Smith rushes to his software partner company, AcmeSoft.
Neo, a detail oriented, product manager, is assigned to the product. He quickly gets in execution mode, and wants to have a working prototype ready for Mr. Smith ASAP. Working with his ace software engineering team, Neo describes a proof-of-concept user story to recognize a single handwritten digit — 7 — provided as an raw image on a white background.
The engineering lead, Jessica, calls a brainstorming meeting with her team.
Traditional Software Design
The super agile, engineering team does fast analysis to break the problem into smaller ones. They determine that a great rule to detect the digit 7.
A horizontal line that could between 0 to 5 degrees tilt, meeting a vertical line between 0 and 35 degree tilt.
They promptly write some code and within a few days, whip up a prototype that they present to the product manager.
As usual, product managers have a knack for breaking stuff that seemed to work during development and QA.
Neo quickly stumbles upon new use cases.
Jessica is glad to have sought early feedback. Given the new use cases, her team goes back to the drawing board, establishes a large number of rules and logic to cover a myriad use cases for recognizing the digit 7, plus other digits, and completes a working prototype in record time.
Disillusionment
Neo calls a meeting with Mr. Smith to show the new IP from AcmeSoft. The prototype consists of a state-of-art, digit recognizing system built from a complex interplay of rules and logic involving computer vision algorithms, geometry, trigonometry and statistics.
At the presentation to the bank, Mr. Smith brings a database of images of handwritten digits available from scans of existing bank checks. The system seems to be working! Unusually, the CEO of the big bank happens to be around and Mr. Smith invites him to check out some cool new tech.
Intrigued by the innovation, the CEO writes the following check as a test.
It fails. The system recognizes the digits above as “7747”. As per the rules around angles and lines, the 7s, 2s and 1s look very similar to the system. Such errors would be catastrophic for a bank. Human intervention would be required to fix it, and that would make the project pointless.
Mr. Smith has a look on his face that Neo finds unfathomable.
Product Manager — Perfect Storm
The sales account manager escalates the issue on the business side. Neo escalates the issue to the VP of Engineering, and expresses surprise that the engineering team missed a use case that a non-technical, bank division head could uncover.
The VP calls a meeting with Jessica and team to discuss.
Jessica pushes back.
Enough is enough. Our team clearly documented the assumptions of the system. The system works beautifully as per spec! The rest is either user error (customers need to improve their handwriting skills), or the product manager must provide better product requirements.
The next day, Neo gets calls from his CEO, from his boss, from the sales head, and the VP of Engineering. There is cross-functional disappointment. The customer is threatening to go to a competitor.
Neo is stuck in the eye of the storm. There are too many human beings with too many ways to write the digit 7. His gift of glib can only postpone the inevitable truth:
The problem has too many exceptions for logic-based rules.
Yet, the customer says that another bank has figured it out. As the impostor syndrome sets in, Neo is discouraged.
The problem is untenable. Maybe I am in the wrong space; maybe I am not technical enough anymore. Do I need a change in career?
It’s that famous scene from the movie — the blue pill or the red pill.
AI and the Essence of 7
Traditional software design must foresee ALL possible inputs. In contrast, the success of AI software is measured by how well it handles previously unseen data.
For the neural network used by banks today, 60000 images of 10 handwritten digits were SHOWN to AI. Yes, the same images that you would see in a digital photograph, represented by a grid of numbers between 0 and 255 representing intensity of light. The AI extracts patterns from the experience of 60K images to build its intuition for handwritten digits. Such a large amount of training data is still tiny compared to all the possible ways human beings can write 10 digits.
The AI extracts the essence of 7 by experiencing what is a ‘7’ and what is not a ‘7’, across 60000 images of handwritten digits.
Humans show, AI learns.
Humans do not show describe what makes the image of 7 the concept of a 7. AI learns by itself.
Just like teaching a child to play catch.
In the new new world of AI-based software, rule specifications are OUT, experience is IN. Describing all ways a 7 could be written is OUT, and actually untenable. Providing a rich training dataset with enough different 7 and other digits is IN.
The Data Scientists’ Hippocratic Oath
One unbendable principle of professional integrity for data scientists:
Keep data used for training AI separate from data used for testing how well the AI handles unseen data.
In the case of handwritten checks, as a fact, 10K images of handwritten digits were kept separate from the 60K images used for training.
Increasingly, as part of product ownership, product managers are being tasked to find and own the training and test sets used by the engineering team. This is because that data set is the new oil. And it is even more important for the product owner to know the market and the domain, than the product itself.
In the next post, using very basic, pre-high school algebra, we will review what the “essence of 7” looks like to AI.