Sep 6, 2022Spotlight on the bias-variance trade-offWhy more data is not always better TL;DR The bias-variance trade-off explains why smaller models– having fewer parameters to train on– can be better. It is intuitive that more data is better. A model trained on more data should be more accurate than a model trained on less data, right…Machine Learning6 min readMachine Learning6 min read
Jan 3, 2022Even linear regression can escape the bias-variance tradeoffLinear regression is like deep learning when you have a massive number of mis-specified features TL;DR The “impossible” phenomenon of minimizing test error with “too many” parameters does not just appear in deep neural nets– it also appears in linear regression. A recent paper sheds light on why, with a key role played by misspecified features. UPDATE: There is now a video version of this…Machine Learning12 min readMachine Learning12 min read
Nov 8, 2021How machine learning models are embedded in social chessFinding stakeholders with impactful feedback loops If there were a mistake in your model’s inputs, would real-world usage amplify or dampen that mistake? This discussion of model stakeholders and feedback loops can help you assess that. When a machine learning model learns to play a game like chess, the model does not just look for the…Machine Learning14 min readMachine Learning14 min read
Nov 2, 2021When model builders can — and can’t — rest on their laurelsJennifer Lopez’ dress was exogenous change while image spam was endogenous change Actress Jennifer Lopez broke Google Search in 2000. She wore a unique green dress to the Grammy Awards, but when users tried to search for the dress, they could not find an image of the dress. This failure spurred the creation of Google Image Search in 2001 [1]. After the…Machine Learning10 min readMachine Learning10 min read
Oct 23, 2021When the irrelevant is made relevantBeyond “distribution shift” in machine learning I have argued that even the most well-trained machine learning model can be “broken.” But that is non-standard terminology — what does it really mean to “break” a model? In short, it is a change that obviates a model’s goal. The model was attaining its goal, and then something changed…Machine Learning14 min readMachine Learning14 min read
Oct 8, 2021When adaptability breaks machine learning modelsWhat Facebook could learn from New Zealand mud snails I lied in my previous post — or rather oversimplified. You don’t really need creativity to break a machine learning model. In fact, any adaptive system is capable of breaking a model of its behavior. I will illustrate this point with the example of cancer evolving to escape treatments. I…Machine Learning6 min readMachine Learning6 min read
Sep 27, 2021When creativity breaks machine learning modelsWhat Facebook, Wells Fargo Bank, and email spam blockers have in common People can be creative, imaginative, irreverent and inventive. But you would never guess that if you looked at our scientific models of people. Our models treat people like inert physical objects, the same as billiard balls or dice. I know because I worked for more than a decade deep in…Machine Learning10 min readMachine Learning10 min read
