Remember those flashing, vibrating banner ads you used to see on lyrics sites in the mid 2000s that nearly gave you a seizure while insisting that you had won a prize? Did they make you want to punch someone?

Yeah, sorry about that. I was running one of those sites — SongLyrics.com. If you’ve looked up lyrics, you’ve probably been there. We really didn’t want to piss you off, but we had to pay the bills, you know? And in those days, the unscrupulous ad networks were offering the best rates.

But you don’t see those kinds of ads anymore. The experience on sites like SongLyrics isn’t perfect, but it’s a lot better. What happened?

The pithiest explanation for what’s happening comes from Marc Andreessen. “Software,” he wrote in 2011, “is eating the world.”

With that evocative phrase, Andreessen put his finger on what may be the most illuminating trend of the information revolution. The idea is simple. Take any problem anywhere within the human enterprise. If software can do something better than the old methods, then inevitably, no matter how recalcitrant and resistant to change the incumbent powers may be, software-based methods will displace traditional methods. For conventional companies, when software comes knocking, the choice is to evolve or die. There’s no third way.

“Companies in every industry need to assume that a software revolution is coming,” Andreessen says. “This includes even industries that are software-based today.”

Which brings us to advertising. Andreessen’s principle is well at work in the $500 billion global ad economy. Traditional forms of advertising like print news and yellow pages are dead or dying. Even media channels that are already powered by software, like the display ads you see on SongLyrics, are being consumed by newer, smarter, and more dynamic software-powered methods. And for the three key players in the advertising economy — consumers, publishers, and advertisers — the world is getting better.

Ad targeting the old way

Lt. Dan: Where the hell you think you’re going?
Forrest Gump: Get Bubba.
Lt. Dan: I got an airstrike inbound right now, they’re gonna nape the whole area! Gump you stay here goddammit that’s an order!
Forrest: I gotta find Bubba!
[Runs back into the jungle]

In the advertising world, “targeting” is the practice of delivering an ad to a desired audience. In the old days, targeting was done with about as much precision as the napalm bombs that Lieutenant Dan is worried about. It was wildly imprecise, and so to hit your target, the best you could do was drop a bomb on a wide area where you had reason to believe he’d be hanging out. You knew that you’d waste a lot of ordnance blowing up the whole area, but in the old days the napalm bomb was the best you could do. Today, software-powered advertising enables targeting at a level of precision that would have been unimaginable twenty years ago. This is the key to understanding the improving relationship between consumers, publishers, and advertisers.

To see how this works, let’s go back in time. It’s a sunny afternoon in 1994. Ace of Base is dominating the radio, Forrest Gump is running all over the world, and you’ve got 500 free hours of AOL burning a hole in your pocket. You’re the marketer in charge of selling the Venus brand of leg razors. You have an ad budget, and you’re looking to move some product.

If it didn’t cost you anything to tell consumers about your product, you’d tell everyone under the sun about your mango-infused lotion strip. But there’s no free lunch. Audience attention is valuable, and the publishers who control it aren’t going to give it away for free. You’ll have to spend money for that advertising space. And since you have a limited budget, you’ll want to get as much bang for your buck as possible. That means you’ll want your ads to appear in front of the people most likely to buy. What you don’t want is to spend good money showing your ad to, for example, teenage boys. They’re not going to buy, no matter how many times you tell them about your lotion strip.

From your research, you have detailed demographic specifications for your ideal audience — characteristics like gender, age, and income. Your major limitation (read: opportunity) is the technology to differentially target the individual consumers that maximally meet your specifications. The advertising media available to you in 1994 — things like billboards, television, magazines, and newspapers — do not provide the means to target individual consumers. Your strategy, by necessity, will be to blast your message at mass audiences, knowing that contained within those groups are some desirable consumers. As with Lieutenant Dan’s airstrike, there’s going to be a lot of collateral damage. Alas, it’s 1994 and this is the best you can do.

Let’s say you know that People magazine has about one million readers per issue, sixty-five percent of which are female. If you buy an ad, that gives you a shot at 650k women. Not bad. But to secure that ad, you’re also paying for 350k men, most of whom are of no value to you. And of those 650k women, how many do you really have a chance at converting? Many could be eliminated based on age and socioeconomic status, for starters. Wouldn’t it be nice if you could filter out all those mismatches from your ad buy? With traditional media, that’s not possible. It’s an all or none deal.

The publisher, People magazine, suffers too. Since People has no choice but to bundle and sell its ad impressions wholesale, it effectively sells a large portion of its inventory for less than those impressions are worth on the open market. If People could sell you just the 650k women, and sell the 350k men to some other advertiser who can get more value from the segment, People would bring in more revenue on the exact same inventory.

What about our third key player, the consumer? Suppose that Jack, a 22-year-old male from Chicago, picks up a copy of People. He’s going to see your Venus razor ad. But he’s not going to care, and he’s certainly not going to buy. His magazine reading experience was degraded, even if by just a fraction, because of this wasted use of his attention. And since you, the advertiser, didn’t pay People very well for that attention, the magazine is going to have to make up for it by showing Jack more ads, further degrading his reading experience.

This is ad targeting like a dumb bomb. It’s not great for anybody. Software can do better.

Ad targeting the programmatic way

“The two major trends in the ad world right now are the rapid consumer shift toward mobile usage, and the industry shift to programmatic buying.”
—Kevin Weil, VP, Twitter

“That’s absolutely right. Those are the only trends in the ad world that matter.”
—Antonio Garcia, the product manager who created Facebook Exchange

Conventional ad targeting is dumb. Software-powered ad targeting can be shockingly precise. The key difference is that conventional targeting happens at the level of the mass audience, whereas targeting in the digital world happens at the level of the individual impression — just the right ad, for just the right person, at just the right moment.

The word for this super-precise delivery of advertising is “programmatic”. This term has a loose definition, but the general idea is that programmatic advertising uses increasingly sophisticated software systems to let advertisers buy and publishers sell highly-targeted ad impressions automatically, in real time. More specifically, those software systems recognize and learn about individual consumers as those consumers move through the digital universe, and use that information, on both the aggregate and individual levels, to inform targeting decisions. The term has associations with specific technologies like ad exchanges, real-time bidding, and machine learning, but for our purposes, most important are the general ideas that programmatic advertising is largely automated and highly targeted.

What this means is that no longer must ad impressions be bought and sold wholesale, as is the case with print publications. In the digital world, each impression is generated on the fly, giving us the opportunity to choose the highest-value ad to serve every time.

So for example, let’s say that Jack, now a middle-aged man, visits People.com. You, the advertiser for Venus, know that you don’t want to waste your ad bucks on him. Now you don’t have to. Using cookies to track an individual’s web behavior in an anonymous and automatic way, it’s possible to build a profile of an individual user. Jack has recently visited the website of the Atlanta Braves, a review of Eminem’s new album on Pitchfork, and the product site for the Ford F-150. Using this information, an ad tech company can infer a wealth of demographic data about this consumer. When Jack shows up at People.com, the algorithms determine that he’s not a good target for a moisturizing leg razor. He might instead be served an ad for, say, Old Spice.

Compare this to 1994. It’s a better experience for Jack, more money for People.com, and a more efficient use of your Venus ad budget. All we do is win, win, win.

Software utilizes data that goes beyond demographics. For example, on the internet, we can often see when a consumer expresses interest in a particular product, and use that information to retarget him later. Suppose that Jack goes and looks at a winter jacket on the Patagonia website, but doesn’t buy. The next day he shows up at People.com. To Patagonia, Jack is a golden prospect. Software makes it possible for Patagonia to find this consumer and serve him with a custom ad — maybe one promoting the same coat that he was looking at the day before.

The advertiser and the publisher are happy, because they’re making money. But Jack might be the biggest winner of all. He gets People.com for free. His only cost is the nominal inconvenience of having to see a few ads. But even that is pretty good, because those ads are surprisingly relevant to him — they might even help him solve his jacket problem.

Retargeting is extremely powerful, but even that is just the beginning of what we can do with data. Let’s focus in on one key idea: identity.

You do all kinds of things online. Email, browse, search, use apps, maybe tweet, and make purchases. That creates an enormous digital trail. As more of that data gets consolidated under a single unique identity, it increasingly models you: who you are, what you believe, what you care about, and what you’ve done.

I’ll use myself as an example. For email, I use Gmail. My browser is Chrome. My mobile operating system is Android. For maps, I use Google Maps. I read books on Google Play, I use Hangouts for text messaging, I use Google Calendar for my schedule, and I back up my files on Google Drive. Oh yeah, and search — I only use Bing when I want to make fun of it. You could say that I’m a Google fan. And since I use all of these products while logged in to my Google account, Mountain View knows quite a bit about me. They probably know a lot about you, too.

And Google can use that information, along with its world-leading machine intelligence expertise, to find every last statistically significant correlation between who I am and what I might want.

Gimme what I don’t know I want.
— Justin Timberlake

We are not at the end of the software revolution. We’re just getting warmed up. Hardware and software innovations are emerging at a pace which, if we step back and look at a look a long-term timeline, is staggering. The data that these services collect will model our interests, our beliefs, and our very personalities. And that’s a good thing, because it allows the ad business to give me what I do and don’t know I want.

But this talk of tracking our identity for the purposes of ad targeting shines the spotlight on the elephant in the room: ethics. This is an enormously important issue, and one not to be brushed aside. However, a thorough discussion is beyond the scope of this article.

For now, it must suffice to say that the ethics of tracking is a composite issue. There are risks and dangers, but there is also enormous value to be gained. We can imagine a future like 1984, in which humans are surveilled and oppressed in their every action. Or we may see ourselves floating around in stupefied bliss like the humans of Wall-E. But we can also imagine a future like Star Trek, in which intelligent technology assists us and enriches our lives. That happens to be the future that the two most powerful people in this universe, Larry and Sergey, have dreamt about since before founding Google.

Our task going forward is to intelligently, consciously, and transparently decide what kinds of norms and laws we’ll enact in order to maximize the good and minimize the bad.

Now this is happening.

Software-powered advertising will never make up 100% of the enormous $500 billion global advertising economy. Blimps will fly. Bill stickers will stick bills. Where there’s attention to be bought, someone will buy it.

But software’s consumption of advertising is well underway, and the data shows it. Digital advertising in the US is now a $42 billion business, accounting for almost a quarter of total US ad spend in 2013. Digital is growing at 15% per year, faster than any other category.

Print advertising, comprising newspapers and magazines but excluding online ad revenues, continues its steady decline at the hands of digital.

Television advertising (cable and broadcast combined) is the last big ‘traditional’ medium still going strong. It is double the size of digital advertising and is itself growing, though at a much slower rate than digital. However, the television business is changing. Consumers are switching to on-demand services like Netflix, and are increasingly consuming video content on connected devices. At the same time, ad tech is moving into the living room as groups like Xaxis and Comcast establish the infrastructure for targeted TV ad buying.

The bottom line is that TV will go programmatic. It’s only question of when. A report from Nielsen and Simulmedia projects that nearly half of TV ad buying will be fully programmatic by 2020.

“We see programmatic as the future. All channels can be bought programmatically.”
—Christina Beaumeier, VP, Xaxis

With all channels going programmatic, we’re moving towards the promised land of ad targeting: an advertiser makes one buy, which finds her audience across all media channels, devices, and publishers, with no wasted buy and no target missed. This is the future of advertising.

Done right, it benefits everyone. Advertisers maximize ROI. Publishers maximize yield, and use that revenue to build great user experiences. The deal that the publisher makes with the consumer is this: I’ll build an awesome product and let you use it for free, if in exchange you let me responsibly track your data and use it to show you some ads.

That’s the deal, and I like it.