Going Peer-to-Peer: Dramatically Simplifying Programmatic Supply Chains
On February 27, 2018, Mike Goldin delivered this talk at the Blockchain Xplore educational conference at the Microsoft Conference Center in New York City. It covers his vision for the adChain project as it approaches its mainnet launch date in April, and the necessity of a public blockchain to realizing that vision.
Who here is an advertiser or someone who works on the demand side? Keep your hands up if you think you have ever bought a non-human impression. Okay, thank you. Now who is on the supply side? Keep your hands up if you think you have ever sold a non-human impression.
In web advertising, billions of dollars are stolen every year. Probably tens of billions. Perhaps a greater share of the overall ad spend than anyone in the industry is willing to admit, because if advertisers knew what was really going on they would stop spending so much money in this way and many of us might be without jobs.
Increasingly, advertisers do know what’s going on. Or at least they seem to be developing the same terrible feeling in the pits of their stomachs which I began to develop relatively early in my involvement with this space: that programmatic supply chains are completely untrustworthy, that the scaling up of web advertising since its inception cannot be fully accounted for, and that the proportion of that scale deriving from non-human traffic might be terrifyingly substantial.
There’s this thing going on right now where advertisers are telling their suppliers to shape up, because they don’t want their family-friendly brand content showing up on offensive properties, or next to offensive content. They’re gonna pull the budget! This sudden moral crisis spurred by the rise of fake news I think is the convenient malady which executives controlling advertising budgets can point to while standing on moral authority as to why they suddenly need to control their ad spend much more carefully. In reality, everyone has known for a long time that the web is a super gross and messed up place, but CMOs can use this moment as an excuse to reign in their ad spend dramatically without admitting that they have been throwing billions of dollars into the void for decades buying impressions for non-humans! Suddenly, they’re worried the boss is going to find out!
Most people here are probably not advertisers, most of us are downstreamers. Downstreamers as in downstream from the money. The advertisers toss dollars into the river, we all fish out as many as we can while leaving just enough for whoever’s downstream of us that we can credibly claim, should an audit ever be conducted, that we actually used the funds in service of sourcing impressions. In exchange for us letting some money float downstream to the next party, they pass back a little “unverified truth” about what they did with the money. At the end of this river are publishers. The publisher’s role is to return the ultimate proof that an impression was viewed by a real human, but nobody can really audit these attestations, and they’re all we have to rely on. This isn’t like telephone where we start with the truth and then lose it — we often will start with a falsehood, and then it’s still telephone anyway.
Because advertising is a volume business for downstreamers, however, a lot of the supply chain turns a blind eye to this. For them, it’s better to collect a 1% rent on a billion dollar spend than a million dollar spend, regardless of how inefficiently that spend is being allocated.
If you’re an executive with an ad budget, and you want more budget to increase the size of your corporate fiefdom, you need to prove that the million dollars you spent last year exceeded the KPIs and that untapped mindshare is being left on the table because the spend isn’t big enough. In a way, you don’t mind turning a blind eye either because it lets you provide inflated numbers to your CEO with plausible deniability. But you’re also nervous the CEO might find out what’s really going on, and then you either have to admit you knew you were throwing money into a pit all along, or pretend that you’re just incompetent.
Publishers drive all this. If you’re a publisher and you can make a penny per thousand impressions, and it only costs half a penny to buy a thousand impressions worth of traffic from a bot farm, and the odds of you getting caught are one in a billion, and the odds of facing any consequences if you are caught are miniscule because you’re in a country that isn’t going to be bothered to prosecute international cyber crime when the victims are mostly from countries your government has poor relationships with anyway and so to them it just looks like a nice cash transfer into their own economy… You’re going to do that all day.
A criminal (or “alternative entrepreneur”, you might say) can spin up some website that looks like a news website. They’ll even run a little daemon on the server that generates new content periodically throughout the day. These bots are pretty good! They can generate timely stories based on what’s happening on Twitter, say. But everything they generate is just artificial regurgitation of whatever else is trending on twitter, which means they artificially regurgitate a bunch of fake news. That’s a different problem. The point is you can generate a website that looks real but which in reality has zero human readership.
With this website you’re going to join some ad network, which might have like 5% real traffic, and this ad network is plugged into a few exchanges. The ad network isn’t going to audit you very carefully because they’re making money on volume. If you’re not egregious, you can play. If you as a buyer go through a US-based exchange, they’re completely unregulated and making money on volume also. They might do a cursory audit of whatever ad network wants to join their supply pool, and if it looks real enough and will add volume, they’re happy. Now advertisers are buying it.
And if you don’t like this version of the story, do you know how easy it is to get a Google AdSense account?
This is all about incentives, so let’s just briefly notice what user incentives look like here. You’re enjoying free content on the web, but these terrible advertisements are annoying you, slowing your computer down, chewing up your data, playing sounds, creeping you out by stalking you with reminders to buy whatever you left in your Amazon cart, mining cryptocurrency in your browser… Ads are malware. So you block them. You still get the web for free, but now your privacy isn’t being invaded and your computer doesn’t get hot when you go to forbes.com.
Ad blocking is an existential threat to web advertising. It happens on the client, so at best you’re gonna get endless cat and mouse if you want to devise a technical workaround. Blockers for the most part have the moral high ground, because ads are so invasive. So real humans are going to be running ad blockers, and you’re left with the bots. Nice market. Self-regulation has worked really well here.
This whole industry might be a house of cards. It’s potentially massively fraudulent.
I think regulation is a lazy answer. In web advertising in particular, a lot of regulation would be hard to enforce since the web is huge and international. Governments of the world aren’t going to come together to fight click fraud. But that’s okay, because technology exists now with which we can actually change the underlying incentive system of web advertising by completely reimaging how supply chains work.
Quick recap, what are the problems in supply chains today?
- Opacity means everyone can plausibly deny knowing that there was fraud in the supply they sold.
- Because there are so many black boxes between a buyer and a seller, advertisers can’t effectively audit publishers because they simply don’t know who they are.
The structure of programmatic ad commerce aligns incentives against social goods in this way. It makes fraud not just easy but rational. You don’t want to know how the sausage gets made in ad tech.
Let’s imagine a different world. What if advertisers always knew, with perfect certainty, exactly who they were buying from on a per-impression basis? What if publishers could be held accountable to the quality and veracity of data they provide?
If an advertiser knows who they are buying from, they can price risk. This is something that their intermediaries should be doing for them right now, but because of the bad incentives we talked about earlier, they’re not. If they were, DSPs would never ever buy ad networks and exchanges would never list ad networks. Because of the plausible deniability thing and the volume-centric profit motive, there’s really only going to be a bare minimum of that. That chief marketing officer who wanted to increase their budget and was willing to turn a blind eye to fraud to do that? In the fantasy world where advertisers do really know exactly who they’re buying from there would be no remove between a CMO and accountability for what is being purchased. Plausible deniability is gone. The CMO cannot say they were deceived in choosing exactly what impressions to buy. The CMO now has nobody to blame if the CEO finds out they’ve been buying bot traffic. Malfeasant or incompetent, which will it be?
So in this fantasy world where we have both really good information and a really good incentive to use it, what happens? Incentives have changed. Advertisers begin discriminating in what sites they’ll buy impressions from. As a thought experiment, what’s a simple filter you might apply just as a first pass to start eliminating fraud in the supply you buy? Maybe you only serve ads to the Alexa majestic million. That seems reasonable to me. Any site not in the top million is a mom and pop at best. Not saying they’re all fraudulent, but there’s gonna be a lot of fraud in that category. First pass.
You can do as many passes of this as you want, trying to apply filters that will omit fraud while blacklisting as few high quality sites as possible. If you want to be really good at this, you would still use filters, but you might introduce exceptions to make sure that certain sites you know are high quality which might otherwise get filtered out keep their spot on your list. Advertising will still be a volume game, but the dominant incentive will be for advertisers to find volume rather than for suppliers to fake volume, since when publishers know that their buyers know who they are, that one in a billion billion discovery chance becomes only as good as their scam artistry. The jurisdictional thing would still be a problem, but attribution would not. Weighing these factors is part of the risk pricing advertisers would be empowered to do.
Doing this filter research is work, and data about site credibility would certainly be a hot commodity since advertisers do still want to broadcast their messages to as wide an audience as possible. The best list would be the whitelist of sites that report impression data with the highest degree of accuracy, such that decisions about what categories and consumers to buy could be made by the advertiser at the impression level. Building these lists or acquiring them from service providers is a cost of doing business for advertisers, but they’re still going to be saving a lot of money in this new world because there’s far less fraud.
For publishers, the game now becomes to proactively endeavor to get listed in the most important whitelists, which would probably be a small number of baseline known non-fraudulent lists produced by large firms as products or curated in an open-source manner. What would it take to get a listing as a publisher? Well, the market would decide. If you’re a firm producing one of these lists, that list’s quality is your product, and you’re going to do your best to balance rigor and the drive for reach in curating your list. Advertisers will decide based on a firm’s reputation what lists they want to buy, so these firms will always be in competition with one another to produce the best lists. An open-source list could exist, provided there were proper incentives for the curators to curate it well.
So as the free market competes itself into a frenzy to produce the best whitelists possible, new best practices will emerge in reporting standards. There will be a natural drive for publishers to satisfy those standards so that they can get listed, and charge a premium for their CPMs. We get a market for honesty.
Could we do this? How could we get to a place where advertisers know exactly who they’re buying ads from? How could we dramatically simplify programmatic supply chains?
Well, what if we could get rid of supply chains entirely and connect advertisers to publishers directly? That would let advertisers know exactly who they were buying ads from, and would dramatically simplify programmatic supply chains. Could we do that with lists? Sure we could.
Imagine you’re a publisher. You subscribe to a list of high-quality advertisers who aren’t going to serve inappropriate or invasive content to your users. What does this list actually contain? Say you trust the list at face value, what you need then are DNS names (ads.coca-cola.com) where you can send RTB bid requests. RTB is real-time bidding, it’s just a standard way of serializing or formatting bid requests. The list you need is a list of DNS names for trusted advertiser bidders, where when I say “bidder” I mean a bidder server that’s going to be listening for bid requests on some known port or route. So you’ve got the numbers for a bunch of advertisers, and now you can call them up whenever you have an impression to sell them!
If we stop here, we’ve just created a new direct route for botters to scam advertisers. Botter calls up, says “hey, this is the New York Times, I have an impression for you!” and the advertiser immediately has to hang up because it’s probably not really the New York Times. Advertisers would never consent to being cold-called in this way, because 99% of those calls would be fraud. Simple domain spoofing. We need to give advertisers a means of authenticating that the inbound bid requests they are receiving are non-fraudulent. There would be two steps to this.
First, the advertisers themselves need a list of non-fraudulent publishers. And to be specific, they need a list of DNS names for publisher ad servers. So Coke is running a bidder at ads.coca-cola.com, and the New York Times is running an ad server (the thing that makes bid requests and processes bid responses) at ads.nytimes.com. This way, when Coke’s bidder gets a call from some random ad server, it can say “is this caller in my contact list? Do I know them?” and if they do, if the publisher is in the trusted list, they can pick up the phone. That’s step one.
Step two, we need to make sure the person calling is who they say they are. For this we can do some sort of mutual TLS scheme. We could establish a convention where, for example, publishers who want to participate in this scheme store a “CERT” record in their DNS listing, and have to sign a challenge message provided by the advertiser. That’s just an example, there are a number of ways authentication could be implemented.
So now we have a system where advertisers get to talk directly to publishers, and where both sides have strong cryptographic proofs of authenticity on all the messages that they exchanged. There is no more blind faith in opaque supply chains with track records of lying. We know exactly who we’re dealing with, and both parties have mutual records of exactly what the other said on a per-impression basis.
Just to round this story out: the publisher blasted out bid requests to all the advertisers on their list. Advertisers authenticated the publisher who sent the bid request, and some may have chosen to send responses. Others may have said, “you are who you say you are, and I trust you, but I don’t want this impression anyway”. The publisher selects the bid they like best, serves the ad and the tracking beacon fires.
A lot depends here on the quality and maintenance of these lists. It would be bad if a company providing a list was taking kickbacks to admit fraudulent sites. It would be bad if the company was not responsive in investigating allegations of misbehavior by entities that it was listing. It would be bad if the company went out of business and its customers were left with stale lists until they could sign a new contract and integrate with a new list provider. This system works if we have really high quality lists that are continuously curated, and reliable.
Earlier, I talked about firms competing to produce the best lists. The presence of competition should itself mitigate some of these bad behaviors or outcomes. If one list provider is putting out a low quality product for any reasons, its customers will go somewhere else. What would the peak predator of list-making look like in such a market? Who could put anybody else out of business providing the ultimate list? How do we build the best list possible, that would work for web scale? Let’s imagine a system, lets talk about building a list of publishers.
As a list provider you need to make money so that you can have employees to continuously curate your list, vet new applicants or maybe go out and hunt for candidates depending on how your business is structured, and you probably need to bill either advertisers who are consuming the list, or publishers who want to be on the list, or both. I think the provider which wins the market here is the one which gives away its list for free to consumers, and bills publishers to apply for consideration. Advertisers are the ones who pay, and if we can give them something for free some of them might try it out, and that will entice publishers to apply for consideration because if they get listed, then long term they can make back their application fee in the new revenue they otherwise would have been locked out from. So the winner here I think is giving their list away for free, totally open-sourcing it, and billing publishers to apply for listings because they have an incentive to be listed.
Now, this business may have something of a problem. Let’s say it’s super successful, and all of the credible properties have been listed. After that initial glut, if you’re not billing advertisers, then your revenues are reduced to a trickle as new credible properties emerge slowly over time. You might start billing advertisers for access to the list, or you might start billing publishers on a subscription basis to stay on the list, or if neither of those worked you might just choose to go out of business. Which would suck for your customers, because then the list goes stale. Listed entities can start defrauding people and there’s nobody to investigate claims or coordinate a response. It’s every advertiser for themselves… Again.
We want a list that never goes stale and never becomes unavailable, and which ideally drives costs as close to zero as possible, because that’s how capitalism works and we might as well just rush to the endgame now. So let’s create an organization that can’t go out of business. Lets create an organization that has no employees, no operating costs, not even server costs to host the list. That sounds easy, right? How would that work?
Well, let’s imagine such a list exists. Advertisers are using it to authenticate publishers, and publishers who are listed are getting a premium for their CPMs relative to publishers whose impressions are only exposed through opaque supply chains. So if such a list existed it would be valuable to advertisers, because it helps them source credible supply, and being listed on it would be valuable to publishers because they can charge a premium for their CPMs. Publishers are willing to pay to be considered for listing, but who is the curator here if there’s no company?
Just for kicks, let’s say that once upon a time when the list first began, I was the curator. And what I did was I made this kind of monopoly money which I would sell to anyone who wanted some. The reason someone might want some is that it’s the only payment I will accept to consider you for curation into this list. So at time zero my fake money isn’t worth very much because this system is very experimental, maybe I’ve convinced a few advertisers to try it out and maybe I’ve fronted some of my monopoly money to a few publishers to convince them to try it out.
But after a few months, it looks like this is kind of working. Its created a reliable revenue stream for listed publishers, and advertisers are happy with the transparency of this supply chain. Now interest is becoming more organic. It appears less risky, so there’s more demand for my monopoly money, and I can charge more for it. The price to actually apply I can adjust down. So if it cost 100 monopoly money to apply when I first started doing this, and I was charging one dollar per fake money, but now demand is such that I am charging two dollars per fake money, I can adjust the application cost down to 50 fake money. So the USD cost of an application stays the same, but my net worth on paper is going up. The pile of monopoly money I created is now valued at twice as much as it was originally.
I’m pretty happy. If I keep doing a good job by keeping fraudulent publishers off my list, interest in my list is going to continue growing. Demand for my fake money will continue to grow, and my net worth will increase, which makes me feel good. Now it’s worth mentioning that publishers might not be the only entities interested in acquiring this fake money. This fake money is appreciating in value as I do a good job of curating the list. Somebody might look at this project and say “man, this thing is going to be huge and it hasn’t even penetrated one percent of the market yet! I want some of this fake money today because it’s going to be more valuable tomorrow!” And I’d be happy to sell to them. I don’t care who owns the money. Fundamental demand comes from publishers, and as long as I keep advertisers happy by curating the list judiciously, they’ll keep buying. If other people want to buy also that’s fine by me. I just like making money.
Then I get hit by a bus.
But that’s okay, our list is going to be just fine! Because those people who bought my fake money have an incentive to see their fake money continue appreciating in value rather than go to zero, which it would if the list went stale because it would stop being useful for advertisers and so publishers would stop acquiring monopoly money to apply to it. There won’t be any more fundamental demand if the quality of the list doesn’t stay high. So these purchasers have an incentive to step in and take up the mantle if they weren’t participating already. This is why we have to use an application-specific monopoly money, because it decentralizes the incentive to maintain the system over a large number of people who can coordinate around a single idea: protecting and growing the value of their holdings.
But I’m dead, so how do they come in and take over my system? Well, I architected my system in a certain way, you see. The monopoly money I was using was all digital. I signed one billion unique files with a private key, and my server will actually accept input from anybody who has valid monopoly money. If you bought monopoly money from me, I signed them to you, meaning you can now sign them to other people, or to the server, which itself can sign them back to other people. And anybody holding the monopoly money can follow the signature chain all the way back to the original signer, me, to prove that they’re not forgeries. So as far as my server is concerned, you’re just fine. It never cared about me, it only cared about my monopoly money.
Quick recap: we have unforgeable digital monopoly money which anybody can verify the provenance of without needing to access a central banking server of any sort. We have a server that anybody possessing this monopoly money can interact with, either as an applicant or as a curator. And we have this incentive scheme where people who have purchased the monopoly money want to use their curation rights to keep the list quality as high as possible such that demand for their money stays high and its value appreciates, or at least doesn’t decline in value. That’s what we have so far.
But now I’m dead, so my AWS bill stops being paid and the server eventually goes offline. Also, when I was alive, nothing would have stopped me from secretly creating more monopoly money. Only I could do that, because I had the private key with which it was originally created and against which it is authenticated, but people who were buying this monopoly money as investments, thinking they were getting some defined slice of a finite pie, would be unhappy to discover that I had been secretly inflating the money supply behind their back. This is how the US Dollar works.
So we don’t want our server to go offline, and we don’t want anybody to be able to create new money, not even me if I hadn’t been hit by a bus. That second problem we could encode as software rules. I could have had the server create the money and give it to me at time zero, and then not have included any logic for creating more, and open-sourced that software for all to see. But even if I did that, you can’t actually prove that’s the software which is running on the server itself.
So we need some kind of compute platform where our software will live forever, and where the software is totally transparent to anyone using it, meaning you can read the source code and verify that program is the one you’re interacting with. For that we can use something called a blockchain, in particular a programmable blockchain, and even more specifically a programmable public blockchain. Blockchains are computers where anybody in the world can compete to get to make an update to the computer’s state (this is called “mining”). Blockchains have this very useful incentive game where every time a miner wins the game and gets to make an update to the computer, they also get to create a small amount of their own monopoly money that all the miners share which, if you as a user want to update the computer, have to include some of this monopoly money as a fee. And so miners have this nice incentive where they want the computer to behave reliably such that people want to use it and the value of their monopoly money goes up. Interestingly, the more miners are participating in this, the harder it is for any of them to misbehave. If we use a vibrant public blockchain that hosts lots of applications, users of our applications can be assured that it will always be available even if I gets hit by a bus. And I mentioned that anyone can be a miner, absolutely anyone, and that means anyone can inspect the state of this computer and see exactly what any particular program on it looks like at the source code level.
So let’s put our list on that instead. Now we can prove things about the money supply and if I get hit by a bus there’s no interruption of service. This is pretty good so far, and I think we’re hashtag winning, but we can do even a little bit better in creating the master list. So for good measure, as a final step, let’s drive the costs down as close to zero as possible. No subscriptions, no rents, no recurring fees, just the bare minimum of money moving necessary to incentivize people to make this work.
So what’s up with these application fees that I’ve been talking about? We have no employees, so who collects these fees? I’ll be honest with you, I’ve been saying fees just so we could defer thinking about something until this moment in the talk. Let’s drop the whole idea of fees. Holders of the monopoly money are realizing upside by seeing demand for their money increase, they don’t need arbitrary revenue. At the same time, we can’t just let applications be free, because then publishers would just spam us, why not, it’s free, and that’s going to give the token holders trouble curating the list. Also, it would be nice to provide people with no capital to buy the monopoly money some means of acquiring it in exchange for useful work.
What if instead of fees, applicants (publishers) made deposits? They make an application by putting down a deposit in monopoly money, and if they’re credible and would make the list more valuable by being in it, they can just keep that deposit locked up in the software for the duration of the listing, and if they ever decide being listed isn’t useful for them they can exit the list, withdraw their deposit, sell it and recoup some of their costs, or even make money depending on the market for the money. But to prevent spam we need to make it painful, financially, for people making ill-considered applications. What happens when somebody applies massivefraud.com?
The way this application process actually works is that the publisher locks up some monopoly money, puts it at stake, and then people look at that applicant and say “Do I personally believe the list is more valuable with this applicant on it, or off it?” If I think the candidate would detract from the list’s quality, and I feel pretty confident that other people would agree with me, I’m going to challenge it by putting down a matching deposit using the same monopoly money, so now there’s two pots of money at stake. With the challenge initiated and two pots of money at stake, the money can vote. Now this is not one person one vote, this is one money one vote. People with more exposure to the money’s value have greater say in the curation process. People with the most to gain or lose by the list’s curation quality have the most say. What’s powerful about this is that only loose coordination is necessary, since all of our financial incentives are aligned. Any actor can take rational actions based on how they imagine other actors will rationally respond.
So the vote happens and the loser’s deposit is going to be forfeited. This goes to the winner as a reward to compensate them for the capital they put at risk, and voters have defended the quality of their list and thereby the value of the money they hold. Notice too that the capital risk in this process that prevents application spam also prevents challenge spam. And remember that a publisher who gets listed has their deposit locked up for as long as they are listed, which means we can challenge them even after they’ve gotten their listing. If they get a listing and they decide to go rogue and abuse it for profit, we can challenge them out and confiscate their deposit. In fact, advertisers who might believe they have been defrauded by a listed entity can do this, and if they are justified, they can recover some of their loss totally in-band to the program without ever having to get a lawyer involved.
This is how AdChain works. It is the peak predator of list-making powering a peer-to-peer ad economy. It extracts no unnecessary rents, and requires no trust in anybody’s honesty or goodness. It aligns the financial incentives of participants towards curating the highest quality list they can and making it available to anybody at no cost, and operating on an immutable, incentivized ledger which will outlive any of its applications.
Now notice: we’re not putting any impressions on the blockchain, this is a low-throughput system that works well at blockchain speed. We’re not requiring end users use a special browser or have any idea that any of this is going on behind the scenes. We’re not requiring that publishers or advertisers use cryptocurrencies to pay one another, they can keep using US Dollars! Publishers just need a small amount of monopoly money for a one-time application process. And notice that we do this at the lowest possible cost, with zero overhead. If another list enters the scene it could compete by coming in with a lower deposit requirement, and if that deposit requirement suffices to incentivize curation, token holders can vote to lower their deposit requirement to match it.
This simple thing, making lists in a transparent, trust-minimized way, opens the door to a peer-to-peer ad economy without middlemen, without opaque supply chains, and without tens of billions of dollars in fraud!
We’ve talked a lot about publishers, there’s no time to talk about advertisers, but we’re going to curate that list with the same transparent, public process. That means users can have a say in what kinds of ads they are okay with seeing. If users don’t like coca-cola tracking them all over the Internet, they can form a coalition and summon the funds to boot them off! They can run ad blockers that permit only ads from listed advertisers because those advertisers serve ads that support publishers but are small, respect privacy and don’t mine cryptocurrency in users’ browsers! And if they violate that trust users can remove them and receive a financial reward for doing so!
A system like this could be the people’s fist against surveillance capitalism. There could be a peer-to-peer ad network, owned by the Internet. There could be an ad network accountable to its participants. There could be an ad network that admits no fraud. There could be a people’s ad network, and this is how I would build it.
