Developing Frameworks for ICO Investing

Adam Kaplan
May 26, 2018 · 18 min read

In the first quarter of 2018, Initial Coin Offerings (ICOs) have raised over $3.4 billion, exceeding the total amount raised from January to November of 20¹⁷¹. This explosive growth has fueled excitement from speculators, scammers, and smart investors alike, and shows no sign of slowing down halfway into Q2. While the ICO has become the new paradigm for fundraising, its infancy means there is a severe lack of objective time-tested data-points to analyze or draw meaningful conclusions from. While we have some preliminary statistics on the successes and failures of various ICOs, it will take several years to fully explore and begin to understand the different forces which impact ICO success. It remains to be seen whether particular tokenized models will be successful, and it remains to be seen whether companies may run out of funding and need to raise additional rounds in the future.

In this post, I will discuss ways we can begin to develop frameworks around ICO investing, both at a macro-level (all ICOs) and a more detailed level (specific ICOs), by applying lessons from the Venture Capital (VC) industry, as well as tools built from a historical analysis of the philosophy behind the development of AI. These frameworks can and should apply both to individual investors, as well as larger firms looking to understand and enter the space. None of the content in this article should be interpreted as investment advice. This post was inspired by Coinlist’s article, The Current State of Initial Coin Offerings, and an assignment for Nan Li and Zavain Dar’s excellent Stanford seminar on Venture Capital, SYMSYS 161.

The ICO as Distributed Venture Capital

With the help of this Dilbert cartoon, it is important to note that ICOs differ from VC several important ways:

  1. ICO investors are traditionally not buying shares (or a stake) in the company; rather, they are buying tokens which are to be used in the company’s ecosystem (whether as a security, financial instrument, or utility token). While there are exceptions, i.e. Siafund, the majority of ICOs raise funding without releasing company equity.
  2. ICO investors traditionally receive liquidity much faster than traditional VC. For a traditional VC, most companies take at least 7–10 years before reaching a point of liquidity where investors can realize a return on their investment. While ICOs are beginning to move in this direction, with longer lock-ups for institutional and pre-sale investors, the time-to-liquidity is still substantially shorter.

While these differences are substantial, they do not invalidate the underlying philosophy that ICOs are best viewed as distributed VC. Rather than playing a role exclusively via funding, VCs must now bring more to the table to receive allocations for the most competitive projects — connections, resources, and partnerships are king in the current landscape. With the widespread availability of funding for quality projects, early investors must now offer more than just cash — and this is where VCs can excel when they find promising projects.

In the Darkness of the Cave

The Typical ICO

The ICO is yet another escalation of a growing long-term trend across VC: The commoditization of fundraising rounds. As more VCs vie for the same promising companies, the race to secure a position in a promising company early drives larger investments in earlier rounds. While companies in the mid-2000s would raise $10–15 million in a Series B or C round, these rounds now have much higher raises, while Series A has become the new Series B, at least for fundraising. Higher early rounds lead investors to find their portfolio gems at even earlier stages, and this requires more risk-taking, as companies are younger and less-established at these earlier rounds. The ICO is the next step in this process: ICOs are raising $15–50 million, yet often the company maturity is closer to a seed or early Series A round. While it remains to be seen whether companies that did ICOs will raise additional rounds in the future, the current trend seems to be to raise sufficient funds all at once.

In addition to the higher initial rounds, to build a portfolio of high-quality blockchain-based companies requires a global reach, as well as a clear thesis about the industry itself. Moreover, within the blockchain industry, there are numerous verticals, each with their own winners and losers. As an investor or VC, it is thus essential to not only build proper position sizes to support companies and hedge risks, but also identify and select promising verticals within blockchain technology. It is insufficient and unwise to simply “invest in blockchain” because of the wide array of sectors under the same umbrella. This is a complicated problem, made even more difficult by uncertainty, volatility, and the early stage of the industry — to begin to tackle the problem, it is necessary to develop short-term and long-term theses which guide investments.

Within this context, we want to define a set of rules and guidelines, a “function,” which takes an ICO as input, gives an investment decision, amount, and timeline as output. To do this, we need to build an understanding of what a “successful” ICO looks like, and then boil this success down to a series of if-this-then-that statements which could help us make investment decisions. This is similar to a canonical problem in artificial intelligence — finding whether a picture contains a cat, given a random picture as input².

Each of these images contains a cat. Is it possible to write a set of rules that a computer could follow to determine this?

Platonism and Nihilism: AI as a Model

  • A cat has four legs
  • A cat has pointy ears
  • A cat has a tail
  • A cat has whiskers
  • A cat has paws

… But in the middle picture, we can see only one of these five “defining” characteristics despite us knowing that it contains a cat, and in the third, we might mistakenly think that the picture contains two cats! How can we do better? This was the problem that early AI researchers faced.

With more advanced algorithms and machine learning tools, we can now create ways for computers to “learn” what a cat looks like. In fact, we may not be able to explain the features that the computer learns when classifying an image as “cat” or “not cat.” Many of these features may take place at the pixel-level, and may not be meaningful for a human to classify a picture, despite being very helpful for computers to do so. This is a Nihilistic, results-driven approach — as humans, we do not care how the computer identifies a cat, so long as it is correct. This unexplainable process is viewed by some as deeply unsettling, while it is championed by others as evidence of the power and success of AI and machine learning tools. The EU’s “Right to Explanation” regulations directly address this point, banning fully automated and unexplainable decision-making when it impacts real people.

While AI has made great strides in cat-finding, replicating the approach to machine learning in other industries is a difficult challenge. Most notably, cat-finding requires a great deal of training data in order for an algorithm to learn about what is a cat and what is not a cat. Without sufficient data, this would not be possible.

The problem of insufficient data plagues many industries.

What have we learned from this example? In fact, “cat-finding” is an analogy for a substantial portion of what VCs do today — in a space with lots of noise, uncertainty, and constantly changing landscapes, VCs are attempting to find their “cats:” Successful companies which will become portfolio-leaders. Unfortunately, VCs, unlike cat-finders, generally do not have access to large amounts of useful data to use in their decision-making processes. They are thus reduced to the earlier method of cat-finding: Defining heuristics, based on “ground truths” they believe to be important, to identify characteristics and features of successful companies, then using those heuristics to evaluate each new investment opportunity they come across.

What is a Cat? Defining Success

Yes, this did (does) exist!

ICOs take the idea of cat-finding to an even more challenging level: Currently, we do not know what a cat (or success) looks like! Before we can attempt to predict any success, we need to know what we are looking for. We have short-term information about successful projects, but even the earliest ICO, Mastercoin, has faded into obscurity over the last 5 years. For the typical VC, this is only half the lifespan of a fund! Even quintessential success stories such as Ethereum are now facing a barrage of new competitors, offering new solutions to smart-contracts, scalability, and trustless transactions. No doubt, Ethereum’s 1500x return to date from its ICO should be defined as a resounding success, but most projects do not even come close to achieving these massive returns, and many fail before even launching anything.

In the world of ICOs, then, how should we define success? Is it simply returning an amount greater than invested? Is it that the coin never trades below ICO price? What time-horizons should be used to determine success? Arguments could be made for countless different answers, but ultimately defining success depends on both perspective and risk. Consider the following challenges:

  • Success for investors may not mirror success for projects, especially in the long-term. For investors, ROI is the primary (and often, only) focus, whereas for projects, building and releasing a successful product may be more important, especially for long-term growth.
  • Given that many ICOs fail, success for an investor must offset the risk of failure, as well as counteracting other promising investments which ultimately failed. Like other industries dominated by startups, we can expect the majority of projects to fail, so the ROI for success must offset these losses for investing to be worthwhile.
  • Success in ICOs can be measured on many different time-frames thanks to token liquidity, and each approach to measuring success has many tradeoffs. For example, did the project increase in value at initial exchange listing? After 1 month? 3 months? At its all-time high? Is the project still in development? Are road-map goals being met?

A case can be made, in fact, for success criteria to be specific to each investor. Everyone has their own risk tolerance, expectations, and philosophies toward investment. While 1500x cases such as Ethereum would be included in everyone’s lists, projects that increased 20% or 100% over 1 month or 1 year may or may not be. Similarly, an ICO which trades at 5x but does not ever (or is extremely slow to) develop a product may not be an unequivocal success.

I will share my personal definition of success and its predictors below, but keep in mind I am not here to argue for any singular definition of success; rather, I encourage everyone to build their own robust definition of success which could be programmed into a computer using simple rules. Just like cat-finding, success-finding requires a clear understanding of what success looks like before you can even begin to build a reliable framework. Only once you are able to define success can you begin to analyze or understand what factors may impact it.

For me, as an investor, my general definition of success involves financial returns. As a young investor with (very!) limited capital, I have a higher risk-tolerance and longer time-horizon than many others. My goal is to pick projects that are likely to be successful in 1–2 years which I believe will contribute to the blockchain ecosystem. Because of the high failure-rate of start-ups broadly, I diversify my capital across numerous projects — as a consequence, to expect positive returns, I look for projects that I expect, if successful, to provide at least 5–10x returns. This framework is not intended to be a boiler-plate, and it will likely evolve over time, but it provides a more concrete definition for “success.”

Cat-Finding in ICOs: My Framework

Thanks to Nan and Zavain again for the slide framework

For me, to be a successful in 1–2 years and achieve an outsized ROI, a project must achieve all of the following. The sub-bullets are potential inputs to each feature, with some impacting short-term ROI, and others impacting more long-term ROI. Since my primary focus is the latter, I naturally weight those factors more highly.

  • An active and supportive community • How is expert sentiment? Twitter sentiment? Newbie sentiment? • What is the quality of discussion in their communication channels? • How often do team members share updates and project details? • Did they do an airdrop? To whom? • Does the community evangelize or promote the project? • How is the project’s marketing and public communications?
  • A clear and ongoing use-case and/or utility for any token/coin involved • Why is the token/coin necessary instead of using BTC/ETH/Fiat? • What incentives are there to hold the token/coin? • What is the supply of the coin, and how does that change over time? • How compelling is the use-case versus other alternatives?
  • A clear reason for utilizing blockchain over other existing solutions • Does the problem need decentralization? • Does the problem need a secure, unalterable record of transactions? • Does the problem have an easier or less-onerous solution? • Has someone else solved the problem using something non-blockchain?
  • Large SAM and a clear plan for achieving market penetration • Is the ICO’s vertical positioning in the market lucrative? • Will the current SOM be sufficient to sustain the project for 2–3 years? • What is a reasonable market-share for the industry? • How have non-blockchain competitors performed? • How have blockchain-based competitors performed? • How will the project onboard new customers and capture competitors’?
  • Regular Github updates and improvements • Is there active and ongoing development? • How old is their Github? How long have they worked on the project? • Are updates exclusively made by 1 person? The entire team?
  • A growing, well-qualified and/or accomplished team • Where did the team work before joining the project? Is it verifiable? • Is the team academically qualified? Technologically qualified? • Have new advisors or team-members joined after the project began? • What have team-members accomplished before starting the project? • How big is the team? How many developers?

Additionally, I believe most successful projects have at least two of the following qualities:

  • Meeting or exceeding projected roadmap deadlines
  • Regular product updates and/or press coverage
  • Working product or prototype

These criteria are quite similar to what many VCs examine when evaluating any startup. When projects meet these criteria in a promising manner, financial success will likely follow. While some projects may dip below ICO price initially, I am more concerned with the long-term viability of each project. By using this heuristic and evaluating the likelihood of each of the above factors (using their inputs), I have found that most projects’ financial success beyond their initial hype aligns nicely. Of course, this is merely a heuristic: My attempt to define what a “cat” looks like, and how we can go about finding those cats.

A Data-Driven Look at ICO Success

While more in-depth data does not exist, I did manage to gather some data from other research sources to attempt to take a more data-driven approach, but this data is lacking in many ways — its depth, process, and verifiability all leave some clarity to be desired. With these caveats out of the way…

In their article on ICO Developments and Trading, Sherwin Dowlat and the Satis Group define ICO success using three primary heuristics: Product/chain deployment, transparent roadmap, and Github code contributions in the past 3 months. Using these metrics, in addition to classifying ICOs as scams and failures, they develop a framework for examining the current ICO landscape. Unfortunately, the raw data and time-horizon of their analysis is not outlined, but their conclusions (assuming their analysis is/was valid) are quite interesting:

Image from Satis Group and Sherwin Dolat

The first piece which should jump out is, of course, the astonishing number of scams. While they were unclear on their precise classification for a scam, if we include the hundreds of Bitcointalk ANN posts (as they say they have done, at least to an extent), this ratio seems accurate. Of the scams, we only hear about a few — but our availability bias does not mean they do not exist. Scams are rampant enough that the SEC even launched a fake ICO website, Howeycoins, aimed at better informing and protecting potential investors from the countless money-grabs popping up on a nearly daily basis. While most scams can be relatively easy to spot, some promising-looking projects have also ultimately turned out to be scams — an unfortunate problem which remains unresolved.

Let us leave aside the scams, however, as without knowing the precise data used, it is difficult to determine its precise accuracy. Instead, let’s focus on the non-scam cases: Given that an ICO was not a scam, about 42% went on to trade. Of this 42%, just under half met their criteria for success. If we assume that we can exclude scams, this success-rate is approximately on-par (or slightly above) with start-up industry standards of ~20% success rate. This validates diversification of investment, as well as the risk inherent in ICO investing.

In the case of these authors’ analysis, their definition success focuses on the broader market reaction to the ICO, and its continued development and improvement. This framework similarly focuses on long-term returns: Even if price performance is not stellar immediately, projects that are building out their products are more likely to succeed and grow over time. These measures, thus, do not directly address profits, rather, financial success is implicit in their criteria. This is particularly evident in their visualization of ICOs by current market-cap:

Image from Satis Group and Sherwin Dolat

As we can see, “successful” ICOs are more likely to be valued higher than dwindling ones. This is not a coincidence: While financial performance in the short-term is driven by far more volatile and unpredictable factors (hype, supply-squeezes, Twitter influencers, pump and dump schemes etc), overall long-term performance is driven by fundamental factors such as team quality, product-market-fit, meeting established deadlines, and continual improvement — all factors in my initial input function. While both the short- and long-term frameworks are potentially profitable opportunities to take advantage of, focusing on the latter provides a more useful framework for strategic investing.

AI and ICO Winters: The Cyclic Nature of Technology

In VC, winter is coming. As deals become commoditized, more expensive, and scarcer, VCs must find new high-yield outlets for their capital. As exits decline and IPOs become rarer, VCs must find new avenues for liquidity. Cryptocurrency and blockchain are potential outlets for these problems, but they are far from immune. In fact, we have already seen at least one “ICO winter,” and we may see another soon: As more money flows into ICOs, deals become similarly commoditized, and competitors vie for an all-too-slowly expanding pool of precious resources (community, developers, government support and more). Projects that fall out of the “hype cycle” struggle to stay afloat and keep their high valuations at ICO. We saw this process last summer, as Ethereum’s price skyrocketed to nearly $400 before crashing again, with numerous projects doing ICOs. We saw it again in February-April of this year, as the poor market conditions added a further challenge.

The good news and bad news are ultimately similar: Hype, markets, and even technologies have a cyclical nature. We can learn from past cycles and behaviors, and generalize to help inform future decisions. Increasing the availability of data-driven resources in the cryptocurrency space will be an important component of ensuring future cycles can be capitalized upon as much as possible.

Recap and Summary

To fuel this process, we must work toward building long-term robust data-sets of information about cryptocurrency markets and ICOs, and we are still in the early stages of doing so. Until then, our approaches to investment will continue to take unverified and potentially inaccurate axiomatic and heuristic approaches — while these may often be “good enough,” they are hardly satisfactory.

Thus:

… To be continued at a later date, with later data.

Footnotes and References

²SYMSYS 161 — Week 3, Nan Li and Zavain Dar

Adam Kaplan

Written by

Kindergarten dropout, Stanford '18 graduate. World champion bridge player, travel and aviation geek, and blockchain enthusiast. http://adamjkaplan.com/