AMD is not the problem, your Web tech stack is

The AMD debate is back, and I have something to say.

It’s half past 2013 and Web development is still a mess. Javascript still has no sensible way to import and export names by default. Still virtually no one is talking about how to package up Web components for the client-side and the JS community is as fragmented as ever.

First let’s begin with some history. Skip to the AMD section at the bottom for the TL;DR; version.

The <link> and <script> tags

For more than a decade since the introduction of CSS and Javascript, people have had to make do with manually managing CSS and Javascript dependencies by juggling <link> and <script> tags. This is not a problem at the beginning, since webapp was still not a word (my dictionary tells me it still isn’t, but it should be) and few people had websites with more than a dozen pages, most likely all include the same flat CSS file with 100s of rules, and a JS file with 100s of functions. People still had no idea how to write CSS and Javascript back then, let alone organize them.

But then things started to improve. CSS come out with an @import statement and is supported first in IE 5.5. Javascript has had virtually no improvement on this front even to this date.

In 2001, the Dot-Com Bubble bursted and Mr Crockford enlightened the world with his reverse-engineering/reverse-understanding of Javascript. At around 2004, 2005, Google released Google Suggest. The term Ajax was coined. Soon after, Prototype was released and bundled into Rails, then jQuery came along in 2006. YUI invented the module pattern also in 2006. We’ve entered the Web 2.0 era, where the browsers had started playing host to a new wave of Web apps that seek to emulate desktop apps. It is at that time that the amount of unoptimized CSS and Javascripts started to explode. It caused not only logistical problems, but also performance problems.

Steve Souders told us to concatenate and minify the CSS and Javascript files in 2007 to speed up the Web.

Now we have a new problem — how do we find out the dependencies of all the open-source Web components, linearize the assets into one big file for each kind, and pass them to the minifiers without doing too much work?

Enter asset management solutions.

Existing Solutions and their problems

Sprockets & Friends

In the years that followed, various server-side libraries, most prominently Sprockets, sprung up to help us deal with this somewhat, but there was still not a centralized place or agreed-upon method to distribute and consume web components. You still had to manually manage the file dependencies outside of CSS and Javascript mostly because these tools don’t understand CSS syntax, so they can’t linearize the CSS files just by following the @import dependency graph. The reason for that escapes me, but I suppose most tools chose to consume dependency metadata externally because they’d have to do it for Javascript anyway. Also, it’s 2013. Lots of newer webapps have moved most of the UI logic to the browser, some even have attempted to write single page apps that completely decouple the UI from the server. The Web needs a new asset management toolchain designed solely for front-end developers for an environment they are most familiar with — Javascript.

CommonJS & Node.js

A first true attempt at solving the module definition and importing problem is the CommonJS effort (Actionscript 3 notwithstanding). At around the same time Sprockets came about, a group of well-meaning renegades started the CommonJS and Node.js project. Finally we have a sane way to distribute and consume Javascript with require, exports and npm.

var sanity = require(‘sanity’);
export.happiness = “npm is awesome”;
export.doubleHappiness = “I can export more than 1 thing too!”;

require looks and works like the require/import statements from other languages you are used to. exports is a little weird, but no weirder than the static and extern keywords in C.

This should lead straight to a straight-forward implementation of a sensible asset management system completely decoupled from any server-side frameworks right? Not so fast.

Client-Side Web Components

At last, CommonJS is a server-side solution. The browsers don’t speak CommonJS. Specifically, while there are various solutions such as Ender and browserify and that can wrap CommonJS modules seamlessly for the browser, they are in some ways even worse of a solution than Sprockets because they are completely useless for packaging up CSS/LESS/SASS. CommonJS also currently does not define any package.json extension that allows components to expose anything other than Javascripts for other components to consume.

AMD & require.js

I’m just going to equate the two here because for all intents and purposes, require.js is the de-facto reference implementation of AMD.

Require.js is by far the most popular attempt to bring some order to the world of client-side asset management, but based on my couple of trials with it, I think it suffers a number of serious flaws which I think will render it a passing fad fairly soon. (Do correct me if I’m wrong because I’ve only tried a couple of times and I still feel I’m not totally comfortable with it)

  1. Too many ways to define or require a module. Too many patterns that are essentially work-arounds and too many options. Too much boilerplate code.
  2. Still can’t define a Web component that exports more than one kind of resource. You’d think with the number of options require.js gives you, you’d be able to do this, but it doesn’t.
  3. Hard to debug. This is the most annoying issue that I keep running into. require.js fails silently when there’s a circular dependency . Errors are often not detected until somewhere deep down the call chain where I reference that file. Often times, I can’t even find out which file couldn’t load or which file was trying to load it based on the error messages.
  4. Pointless asynchronous loading. This has a tendency to lead people believe that they could just require modules on the spot whenever and wherever, this is just wrong. This leads to callback hell and if you are not careful, lots of choppy incremental rendering on the screen. Making lots of requests also goes against common Web performance best-practices.
  5. r.js has yet more inscrutable configuration options. (50+ options for essentially a thin layer of asset management code on top of the minifiers, many options are duplicates of require.config).
  6. r.js doesn’t compile require.js into the output bundle by default. Why must the loader be separate?
  7. Reinventing the wheel of module definition syntax, but not compatible with CommonJS tools and ES6 syntax. This is the most important reason of why I think require.js will fade out in a few years time because it is significantly more complex than ES6 or CommonJS. Even with the ES6 to AMD transpiler, it’s still only a temporary solution until the libraries and tools catch up with ES6.

There are many more idiosyncratic problems with require.js based on my experiences, but I’m too lazy to list them all here.

The Community and The Way Forward

AMD is an admirable attempt to solve a number of issues plaguing client-side Web development, but it is also true that it remains an admirable attempt at best. Its complexity and deficiencies are actually a reflection of the state of affairs for front-end Web development — everybody has been busy reinventing the wheel, nobody asks what the wheel is suppose to roll on, and nobody can agree to combine their efforts to work on anything.

What this ends up with is of course a number of mutually incompatible, half-solutions with varying community sizes.

Is There a Solution?

Well, in the long term, I think the solution is definitely for the browser vendors to agree to what a Web component constitutes of, and come up with a cross-platform packaging solution that enjoys an overwhelmingly wide acceptance. Ideally, visiting a URL will automatically read a dependency metadata file, download the dependencies with WebRTC peer-to-peer, or load them from the cache. This way multiple websites/apps can share library resources. Essentially, a distributed CDN + NPM on steroid, on the browser.

In the medium term, I think quite certainly that ES6's module syntax needs to be finalized as soon as possible so browser vendors can start shipping it, and developers can solve half of the problem first.

Short Term Band-Aid

As far as I know there is only one tool that is close to a solution to packaging Web components — Component. It not only can manage and build the assets for you, it also has a clear vision and definition of what a Web component should consist of. It also has a number of fairly well done plugins that can manage your LESS and SASS files and more. While I’m still waiting for it to support CSS sprites/icon fonts, and image optimization, they shouldn’t be a serious enough problem to hold you back. If you have a large modern day webapp with lots of static assets, I encourage you to check it out.


Next Story — Measuring Programming Language Insanity
Currently Reading - Measuring Programming Language Insanity

Measuring Programming Language Insanity

(Unscientifically)

I saw a tweet yesterday that someone from the UK ran the Perl interpreter against a SSL certificate:

Since I'm bored out of my mind during this holiday season, I thought it might be fun to run all the programming language compilers/interpreters on my Mac to a self-signed SSL certificate with a real CA cert bundle chained to the end. Here’s what I found, in order of insanity:

Java 1.7

Stops right at the beginning, and not just tells me there’s an error, it even tells me what it was expecting. This is truly professional. Very impressive.

Clang and Clang++

Not as pretty, but pretty damn good error messages too. Good job.

Python 3

Python is such a clean language, even its syntax error is a minimalist language designer’s dream, though it wouldn't hurt if I could know what the parser was expecting. In case you were wondering, Python 2's output is exactly the same.

Tcl

Not bad either for a language where everything is a string.

Ruby 2

We are starting to get into the insane asylum here. The first + is making Ruby going a little haywire.

Node 0.10

Alright I get it Node! You need to wrap every module in some mysterious “tion” function and it takes a couple of steps for you to get from the CLI to the _compile method. Now get out of my error message.

PHP 5.4

Lazy much PHP?

PostgreSQL 9.3

Shit what did I just do?

Scala 2.10

Ok Scala, you are drunk. Go home.

Perl 5.12

There is nothing scarier than a persistent madman refusing to give up. Someone call 911 please, I’m afraid Perl has a violent tendency to throw garbage at people’s faces.


Thank you for reading this far. Just a reminder, this blog post measured and compared nothing of scientific importance. In fact, it didn’t even compared every language on equal footing.

Next Story — The Conservation of Hairiness
Currently Reading - The Conservation of Hairiness

http://www.nileguide.com/blog/files/2011/11/wnger-16999-swiss-army-knife.jpg

The Conservation of Hairiness

(I originally wrote this post on my old Posterous blog sometime in 2011)

Recently, or perhaps for as long as there have been technical decisions to be made, I’ve been hearing endless arguments about this versus that that lead to nowhere at all. Emacs vs Vi, PL X vs PL Y, NoSQL vs RDBMS, DjangoORM vs SQLAlchemy… etc. I hope I can shed light on what’s fundamentally going on here. I haven’t blogged in a long time, this is the first post in which I hope will be a long line of semi-regular posts that document my thoughts, philosophy and learnings from programming, and anything barely related.

Back in the days when I was a grad student at Tufts University, I attended a guest lecture by then Project Darkstar tech lead Jim Waldo (now Harvard’s CTO). At the end of the lecture, when someone asked him why he made the technical decisions in Jini, he introduced the term — The Conservation of Hairiness. Lightning bolts struck. He didn’t spend too much time explaining what he meant, but here’s how I came to understand this term later (adjusted by my own interpretation, so don’t misattribute its entirety to Dr. Waldo please):

For any given problem, there’s usually more than one subproblem

Suppose you are to design a templating library, there are a number of smaller problems you need to solve. You will need to be able to load the template files from somewhere, parse them, allow the users to do some manipulation to fill in the contents, and return the results in some representation. You can obviously break these subproblems into sub-subproblems. For example, you may want to be able to load the templetes from the file system, or memory. You get my point. I will postulate that for any given problem that takes some input and produces some output, there’s at least one subproblem that is either the problem itself, or a set of smaller problems.

The solution of a problem is the sum of the solutions of its subproblems

Naturally, if you manage to solve all the subproblems, you have a complete solution to the problem as a whole. I assume you have heard of the Single Responsibility Principle, Do one thing only and do it well, and the Rule of Composition, so I’m not going to dwell on this. What I’m going to tell you though, is that there are solutions out there that attempt to solve more than one problem at the same time, and there are solutions out there where the solutions for the subproblems are pretty much the same, but arranged in different orders.

If you have a different set of subproblems, you have a different problem, and requires a different solution

This is probably the most subtle point in the entire post so I’m going to explain a bit more. Let’s say you are to build your startup company’s website, you have limited time, money and you don’t really know the constraints, you should probably go with the most popular technologies out there right? Wrong! The correct approach is always going to be to understand the problem first, even if you have to guesstimate on some number. How many users are going to be visiting your site? What’s an acceptable response time? Do your data have schemas or are they free-form? Is it going to have lots of dynamic interactions or is it just mostly static with some dynamic content? These are all proper subproblems. Once you understand the constraints, then you can go out and choose a technology stack that enables that solution.

If your requirement is simply to be able to serve 1000 requests at a time, with less than 3 seconds response time, with a predefined data model and the pages are mostly static with some dynamic content, Spending days and weeks to evaluate Django vs Pyramid or Django ORM vs SQLAlchemy is probably going to make very little difference as they all solve the same problem, with slightly different arrangements of complexity spread out across basically the same layers.

However, if your data is free-form, using a schema-less datastore is probably a more attractive solution. Picking a framework that will enable you to use a different data access layer easily is probably a better idea. In this case, you may opt for Flask + MongoDB. By the same token, evaluating whether CherryPy + MongoDB or Bottle + Riak is better is going to make very little difference unless you have more nuanced requirements.

If you don’t, or don’t know, it’s probably better to choose a stack that doesn’t prevent you from solving other subproblems. Fortunately, most tools out there, especially those that adhere to the “Do one thing only and do it well” principle fit this description.

The complexity of any given problem is a constant

I would argue that how you choose your technology stack is largely dependent on how well you can understand the problem at hand. In reality, most of us most of the time simply don’t know enough about the problem to be able to completely enumerate all the subproblems, so sitting here arguing about the same nuanced points days after days for two different tools that pretty much solve the same problems with different tradeoffs is going to make very little difference. At last, you still have to take your comfort zone into account. If you’ve found the perfect solution but requires you to learn the pieces for months, is it worth it?

This post has gone on long enough, I hope this post can give you some perspective in how to pick the right tool to do the right job. Now please feel free to comment!

Next Story — This is Your Life in Silicon Valley
Currently Reading - This is Your Life in Silicon Valley

Silicon Valley — Photo Credit — Vadim Kurland

This is Your Life in Silicon Valley

You wake up at 6:30am after an Ambien-induced sleep. It’s Friday. Last night at The Rosewood was pretty intense — you had to check out Madera and see if there is any truth to the long running Silicon Valley rumors. You were disappointed, but at least you did get to see a few GPs from prominent VC firms at the bar. Did they notice you? Did you make eye contact? You remind yourself they are not real celebrities — only well known in a 15-mile radius to the Techcrunch-reading crowd.

Your non-English-speaking nanny shows up at 7:30am on the nose. You are paying her $24/hour and entrusting her (and Daniel the Tiger) with raising your child. You tell yourself that it’s ok for now — when he’s old enough he’ll (someday) be in public school in the Palo Alto school district.

You commit to being a better parent this weekend and spending more quality time with him as you browse through the latest headlines on Flipboard. You recently realized he may not be the next Mark Zuckerberg after all — still you send him to a music school even though he’s only 3. You swear he’s a genius because he can say a few 4-syllable words and can clap perfectly to the beat of “Call me Maybe”. He’s special. He is destined for greatness and you’ll make sure he achieves every ounce of it. After all, both of you are so smart and accomplished.

Cal Academy of Sciences — Photo Credit: Brook Peterson

You ask your nanny if she has any availability to watch your son this weekend. Bummer — you wish Cal Academy of Sciences hadn’t sold you on the annual pass 11 months ago. You figured you’d be going there every weekend, but only ended up going the one time. Not a break even proposition for you.

Your wifi enabled coffee maker downloads the perfect instructions to brew a cup of Blue Bottle — and you don’t have to do anything. The Roomba purrs in the background while you continue to read from your smartphone. You see a few articles about Trump and how crazy he is — somehow this comforts you.

You decide to share an article about Brexit from “The Atlantic”, which will somehow shed light to all your friends as to why it happened. The article is 1,000 words long — you only read half of it, but that’s good enough. It captures all the arguments you’ve been wanting to make for the past two months to your friends. Will this be the Facebook post that finally spurns your friends into action? You realize your Facebook friends all agree with your political views and social views already.

Fifteen minutes — only 3 likes — better luck next time. The Facebook Newsfeed algorithm totally fucked you — you should have shared from your browser, not your phone, and perhaps at a more optimal time.

But then you realize another friend already shared the article. You feel stupid.

Youtube office in San Bruno — photo credit Travis Wise

Your spouse hurriedly gets ready for work — you are a two income family and you have to be one for now. The spreadsheet shows that with only three more years’ savings, you can finally afford that 2 bedroom condo in San Bruno. So what if the weather is shitty 340 days out of the year? At least you’ll be homeowner in the Bay Area — and nothing says you’ve “made it” like being able to afford a down payment. Besides, San Bruno is “up and coming” — and Youtube has an office there.

Your commute to work sucks, but at least its an opportunity to catch up on Podcasts so you can have great conversations over cocktails with your friends. Should you listen to “Serial Season 2” today? Or should you listen to that amazing “Startup” podcast? So many choices, so little time. You instead decide to expand your horizons by trying a new playlist on Spotify — something about Indian-infused-jazz music. It sounds great. It makes you feel cultured.

You decide to park your car using “Luxe” today. You justify it to yourself by saying that parking garages are only $10 less expensive. And you have to spend all of that time walking back and forth. And besides — today you are meeting some friends after work for dinner and you’ll be on the other end of town. You can’t decide whether you’ll take Uber or Lyft to the dinner from your office — decisions, decisions.

You are the Director of Business Development at your startup. You aren’t even sure what that means, but the startup seems to be doing well. Your company recently raised a round and was featured in Techcrunch. You have 5,000 stock options. You aren’t exactly sure what that means, but that must be good. If you exit, maybe that will mean money toward a down payment.

Your day starts in Salesforce. You have to email a bunch of people. You briefly contemplate a business idea you have that will totally kill Salesforce and Facebook at the same time. But you need a technical co-founder. Eventually you’ll get to it — after all, you’re smart and destined for greatness yourself. And your friends all tell you how you should start something someday.

Your 27-year-old CEO calls an ad-hoc all-hands meeting and regales about company culture and how your mission is to “kill email because it’s broken”. He wants to make every enterprise company in the world switch to your product. He’s never worked for an enterprise company, or any other company at all.

The sales team got rowdy the night before. They missed their quota, but it was not their fault — it was implementation’s fault for fucking up a major deal. Also — marketing didn’t send them enough inbound leads for them to hit quota. Maybe next quarter. You trade emails with your college buddies on Gmail about how ridiculous Kevin Durant is for joining the Warriors. You come to realize email is working just fine for you. You feel depressed for a moment. Your summer intern is trying to figure out a Snapchat strategy.

Philz Coffee — photo credt: Rick.

It’s time for that afternoon coffee to keep you going through the day. You head over to Philz with some co-workers. You order a vegan donut and very clearly ask the barista for 3 Splendas. He was clearly a Splenda short, but the line is long and you want to be civil. You are above mentioning something like this to the barista — you let it pass and feel a “micro aggression” bubbling inside.

You have to decide where to go for dinner tonight. You look at Yelp for a place that’s within 1 mile and is rated at least 3.5 stars. But really you’re looking for something 4 stars plus and at least $$$. What will your friends think of you if you pick a place that’s too cheap? But you also don’t want to go $$$$ because that’s too expensive. You have good taste. This comforts you.

You realize your reservation with your spouse at the French Laundry is coming up this weekend. Your calendar app reminds you of this. You’ve been looking forward to it for months. You can’t wait to take perfectly Instagrammed photos of the meal to go along with your perfectly Instagrammed life.

#San Francisco is trending on Twitter. You realize the San Francisco journalism community is angry about something — they are full of rage at the way a homeless person is being treated. The reporters all share photos and videos of the homeless person, but no one talks to him.

It’s time for some afternoon Facebook browsing. Your friends are all doing SO well. You are secretly jealous of your friend who just bought a house in the Noe. You speculate as to how rich they must be after their exit from LinkedIn. Even though they were only employee #500 they must have done well. You briefly try to do the math in your head. Maybe that can be you at your current startup. It’s only a matter of time.

More browsing. One friend was employee #5 at a company that just sold to Twitter. They must have made so much money, you think. You like the status, but you are jealous. Another friend’s kid seems to be more advanced than your kid based on the Vine they just shared of them playing the piano. Damnit, need to be a better parent.

You go to Redfin to see how much they paid for their house.

You briefly daydream about how you once had an opportunity to work at Google pre-IPO. And that you could have joined Facebook right after IPO — and imagine that — the stock price has tripled in a short amount of time. Would that have been the big break you needed?

Your CEO grabs you in a panic and asks you to do a quick analysis for a board member. The board member was base jumping in Mexico and panicked about something related to burn rate and strategy. The CEO’s job is at risk.

Microsoft Excel — photo credit Collin Anderson.

You do the grunt work and analysis, and finish it just in time for him to breathe a sigh of relief and tell you what an “Excel Ninja” you are. Your analysis makes you realize the company maybe should have saved money on office space, and perhaps the rock climbing wall and Segways. You realize your CEO knows nothing about your business.

Your mind briefly drifts off and you think — “is this all really worth it? should I move to Seattle, Austin, or maybe even Florida?” After all there is no state tax and you could live a great quality of life there with an actual house with your beautiful family.

You browse Redfin again. Hmmm. Maybe not Austin — what about something less ambitious like Fremont, Morgan Hill or Milpitas? That wouldn’t solve your commute problems, you think. It would be more affordable though.

Delicious looking cupcakes — photo credit Frederic Bisson.

You know what? If you move to Austin you could somehow get by. After all your spouse is so amazing at baking. She could easily make a living selling her cupcakes — she has so much talent as a cook and you could afford culinary school. Worst case, she also has an amazing knack for craft jewelry. The three pieces she sold on Etsy last month are evidence of that. How talented both of you are.

And hey — if you move to Austin, you can finally build that home with a “Zen minimalist” theme you’ve been dreaming of. You go to Bluhome’s website — their design aesthetic perfectly matches yours. You just need to save the money to make it happen. You browse Pinterest and Houzz for ideas on how to decorate the interior. Is Red or Navy Blue TOO bold of a color? You don’t know. Maybe you should use an on-demand service for that.

You forgot to order groceries and the nanny needs milk for your kid ASAP. She texts you frantically in broken English. Thank goodness for Instacart — you spend $10 in delivery costs, but you need to add a bunch of items to your cart to hit the minimum threshold. You add a few squeezies, some bananas and a few artisan cheeses to hit the mark. You realize you haven’t stepped into a grocery store for months — but don’t worry — your opportunity cost of time is way too high at the moment. Especially if you factor in those stock options.

Almost time for dinner. You are having dinner tonight with the “Chief Hacking Officer” at the company and the “VP of Awesomeness”. You arrive at the restaurant, and they marvel at your taste — nice job surfing Yelp.

Your dinner conversation centers around how autonomous vehicles are going to be better in the long run than ordinary cars for a variety of reasons. And something about how Elon Musk handles meetings. You are all too busy making your own points and citing articles to really listen to each other. You order the $17 dollar Risotto and the $9 glass of Pleasanton-brewed IPA.

On your ride home you find the time to catch up on the Malcolm Gladwell podcast. What an interesting guy he is — he’s so smart and he makes you think about things.

After coming home you briefly use that “7 minute workout” app, which scientists have proven is way more effective than a one-hour cardio workout. You got your exercise in for the day — nice work.

You and your spouse get ready for bed. What’s in your Netflix queue? Well, you have to catch up on “Making a Murderer” since it’s been all over the news lately. And let’s not get too far behind on “Mr. Robot” since it’s so critically acclaimed. For lighter fare, and if you have time, you can always try “Last Week Tonight” — John Oliver always says exactly what you’re thinking in your head — just funnier than you would have said it.

You quietly shuffle to bed, tired from the long, hard day. You check your email, Twitter, Facebook and Snapchat one last time before bedtime. You don’t think you’ll have enough energy to check LinkedIn today — and besides — their mobile UI is not very good. Maybe you can start a company that will disrupt LinkedIn? They did just sell for a bunch of money after all.

Your last thought before bed — should you switch to the Android ecosystem? You are on the “S” iPhone replacement cycle and you are getting impatient. But then you realize you are so heavily invested in the Apple ecosystem that it may not make sense.

Vipassana Retreat — photo credit kinnla.

You briefly use mobile Safari to browse for Vipassana retreats — you hear a 10 day retreat in Soquel may be the ticket to shake things up. You realize it’s not going to be possible. You download a meditation app. You turn it off. You don’t have time.

You briefly recall your ride home on the 280 tonight. The sun was setting. It was beautiful. You realize you live in paradise.

Next Story — Eleven Reasons To Be Excited About The Future of Technology
Currently Reading - Eleven Reasons To Be Excited About The Future of Technology

Eleven Reasons To Be Excited About The Future of Technology

“The strongest force propelling human progress has been the swift advance and wide diffusion of technology.” — The Economist

In the year 1820, a person could expect to live less than 35 years, 94% of the global population lived in extreme poverty, and less that 20% of the population was literate. Today, human life expectancy is over 70 years, less that 10% of the global population lives in extreme poverty, and over 80% of people are literate. These improvements are due mainly to advances in technology, beginning in the industrial age and continuing today in the information age.

There are many exciting new technologies that will continue to transform the world and improve human welfare. Here are eleven of them.

1. Self-Driving Cars

Self-driving cars exist today that are safer than human-driven cars in most driving conditions. Over the next 3–5 years they‘ll get even safer, and will begin to go mainstream.

The World Health Organization estimates that 1.25 million people die from car-related injuries per year. Half of the deaths are pedestrians, bicyclists, and motorcyclists hit by cars. Cars are the leading cause of death for people ages 15–29 years old.

Just as cars reshaped the world in the 20th century, so will self-driving cars in the 21st century. In most cities, between 20–30% of usable space is taken up by parking spaces, and most cars are parked about 95% of the time. Self-driving cars will be in almost continuous use (most likely hailed from a smartphone app), thereby dramatically reducing the need for parking. Cars will communicate with one another to avoid accidents and traffic jams, and riders will be able to spend commuting time on other activities like work, education, and socializing.

Source: Tech Insider

2. Clean Energy

Attempts to fight climate change by reducing the demand for energy haven’t worked. Fortunately, scientists, engineers, and entrepreneurs have been working hard on the supply side to make clean energy convenient and cost-effective.

Due to steady technological and manufacturing advances, the price of solar cells has dropped 99.5% since 1977. Solar will soon be more cost efficient than fossil fuels. The cost of wind energy has also dropped to an all-time low, and in the last decade represented about a third of newly installed US energy capacity.

Forward thinking organizations are taking advantage of this. For example, in India there is an initiative to convert airports to self-sustaining clean energy.

Airport in Kochi, India (source: Clean Technica)

Tesla is making high-performance, affordable electric cars, and installing electric charging stations worldwide.

Tesla Model 3 and US supercharger locations

There are hopeful signs that clean energy could soon be reaching a tipping point. For example, in Japan, there are now more electric charging stations than gas stations.

Source: The Guardian

And Germany produces so much renewable energy, it sometimes produces even more than it can use.

Source: Time Magazine

3. Virtual and Augmented Reality

Computer processors only recently became fast enough to power comfortable and convincing virtual and augmented reality experiences. Companies like Facebook, Google, Apple, and Microsoft are investing billions of dollars to make VR and AR more immersive, comfortable, and affordable.

Toybox demo from Oculus

People sometimes think VR and AR will be used only for gaming, but over time they will be used for all sorts of activities. For example, we’ll use them to manipulate 3-D objects:

Augmented reality computer interface (from Iron Man)

To meet with friends and colleagues from around the world:

Augmented reality teleconference (from The Kingsman)

And even for medical applications, like treating phobias or helping rehabilitate paralysis victims:

Source: New Scientist

VR and AR have been dreamed about by science fiction fans for decades. In the next few years, they’ll finally become a mainstream reality.

4. Drones and Flying Cars

“Roads? Where we’re going we don’t need… roads.” — Dr. Emmet Brown

GPS started out as a military technology but is now used to hail taxis, get mapping directions, and hunt Pokémon. Likewise, drones started out as a military technology, but are increasingly being used for a wide range of consumer and commercial applications.

For example, drones are being used to inspect critical infrastructure like bridges and power lines, to survey areas struck by natural disasters, and many other creative uses like fighting animal poaching.

Source: NBC News

Amazon and Google are building drones to deliver household items.

Amazon delivery drone

The startup Zipline uses drones to deliver medical supplies to remote villages that can’t be accessed by roads.

Source: The Verge

There is also a new wave of startups working on flying cars (including two funded by the cofounder of Google, Larry Page).

The Terrafugia TF-X flying car (source)

Flying cars use the same advanced technology used in drones but are large enough to carry people. Due to advances in materials, batteries, and software, flying cars will be significantly more affordable and convenient than today’s planes and helicopters.

5. Artificial Intelligence

‘’It may be a hundred years before a computer beats humans at Go — maybe even longer.” — New York Times, 1997
“Master of Go Board Game Is Walloped by Google Computer Program” — New York Times, 2016

Artificial intelligence has made rapid advances in the last decade, due to new algorithms and massive increases in data collection and computing power.

AI can be applied to almost any field. For example, in photography an AI technique called artistic style transfer transforms photographs into the style of a given painter:

Source

Google built an AI system that controls its datacenter power systems, saving hundreds of millions of dollars in energy costs.

Source: Bloomberg

The broad promise of AI is to liberate people from repetitive mental tasks the same way the industrial revolution liberated people from repetitive physical tasks.

“If AI can help humans become better chess players, it stands to reason that it can help us become better pilots, better doctors, better judges, better teachers.” — Kevin Kelly

Some people worry that AI will destroy jobs. History has shown that while new technology does indeed eliminate jobs, it also creates new and better jobs to replace them. For example, with advent of the personal computer, the number of typographer jobs dropped, but the increase in graphic designer jobs more than made up for it.

Source: Harvard Business Review

It is much easier to imagine jobs that will go away than new jobs that will be created. Today millions of people work as app developers, ride-sharing drivers, drone operators, and social media marketers— jobs that didn’t exist and would have been difficult to even imagine ten years ago.

6. Pocket Supercomputers for Everyone

By 2020, 80% of adults on earth will have an internet-connected smartphone. An iPhone 6 has about 2 billion transistors, roughly 625 times more transistors than a 1995 Intel Pentium computer. Today’s smartphones are what used to be considered supercomputers.

Visitors to the pope (source: Business Insider)

Internet-connected smartphones give ordinary people abilities that, just a short time ago, were only available to an elite few:

“Right now, a Masai warrior on a mobile phone in the middle of Kenya has better mobile communications than the president did 25 years ago. If he’s on a smart phone using Google, he has access to more information than the U.S. president did just 15 years ago.” — Peter Diamandis

7. Cryptocurrencies and Blockchains

“If you asked people in 1989 what they needed to make their life better, it was unlikely that they would have said a decentralized network of information nodes that are linked using hypertext.” — Farmer & Farmer

Protocols are the plumbing of the internet. Most of the protocols we use today were developed decades ago by academia and government. Since then, protocol development mostly stopped as energy shifted to developing proprietary systems like social networks and messaging apps.

Cryptocurrency and blockchain technologies are changing this by providing a new business model for internet protocols. This year alone, hundreds of millions of dollars were raised for a broad range of innovative blockchain-based protocols.

Protocols based on blockchains also have capabilities that previous protocols didn’t. For example, Ethereum is a new blockchain-based protocol that can be used to create smart contracts and trusted databases that are immune to corruption and censorship.

8. High-Quality Online Education

While college tuition skyrockets, anyone with a smartphone can study almost any topic online, accessing educational content that is mostly free and increasingly high-quality.

Encyclopedia Britannica used to cost $1,400. Now anyone with a smartphone can instantly access Wikipedia. You used to have to go to school or buy programming books to learn computer programming. Now you can learn from a community of over 40 million programmers at Stack Overflow. YouTube has millions of hours of free tutorials and lectures, many of which are produced by top professors and universities.

UC Berkeley Physics on Youtube

The quality of online education is getting better all the time. For the last 15 years, MIT has been recording lectures and compiling materials that cover over 2000 courses.

“The idea is simple: to publish all of our course materials online and make them widely available to everyone.” — Dick K.P. Yue, Professor, MIT School of Engineering

As perhaps the greatest research university in the world, MIT has always been ahead of the trends. Over the next decade, expect many other schools to follow MIT’s lead.

Source: Futurism

9. Better Food through Science

Source: National Geographic

Earth is running out of farmable land and fresh water. This is partly because our food production systems are incredibly inefficient. It takes an astounding 1799 gallons of water to produce 1 pound of beef.

Fortunately, a variety of new technologies are being developed to improve our food system.

For example, entrepreneurs are developing new food products that are tasty and nutritious substitutes for traditional foods but far more environmentally friendly. The startup Impossible Foods invented meat products that look and taste like the real thing but are actually made of plants.

Impossible Food’s plant-based burger (source: Tech Insider)

Their burger uses 95% less land, 74% less water, and produces 87% less greenhouse gas emissions than traditional burgers. Other startups are creating plant-based replacements for milk, eggs, and other common foods. Soylent is a healthy, inexpensive meal replacement that uses advanced engineered ingredients that are much friendlier to the environment than traditional ingredients.

Some of these products are developed using genetic modification, a powerful scientific technique that has been widely mischaracterized as dangerous. According to a study by the Pew Organization, 88% of scientists think genetically modified foods are safe.

Another exciting development in food production is automated indoor farming. Due to advances in solar energy, sensors, lighting, robotics, and artificial intelligence, indoor farms have become viable alternatives to traditional outdoor farms.

Aerofarms indoor farm (Source: New York Times)

Compared to traditional farms, automated indoor farms use roughly 10 times less water and land. Crops are harvested many more times per year, there is no dependency on weather, and no need to use pesticides.

10. Computerized Medicine

Until recently, computers have only been at the periphery of medicine, used primarily for research and record keeping. Today, the combination of computer science and medicine is leading to a variety of breakthroughs.

For example, just fifteen years ago, it cost $3B to sequence a human genome. Today, the cost is about a thousand dollars and continues to drop. Genetic sequencing will soon be a routine part of medicine.

Genetic sequencing generates massive amounts of data that can be analyzed using powerful data analysis software. One application is analyzing blood samples for early detection of cancer. Further genetic analysis can help determine the best course of treatment.

Another application of computers to medicine is in prosthetic limbs. Here a young girl is using prosthetic hands she controls using her upper-arm muscles:

Source: Open Bionics

Soon we’ll have the technology to control prothetic limbs with just our thoughts using brain-to-machine interfaces.

Computers are also becoming increasingly effective at diagnosing diseases. An artificial intelligence system recently diagnosed a rare disease that human doctors failed to diagnose by finding hidden patterns in 20 million cancer records.

Source: International Business Times

11. A New Space Age

Since the beginning of the space age in the 1950s, the vast majority of space funding has come from governments. But that funding has been in decline: for example, NASA’s budget dropped from about 4.5% of the federal budget in the 1960s to about 0.5% of the federal budget today.

Source: Fortune

The good news is that private space companies have started filling the void. These companies provide a wide range of products and services, including rocket launches, scientific research, communications and imaging satellites, and emerging speculative business models like asteroid mining.

The most famous private space company is Elon Musk’s SpaceX, which successfully sent rockets into space that can return home to be reused.

SpaceX Falcon 9 landing

Perhaps the most intriguing private space company is Planetary Resources, which is trying to pioneer a new industry: mining minerals from asteroids.

Asteroid mining

If successful, asteroid mining could lead to a new gold rush in outer space. Like previous gold rushes, this could lead to speculative excess, but also dramatically increased funding for new technologies and infrastructure.


These are just a few of the amazing technologies we’ll see developed in the coming decades. 2016 is just the beginning of a new age of wonders. As futurist Kevin Kelly says:

If we could climb into a time machine, journey 30 years into the future, and from that vantage look back to today, we’d realize that most of the greatest products running the lives of citizens in 2050 were not invented until after 2016. People in the future will look at their holodecks and wearable virtual reality contact lenses and downloadable avatars and AI interfaces and say, “Oh, you didn’t really have the internet” — or whatever they’ll call it — “back then.”
So, the truth: Right now, today, in 2016 is the best time to start up. There has never been a better day in the whole history of the world to invent something. There has never been a better time with more opportunities, more openings, lower barriers, higher benefit/ risk ratios, better returns, greater upside than now. Right now, this minute. This is the moment that folks in the future will look back at and say, “Oh, to have been alive and well back then!”

Sign up to continue reading what matters most to you

Great stories deserve a great audience

Continue reading