The ‘Overload’ Era

Sandeep Mangalath
Jul 24, 2017 · 4 min read

The rate at which information perpetuates around is overwhelming. For a perspective research says by end of 2016 global internet traffic will reach 1.1 Zettabyte per year and by 2019 it is expected to go up to 2 Zettabyte. If you zoom in, it can hold 36,000 years of HD video. For things to get worse (hmm not really! Good for Artificial Intelligence, aka AI which is yearning for data) the data is dynamic and unpredictable. Our interests, hobbies and leisure end up with mammoth data which is posted, uploaded and streamed over the Internet.

The scale is astronomical and the cause of this is predominantly we are human beings and we love to share. Information overload we experience is like a raging bull which we are ready to take head on. Social media networks streamlines this chaos we create with our own consent on news feeds personalized for our devices identified by our logins. Google is working on password-free login which will void the idea of ‘taking our consent’ for using our private information. I didn’t intend to be a privacy prick in this write-up, despite of that I had to point this out.

A fleet of Aquila airplanes communicating via free space optics — aka lasers. (Courtesy — Wired)

Our telecom last mile is very critical and we always take it for granted but sadly our provider does is more than we do. Traffic aggregation and management has been historically poor and still continues with the virtue of our Government regulations which are Victorian and seamless to purge any prohibition against telecom providers. Google’s project Loon, Facebook’s behemoth idea of Internet connectivity using drone (Aquila) is aimed at the less fortunate (due to political or geographical reasons) population of the world. Facebook just released an Open-source map of Internet connectivity of certain cities of the world and the data is staggering. Another project ‘Voyager’ (TIP — Telco Infra Project) by Facebook attempts to handhold telecom providers with a platform (powered by Open-source hardware and software) and cheaper infrastructure in African nations.

Social media networks drive this age and they have planted the seeds deep and rooted hard in challenging the status quo of service providers and their monopoly. Disruption is caused and service providers can only adopt and tether in. “Time” is only relative and the analogy to prove it just happens. Time Warner would be sold for $85.4 Billion and Spanchat files IPO at a valuation of $25 Billion. One a service provider and another social media network with two decades in between (Time Warner founded in 1992 and Snapchat in 2011). Industry wishfully call them OTT player which is sought of a weird name for these cool Social media companies. Social media would be the lifeline of Internet and run networks of future.

Facebook and Microsoft are laying a giant submarine cable across Atlantic.

Google and Facebook building super high-speed cable between LA and Hong Kong

Though data is quantifiable for a large extent; classifying, cataloging, indexing, querying and co-relating it to a meaningful logic needs sophisticated computing. The scaling out of compute by Cloud powers all the data chugging in and out from those shards. There are robust applications which make sense of all this data and give you meaningful visualization. Open source applications have heeded to this need and are core to many ISVs who ran licensed proprietary software for years from Operating Systems to Databases.

Social networks possess the compute capacity and connectivity tuned to serve this colossal amount of data. Physically they are placed in different PoP across the world and the content is always locally served to a user who is local. The dire requirement is to be available at any cost to the user who just spins in or out of the service at no price. They make money by Ads served, what Google mastered so elegantly. There is no social media network that hasn’t traveled the path which Google did few years back.

Four years before Facebook was founded Larry Page said “Artificial Intelligence would be the ultimate version of Google”. Six months back in March 2016 Google’s Deep Mind sealed victory challenging Korean Lee Sedol, who is considered to be the champion of the decade in the game Go. AI has evolved since the days of DeepBlue. It’s no more the computer which was configured for an algorithm for playing chess against Grand masters. Today AI is programmed to learn, see and listen. Deep Mind’s neural network uses tons of data to learn and unlearn from pattern/face/voice recognition to aiming a spec on a block game.

“The Overload Era” should be more surprising than shocking in the world of connected things. The Ethical reality of any technology had been exploited by the in-correct means in which it is explored and that very reason can’t ground it. Machine learning and Deep Learning are technology avenues that distill Data into engineering marvels.

So the next time you Post, Tweet, Upload, Update, Like, Comment, Blog or even raise a security alarm, remember your data is absorbed by some neural network to teach itself and realize that after-all you were not a douche-bag :)