The Epic Moment of Hybrid Cloud

On the wine-dark sea …

Yes. This starts with a conjuring of the famed night sea to which a distraught and betrayed Achilles beckoned, a conjuring of the tumultuous waves Odysseus faced at the twin monsters, Scylla and Charybdis, a conjuring of the hurricane in the open deep that engulfed Aeneas’ disoriented crews then spit the survivors out onto a tangled, unfamiliar shore. The wine-dark sea was, by turns, an alluring or stormy place in the ancient world, conveying goods and commerce, then swallowing ships whole.

Cloud computing, in its unwieldy definitional grandeur, opens itself up to all kinds of adventurous, secondary analogies. Because they work. The cloud is an alluring landscape (or skyscape or seascape), populated by heroic journeys into unknown depths. It is a jungled coast, tangled and teeming. Its old growth and exotic new features are being hacked away by an elite DevOps troop on the one hand, by powerful venture kings launching a thousand ships on the other. As in a good epic, our heroes cut away the cultural and technical norms of computing and call into question foundations, the universal ‘on-premise’ architectures beneath everyone’s feet.

In May, in a keynote at Beyond Tellerrand, Maciej Cegłowski, was a bit like a modern Laocoon. With prophetic insight, he paused to survey the unknown and described the cloud this way:

“And then there’s the cloud. The cloud fascinates me because of the distance between what it promises and what it actually is. The cloud promises us complete liberation from the mundane world of hardware and infrastructure. It invites us to soar into an astral plane of pure computation, freed from the weary bonds of earth. What the cloud is is a big collection of buildings and computers that we actually know very little about […]. Who knows what angry sysadmin lurks inside the cloud? The cloud is a fog of sweet, sweet promises. Amazon promises eleven nines of durability. Eleven nines! The Sun will be a charred cinder before a single bit gets flipped in a file you’ve stored on S3. Amazon promises no single points of failure. Instead, you get a single cloud of failure, the promise that when the system comes crashing down, at least you won’t be alone. The problem with these points of control, of course, is that they make it very easy to spy on everyone.”

Mr. Cegłowski is trying to tell us something important here. Laocoon was right about the Trojan horse after all, as Code Spaces might tell us too if the dead could speak. But, he was wrong about saving the city. Troy, fortress of self-sufficiency and the very model of rooted infrastructure, was falling whether he liked it or not. It burned to the ground, at the will of the gods, never to be rebuilt in the same way, nor in the same land. And, so it is with the advent of the cloud. Its new territory is destined for empire, as surely as the remnants of Troy fused with fierce Italic houses to engender Rome. Likewise, its people and culture are in transition.

With transition comes equivocation of every sort. Cloud culture has been aptly characterized as something that “has become over the past decade a mess of choices and fixes.” The pace of change, the rate of tool proliferation, the sheer volume of contributions — all are humbling. Every day new caveats appear in new equations. And, we face a growing, valid trend to move beyond the language of virtualization in order to genuinely describe “cloud-first” building, to meaningfully document distributed, automated, ephemeral environments, disposable immutable infrastructures, agile development. Perhaps most troubling–because the need is so primary–is the wide definitional ambiguity of the cloud itself.

Precision matters in computing, like philology. So, let’s parse.

A few years ago, NIST (National Institute of Standards and Technology), the last word in the U.S. on the sweeping and sophisticated measurements that touch every aspect of life and commerce, (new atomic clock and quantum droplets aside!), took a valiant stab at defining the cloud and its attendant nexi.

“Cloud computing is an evolving paradigm. … Cloud computing is a model for enabling ubiquitous, convenient, on-demand network access to a shared pool of configurable computing resources (e.g., networks, servers, storage, applications, and services) that can be rapidly provisioned and released with minimal management effort or service provider interaction.This cloud model is composed of five essential characteristics, three service models, and four deployment models.”

Five essential characteristics … three service models … four deployment models … evolving paradigm. For Zeus’s sake! Cloud computing gives us on-demand self-service (DIY 24/7), broad network access (by heterogenous clients, be they mobile device, laptop, or workstation), resource pooling (with location independence), rapid elasticity (commensurate with demand), and measured service (with metering capability much like utilities). That’s five. And, Martin Fowler, among others, approves of them.

Cloud providers are service providers. They might be in the business of giving the consumer software as a service (SaaS), platform as a service (PaaS), infrastructure as a service (IaaS), variations and combinations of those (i.e., tying PaaS to an operating environment like AWS’s Elastic Beanstalk or PaaS as backend platform for mobile apps, called mBaaS), and even sexy, “bare-metal” services (MaaS). So, perhaps three-plus service models. To further refine it, some providers of these services act as vendors; some are inventors embracing open source and open stacks; many start-ups and DevOps types are certainly both. Rackspace, with its new “managed cloud” offering, even wants to be on your payroll: “We’ll run your cloud ops for you. For $100/ month, get 24x7x365 service from a dedicated engineering team who’ll manage your environment as if they’re a member of your team.” Sounds like a deal. But, the real deal is that this taxonomy is becoming anachronistic. Services themselves are the new unit of cloud and faceted classification will rise as an approach far more useful than the IaaS-PaaS-SaaS triumvirate. (See: http://luminal.com/blog/2014-07-29-something-as-a-service.html)

What about the deployment models? How is the cloud infrastructure–the shared pool of computing resources–deployed quadruply per NIST? From a user’s perspective, you’re swimming in a Private pool, a Community pool, a Public pool, or, of course, a Hybrid–the swanky hotel pool with two sections, the spa area for special clients and the sprawling waterworld of noisy splashes and occasionally clogged filters for everyone else.

The hybrid cloud, says NIST, is two or more of the former infrastructures bound together by standardized or proprietary technology that enables data and application portability, like load balancing between clouds. It may or may not involve legacy systems and on-premise data centers.

Hybrid is the emerging superstar. That’s logical. Its compromise is winning over proponents exponentially and on a global scale. Gregory Ness, writing for Nasdaq online earlier this year, made bold but prescient projections:

“At stake in the short term is a potential $60B market based on a 3-year server refresh rate, for hybrid cloud software that would run on servers and allow apps to scale into one or more public clouds.Then there is the $1T+ spending on traditional IT, that could eventually shift to the cloud when one looks out 10+ years. That takes us to the second major development: the rise of hybrid cloud automation in the increasingly critical middle ground between the data center and each IaaS offering. … When the hybrid cloud takes hold -and many think that 2014 will be the year of hybrid cloud- the market power will shift from hardware providers monetizing complexity to software and service providers monetizing agility.”

Hybrid is the very heart of transition. Industries don’t generally quit cold-turkey. If they’re viable, they transform. The challenge always is to see the transformation while you’re living it, predict and strategize accordingly.

NYT’s Quentin Hardy, citing industry analysts, put more numbers on the transition: “ … if largely cloud-based things like mobile apps, big data, and social media are counted, over the next six years almost 90 percent of new spending on Internet and communications technologies, a $5 trillion global business, will be on cloud-based technology.”

The general optimism in all this is impressive; there may be lots of hype, but there’s palpable hope. Hybrid actually may deliver the advantages of a diversified portfolio. An enterprise might slot more sensitive data within its virtual private cloud (VPC), a distinct, isolated network within a cloud according to Amazon (AWS). It might then leverage access to robust applications and to the sheer power and scope of compute resources in the public cloud for big data queries and for tasks that don’t expose financial and personally identifiable information. It might spin up in a cloud and augment its work to deal with some big external event or seasonal demand (the World Cup, a summer tourism jolt, etc.). A small business might scale up and extend its reach to regional and global markets with the Google Cloud Platform (growth not feasible a decade ago for a little guy), while using a community cloud to meet contractual and compliance obligations for another part of its business function. But, the diversification isn’t painless. In epic, every time three or four gods, with conflicting priorities and different spheres of control, got mixed up in a hero’s journey, our man and his mission got lost.

The hubris-inducing challenge of hybrid is to ensure smooth connections across big systems with internal infrastructure dependencies, seamless provisioning of activity across data centers and clouds, uninterrupted service in an environment of complex networking, consistent real- time interface data for both consumers and the employees helping them, and security, security, security. Failures of integration and communication can batter and sink the ship. With shipwrecks all too commonplace, the fear and pollution of being lost at sea is legitimate. The salience and wisdom of Mr. Cegłowski’s words re-emerge.

Nevertheless, the challenge is being faced with a degree of gravitas perhaps previously scant. This spring 2014, Cisco Systems unrolled their $1 billion initiative to introduce the global intercloud, that leverages OpenStack and allegedly will “support any workload, on any hypervisor and interoperate with any cloud.” VMware’s NSX, introduced in the fall of 2013, provides virtual networks through any cloud management platform leveraging the NSX APIs; they “can be arranged in any topology with isolation and multi-tenancy” and “deployed non-disruptively over any existing network and on any hypervisor.” Even Andy Jassy, at the AWS Summit in San Francisco in March, changed tone and specifically pointed to AWS features and third party partners like NetApp that enable hybrid. “We know that many enterprises have data center assets they’re not ready to retire yet,” Jassy said in a marked departure from his stance at re:Invent 2013.

This particular transition in human technology is only a flashing moment in measures of human invention. That it is one of at least a dozen major shifts in foundation already defining the century makes it epic. Cloud technology, along with mobile internet, IoT, advanced robotics, autonomous vehicles, next generation genomics, energy storage and renewables growth, advanced materials development, 3D printing, et alia, marks powerful economic disruption that the hero in each of us is facing, ready or not.