Building Like Google
Discovering How to Build Solutions When You Are Not Google
I must admit, I have somewhat of an odd hobby. Although for some my walking uphill with a snowboard might qualify, that is not the one I am aiming at today. No, I like discovering and applying the principles by which, I think, Google develops software.
But this hobby is hampered by two problems. One: I do not, nor ever did, work at Google. So I was never surrounded by senior engineers that would patiently teach me (or company chefs serving awesome lunches daily). Two: my customers are usually not Google, which brings another set of limitations (and less fancy lunches).
The largest (and arguably most exciting) part of my hobby revolves around reverse designing Google’s workflow. To this end I read (between the lines of) their blogs, observe systems and tools they release in Google Cloud and analyse the structure of (open source) code they write. I try to discern that grand unifying principle, if there is one to be found. And I think I might be on the right track.
But in the second part of my little hobby, applying those principles, I still run into the other problem. Because one downside of neither me nor my customers being Google, is that we have considerably fewer resources. Now when I say resources, you might immediately think of the raw computational power of their data centres, or the enormous collection of quality data therein.
But provided there is value in a computationally demanding workload, elastic public clouds can accommodate for the former. And while the latter is definitely more difficult to match, data in and of themselves do nothing. No, I primarily think of the disparity in human resources; the skilled professionals that do the work. Here are two simple examples to illustrate my point.
On the application level, we non-Googlers have all kinds of unstable, faulty legacy software that feeds us data in obsolete formats through cumbersome means. Yes, I am looking at you, nasty SOAP XML interfaces. And even when we choose to ignore (generally sensible) guidelines like ‘minimise investments in legacy’, we still would not have the people available to (reasonably) swiftly perform the required changes. We have to make do.
On the security level, while we we may have security and penetration testing teams, they are not on the same level as aforementioned company. So to keep things reasonably secure, we run our (legacy) backend application servers in private, segregated networks shielded by firewalls. We then pray that no one manages to get into those as he or she would be shooting fish in a barrel.
So things might seem pretty bleak. But when I started with developing on Google Cloud late 2015 (incidentally kick-starting this odd hobby of mine), hope sparked.
Back in those days (feels like ages), Google Cloud was a pretty spartan affair. We had basic VMs, AppEngine, BigQuery, Blob Storage and a handful of other services. We did not even have any kind of managed Hadoop. Life was simpler, but we were happy. For the sake of accuracy though, do replace ‘life’ with ‘documentation’ and ‘simpler’ with ‘spotty’.
But thanks to this frugality, I was able to discern a system, a pattern. Google Cloud has always had a strong offer of managed services, dubbed platform-as-a-service (or PaaS). Developers can simply push code to these services and receive web servers, big data pipelines and such in return. These will operate according to professional (reliability, security, et cetera) standards without involvement of any system operator.
I noticed that each of these PaaS services could be plotted somewhere in what I now like to call the technology triangle of compute, storage and transfer. But more often then not, they would reside in the corners of a triangle; they only took care of one concern. AppEngine only provided a web server (compute). Datastore only provided quick read-write storage. PubSub only moved data around (transfer).
Combining services from across the triangle, I could quite rapidly build scalable and maintainable (and secure) landscapes, all by my little self. The need to include several different services to achieve things other technologies provided as a one-stop-shop initially seemed inconvenient. But I quickly realised, and leveraged, its value. I could nimbly swap or add components or services as requirements changed.
If I for instance was using BigQuery for storage and real-time access became necessary, adding a simple write to Datastore would make that happen. However, unlike BigQuery, Datastore does not do compute. So while I could subsequently implement a data transformation pipeline in BigQuery using just queries, I could not do so in Datastore. There are plenty of solutions to this ‘problem’; the point I am trying to make is that the further a technology is from a corner, the harder it can be to replace. Plotting technologies in the triangle can make you better aware of this, aiding your decision making.
But the triangle does not capture another, important dimension of good design. This one was taught to me (rather explicitly) as I was introduced to Google Cloud and platform-as-a-service. Because it was all about layering; software-as-a-service on top platform-as-a-service on top of infrastructure-as-a-service. I now apply a similar layering scheme to any system I analyse or design. From bottom to up: infrastructure (VMs and networking), application (web services, databases, applications) and business (people using applications). Like with the OSI model, each layer extends on what is provided by a lower layer.
You could also insert extra layers; for instance platforms like Kubernetes and Hadoop. These tend to reside between infrastructure and application. When plotted in my technology triangle, they would likely end up in the centre; they could provide everything, often doing so in a reasonably well integrated fashion. But for setting up and maintaining these on a professional level, you do require people with specialised skill sets. This combination of benefits and investments can often make migrating to and from them non-trivial (aka: lock-in). Do note that paradoxically Kubernetes does decrease lock-in with infrastructure providers (clouds) or hardware suppliers.
A final ingredient to the mix is to design for and with microservices. A paradigm shift that has been hard to miss, regardless of whether you were following Google or not. Unlike with monolithic applications, developers in microservice landscapes do not need to concern themselves too much with negatively impacting or getting impacted by other parts of the application. Their microservice is the entire application and as long as it fulfils its contracts with its downstream clients and upstream services, everything should be peachy.
This, once again, is a separation of concerns. But now it is horizontally across (business) features or services rather than vertically, as with the technology stack. You could plot them together in a grid where the services form the capstones of the technology stacks. We can even fit software-as-a-service solutions (like for instance G-Suite) in there.
So separation of concerns is going to be our theme. Because no one is an expert at everything. Not being Google, at times we might even struggle to get sufficient people to handle basic hygiene (like system maintenance). We must then force such vital roles upon the people we have available. But developers often do not like to nor are they good at maintaining VMs (or fixing computers). Data scientists do not like to be doubling as database administrators (or computer fixers). And system operators probably do not care much for implementing business logic (but they might like fixing your computer —higher chances if it runs on Linux).
I believe that people are happiest when they can do their own thing. The thing or things they like doing and (consequently?) are good at. So let us prevent placing ourselves in situations where we need to foist chores upon our fellow engineers (or ‘underlings’ if you so prefer). Carefully decompose your problem into distinct functions. Carefully design those functions with plugable compute, storage, and transfer technologies in mind. And then carefully select infrastructure or platform-as-a-service to match your current talent pool.
To me building like Google is about enabling engineers to focus on building the parts they like to build. You do not need to be Google to achieve that. So let’s start making it happen.
