Understanding the Pattern of Technology Evolution

I recently started writing a book about Web Architecture Evolution, Its about how Web Architectures transformed from one server for everything to Microservices and Functions as a service. When I was learning about Kubernetes, I noticed that there is a Pattern in general how things and particularly Technology evolved and evolving.

That Pattern is unified across everything, whether its Web Architecture or Disks or Computers or Mobile or Web Frameworks or anything. That pattern is to solve immediate problems then identify Patterns in solutions than either map that to another field(s) or make a base of those solutions and build upon that. This holds for entire human evolution, humans don’t invent things. However, Humans solve their problems or make patterns out of them.

In the Business community, these Patterns called Processes. In the Design community, these Patterns called Design Systems/Systems, components, etc. In the Tech community, these Patterns called libraries, frameworks, etc. In most of the other business communities, these patterns called Innovation, breakthrough, New Product, etc.

We, Humans, identify and categorize things based on certain patterns not only today but even before starting of civilization even civilization is part of a pattern. Other examples of patterns are, in Dance, we have Contemporary, Ballet, Jazz, Bhangra, Kathak, etc or in Music, we have Folk Music, Rock, Jazz, Heavy Metal, etc or in Cooking, we have dry heat method, moist-heat method, or mixture of both or in Pork, we have Ham, Bacon, Rib, Loin, etc. By these examples, we established that these patterns are everywhere and we categorize things according to these patterns that’s how we Humans became Intelligent. From identifying and forming patterns for speech to identifying and forming patterns to emulate our intelligence through Artificial Intelligence, we came a long way and formed countless Patterns in the process.

Now let’s see some examples to understand how innovation in Technology happened over time with these patterns. And, how going forward with an understanding of how things evolve will help you in a long way.

Web servers started with single computer serving HTML/CSS pages. Then, it got integrated with another Pattern i.e. Database. That also required it to be integrated with one more pattern already exists i.e. Programming language. Since one server/computer got limited capacity so we shifted to 2-tier and 3-tier Architecture that was need-based evolution. That n-tier Architecture needs dedicated teams to manage all the things reliably. That required money that only few can afford. So, that need leads to another pattern creation i.e. VPS, reliability of infra with a fraction of cost. After VPS become common, everyone needs to do the same tasks again and again. That leads us to Cloud where we got many common things managed. Now, we have so many cloud operators and everyone provides their own flavor of managed services. Vendor lock-in becomes a concern for Corporates. That’s why Cloud neutral managed service cluster service like Kubernetes got picked up early and thriving nowadays. Kubernetes brought many patterns of cloud management and deployment (Rolling updates, Replication, Deployment, etc.) into its core and vendors brought in various other managed services patterns in a cloud-neutral way.

Now, what next pattern will be? There is one growing need that is already there for IoT devices. The problem statement is they need to show results in near-real-time and network can be unreliable. So, they need to process some of the data on the device and latency-tolerant data on Cloud. There is a concept of Edge computing in the Industry but currently we are experimenting with the idea there is no pattern coming out of that. Once we figured out the pattern in Edge computing then we will create a platform for that and build for next. That next might be decentralized data processing that some are already talking about and Cryptocurrency patterns might help there along with edge computing platforms.

We learned about just the Web servers growth journey these patterns are everywhere. Even, Governments have Policies, Laws, etc that are also patterns that are driven by certain other patterns.

Let’s see it from another perspective of the Hype cycle of new technology. BTW, Hype cycle in itself is a pattern. Hype cycle looks like the following,

Certain idea looks cool and got media attention. Then, early adopters started working on that. Those early adopters create certain applications or businesses or other things that field expected to be with that idea. However, that all looks random development in starting and it reaches the peak and people realize it might not be useful so the majority left. At this phase, further development happens based on need. If there is no need for that idea then it dies otherwise people started creating patterns so that majority that left don’t go into details of the previous pattern but utilizes those new patterns and builds upon those patterns. So, Trough of Disillusionment is nothing but making base pattern out of initial idea and building on top of that and the pattern cycle continues.

What do we do as humans for development? we learn from one phase then build patterns around that and those being abstracted out to reduce complexity for the next phase. For e.g., you don’t know how fire needs to be generated, you just use abstracted patterns like lighter, oven, stove, etc. For Tech, most of the programmers don’t know how compilers or interpreter work but still they are good programmers because we abstracted out compilers with many layers of patterns above it.

Please comment, if you find patterns in your industry or any other pattern that is interesting.

Footnote — To build AI similar to us, we need to teach them how to recognize these patterns and how to baseline them and build upon that just like us.