The Future of Application Development
How PostgreSQL impacts the database landscape
By Jan Karremans
Application development
Then
It somehow comes naturally to everyone who starts coding, at least it did to me when I wrote my first program back in the early 1980s, to write something and then to test it. You find problems that you fix and you try again. Later on, when you transition into coding professionally, you find this trial-and-error programming evolved into a real methodology that has an actual name, the waterfall programming methodology.
And, even though there is nothing fundamentally wrong with such an approach, it does tend to lead to a development lifecycle that is a little less flexible. It is more aimed at the development of traditional, more monolithic applications that are used in a world with rigorous demands that typically leave less room for modern popular mantras such as “try often, fail fast”.
Now
On the other hand, the IT industry is swiftly moving away from traditional concepts. To be able to answer the enormous “need to speed” in getting new applications, features and functionality done, our tried and tested methodology no longer suffices. This is where “trying often, failing fast” comes into play as one side of the coin, the agile approach. And, as every coin needs to have two sides, so does the agile methodology. Even though we want to have an agile approach to building our features, the application shall not be unavailable.
Both of these requirements create a very nice contradiction for us to resolve. Evolve at hyperspeed, but only let your users experience improvement, preferably without any side effects.
Change fast
Microservices
This is not a hill too steep to climb! We only need a fundamental new way of looking at things. You might even call this the “Cloud Native” approach, which is not necessarily directly connected to the Cloud as we know it today, but rather to a way of thinking and methodologies used.
The agility that is required to address the “heads of this coin” forms an interesting challenge. It is the greatest change that is needed, because it addresses the application, and the application building methodology more specifically. Saying you are now agile does not mean that you actually are, just stating that obvious thing. Building a microservices-based application requires moving away from a decades old way of thinking about application architecture. You do not design an application from back to front and build it all at once in one big effort.
An application is a gathering of services, hence the name microservice architecture. Every function of the application, every knob that you can press, every section of the screen, becomes a service that, together, makes up the application.
Just imagine the flexibility that this will give you. You can rearrange an application super easily by re-using the microservices in another constellation, perhaps combining several of these microservices with others, to get an application for a specific group of users.
And that is only part of the benefit. Typically, each microservice has its own (micro) development team, and it can have its very specific release cycle with very specific feature requirements, etc. Basically, this is the foundation of what we know today as DevOps. Again, just imagine what this could do for your application and its continuous development. It is an evolution in application development, bringing DevOps, CI/CD, continuous availability, continuous development and all of those benefits together.
Microservices allow you to build bigger and more complex applications quicker and with less risk. Much less risk and much less time than it would for a far less application in “the old ways”.
Sagas
However, all benefits come with drawbacks. With that traditional approach, all data lives in one gigantic, monolithic database. All of the functions of the application can interact with all the data all of the time. That is an advantage, as paired functions will need to be able to read and manipulate the same categories of data.
This is no different in microservice architectures.
In this brave new world, these sequences of transactions are called Sagas. Thinking back of this single database approach, that would not be an issue. But, that would not fit in with the ideas of Cloud Native as this would really need the database to be part of the microservice, or perhaps of the Saga.
Databases
As discussed in the previous paragraphs, Cloud Native principles preferably would be honored all throughout the application architecture. This means that the more old-fashioned paradigms of having an agile application, of which all parts still connect to this one monolithic datastore would be “unheard of”. The store that holds the data for any given micro application, should live within that same realm, within the confines of that microservice. This obviously poses all kinds of challenges, from the original paradigmal contradiction of databases in containers to sharing data between microservice — if only to answer to the requirements of the Sagas, as discussed before.
Do not go down
As we have seen and heard in many articles and in many discussions, the demands for applications changed dramatically over the last decade. Where we could afford to take a service down for a day, or even for a few hours to do patching or run upgrades, that is totally unheard of today. Having an application down can cost up to tens of thousands of dollars per hour in lost revenue, decreased vendor confidence, and overall reputation. And building applications that are able to cope with being always-on, is hard, time-consuming, and above all, expensive. Or, is it?
The introduction of Open Source solutions has had a profound impact on how we run and administer IT environments today. Where, in the first wave of Open Source, much of this was focused on “easier to replace” systems, like application servers, integration software and the like. Today, we see the second wave of Open Source, where the focus is really on the more complex — and thus harder to replace — systems, like relational database management systems. Postgres being a prime example of that.
Postgres
The first and most important part of our journey to the new paradigm is Postgres. With its extensibility system, its small installation footprint, Postgres is the most agile and flexible data processing platform available. It is one of the reasons why so many projects are based on Postgres, and so many new initiatives fork off of Postgres.
Having the ability to run both structured and unstructured, both SQL and NoSQL workloads at the same time in the exact same platform, intermingled is mind boggling. Add to that the ability to make Postgres do most anything through extensions you can create as an application visionaire, this platform is your one-shop-stop solution for any data processing you might imagine.
Server side application code
In the ‘80’s of the previous millennium, there was (yet another) transformation in how application development was approached. Mind you, this shift was in the life and times of Waterfall Application Development, the thing we are really moving away from. It stated that no business logic should be inside the database, and that all data transformation should take place in this layer dubbed “middleware”.
Fortunately, Postgres stems from a time and place where a few smart people decided that Postgres should support data processing in the database. Postgres supports application code inside the database, and it’s exactly that which helps us in our journey towards microservices. By adopting server-side application code, you can “pick and choose” and “divide and conquer” on where you want your application to do which bit of work, reducing the risk of scalability and aiding reusability of specific functions from your application.
CNP and BDR for microservices architectures
So, now, how do we solve this?
There are a few patterns here, either inherent to (relational) databases or perhaps specific to Postgres. How can we tame this beautiful beast that we have come to know to make it run on these large servers in a basement, crunching away at great loads of data, to fly as this proverbial bird in the flock. Let’s look at a few answers.
CNP
Back in the eighties, when Dr. Michael Stonebreaker conceived his Post-Ingres project, extensibility, and by extension (pun intended) the modular build-up of Postgres, the foundations for Cloud Native Postgres (CNP) were laid. By having a system like Postgres being able to “fit” in a containerized architecture, we can start thinking about creating a Cloud Native Postgres approach.
By implementing the fundamentals of Cloud Native, you now have the ability to run and use Postgres without having to worry about the intricacies of container management, high availability and so on. The Cloud Native Operator abstracts this away and leaves much of the heavy lifting to the Kubernetes platform, as intended. This allows Postgres to develop and flex as Kubernetes evolves.
With this, we have the basis for our database as a part of the microservice, as intended in the graph above.
BDR
As we have now solved the challenge with the microservice in its esses, the next challenge dawns. Some microservices really have to work together on specific data-sets. You can compare this to the movements of the individual birds in the flock. If one bird veers to the left to avoid collisions, all of the other birds will have to make greater or smaller changes to their direction as well.
Introducing BDR, allows Postgres to do just that.
As BDR gives us the ability to pick and choose which database belongs to what replication groups, we can define the flexibility of movement between these microservices.
This way we have the tools to give our microservices the magic glue to start moving and flexing to the beat of the business it services.
Tying this together
We picked up our story at sequential application development. We then looked at this brand new way of approaching this challenge and found that microservice architectures could just be this golden nugget, as it allows us to break applications into smaller pieces.
With this in mind, we looked at the power of Postgres and how it can help us in creating fluent applications that have the ability to scale and remain available, while changing and growing to flex with changing requirements.
Zero downtime application lifecycle management
From this place, it is a relatively small step to zero downtime application lifecycle management. It sounds ominous, but it can be done. An application that, in its lifespan, is simply never unavailable, whatever the circumstances. Okay, when the Sun engulfs the Earth or when you are not connected to the world, the application will be unavailable to you, but other than that it should stay available…
Postgres and the surrounding tools bring a wealth of functionality that allows for the development of an application that can technically remain available. It allows for the development of an application that can absorb functional changes without the necessity of being unavailable, at all.
Conclusion
EDB has this slogan: “Power to Postgres”. If you look at everything that is required in a modern day business, it is scary. If you then look at everything that is possible with a solution as modest and seemingly simple like Postgres, all of a sudden, fear subsides.
With Postgres, you can do it. It gives a whole new meaning to “technically, it is never a problem”. If you get on board with the new application development paradigms, truly implement what is meant by the buzzwords; Agile, DevOps, Microservices and what have you, it can be done. We will swiftly move to a new reality where this is not just a requirement, but a true necessity!
By Jan Karremans, Product Marketing Director at EDB. With over 25 years of experience in the database market and a former qualified Oracle ACE, Jan is a well-respected expert in relational database technologies, speaking regularly at conferences around the globe.
