Open source has stormed the software world for ages. Plenty of companies not only use open-source projects but also contribute back to the communities. And the open technology world is growing every day. We can not imagine the software world without some projects like Linux, Postgres SQL, Hadoop, Kubernetes, Cassandra, Python, Tensorflow, and more — the list is unending.
In an enterprise world, it is essential to understand how open-source projects are being used. Some of these reasons are:
Envoy is an open-source proxy created at Lyft. As the world is moving towards a distributed microservice architecture, practitioners are observing difficulties in running operations in two main areas:
Envoy tries to solve these problems by making networks more transparent to applications. It also provides a way to better collect stats so that it’s easier to debug and trace the issues.
The most common way the Envoy proxy is deployed today is alongside applications as a sidecar. Just in case you are not aware of a sidecar pattern, then you can read about it in my previous article.
In this article, I am going to talk about how to get started with Envoy Proxy with a DIY tutorial and how to use Envoy’s various features to improve the reliability, security, and observability of your applications. …
Many modern-day applications need to be built at an enterprise scale, sometimes even at an internet scale. Each application needs to meet scalability, availability, security, reliability, and resiliency demands.
In this article, I’m going to talk about some design patterns that can help you achieve the above-mentioned abilities with ease. I’ll be talking about each pattern, how to use that pattern in a cloud-native environment, and when to use it and when not.
Some of these patterns aren’t so new but are very useful in the current internet-scale cloud world.
Here’s the list of patterns I’ll be discussing in this…
If you’re into microservices, then you might’ve heard about these two terms multiple times. Often people get confused between the two. In this article, I’m going to talk about service meshes and API gateways in detail and discuss when to use what.
Before diving into service meshes and API gateways, let’s revisit the network layers. The following is the OSI network layer model:
As a software engineer, it is very important to acquire the skill of writing high-quality documentation. Due to the recent increase in remote work, it has become even more important to be better at asynchronous communication. Remote company GitLab does an excellent job of defining asynchronous communication:
“Asynchronous communication is the art of communicating and moving projects forward without the need for additional stakeholders to be available at the same time your communique is sent.”
High-quality documentation is an easier way to achieve effective asynchronous communication. …
Artificial Intelligence (AI) has become an integral part of everything we do. Right from suggesting what to buy on Amazon to recommending the songs on Spotify. Chatbots have made our customer service operations way too simpler. In the recent pandemic, we have even seen governments using AI to detect face mask violations.
The advanced practices of AI have even seen success in building autonomous vehicles, where companies like Google, Baidu, Uber, etc. are quite advanced in this direction. AI has made its mark in various other fields like healthcare, banking, retail, and many more.
As we start using AI to get complex tasks done, we need to understand why the algorithm made certain decisions and why not the other. The higher transparency we can get into the AI black box, it would easier for people to believe and use it for doing better things. This exact thought gave rise to the field of eXplainable AI (XAI). …
Sometime you may get into a situation where someone might accidentally or unknowingly end up committing credentials, passwords, secrets, etc. in a
git repository. In one of my previous articles, I had talked about how to prevent committing secrets using
pre-commit hooks. Even by taking such precautions if the credentials get committed then there is no easy way to remove these credentials. If you just remove the credentials or the file itself and commit again, the credentials can be seen in the history easily.
There are other options like
git-filter-branch but you need real good git expertise to use it. …
As per Wikipedia,
“Continuous testing is the process of executing automated tests as part of the software delivery pipeline to obtain immediate feedback on the business risks associated with a software release candidate.”
In my previous article, I wrote about how to perform functional testing of your APIs using Postman/Newman and integrate it into a DevOps platform. Another important aspect of automated testing is load testing, which helps us analyze the application behavior under stress.
There are various tools available in the market which can help us perform load tests. …
REST APIs are part of everyday software engineering practices now. Sometime back, I wrote an article talking about how to design REST APIs. In this article, I am going to talk about how to do automated testing of APIs and integrate this into CI pipelines.
This article assumes you have basic knowledge of REST APIs.
Postman is a collaboration platform for API development. Postman provides a GUI-based, easy-to-use, and secure application for API usage and testing. It simply acts as an HTTP client.
You need to download and install the Postman app, to begin with.
I have been closely working with the Data & Analytics domain for almost a decade and have seen a lot of interesting trends around analytics, big data, data engineering in general. Being a hardcore software engineer, I always wondered how to bring the principle of software engineering best practices to the analytics world. Recently I came across a very interesting open-source tool — dbt ( Data Build Tool).
dbt applies the following principles of software engineering to analytics code —
dbt is an open-source project under Apache 2.0 …