Define and control the way you ship software applications to production

Image for post
Image for post
Photo by Jacalyn Beales on Unsplash

Stability and governance are among the hottest topics that should be handled in any software development process. Both topics need to be addressed from the development stage to the deployment and running stages in the production environment.

One of the processes or techniques that can help in mastering software development, deployment, and running of the services is the production readiness process. This process is a bunch of tasks and verification of items that need to be checked during the software development process to ensure that the services:

  • have proper documentation
  • are prepared for running in production environments
  • are compliant with all adopted standards and non-functional…

Manage Docker Swarm and Docker Compose deployments with GitHub Actions

Image for post
Image for post
Photo by Chris Sabor on Unsplash

There are plenty of continuous integration and delivery tools that can be used to run CI/CD pipelines, such as Jenkins, Travis CI, and CircleCI. When it comes to projects hosted on GitHub, there is another native option available, which is GitHub workflows.

GitHub workflows can be used to achieve several goals, such as building Docker images, running unit tests, and even deploying to a remote server. The workflows can be triggered by GitHub events as well as external events. For instance, it is possible to create a workflow that will be executed once you create a new release on the repository. …


Extract key-value pairs from log lines using the FluentD parser

Image for post
Image for post
Photo by David Kovalenko on Unsplash

“Fluentd is a cross-platform open-source data collection software project originally developed at Treasure Data. It is written primarily in the Ruby programming language.”

— Wikipedia

In my previous blog posts, I explained how to use FluentD to parse and ship logs to a centralized logging server (EFK stack), Parse multi-line logs for Rails applications, and provided an overview of some of the most used plugins for parsing logs with FluentD. You can check the mentioned blog posts using the links below:

In this post, I will focus on another issue that can be solved with Fluentd. It will result in improving the quality of the logs and provide a better chance for building visualizations based on the parsed logs. …


Docker Swarm Clusters

Tips for running a Swarm cluster in a highly available mode for production environments

Image for post
Image for post
Photo by Alex Iby on Unsplash

Start running Docker containers with Docker Swarm is a simple task especially if you have the knowledge and experience of using docker-compose files. Creating a Docker Swarm cluster can be easily done by executing a single command line. Moreover, extending the cluster with more servers can be done using a single command line.

On the other hand, Running Docker services on Swarm clusters for production environments could be a challenging task, due to the fact that production environments should be stable and sustain a certain level of quality. …


Publish your Markdown-powered blog on GitHub Pages for free

Image for post
Image for post
Photo by Belo Rio Studio on Unsplash.

There are many options for building and hosting personal blogs. It is possible to use one of the well-known blogging platforms such as WordPress, Blogger, and Tumblr to host your blog. With this option, you don't need to worry about maintaining the blog’s infrastructure and the blogging platform.

On the other hand, you can host your own blog on one of the cloud providers. Below is a well-documented article that describes how to achieve this task using Ghost and DigitalOcean:

All the options mentioned above have some drawbacks:

  • You have to maintain the infrastructure.
  • It is not free.
  • Editing the blog posts is not comfortable (at least for me). …

Running a Traefik cluster and store traffic configuration in a key-value store

Image for post
Image for post
Photo by Adrian Schwarz on Unsplash

Traefik is an open-source Edge Router that makes publishing your services a fun and easy experience. It receives requests on behalf of your system and finds out which components are responsible for handling them.

https://docs.traefik.io/

Running a single Traefik instance in production environments is not recommended simply because Traefik will be a bottleneck of the environment and the environment will be highly available. If Traefik instance is down all services behind it will be down.

Therefore, I started exploring the options available to host Traefik in a high availability mode, and since I am using Docker Swarm for hosting Traefik the backend services, the first idea that came to my mind is to scale Traefik service with more replicas or to deploy it in a global mode (It will be deployed to every swarm worker node). However, It turns out that running Traefik with this setup will introduce the risk of generating the SLL certificates multiple times by different instances (if there is no need for generating SSL certificates, this Idea is enough to host Traefik in HA mode). …


Move your services from docker-compose files to Kubernetes resources and deploy them

Image for post
Image for post
Photo by Andrew Ruiz on Unsplash.

Docker Compose is a great tool that can be used to manage Docker container creations and deployment in development environments. However, it is not the best option for deploying containers into production environments because it lacks a lot of features that are needed for production deployment, such as building container clusters to support high availability of the containers and supporting zero-downtime deployment.

On the other hand, container orchestrators such as Marathon, Nomad, Docker Swarm, and Kubernetes are built to manage container deployments in production environments. These systems are built to handle:

  • A large number of microservices running in containers.
  • Hosting services in a highly available mode using clusters and service replications. …

Build and deploy a GitHub action to the GitHub marketplace

Image for post
Image for post
Photo by Leonardo Yip on Unsplash

GitHub supports several third-party tools for continuous integration and continuous deployment tools, such as Travis CI and Circle CI. On the other hand, GitHub offers its own CI service, which is GitGub workflows.

Workflows can be created to achieve several goals, such as building docker images, running unit tests, and even deploying to a remote server. The workflows can be triggered based on GitHub events as well as external events. For instance, It is possible to create a workflow that will be executed once you create a new release on the repository. …


Secure your application's source code and environments by mitigating known security risks

Image for post
Image for post
Photo by Loic Leray on Unsplash

Security is one of the most important topics in the computing world, including software engineering. Security topics are covering a wide range of areas, starting from the physical security of the infrastructure to the application security areas. Addressing these security areas and protecting against the known security issues is a must for every single software project.

Therefore, security needs to be addressed early in the project life and should be integrated with the development process to ensure the development of secure software applications and to reduce the effort needed to improve the software quality from a security point of view.

In other words, it’s recommended you consider security topics during the software development cycle in an agile way instead of considering it later in a single one-time phase. …


CENTRALIZED LOGGING

Improve your FluentD configurations and scripts by applying some of the best practices

Image for post
Image for post
Photo by Sam Dan Truong on Unsplash

Fluentd is a cross platform open source data collection software project originally developed at Treasure Data. It is written primarily in the Ruby programming language.Wikipedia

Fluentd is a tool that can be used to collect logs from several data sources such as application logs, network protocols. And third-party services. In addition, Fluentd allows you to build a unified logging layer across all supported services and applications. …

About

Al-Waleed Shihadeh

Team Lead & Product Owner

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store