The system I work on is about to undergo a significant tech-refresh. The goal is to improve the current hindrances that prevent ingesting more data. The remedy requires a product that receives inputs from external sources, processes them, and disseminates the outcomes to their destinations.

Apache Nifi implements the flow-based programming (FBP) paradigm; it composes of black-box processes that exchange data across predefined connections (an excerpt from Wikipedia).

In short, Apache NiFi is a tool to process and distribute data. Its intuitive UI supports routing definitions, a variety of connectors (in/out), and many built-in processors. …

Two years ago, I published my first technical blog.

Back then, my career was on cruise control while I was leading an IT project, I was in a complacent period, and everything was under control. Nevertheless, I felt something was missing; I was doing more of the same. I lacked the enthusiasm I had had until a few months back.

I have never felt such fatigue before; it was totally unfamiliar. Instead of dwelling on this unsatisfying situation, I stopped to think of what can help me fill in the void.

In the beginning, I didn’t know exactly what I…

I recently found myself explaining the concept of thread and thread pools to my team. We encountered a complicated threads-problem at our production environment that led us to review and analyse some legacy code. Although it threw me back to the basics, it was beneficial to review .NET capabilities and features in managing threads, which mainly reflected how .NET evolved significantly throughout the years. With the new options available today, tackling the production problem is much easier to cope with.

TL;DR

So, this circumstance led me to write this blog-post to go through the underlying foundations and features of threads and…

These are times that we are compelled to avoid socialising face-to-face, a fundamental behaviour we have been doing since childhood, and stay at home. Everyone should dwell at his cave, with the nuclear family. That is definitely a drastic change for most of us, which brings many difficulties.

Like any transition, we can try to fight it or choose to accept the situation and mourn; however, not much can be done to change things. We have zero influence on the government’s decisions right now, but we can control our perception and adjustment.

As I try to find the benefits in…

Our monolith solution does not cope with the increased load of incoming data, and thus it has to evolve. This is the time for the next generation of our product. Stream processing is the new data ingestion paradigm, as compared to the batch processing we’ve implemented so far.

So, my team embarked on processing information using Flink. There is an abundance of articles about Flink’s features and benefits, Cloudera shared an excellent slide about Flink; this article is a practical hands-on guide on how to build a simple stream processing application starting from the basics.

Apache Flink in Two Sentences

Apache Flink is a scalable…

My son has a weekly rugby training. It started when he requested to try out rugby after watching the last Rugby World Cup, and I jumped on the wagon as I saw an opportunity to instil some discipline in him. So far, over the previous two months, he has persevered these training.

While I was watching him learning some new drills and movements, I started to compare him with the other kids. You can easily see who is performing the plays well and who needs to continue practising. Well, I’ll be honest; my son belongs to the latter.

This understanding…

The book Good to Great (by Jim Collins), was hiding at my library for almost a decade. Back then, Elbit Systems, the company I’m working at in the last decade, has handed out this book to all graduates of a Team Leads course. It was forgotten and blended with my other must-read books, while I was focusing on my career and my family. Recently, I encountered this book when I was refreshing my library; this time, the book hooked me.

The book unfolds a comprehensive analysis why some companies have done the transition from being a good company to a…

I am an avid follower of AWS Online Tech Talks YouTube channel. It is a useful way to keep being updated with new or existing AWS features and services; I find it helpful to refresh and retain knowledge. Recently I encountered a webinar about AWS S3 security, which triggered me to relook at my S3 policies and settings. I decided to consolidate some S3 security features and properties while adding some examples along with my two cents.

What’s the Incentive?

Typically, on the daily usage of S3, security and permissions are not being changed regularly. In most cases, we set the security definitions…