Image for post
Image for post

At ING’s Wholesale Banking Advanced Analytics team we’ve been using Apache Flink and more specifically the DataStreams API for over 6 months now. Although we’re very enthusiastic about this open source project, we’ve identified one major pain point in using it in a production environment: managing job deployments. This post will highlight the challenges and the open source solution we’ve built to tackle this.

#TL;DR
Managing job deployments and state recovery with Apache Flink is tedious and tricky within a production cluster and sometimes requires chaining operators that are provided by the native Apache Flink CLI. The deployer CLI that we’ve developed will provide several methods which encapsulate this process for you. …

About

WB Advanced Analytics

ING’s Wholesale Banking Advanced Analytics team

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store