A Quick Demo of Apache Beam with Docker

Emanuele Cesena
Jun 22, 2016 · 3 min read

Deploy Flink & Beam with Docker

git clone https://github.com/ecesena/docker-beam-flink.git
cd docker-beam-flink
docker-compose up -d
docker-compose scale taskmanager=2
docker psCONTAINER ID IMAGE      ... NAMES
3d59d952d152 beam-flink ... dockerbeamflink_taskmanager_2
4cce6219be80 beam-flink ... dockerbeamflink_taskmanager_1
3b7b6b32b4de beam-flink ... dockerbeamflink_jobmanager_1

Run HelloWo — ehm, WordCount

open http://$(docker-machine ip default):48080
docker exec -it dockerbeamflink_taskmanager_1 /bin/bashcat /tmp/output.txt*
...
live: 13
long: 15
look: 14
lord: 90
lose: 6
...

Build a Beam Pipeline

git clone https://github.com/ecesena/beam-starter
cd beam-starter
mvn clean package

Emanuele Cesena

Written by

Security eng at Pinterest, co-founder at SoloKeys

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade