Apache Nifi

Sibasish Brahma
Nov 8 · 3 min read

Moving data effectively is hard

Dataflow challenges:

-Data ( standards, format, protocols)

-Infrastructure ( security, credentials, network, exactly once delivery )

-People ( compliance, consumers change, requirements change, person or group, GDPR )

  • Requirements : what is noise today becomes signals tomorrow.
Running Apache Nifi on windows
http://localhost:8080/nifi/

The above image of Nifi UI contains a data flow, where GetFile processor gets a file from a folder and sends to Putfile processor. Putfile takes the file and drops it to a Kafka topic using PublishKafka processor.

there is another process called ConsumeKafka which receives the file from the topic where the file was dropped by previous process and then at last there is another Putfile processor which takes the file and stores it in a location.

currently the processors are in stopped state. below image shows they are running , as you can see a green button to all process names. The workflow is active and processing data files

Sibasish Brahma

Written by

Technology evangelist | Public speaker | Social worker | Spiritual Researcher | Socio-Political Analyst | BigData & AI Systems-Solutions Architect | Optimist

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade