Installation of Apache NiFi

Step-by-Step guide for installing Apache NiFi 1.25.0 and other essential dependencies.

Image Generated by DALL-E 3 Illuserating Apache NiFi Installation Process.

I. INTRODUCTION

Installing software can often be a challenging and complex process, fraught with potential difficulties and conflicts. This is particularly true when step-by-step instructions are lacking. Recognizing the importance of clear and detailed guidance, this tutorial is designed to provide an extensive installation guide for Apache NiFi 1.25.0, covering all necessary prerequisites such as Java, as well as essential configurations and security settings. It’s important to note that this tutorial focuses only on the standalone mode of Apache NiFi.

II. APACHE NIFI

Apache NiFi can be considered as a workflow tool equipped with an interactive user interface. It has high ability to automate cybersecurity, observability, event streams, and generative AI data pipelines and distribution for thousands of companies worldwide across every industry [1]. The interface of Apache NiFi provides several useful information related to each processecor such as Queued, In, read/write, and Out quantities. Refer to the “An Overview of Apache NiFi” article [2] for more information about Apache NiFi. Additionally, Apache NiFi is highly optimized for efficient utilization of IO, CPU, and RAM resources, ensuring optimal performance. It offers the flexibility to operate in either standalone or clustered modes, providing manageability, availability, extensibility, and tolerance [3].

III. SYSTEM REQUIRMENTS

Apache NiFi, generally, can be installed on only one node or machine and at the same time it can be installed on a cluster that consists of master node(s) and slave node(s) so that it can handle the pipleines more tolurent and more effictively. Based on that, the architecture of Apache NiFi is completely depending on its goal. If the pipleines that will run on Apache NiFi are simple and small, then standalone mode is enough, otherwise, cluster mode must be selected. Disk size is also an important feature that must be taken into consideration as the running pipleins will use the machine disk [4].

The minimum requirments of Apache NiFi are as follow:

• Java:

Java 8 or Java 11

• Web Browsers:

Microsoft Edge, Mozilla FireFox, Google Chrome, or Safari

• Operating Systems:

Linux, Unix, Windows, or macOS

In this tutorial, as a Java; Java 8 will be installed, as a web broswer Google Chrom will be used, and as an operating system Linux Ubunutu will be used. Aditionally, some other packages such as unzip must be installed.

IV. INSTALLATIONS

A thorough explanation of the installation processes for Java8, the Unzip package, and Apache NiFi 1.25.0 is supposed to be provided in this section.

A. Java 8

The installation process for Java involves the following sequential steps [5]

1. Update the package lists

sudo apt-get update

2. Install Java 8

sudo apt install openjdk-8-jdk

3. Prepare the enviroment variables for Java 8

3.1. Go to main directory and open .bashrc file

cd && nano .bashrc

3.2. Put the following lines at the end of the file, you can go to the end of the file using (alt+/) shortcut.

export JAVA_HOME=/lib/jvm/java-8-openjdk-amd64
export PATH=$PATH:$JAVA_HOME/bin

3.3. Apply changes to .basrch file

source .bashrc

3.4. Ensure that the JAVA_HOME variable is correctly configured

echo $JAVA_HOME

Figure 2. Obtained result from “echo $JAVA_HOME” command.

4. Ensure that you have the correct Java version

java -version

Figure 3. Obtained result from “java -version” command.

B. Unzip:

The installation of unzip package can be easily achived using the following two commands:

1. Install Unzip package

sudo apt install unzip

2. Ensure that you have the installed Unzip package correctly

unzip --version

Figure 4. Obtained result from “unzip — version” command.

C. Apache NiFi 1.25.0

1. Install ِA]pache NiFi 1.25.0

cd && wget https://dlcdn.apache.org/nifi/1.25.0/nifi-1.25.0-bin.zip

2. Uncompress the installed Apache NiFi 1.25.0 package

unzip nifi-1.25.0-bin.zip

3. Move Apache NiFi folders to /opt/nifi

3.1 Make the directory of /opt/nifi

mkdir -p /opt/nifi

3.2 Move all Apache NiFi folders to /opt/nifi directory

sudo cp -r nifi-1.25.0/* /opt/nifi/

4. Modify the permissions for the Apache NiFi directory and its bin subdirectory to ensure proper access control.

4.1 Change the permissions of sudo for /opt/nifi folder

sudo chmod -R 777 /opt/nifi/

4.2 Change the permisions for /opt/nifi/bin/nifi.sh file

sudo chmod u+x /opt/nifi/bin/nifi.sh

4.3 Enhance the accessibility of the nifi.sh file by enabling its direct execution using the nifi command. It can be achived by appending the Apache NiFi bin directory to the PATH variable in the .bashrc file

4.3.1 Go to main directroy and open .bashrc file

cd && nano .bashrc

4.3.2 Add the following line to the end of .bashrc file

alias nifi="/opt/nifi/bin/nifi.sh"

4.3.3 Apply changes to .basrch file

soruce .bashrc

5. Set the JAVA_HOME environment variable within the Apache NiFi enviroment file.

5.1 Go to Apache NiFi bin Directory

cd /opt/nifi/bin

5.2 Update the nifi-env.sh file by uncommenting the export JAVA_HOME line and setting its value to /lib/jvm/java-8-openjdk-amd64.

nano nifi-env.sh

Figure 5. Defining “JAVA_HOME” inside “nifi-env.sh” file

6. Open port 8443 to allow the Apache NiFi interface to utilize it.

sudo ufw allow 8443/tcp

7. Verify that the nifi command is functioning properly.

nifi status

Figure 6. Obtained result from “nifi status” command.

8. Generate a new user and password. This is important in order to avoid the generated user and password by Apache NiFi which lists them under logs/nifi-app.log

8.1 Go to Apache NiFi bin directory

cd nifi-1.25.0/bin

8.2 Run the following command where:

admin is the username and admin 111222333 is the password.

Feel free to change them based on your requiremnts

./nifi.sh set-single-user-credentials admin admin111222333

9. Start Apache NiFi service

nifi start

10. Check Apache NiFi status

nifi status

Figure 7. Obtained result from “nifi status” command.

11. Access the Apache NiFi interface by pasting the following link into Google Chrome: https://127.0.0.1:8443/nifi/. The page must look as the one in Figure 8. Then, enter the username and password created in step 8.2. In this case, they are:

* username: admin

* password: admin111222333

Figure 8. Apache NiFi login interface

V. CONCLUSION

In conclusion, this tutorial has been carefully structured to provide a detailed, step-by-step guide for the installation of Apache NiFi and its essential accompanying software. Additionally, the necessary configurations and security permissions have been explained to ensure a smooth and secure setup. Furthermore, we have provided an overview of Apache NiFi, including its performance and minimum requirements, to offer a comprehensive understanding of this powerful workflow management tool. For more information about Apache NiFi, please refer to the “An Overview of Apache NiFi” article [2]. By following this tutorial, any user can confidently establish Apache NiFi and be equipped with a solid foundation of knowledge about the system. For case studies that involve executing Python scripts and SQL queries, please refer to the “Executing Python Scripts and SQL Quries In Apache NiFi” article [6].

VI. REFRENCES

[1] Apache NiFi (n.d.). An easy to use, powerful, and reliable system to process and distribute data. Accessed on [14.03.2024]. Retrieved from: https://nifi.apache.org/

[2] Alkhanafseh, Y. (2024). An Overview of Apache NiFi. Accessed on [17.03.2024]. Retrieved from: https://medium.com/@alkhanafseh/an-overview-of-apache-nifi-44819186674c

[3] Apache NiFi Team (n.d.). Apache NiFi Overview. Accessed on [18.03.2024]. Retrieved from: https://nifi.apache.org/docs/nifi-docs/html/overview.html

[4] Apache NiFi Team (n.d.). NiFi System Administrator’s Guide. Accessed on [17.03.2024]. Retrieved from: https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#system_requirements

[5] Alkhanafseh, Y. (2024). Real-Time Stream Processing Using Flink and Kafka with Scala and Zeppelin (Part 1): Installations. Accessed on [19.03.2024]. Retrieved from: https://medium.com/turknettech/real-time-stream-processing-using-flink-and-kafka-with-scala-and-zeppelin-part-1-installations-f2bd906f47fb

[6] Alkhanafseh, Y. (2024). Executing Python Scripts and SQL Quries In Apache NiFi. Accessed on [19.03.2024]. Retrieved from: https://medium.com/@alkhanafseh/executing-python-scripts-and-sql-quries-in-apache-nifi-4eb49945846f