How to Hadoop at home with Raspberry Pi — Part 3
In the home stretch now to completing my Raspberry Pi Hadoop cluster. But first a quick summary. I started this personal project about 3 weeks ago because of my interest in “Big Data”, data analytics and data engineering. I’m also taking Udacity’s Nanodegree in Data Analytics so I figured getting my hands dirty with Hadoop was a great way to dive in. That being said, this is Part 3 of 3, I’ve setup Raspberry Pi in part 1 then installed and configured Hadoop in a single node configuration in part 2. Now it’s time to get the cluster working.
This is not a tutorial. Think of it more as a journey, there’s no nice step-by-step process here, I’m going to make mistakes, get errors, fix them and try to move on.
- Part 1: Setting up Raspberry Pi and network configurations
- Part 2: Hadoop single node setup, testing and prepping the cluster
- Part 3: Hadoop cluster setup, testing and final thoughts
Hadoop and distributed data processing
Now that we’re talking…
over SSH, it’s time to start things up and make sure all of our services are running properly on all 3 nodes. I log into node1 and start the services.
1. $su hduser
2. $cd $HADOOP_HOME/sbin
3. $./start-dfs.sh
4. $./start-yarn.shFrom node1, the services are up and running but let us see what’s happening on node 2 and 3.
