Build Hadoop source code on macOS

First of all, prepare install environment

• java v1.8.0_161(v9 has errors)
• brew
• cmake(brew install cmake)
• maven(brew install maven)
• protoc (Should download v2.5.0, https://github.com/google/protobuf/releases/download/v2.5.0/protobuf-2.5.0.tar.gz)
• Port (Download from https://www.macports.org/install.php)

Protoc Installation:

tar -zvxf protobuf-2.5.0.tar.gz
cd protobuf-2.5.0
./configure
make & make install

Add env params to .zshrc(or .bashrc)

PROTOC_HOME=/Users/zekexu/Documents/protoc
export PATH=$PATH:$PROTOC_HOME/bin

then test protoc

protoc  — version
libprotoc 2.5.0 ## Success

Port configuration

sudo vi /etc/paths

Add 2 following lines:

/opt/local/bin
/opt/local/sbin

Maybe you need restart terminal, then:

$ sudo port -v selfupdate 
$ sudo port install ncurses 
$ sudo port install openssl

Now start Hadoop build

Download Hadoop2.7.5 source code from https://hadoop.apache.org/releases.html

tar -zvxf hadoop-2.7.5-src.tar.gz
cd hadoop-2.7.5-src
mvn package -Pdist,native -DskipTests -Dtar

Lang time wait…

If success will show:

BUILD SUCCESS

Finally, you could do any one:

cp hadoop-2.7.5-src/hadoop-dist/target/hadoop-2.7.5/lib/native/* to ${HADOOP_HOME}/lib/native

Or just use the directory hadoop-2.7.5-src/hadoop-dist/target/hadoop-2.7.5 as your Hadoop directory, you could move it to your path.