Step By Step Guide on How to install Hive on Ubuntu

4 min readJul 8, 2018

This installation is done on Ubuntu 18 and Hadoop 2.8.4. Please refer previous docs to download Ubuntu and Hadoop set up.

We are going to install hive 2.3.3

Refer to download the Hive: https://hive.apache.org/downloads.html

wget http://www-us.apache.org/dist/hive/hive-2.3.3/apache-hive-2.3.3-bin.tar.gz

Open ~/.bashrc file and set the environment variable HIVE_HOME to point to the installation directory and PATH:

After entering environemnt variables, Activate it.

Creating Hive warehouse directory

Hive uses Hadoop, so we must have Hadoop in our path:

In addition, we must use below HDFS commands to create /tmp and /user/hive/warehouse (aka hive.metastore.warehouse.dir) and set them chmod g+w before we can create a table in Hive:

Verify it

The directory warehouse is the location to store the table or data related to hive, and the temporary directory tmp is the temporary location to store the intermediate result of processing.

Configuring Hive

To configure Hive with Hadoop, we need to edit the hive-env.sh file, which is placed in the $HIVE_HOME/conf directory. The following commands redirect to Hive conf folder and copy the template file:

Edit the hive-env.sh file by appending the following line:

export HADOOP_HOME=/usr/local/hadoop

Hive installation is completed successfully. Now we need an external database server to configure Metastore. We use Apache Derby database.

The following command is used to download Apache Derby

wget http://archive.apache.org/dist/db/derby/db-derby-10.13.1.1/db-derby-10.13.1.1-bin.tar.gz

Let’s set up the Derby environment by appending the following lines to ~/.bashrc file:

export DERBY_HOME=/usr/local/db-derby-10.13.1.1-bin

export PATH=$PATH:$DERBY_HOME/bin

export CLASSPATH=$CLASSPATH:$DERBY_HOME/lib/derby.jar:$DERBY_HOME/lib/derbytools.jar

We need to create a directory named data in $DERBY_HOME directory to store Metastore data.

Now we completed Derby installation and environmental setup.

Configuring Hive Metastore :

Configuring Metastore means specifying to Hive where the database is stored. We want to do this by editing the hive-site.xml file, which is in the $HIVE_HOME/conf directory.

Let’s copy the template file using the following command:

Paste following lines are between the <configuration> and </configuration> tags of hive-site.xml:

Hive-site.xml is the readonly file you can’t change it.So login by root user and then change it.

<property><name>javax.jdo.option.ConnectionURL</name><value>jdbc:derby:;databaseName=metastore_db;create=true</value><description>JDBC connect string for a JDBC metastore.To use SSL to encrypt/authenticate the connection, provide database-specific SSL flag in the connection URL.For example, jdbc:postgresql://myhost/db?ssl=true for postgres database.
</description>
</property>

Create a file named jpox.properties and add the following lines into it:

javax.jdo.PersistenceManagerFactoryClass =org.jpox.PersistenceManagerFactoryImpl
org.jpox.autoCreateSchema = false
org.jpox.validateTables = false
org.jpox.validateColumns = false
org.jpox.validateConstraints = false
org.jpox.storeManagerType = rdbms
org.jpox.autoCreateSchema = true
org.jpox.autoStartMechanismMode = checked
org.jpox.transactionIsolation = read_committed
javax.jdo.option.DetachAllOnCommit = true
javax.jdo.option.NontransactionalRead = truejavax.jdo.option.ConnectionDriverName = org.apache.derby.jdbc.ClientDriverjavax.jdo.option.ConnectionURL = jdbc:derby://hadoop1:1527/metastore_db;create = truejavax.jdo.option.ConnectionUserName = APPjavax.jdo.option.ConnectionPassword = mine

We need to set permission to Hive folder:

Metastore schema initialization

Starting from Hive 2.1, we need to run the schematool command below as an initialization step. In our case, we use derby as db type: