Compiling OmniSci Core: Ubuntu 16.04 LTS (GPU)

Parking ticket issuance in Philadelphia at 5 pm (rendered by OmniSci Immerse Enterprise Edition)

In my role as a Developer Advocate at OmniSci, one of my many responsibilities is to monitor our various ‘Help’ areas like GitHub issues and our Community message board. While doing talks and contributing to open-source is super fun (and valuable in its own right), the task of answering people’s questions or tracking feature requests provides such a rich understanding of what our product currently does, what our users would like OmniSci to do, and where our documentation could be a little bit better.

In the spirit of ‘…where our documentation could be a little bit better’, this blog post was born. In this post, I’ll demonstrate how to compile OmniSci Core (open-source) on Ubuntu 16.04 LTS for NVIDIA GPUs.

This post is focused on those who want to contribute to OmniSci Core (and thus need an actual build environment); if you just want to try OmniSci Core, we provide Docker images, as well as tarball downloads and even OmniSci Cloud (paid, 14-day free trial).

Installing nvidia-docker

After answering dozens of ‘Why doesn’t this compile?’ questions, without a doubt my most frequent suggestion is ‘Can you use Docker?’ While OmniSci Core isn’t a gigantic code base, there are enough dependencies where building inside of a Docker container makes a lot of sense. This is especially true when it comes to the NVIDIA CUDA dependencies.

In order to use Docker with GPU support, you need to install nvidia-docker. After installing nvidia-docker, we can use the following command to build the Docker container interactively:

With the above command, Docker will download an Ubuntu 16.04 LTS image with all of the NVIDIA development tools installed (CUDA, nvcc, headers, etc.), then run the image in interactive mode and provide a Bash shell.

Updating Image and Installing Standard Dependencies

Once we’ve got our Docker image up and running, we can update the image and install common dependencies as follows:

The second command installs git so that we can clone the OmniSci Core repo, which contains a script to build all of the software dependencies. That script in turn uses sudo, so we install that as well.

Installing the OmniSci Core Dependencies

The longest step in this build process is installing all of the dependencies needed to compile OmniSci Core. Because Ubuntu 16.04 LTS is a long-term support release, the base libraries such as the GCC compiler are fixed for a given LTS version. Of course, new releases of libraries continue to come out every day, and rather than constrain ourselves to older versions we can provide a build script that installs the versions of the updated libraries we need:

This step can easily take 20–30 minutes to run as compilers are installed, specific library versions are downloaded and compiled, etc. The final step in the process is to call the script which sets various environment variables.

Before we move on to compiling OmniSci Core, it’s a good sanity check to make sure we have the GPU support we’re expecting in the container:

On my workstation, I have 3 of the same GPU; your output may look different

nvidia-smi shows the state of your GPUs similar to the htop utility in Linux and nvcc is the CUDA compiler. With both in place, we should have everything we need for successfully compiling OmniSci with GPU support.

Compiling OmniSci Core With GPU support

With all of our dependencies installed, it’s time to actually compile OmniSci Core! OmniSci Core uses cmake to manage the build process:

The cmake file has numerous flags that can be set to control what gets built. To see the various flags possible, see the ‘Build’ section of the OmniSci Core repo. The code snippet above should be sufficient for most people wanting to do development on a GPU-enabled build.

Compiling OmniSci from scratch takes 5–10 minutes, depending on the value of -j provided to make (-j sets the number of jobs that can run concurrently). Depending on how many processors your computer has or what else you are running on your computer, you can set the -j value higher or lower.

Note: If you run into build issues while using make -j 4, it can be difficult to trace where the error occurred because the steps will print out of order. Upon error, retry the build using just make with no -j argument, which will give you a clearer indication of where the error occurs.

Running Sanity Tests

After the build has successfully completed, you can validate that the code compiled correctly using make sanity_tests:

When the tests run successfully, you’ll see the following:

What Will You Contribute?

With an OmniSci Core GPU-enabled build environment in-place, the only thing left is to contribute! OmniSci Core is an Apache 2.0 licensed project and we welcome contributions of all kinds, from documentation improvements to the lowest-level code updates.

To contribute to the OmniSci Core codebase, stop by our GitHub repo and submit a pull request. Although not required, if you are planning to submit a large pull request, submitting an issue before you start working will give our internal engineers an idea of what you are working on and allow them to guide you through the submission process.

Happy building!

This blog post is one of several posts demonstrating how to compile OmniSci Core (Open-Source). Check out the others in the series: