Photo by arj. on Unsplash

How to install RAPIDS/Bumblebee from scratch on Google Cloud Platform

Updated May 26, 2021

Bumblebee can run on CPU and GPU. To run on GPU you need to install RAPIDS(instruction below). If you want to run it on CPU just skip the (Only GPU) sections.

In Google Cloud Platform create the instance you need.

Not all zones have GPU available for a full list you can check this page

GPUs on Compute Engine | Compute Engine Documentation | Google Cloud

Also, be sure to check the N1 Series in the dropdown.

Be sure to select an OS compatible with the CUDA drivers (Check the options here https://developer.nvidia.com/cuda-downloads). After the installation, you will be using 30GB so be sure you have enough disk space.

Check the HTTP traffic option.

(Only GPU)

Install RAPIDS Prerequisites (Only GPU)

First,

>> sudo apt update
>> sudo apt install build-essential

To install CUDA. Go to:

CUDA Toolkit 11.1 Downloads

and be sure to select the options that match your system specifications.

With the deb (local) option you should get something like: (Note: the runfile (local) option did not work for me)

For Ubuntu 20.04

>> wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2004/x86_64/cuda-ubuntu2004.pin
>> sudo mv cuda-ubuntu2004.pin /etc/apt/preferences.d/cuda-repository-pin-600
>> wget https://developer.download.nvidia.com/compute/cuda/11.2.2/local_installers/cuda-repo-ubuntu2004-11-2-local_11.2.2-460.32.03-1_amd64.deb
>> sudo dpkg -i cuda-repo-ubuntu2004-11-2-local_11.2.2-460.32.03-1_amd64.deb
>> sudo apt-key add /var/cuda-repo-ubuntu2004-11-2-local/7fa2af80.pub
>> sudo apt-get update
>> sudo apt-get -y install cuda

Reset the instance using the reset button in the top section.

Check if the NVIDIA driver is working using

>> nvidia-smi

You should get something like

Now install the NVIDIA toolkit

>> sudo apt install nvidia-cuda-toolkit

Install Anaconda

Go to https://www.anaconda.com/products/individual

Copy the .sh file URL and use it with the wget command

>> wget https://repo.anaconda.com/archive/Anaconda3-2020.11-Linux-x86_64.sh
>> bash Anaconda3-2020.11-Linux-x86_64.sh

Accept the terms and conditions, the installation path, and answer yes to the conda init question. Then execute:

>> source .bashrc

Test conda using:

>> conda --version

Install RAPIDS (Only GPU)

Select the rapids version you want to install from https://rapids.ai/start.html

RAPIDS release selector

Be sure your CUDA driver version you download matches the CUDA version you select here.

In my case I got:

>> conda create -n rapids-0.19 -c rapidsai -c nvidia -c conda-forge 
    rapids-blazing=0.19 python=3.7 cudatoolkit=11.2

This can take some minutes.

>> conda activate rapids-0.19

Install Jupyter Lab

>> conda install -c conda-forge jupyterlab

There are a couple of handy extensions that let you see the Dask cluster and GPUS operation. For installation:

Dask JupyterLab Extension

>> conda install jupyterlab nodejs
>> conda install -c conda-forge dask-labextension
>> jupyter labextension install dask-labextension
>> jupyter serverextension enable dask_labextension

For more info https://github.com/dask/dask-labextension

JupyterLab GPU Dashboards

>> pip install jupyterlab-nvdashboard
>> jupyter labextension install jupyterlab-nvdashboard

For more info https://github.com/rapidsai/jupyterlab-nvdashboard

Then create a new kernel:

>> ipython kernel install --user --name=rapids_blazing

Run ipython and write import cudf to check that cudf has been installed properly.

Then run jupyter:

>> jupyter lab --ip=0.0.0.0 --port=8888

With your instance IP address found in google console go to your browser(your_ip:8888) and use the token found here:

Now you should get access to Jupyter Lab

Install jupyter gateway

>> conda install -c conda-forge jupyter_kernel_gateway

Run byobu using >> byobu. byobu is a text-based window manager and terminal multiplexer. Be sure you are not in a conda env. Working from conda cause and ugly side effect in which the footer repeats forever. Use >> conda deactivate .

>> byobu

The use:

>> jupyter notebook --ip=0.0.0.0 --port=8888

create another byobu windows F2

>> conda activate rapids-0.19
>> jupyter kernelgateway --ip=0.0.0.0 --JupyterWebsocketPersonality.list_kernels=True --KernelGatewayApp.allow_origin='*'

Test the gateway going to

http://<your_ip:your_port>/api/kernelspecs

you should get something like

create another byobu windows F2 , then install Optimus

>> pip3 install git+https://github.com/ironmussa/Optimus.git@develop-3.0

Install Bumblebee

Install Node.js 10

>> curl -sL https://deb.nodesource.com/setup_10.x | sudo -E bash -
>> sudo apt-get install nodejs

Install and run MongoDB. Go to

https://docs.mongodb.com/manual/tutorial/install-mongodb-on-ubuntu/

Clone Bumblebee from the github repo

>> cd
>> cd ..
>> cd opt
>> git clone https://github.com/ironmussa/Bumblebee.git
>> cd Bumblebee
>> chmod +x start-environment.sh; ./start-environment.sh

You will be asked to introduce the public IP of your server, you may leave that field blank to initialize in localhost mode

Now Bumblebee is running. You can point to your server IP or localhost

From here you may create an account and start using Bumblebee 🚕

Extras:

Install git

>> sudo apt install git

Install curl

>> sudo apt install curl

Install pip

>> curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py
>> python3 get-pip.py

Upgrade RAPIDS

>> conda update -c rapidsai-nightly -c nvidia -c conda-forge -c defaults rapids

Install BlazingSQL

Be sure the version match you current CUDA and python versions:

>> conda install -c blazingsql-nightly/label/cuda10.1 -c rapidsai-nightly -c nvidia -c conda-forge -c defaults blazingsql python=3.7