Setting up a Swift for TensorFlow dev enviroment with Jupyter on Ubuntu

4 min readMar 21, 2019

Edit: Updated with new, simpler approach utilizing conda. Original post utilized virtualenv and required editing of the Swift-Jupyter code base for some users.

Overview

Install Swift for TensorFlow from nightly builds
Clone Swift-Jupyter from GitHub repository
Create a conda env with Python 3.6 and the required packages
Set the $LD_LIBRARY_PATH environment variable
Register the Swift-Jupyter kernel with Jupyter
Run the tutorial Jupyter notebooks to test out Swift for TensorFlow!

Introduction

This post details my account of getting set up for using Swift for TensorFlow with Jupyter notebooks on Ubuntu 18.04. Hopefully, it will serve as a guide for others attempting to do the same and — maybe — even help avoid some of the issues I encountered during the process.

Step 1. Install Swift for TensorFlow from nightly builds

Head over to the “Swift for TensorFlow” GitHub repository to download and install the appropriate nightly build for your version of Ubuntu. Baseline requirements include:

CUDA 10.0 or 9.2
cuDNN 7.1
NVIDIA GPU with compute capability 3.5 or 7.0**
Other dependencies installed via apt-get, as below

**Note: If your GPU has compute capability that is NOT 3.5 or 7.0, you will experience a long delay (~10 min) the first time you run a TensorFlow operation, while TF compiles new kernels for your compute capability.

$ sudo apt-get install clang libicu-dev
$ sudo apt-get install libpython-dev libblocksruntime-dev

Download the *.tar.gz file for the appropriate prebuilt package or nightly build. I went through the following procedure to extract the files and move them to the desired location (for me this was a newly created folder ~/Applications/swift-tensorflow ). The extraction will create a usr subdirectory within the folder to which your tar file was downloaded. After moving the usr folder to the desired location, you will need to add the path to the usr/bin directory to your PATH environment variable.

$ tar xzf swift-tensorflow-DEVELOPMENT-cuda10.0-cudnn7-ubuntu18.04.tar.gz
$ mv usr ~/Applications/swift-tensorflow
$ echo 'export PATH="~/Applications/swift-tensorflow/usr/bin:$PATH"' >> ~/.zshrc

Now, you should be able to run the Swift REPL and LLDB debugger from the command line. See https://swift.org/getting-started/ for a brief overview.

Step 2. Clone Swift-Jupyter from the GitHub repo

Basic requirements include:

Prebuilt Swift toolchain with LLDB Python3 support (fulfilled by Step 1)
Python3 and a virtual environment manager — we’ll use conda
Other requirements are satisfied below

After you git clone the Swift-Jupyter repository and cd into the swift-jupyter directory.

Step 3. Create a `conda` env with Python 3.6 and the required packages

The Swift-Jupyter docs recommend using virtualenv, but conda is what is typically recommended for fast.ai, so I’ll stick with that. We’ll need to create an environment with Python 3.6. We’ll then activate our new environment and install the basic requirements for working with Swift for TensorFlow and Swift-Jupyter.

**Note: The official installation guidelines recommend pip installing the requirements from the files in the Swift-Jupyter repo. However, through experimentation at Jeremy Howard’s suggestion, it’s easier to use conda , as we do here. In addition to being simpler, it avoids a few dependency conflicts that arise if you try it the other way.

$ conda create -n s4tf python==3.6
$ conda activate s4tf
(s4tf) $ conda install jupyter numpy matplotlib

Step 4. Set $LD_LIBRARY_PATH environment variable

Swift-Jupyter expects to find the cuDNN files where the $LD_LIBRARY_PATH is directed during the swift kernel registration process in register.py (addressed below in Step 5). Thus, you should run the following line of code to set this env var to the directory where these files (i.e. libcudnn*) should be found.

$ LD_LIBRARY_PATH=/usr/local/cuda/lib64
$ export $LD_LIBRARY_PATH
$ sudo ldconfig

Step 5. Register the Swift kernel with Jupyter

Next, we’ll run register.py with a couple of options to specify the locations of the Swift toolchain (Swift for TensorFlow, in our case) and the Python library that we want the kernel to use.

(s4tf) $ python register.py --sys-prefix --swift-toolchain ~/Applications/swift-tensorflow --swift-python-library ~/miniconda/envs/s4tf/lib/libpython3.6m.so

If you’re successful, you should see the following print to stdout.