The Newbie Guide to Setting Up a Jetson Nano on JP4.4 Part 1: Running Jupyter Lab Headless Using SSH

As a newbie, my goal is to replicate the learning enviroment used by Nvidia’s Deep Learing Institute courses. I’ve done both the “Getting Started with AI on the Jetson Nano”, and “Getting Started with DeepStream for Video Analytics on Jetson Nano”. I like how those courses were presented. I liked using ssh to remote into the Nano and using Jupyter Lab to learn both AI, Machine Learning, and Coding.

For me right now, as a newbie, this approach to learning makes things so much easier. I know one day I’ll be good enough to get into an IDE like PyCharm or Visual Studio to deploy my projects, but for now my main goal is to breakdown the problem into the most basic and simple steps and build from there. So far I’ve found no better equivalent than Jupyter Lab to make this happen, especially with Python.

The steps I describe below is what worked in setting up the Nano to run “Headless”, allowing me to ssh into it and run Jupyter Lab from a remote computer (my laptop). I’ll try my best to explain the step in detail so it’s clear why it needs to be done this way.

Important note: The steps I discuss below are for an ethernet connection. Some of these steps may be different for Wifi.

Step 1 — JP4.4 Post-Installation configuration

This steps requires that you’ve already done the following:

• Flashed JP4.4 onto a MicroSD card and it’s in your Nano.
• You have a keyboard, mouse, ethernet, and video display connected to your Nano.
• You’ve powered your Nano, and have run the setup and configuration.

Once you’ve logged into the Nano, here are the first things to do:

1. Open up google chrome and set up sync. This step is important, since it’ll help you quickly log into the Nvidia Developer websites, as well as help you access pages where you might need to “copy/paste” instructions, instead of typing them.
2. Download the DeepStream5 SDK. Make sure it’s the “DeepStream5.01 for Jetson” version! Also, makes sure you download the debian version (.deb). This file will automatically get saved into your Downloads directory, unless you specified otherwise.

Step 2— Remove Unecessary Software & Change Swap

Removing Unecessary Software

My Nano is going to be used for ComputerVision and Machine Learning. As such I don’t need the pre-loaded ubuntu games or LibreOffice. To uninstall, simply go to the “Store” icon found on the launcher. Go to “Installed” tab, and remove the following:

• Any games
• Anything related to LibreOffice

I would caution to not remove anything else, unless you know for sure what you’re removing. If you don’t know what it does, leave it alone.

By now you should also be getting some sort of notification that Ubuntu needs updating, run those updates now. If you run the updates before uninstalling the software, the packages updates is larger, albeit only incrementally larger; but larger no less. You’ll be prompted to reboot your machine once complete.

Changing size of SWAP file

For now I’m going to change the size of the SWAP file to 8GB, mainly to test out compiling a new version of OpenCV, but it seems there is some thoughts out there that a larger swap file is better.

Here are the general instructions to change SWAP file size. In my case it’s to 8GB, if you want to change it to anything else , simply replace the ‘8’ with whatever number is appropriate.

$ sudo fallocate -l 8G /mnt/8GB.swap
$ sudo chmod 600 /mnt/8GB.swap
$ sudo mkswap /mnt/8GB.swap

Next, add the following lines to /etc/fstab:

/mnt/8GB.swap swap swap defaults 0 0

I used vi to do this, but you can also use nano or any other editor you feel comfortable with. Just replace vi with whatever editor you care to use:

$ sudo vi /etc/fstab

Reboot your machine and check the swap file size by running the following command:

$ free -h

This is what I got back:

      total    used      free    shared   buff/cache   available
Mem:   3.9G    400M      3.1G      18M       421M        3.3G
Swap:  9.9G    0B        9.9G

I can’t explain why it’s showing 9.9GB of SWAP, but for now I’m leaving it be. I’m planning on installing OpenCV with CUDA support, and they recommend a large SWAP file for the install process.

Step 3— Updating Packages and Installing Jetcam and DeepStream

Once you’ve logged back in, it’s time to update packages and Install DeepStream.

Updating packages

Run the following command in a terminal window:

$ sudo apt update && sudo apt upgrade

In allot of websites you’ll see it this way:

$ sudo apt-get update && sudo apt-get upgrade

This is the old version ‘apt’. However because of backwards compatibility, these commands both do the same thing. I prefer using the new convention so I’m going to be sticking with that format from here onward. Don’t get stuck thinking there is a difference between the two. There isn’t.

Once the packages are updated, you’re ready for the installation of DeepStream5.0. On the Jetson Nano forums, there was a discussion regarding all the setup and configuration you needed to do prior to setting up DeepStream. Long story short, if you follow the official instructions, you won’t have success in installing. They’re confusing.

Here is the solution:

Installing DeepStream5

Technically, you don’t need to do this step now. I did it just because, but other than explore the sample apps, there’s nothing really you can do with this DeepStream right now, if running in a Jupyter Notebook.

Navigate to the folder where you have DeepStream downloaded, in my case it was /Downloads. Run the following command to see what’s in that folder:

$ ls -la

Running that command produced this result for me:

total 418984
drwxr-xr-x  2 jiac jiac      4096 Oct 14 09:50 .
drwxr-xr-x 22 jiac jiac      4096 Oct 14 12:10 ..
-rw-r--r--  1 jiac jiac 429026460 Oct 14 09:50 deepstream-5.0_5.0.1-1_arm64.deb

Note: “jaic” is my username. When you run this command, you’ll see your username in that space.

The following is the full name of the debian package: deepstream-5.0_5.0.1–1_arm64.deb. Copy the full name you get of the debian package you downloaded, and what you see when you use the ‘ls’ command. It’s important that you properly copy the full name of this file, otherwise the next step won’t work.

Next, install DeepStream by using the following command:

$ sudo apt install ./deepstream-5.0_5.0.1-1_arm64.deb

DeepStream installs itself here by default:

/opt/nvidia/deepstream/deepstream-5.0

I’m not sure if you can change the default location. I can’t comment.

Installing Jetcam

“JetCam is an easy to use Python camera interface for NVIDIA Jetson.” (Copied from the JetCam github). This library is needed, it makes working with CSI or USB cameras in your Jupyter Lab far and away easier than anything else I’ve looked at. To learn more about JetCam, including notebooks that provide support for multiple CSI cameras, go to the JetCam github located here.

The setup bellow is copied from the github:

$ sudo git clone https://github.com/NVIDIA-AI-IOT/jetcam
$ cd jetcam
$ sudo python3 setup.py install

The github does a really good job of explaining how to use, I encourage you to learn more by going to the github.

Step 4-Installing Jupyter Lab Dependencies

Here are the dependencies that need to be installed before installing Jupyter Lab:

node.js

Jupyter Lab uses Node JS to manage widgets, amongst other things. The version of Node JS that comes on JP4.4 is an older version and will cause you grief if not updated.

The following steps installs node.js, overriding the version used in JP4.4. Note: This is the only way I’ve found that works when installing node.js. All other means say that node.js is up to date, no need to update. These instructions are based on a discussion on the developers forum regarding installing node.js on the nano.

$ wget https://nodejs.org/dist/v#.#.#/node-v#.#.#-linux-arm64.tar.xz tar
$ tar -xJf node-v#.#.#-linux-arm64.tar.xz
$ node-v#.#.#-linux-arm64
$ sudo cp -R * /usr/local/
$ node -v
$ npm -v

Where #.#.# is the version of nodejs you plan on install. As of this writing, the most current version of nodejs, from the nodejs website (https://nodejs.org/en/) is: 12.19.0. Make sure you’re installing the most current version, or you might run into issues with ipywidgets and ipywebrtc.

libffi-dev package and libssl1.0-dev

To be honest, I don’t really know what these packages do. All I know is that without it, the Jupyter Lab installation failed to build on errors saying it was missing files related to these packages. I installed this packages, re-ran the Jupyter Lab installation, and everything worked!

$ sudo apt install -y libffi-dev
$ sudo apt install -y libssl1.0-dev

Note: I’ve since noticed that you can look at the “info.md” found in the nvidia DLI course: Getting Started with AI on the Jetson Nano, when you launch the Jupyter Lab for that course. There you will see that these packages are needed.

numpy, scipy, matplotlib packages

To update / install these packages, run the following command:

$ sudo apt install python3-numpy python3-scipy python3-matplotlib

It’s important to note python version you’re installing, as noted above. As of 2020-Oct-15, the default python version installed on the Jetson Nano is Python2. Therefore, if you run this command:

$ sudo apt install python-numpy python-scipy python-matplotlib

You’re telling the computer to update / install the Python2 packages, which can wreak havoc on your machine. The OS uses these Python2 packages in it’s set up and operation. Since you’re not creating a virtual environment (more on why latter), you need to be careful and specify the python version you’re going to modify. Specifying the wrong version can have bad consequences.

python3-pip

The Python Installation Packager (pip) is not installed on the Nano by default. You need to install it:

$ sudo apt install -y python3-pip

I’m going to re-iterate that the default python version on the Nano is Python2. Therefore, any commands we use need to explicitly address Python3.

It’s important to rember this fact. Otherwise, you’ll wonder why “pip install” doesn’t work, and do something silly that messes everything up. Only pip3 will be installed by the operating system. It’s a safegaurd measure.

Installing this package also installs other packages, including the python3-dev package.

Step 5-Installing and Configuring Jupyter Lab

Installing

Before you install Jupyter Lab, you must first make sure that you add the user-level bin directory to your PATH environment variable. You do this by running the following command:

$ export PATH="$HOME/.local/bin:$PATH"

You need to do this in order to get the installation to work properly because of how you’ll install Jupyter Lab:

$ pip3 install --user jupyterlab

You’ll get errors if you try and install it using “pip install…”. This installation method is a variation of the official Jupyter Lab installation instuctions found here.

It’s a good idea to reboot after the installation.

$ sudo reboot now

Configure + Password Setup

To configure the notebook, run the following command:

$ jupyter notebook --generate-config

To set up a password, do the following:

$ jupyter notebook password

You need these steps so that you can ssh into the Nano and use Jupyter Lab.

You can find out more information and details on these commands here. I believe there are more complicated steps required if you plan on doing wireless communication with the Nano. Most of these instructions I fot from Andrew Bibian’s github.io page. He has instructions on Running Jupyter Notebooks on the Jetson Nano (via https). If you’re plan is to run wireless, check out his instructions. Note: He writes scripts automating allot of this stuff, which may or may not interest you.

Installing Extension Manager, iPywidgets, and iPywebrtc

Since you’re going to be running Jupyter remotely, you’re going to need to install the widgets and libraries to display images inside Jupyter Lab. Without doing this step, Juypter will execute the command, but you won’t see the display window.

$ sudo pip3 install Cython
$ jupyter labextension install @jupyter-widgets/jupyterlab-manager
$ pip3 install ipywidgets
$ jupyter nbextension enable --py widgetsnbextension
$ pip3 install ipywebrtc     
$ jupyter labextension install jupyter-webrtc
$ pip3 install ipyleaflet
$ jupyter labextension install @jupyter-widgets/jupyterlab-manager jupyter-leaflet
$ pip3 install bqplot
$ jupyter labextension install bqplot

To learn more about the webrtc widget, and some of the things you can do with it, go the the webrtc github here. I encourage you to go look at their github, you’ll be impressed!

Reboot the Machine and Verify the installation

After rebooting the machine, test the installation. First, open a web browser (default web browser), next launch jupyter lab.

$ jupyter lab

The open web browser should launch Jupyter Lab. You’ll be prompted for the password you created, and then to the Jupyter Lab homescreen.

Note: If you don’t launch a web browser before running jupyter lab, you’ll get the following errors in the terminal window you used to launch:

Error: Can't initialize nvrm channel

The error is more annoying than anything else. To avoid getting these errors, simply have a web browser open before you launch.

Get Nano IP Address

You’ll need to determine the IP address of your nano. There are a few ways of doing this, both using the command line (ifconfig) or going to the network manager icon.

Once you’ve noted the IP address, power off the nano. It’s time to disconnect the monitor, keyboard, and mouse. Going forward all you need to have connected is the power and ethernet. Everything else you’ll be doing via ssh.

Power up your Nano, making sure that no display is connected. When a display is detected, the Nano start running the configs necessary for the video display. If no display is detected, the Nano runs in OEM mode, freeing up resources that would be otherwise used for display.

If the Nano does not detects a display, it does not run

Step 6-Testing SSH connection and Jupyter Lab remote

To ssh into your nano, you need to know two things:

1. Username on Nano.
2. Nano IP address(ethernet).

On the host machine, open a terminal window and run the following command:

$ ssh -L 8000:localhost:8888 jiac@10.0.0.92

In this example, jiac is the username on the nano, and 10.0.0.92 is the nano IP address, replace these with your nano username and ip address. This command is a modified version of Mircea Stoica’s instructions found in an article he published on Medium here.

You’ll be prompted for the username password, once logged in you’ll be in a command line terminal inside you Nano. From here, launch a jupyter lab notebook:

$ jupyter lab

This will start the jupyter lab server. On your host machine, open a webpage and type in the following:

http://localhost:8000

You’ll be prompted for the Jupyter Notebook password, and then done! You’ve launched your Jupyter Lab remotely.

You can test to make sure your camera is working by running the following either in a Python cell, or in a terminal inside jupyter lab:

If inside a python cell, run this command:

!ls -l /dev/video*

The output of this command for me is:

crw-rw----+ 1 root video 81, 0 Oct 14 11:35 /dev/video0

I’m using my Nano A02 board, it has only one CSI camera connection. If you have a Nano B01, and you have two cameras connected, you should see both of them when running this command.

From here you can also test OpenCV and any other packages. :

import cv2  #Imports OpenCV into the python notebook.
cv2.__version__   ##Outputs the version of OpenCV that python sees

If you’ve made it this far, then congrats!

You should now have a working Jupyter Lab on a web browser on a remote machine. All the raw processing power of the nano can be utilized in number crunching, instead of on diplays, etc.

Putting it all together and testing the setup

To test your installation, try running the following in a jupyter notebook:

Step 1 — Import the camera library:

I’m using a CSI camera:

from jetcam.csi_camera import CSICamera

I haven’t read what the USB camera would be, but intuition tells me that if you replace csi / CSI with usb / USB in the above, chances are it’ll work! Check the jetcam repo on github for more details, including multi-camera set-up!

Step 2 — Create the camera object:

camera = CSICamera(width=224, height=224)
image = camera.read()

Note, if you have any error messages with the above step, chances are that you have you’re camera stream connected to another notebook. A good way to make sure this isn’t happening is simply restarting the Jupyter Kernels and clearing all outputs.

Another option is adding this script to the very beggining of your code (before Step 1):

import getpass
import os
password = getpass.getpass()
command = 'sudo -S systemctl restartnvargus-daemon' #This is how you invoke any command
os.system('echo %s | %s' % (password, command))

This will release any and all camera connections, and you won’t get that error anymore.

Inspiration for this script came thanks to watching this YouTube video made by Victor Jiang.

Step 3- Display the camera object in a widget

import ipywidgets
from IPython.display import display
from jetcam.utils import bgr8_to_jpeg
image_widget = ipywidgets.Image(format='jpeg')
image_widget.value = bgr8_to_jpeg(image)
display(image_widget)

You should see a box with an image from your camer, just below the cell where you ran this code on Jupyter Lab.

Step 5— Stream Video to the widget

camera.running = True
def update_image(change):
    image = change['new']
    image_widget.value = bgr8_to_jpeg(image)
    
camera.observe(update_image, names='value')

You should now see video from your camera streaming into your video widget.

Step 6— Stop the camera stream

camera.unobserve(update_image, names='value')

If you got this far with no errors, congrats! You now have a Jupyter Lab evironment that you can start developing on your own!

Note: Allot of examples online, and even in some Jupyter Notebooks I’ve looked at, you’ll need to replace the python “print” command with the Widget, otherwise it may not execute properly. What works using a local notebook does not always work when using a remote notebook like this setup is doing.

Why I didn’t set up a Virtual Environment?

Best practise recommend that you always set up a virtual environment, it’s your sandbox that prevents your programming environment from affecting your computer’s system environmnet, or one programming environment from affecting another. Believe me, if it was easy, I would have.

Remember: My main goal is to set up an enviroment to try and see if I can play around with Python DeepStream. To that end, here are the road blocks I’ve encountered whrn trying to perform these steps in an virtual environment:

1. OpenCV no longer works. It doesn’t import into the environment, and as a newbie I’ve had no luck with sym-linking the OpenCV build that comes with JP4.4. Furthermore; I get crashes and errors when I try to build it inside the environment. As a newbie I haven’t yet figured out what I’m doing wrong.
2. Need to re-install Gst Python v1.14.5. Same as OpenCV, as soon as you create a virtual environment, you have to re-install Gst Python. This isn’t a chore, but it is a hassel.
3. With the issue with OpenCV and Gst Python, I wasn’t sure what other links would need to be re-made (CuDNN, CUDA, etc). My goal is to get something set up and running so I can start practising AI and Machine Learning, not figure out how to do an set up within an environment variable

I will eventually figure this out, but for now my mindset is simple: one nano, one environment.

If you do want to venture into setting up a virtual environment, have a look at Adrian Rosenbrock’s blog post: pyimagesearch. He’s a great resource to help you get going with Computer Vision. He’s even got a how-to called Getting Started with the NVIDIA Jetson Nano. These are step by steps instructions, and pictures on setting up your Nano; go have a look! However; I do caution: in these instructions there is no set up for OpenCV. He promises to have another blog that addresses.

Conclusion

I hope you found this article useful, please like and share if you did! I’m hoping this saves you the setup time I had to go through to get here myself. This article wouldn’t be needed if all this information was readily available somewhere else! On that note, here is a good article written by Alasdair Allan called: Getting Started with the NVIDIA Jetson Nano Developer Kit which gives allot more details on the hardware setup.

In Part 2 of this article, I’m going to discuss installing TensorFlow, and Building OpenCV with CUDA support. I’ll also provide a simple Jupyter Notebook that tests both OpenCV and TensorFlow, confirming you now have a means of learning AI, Machine Learning, and Computer Vision!

In Part 3, I hope to discuss how to acutally use DeepStream5.0 python examples and apps inside a Jupyter Notebook.

Please leave me notes in the comments, I’ll try my best to respond ASAP.