How to Set Up the Development Environment for the xArm5 Robotic Arm

xArm5 Robotic Arm with Gripper in a simulated environment in Gazebo

xArm is the most cost-effective robotic arm and has many novel applications. The Intel depth-sensing camera when installed on the arm acts as a sensor that allows the robotic arm to discover and know about the environment. The arm and the camera together function like a hand and an eye setup. The team at UNC Reese Innovation Lab (Chapel Hill, North Carolina, US) is working on making the robotic arm perform tasks autonomously without any human intervention. For example, in the initial project, “Pick and Place,” the robotic arm would detect the correct object from a pile of objects, pick up the desired object, and place the object in a new location. Since there is a lot of different hardware and software being used simultaneously, a typical user runs into compatibility issues. Therefore, this detailed guide has been created to describe how to install all the required software and libraries to set up the development environment.

Installation Versions

• Operating System — Ubuntu 20.04 LTS
• Robotic Arm — UFACTORY xArm5
• Camera — Intel Realsense D415
• Conda — Miniconda3-latest-Linux-x86_64
• Python — 3.8 (most compatible version, DO NOT use the latest version)
• Tensorflow — 2.8
• Cudnn — 8.2.1 (for CUDA 11.x)
• Graphics Card — NVIDIA Quadro RTX 5000 (notebook)

Option 1 Install the Nvidia-rapids library

• Nvidia rapids ubuntu: this installs the appropriate driver automatically
• Nvidia rapids repo: details how to set up a data science conda environment that will automatically install all dependencies
• clone the data-science-stack to xarm conda environment

However, Nvidia-rapids installs a lot of unnecessary software that will conflict with PyTorch and TensorFlow packages, hence avoid this option.

Option 2 Manual Install

Install NVIDIA Driver

1. Checkbox the additional driver install when doing a fresh install of Ubuntu.
2. Check the installation using nvidia-smi command at the terminal. This would output the Nvidia graphics card installed on your system. This would also signal the correct installation of the driver.
3. Install the Nvidia driver from the official download page if after installation nvidia-smi command does not work or show your graphics card.

DO NOT change GPU from Nvidia Settings to Performance… system won’t restart, change from BIOS display setting to use Discrete GPU .

Set up conda environment in ubuntu 20.04

• Miniconda is smaller in size than Anaconda, hence better to use Miniconda rather than Anaconda and install whatever libraries are required.

Replace the link with the latest one if this does not work, no-check-certificate option is required to bypass the firewall if needed.

wget --no-check-certificate https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh
sh Miniconda3-latest-Linux-x86_64.sh

• Create a new conda environment for each project separately to avoid conflicts between the installed packages.

conda create --name <ENVNAME> python=3.8
conda activate <ENVNAME>
conda deactivate

Link to conda cheatsheet

• Say ‘Yes’ to initialize conda during startup? If not, add this to the end of your ~/.bashrc file

# >>> conda initialize >>>
# !! Contents within this block are managed by 'conda init' !!
__conda_setup="$('<your-home-dir>/miniconda3/bin/conda' 'shell.bash' 'hook' 2> /dev/null)"
if [ $? -eq 0 ]; then
    eval "$__conda_setup"
else
    if [ -f "<your-home-dir>/miniconda3/etc/profile.d/conda.sh" ]; then
        . "<your-home-dir>/miniconda3/etc/profile.d/conda.sh"
    else
        export PATH="<your-home-dir>/miniconda3/bin:$PATH"
    fi
fi
unset __conda_setup
# <<< conda initialize <<<

Install UFACTORY xArm (in this case xArm5) software

1. Connect xArm to laptop via ethernet port.
2. Change the IPv4 settings from Network Settings as follows:
   These IPv4 settings might be different for each user but you can try with these settings.
   IP address — 192.168.1.12
   Subnet mask — 255.255.255.0
   Gateway — 192.168.1.1
   Preferred DNS — 192.168.1.12
3. Install xArm software Linux (.deb) from the website. This will download .zip file.
4. Right-click the file and choose Extract Here to extract an .AppImage file.
5. Make this .AppImage file executable by:
   Right-click the icon → Click Properties .
   Go to Permissions .
   Check box Allow executing file as program .
6. Just click this icon to start xArm Studio. DO NOT delete this file.
7. Connect the robot xArm directly to the ethernet port on the laptop and the internet ethernet to the ethernet port on USB C (Somehow the robot does not recognize the ethernet port on the extension).
8. Switch off the ethernet port (internet), connect to the robot via the direct ethernet port and then switch the internet ethernet port again. This is to be done ONLY ONCE.
9. Now install the required software from UFACTORY xArm Downloads Page

Install Intel RealSense Camera software

1. Installing the Intel Realsense Camera on Ubuntu (Use this link only, using other links will double install and cause conflict).
2. Follow this youtube video for a step-by-step guide to install intel realsense sdk on ubuntu.
3. Then install pyrealsense2 using pip .

Bonus install — install oh my zsh shell, plugins, powerlevel10k theme — recommended but not required

• Install zsh shell first.
• Then install oh-my-zsh using the Blog post for installing oh my zsh.
• Initialize conda to use zsh conda init zsh .
• Change default shell to zsh chsh -s <SHELL> .
• Change key bindings CTRL to CMD for mac.
• Add auto-suggestion plugins for zsh.
• Similarly, add the syntax-highlight plugin.
• Add theme powerlevel10k .
• Install font recommended by powerlevel10k.

Linux Tricks

• Set mid mouse button to paste:
  Make changes using nano ~/.Xresources .
  Add line xterm*selectToClipboard: true .
  Select text to copy and set the mid mouse button to paste
• Install tweaks using sudo apt-get install gnome-tweaks .
• Change computer from suspend to do nothing when the laptop lid is closed off from Activites → Tweaks .
• Setting up google drive in ubuntu.

Tensorflow Installation

• Go to the TensorFlow page here and follow the instructions.
• Useful guide here to install the tensorflow-gpuversion.
• Please make sure that PATH includes /usr/local/cuda-11.3/bin or add /usr/local/cuda-11.3/lib64 to /etc/ld.so.conf and run ldconfig as root .
• To uninstall the CUDA Toolkit, run ‘cuda-uninstaller’ in /usr/local/cuda-11.3/bin
• Sometimes you might get this error

WARNING: Incomplete installation! This installation did not install the CUDA Driver. A driver of version at least 465.00 is required for CUDA 11.3 functionality to work.

• To install the driver using this installer, run the following command, replacing > <CudaInstaller> with the name of this run file:

sudo <CudaInstaller>.run --silent --driver
Logfile is /var/log/cuda-installer.log

• Install all the prerequisites mentioned on the TensorFlow installation page.
• For cudnn installation: DO NOT Click oncuDNN SDKlink to install the sdk.debfile.

1. Click on cuDNN versions
2. Find the version of cudnn compatible with cuda 11.x (Here, choose cuDNN v8.2.1 for CUDA 11.x. for the current setting)
3. Download 3 separate files: runtime library, developer library, and code samples for Ubuntu20.04 x86_64 (Deb) and install all of them.

• Check the Cuda version using nvcc -V command.
• Check if TensorFlow is using GPU.

There is the code that prints the message “successful NUMA node read from SysFS had negative value (-1)”, and it is not a Fatal Error, it is just a warning. The real error is MemoryError in your File “model_new.py”, line 85, in <module>. We need more sources to check this error. Try to make your model smaller or run on a server with more RAM.

To get rid of this non-fatal warning use this command:
for a in /sys/bus/pci/devices/*; do echo 0 | sudo tee -a $a/numa_node; done

For details go to, this question in StackOverflow.

Pytorch installation

• Go to the PyTorch homepage and select the version appropriate for your system.
• Check if PyTorch is using GPU.

Some common commands to set up conda environment

• I’ve found that sometimes PyTorch and TensorFlow conflict in the same environment so create separate environments.
• Create conda environment conda create --name <env-name> python=3.8
• Delete conda environment conda env remove --name <env-name>
• Install the following libraries. Use pip if conda fails.

conda install -c conda-forge jupyterlab
conda install -c conda-forge nb_conda_kernels
conda install ipykernel
conda install -c conda-forge jupyter_conda # not needed perhaps
conda install -c conda-forge nodejs # not needed perhaps
pip install opencv-contrib-python # install contrib-python version #not the normal opencv version
pip install imutils

Github

Accessing the GitHub repo using a password on the local machine has been disabled.

1. Register GitHub on Microsoft authenticator or other authenticator apps.
2. Clone GitHub repo using personal access token by following this guide.

Some regular useful commands for creating and maintaining a repo

git clone <url: https simpler >
git add
git commit -m ""
git status
git push
git pull
git log
git reset --hard <commit>
git reset --hard origin/master
git branch
git checkout -b
git merge <branch to be merged> (head should point to `master` branch)
git branch -D (delete a branch)
git push --set-upstream origin <branch>
git commit -am
touch <file>
cat <file>
git config --global user.email "you@example.com"
git config --global user.name "Your Name"

Once you follow all the steps, you should have an environment ready to build exciting applications for the robotic arm. Follow along on our Medium publication and social sites for more articles about using the robotic arm to perform tasks autonomously.