Devendor Tech
Published in

Devendor Tech

Userns LXD, nested Docker, and accelerated GPU graphics

Userns LXD, nested Docker, and accelerated GPU graphics

LXD profile with nested docker and accelerated GPU support for JetBrains toolbox.

Prettier, github, medium.


There are certainly more use cases for this type of container, but in my case, I’m using it to contain development environments without losing the advantages of direct access to runtime system from advanced tools like pycharm or Idea.

There are certainly other approaches to the issue of getting an advanced IDE in the same process space as your running app including remote debugging and various IDEs that have moved toward client server models or a browser model instead of a direct GUI application. I like those ideas, but I don’t like to waste time figuring out new IDEs that usually have more limited language support and very different UI designs.

For my money, jetbrains gives me a consistent layout and functionality across all of the languages I currently program in with deep support in each language. The trade off is often remote debugging or contaminating your workstation with development jobs that come and go.

This approach eliminates most of those issues and keeps things clean and safe.


I strongly recommend checking out my turtles article for background, as I don’t intend to recap nested docker, LXD, and cloud-init here.

See Turtles on github, on Devendor Tech for prettiest formatting, and on medium for a walkthrough of nested container support for LXD.

My Environment

The following details of my workstation environment effect the lxd profile I use for turning up a new development environment.

How To

Starting where Turtles left off.

Get and edit the new devUser.yml profile. .. code-block:

rferguson@myhost$ git clone

Edit the profile to suit your environment as noted in TODO tags. .. code-block:

rferguson@myhost$ vim turtles/devUser.yml

Create a lxd profile and import devUser.yml to it. .. code-block:

rferguson@myhost$ lxc profile create devUser
rferguson@myhost$ lxc profile edit devUser <devUser.yml

Launch a new dev machine. .. code-block:

rferguson@myhost$ lxc launch b dev1 -p devUser -p default
rferguson@myhost$ lxc exec dev1 -- tail -f /var/log/cloud-init-output.log

Map a project dir if desired. .. code-block:

rferguson@myhost$ lxc config device add dev1 myproject \
disk source=/home/rferguson/code path=/home/me/code

Launch pycharm. .. code-block:

rferguson@myhost$ lxc exec dev1 -- runuser me -c "pycharm ~/code" &

Final thoughts

You wil have to do some initial setup unless you also map your IDE setting directory, but it can be nice to use one of the settings sync options of this particular ide and keep per instance settings separated and use the various settings sync options to syncronize or archive IDE settings on a per-project basis.

Mapping user rferguson to user me has pragmatic value since I can now image the entire dev environment and give it to the new guy or push it somewhere else and run it under a different local host user. Me@dev1 is also a shorter PS1 for cleaner looking docs.

There is a lot that isn’t covered here. The docker nesting are already in turtles and there is some good information on the details of GPU features of LXD containers from existing sources.

Happy coding!

devUser.yml Profile

Checkout the devUser.yml on github for the maintained version.

name: devUser
description: LXD profile with nested docker and accelerated GPU support for JetBrains toolbox.
environment.LANG: en_US.UTF-8
environment.LANGUAGE: en_US:en
environment.DISPLAY: :0.0
environment.XAUTHORITY: /home/me/.Xauthority
nvidia.runtime: "true" # TODO only if you have an nvidia GPU.
raw.idmap: | # TODO Set your UID/GID
both 1000 1000
linux.kernel_modules: ip_tables,btrfs
security.nesting: "true"
security.privileged: "false"
user.user-data: |-
all: '| tee -a /var/log/cloud-init-output.log'
package_update: true
package_upgrade: true
- set -xe
- curl -fsSL | apt-key add -
- apt-get install -y apt-transport-https curl
- add-apt-repository
"deb [arch=amd64]\
linux/ubuntu $(lsb_release -cs) stable"
- apt-get update
- apt-get install -y
- usermod -aG docker me
- systemctl start docker
- docker image pull hello-world
- docker run --rm hello-world
- apt-get install -y
libnvidia-gl-390 # TODO validate appropriate gl library.
- "export DISPLAY=:0.0 XAUTHORITY=/home/me/.Xauthority"
- nvidia-smi
- runuser me -c "glxinfo -B"
- runuser me -c "glxgears -info" &
- sleep 12
- killall glxgears
- path: /etc/rsylog.conf
content: |
*.* @log.virtdmz
owner: root:root
permissions: '0644'
- path: /etc/docker/daemon.json
content: |
"hosts": [
"storage-driver": "btrfs"
permissions: '0644'
owner: root:root
- path: /etc/systemd/system/docker.service.d/override.conf
content: |
permissions: '0644'
owner: root:root
- path: /bin/pycharm
permissions: '0755'
owner: root:root # TODO Check your install path.
content: "\
exec $( ls -1c ~/.local/share/JetBrains/Toolbox/apps/\
PyCharm-P/ch-0/*/bin/ | head -1) $@\n"
- name: me
- adm
lock_passwd: true
shell: /bin/bash
uid: 1000 # TODO Swap to your numeric UID/GID
gid: 1000
ssh-authorized-keys: # TODO Add your own keys
oUu43rMsGLjL/gyD5XNJntdSuENYWH rferguson@booger"
devices: # TODO Swap in your home dir path.
path: /home/me/.Xauthority
source: /home/rferguson/.Xauthority
type: disk
type: gpu
uid: "0"
gid: "0"
path: /tmp/.X11-unix/X0
source: /tmp/.X11-unix/X0
type: disk
source: /home/rferguson/.local
path: /home/me/.local
type: disk



Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store