Started using Docker from Dev to Prod yet? Here are some hacky solutions to common problems during development. Some of these apply for VirtualBox, while others are independent.
Thanks to @docker for featuring this in the Docker Weekly!
Edit: Added dumb-init
Port Forwarding to Host machine
Port forwarding only works inside the boot2docker VM. What if you want the same ports to be forwarded to your Host machine as well — say you are testing an Android app’s backend or want to share this with somebody else?
You have two options:
1 — Go to VirtualBox settings and add a Bridged adapter. Now when you boot the docker-machine, you’ll get an official LAN address for that VM. And you can access the VM from any other device on the network, and docker-compose port forwarding can be used.
2 — Wait for Issue #691 (or related) to get resolved. It will allow you to use SSH Port Forwarding. The only problem is it has to be done manually for each port you need.
docker-machine ssh -L <host-port>:localhost:<machine-port>Slow Performance
By default, when you create a VM with docker-machine, it defaults to very low capacity (1 CPU, 1 GB RAM). Make sure you run “docker-machine help create” with flags to increase CPU/RAM. For example:
docker-machine create \
--driver virtualbox \
--virtualbox-cpu-count 2 \
--virtualbox-memory 2048 \
devSymlink Errors
There are various reasons why this kind of error happens, but the most common one with VirtualBox is the lack of Guest Additions. Make sure it is installed (VirtualBox > Preferences > Extensions) and matches the same version of VirtualBox. Mostly happens when people install VirtualBox separately.
Shared Folder not picking up changes
This is a known bug in boot2docker since it caches NFS files for speed. You need to refresh the filesystem cache.
docker-machine ssh <name-of-your-machine>sudo cache-clear
(OR)
sudo su
# echo 3 > /proc/sys/vm/drop_caches
Some commands/languages ignore the FS cache. For example, you won’t find this error when running npm, less, vi, etc. But other commands do use the cache — cat, python, Apache SendFile, etc.
This issue is also related with the Apache SendFile bug in VirtualBox as well.
Internet Errors
Often you’ll get weird internet errors like address not resolved, ping not connecting, etc. This usually happens when your host machine’s network configuration changes (e.g. you switched WiFi networks, etc). Restarting the VM picks up the updated network configuration.
Building and Dependencies
Your Dockerfile will usually compile and build the application, and keep it ready for launching. When you are mounting your folder inside as a volume (as is usually the case in development) this causes various issues.
1 — Some languages (Ruby, Python, etc) install dependencies to a shared folder in the user directory. Others (Node/NPM) install directly on the current app folder. So when building the container, “npm install” happens, but later when you run “docker-compose up” with a shared volume, the “node_modules” folder goes away!
2 — Similarly, the build folders also go away. So the container’s CMD that starts the app will mostly not work.
This is tricky even if you have different Dockerfiles for dev/prod. Because this is not an image build-time issue, its a container runtime issue. The current solution is to rely on a Task runner for your language (rake/ grunt/ gulp/ fabric/ etc) and use that in the docker CMD for development. Don’t stuff application’s “build” logic into the Dockerfile, use the task runner instead.
# gulp/grunt/rake/...
# have a "build" task# Dockerfile
# used for normal production deployment
# here you use separate tasks as needed
COPY package.json /myapp/
RUN npm install
COPY . /myapp
RUN gulp build
CMD ["npm", "start"]# docker-compose
# when developing, combine all the above commands
myservice:
volumes: ".:/myapp"
command: "sh -c 'npm install && gulp build && npm start'"
.dockerignore
Make sure your “.dockerignore” file ignores common special files across platforms. These unnecessary files invalidate your docker build cache. With the right “.dockerignore” file in place, you can simply run “COPY . /myapp” to copy your entire app source code when building, rather than doing it folder by folder.
# Ignore .git folder
.git*# Ignore all Windows, Linux and OSX special files
# https://www.gitignore.io/api/windows,linux,osx,vim# Ignore all your language-specific build folders
build/
target/
node_modules/
.bundler/
etc
Use gitignore.io to get the common list of files for languages/platforms. But keep in mind that “.gitignore” and “.dockerignore” evaluate these file patterns differently. In dockerignore the entire file name has to match. So in gitignore you can say “node_modules” and it will ignore the whole folder, but in dockerignore you have to say “node_modules/*”.
Ctrl+C takes long time to stop the containers
That’s because signals from the shell are not properly sent to your docker container. Just use dumb-init. It is a great, small wrapper over your executable, and properly forwards all signals to your command.
# Simply Change:
CMD ["npm", "start"]
# To:
CMD ["dumb-init", "npm", "start"]And your Ctrl+C and container cleanup problems will magically disapper (in most standard cases). This not only helps in development, but also in production. Read Yelp’s blog post for more details.
My only gripe is, for such a simple piece of software, only a DEB package is provided. A straight-forward executable would have been much better.
Cleaning up
Here is a docker-clean command that removes all untagged images and stopped containers. Periodically run this to keep the VM clean.
# save this in the PATH as docker-clean
docker ps -aqf status=exited | xargs docker rm
docker images -qf dangling=true | xargs docker rmiUnit Tests / CI
Have a separate docker-compose.test.yml where all services “extend” from the normal docker-compose.yml, but just override the “command” set directly to run tests. Then in the CI, run the following:
docker-compose -f docker-compose.test.yml run <service_name>The above will directly start all linked services, run your tests, and will exit with the status code of your test script. Now your CI can simply have docker-compose installed, and for every service it can just run the above command with no service-specific knowledge!
Linking
With docker-compose, its tempting to link all your microservices together when running tests. This is usually not a problem for pet projects, but for larger projects it bites back. Avoid that early on and use stub servers like stubby4node with a simple YAML file that describes the stubbed endpoints.
Clock Sync Errors
When the docker-machine VM clock is out of sync with actual time, you’ll get many weird errors like Signing errors (various AWS services). In this case, just restart the VM to re-sync the clock. This mostly happens after you suspend/resume your laptop or desktop. Your host OS updates the time, but the suspended VM falls behind. The same happens often in Vagrant as well.
Forgetting to shutdown the VM
Shameless plug: We’re developing a Menubar app for Docker because we tend to leave the docker-machine VM running all the time and forget to shut it down. Have a look — github.com/rdsubhas/docker-menu!
