Linux Web Apps on Azure and “no space left on device”
Azure provides next to Windows Web Apps also Linux based Web Apps. Windows web apps are based on multiple IIS Web Sites on a single VM. Linux Web Apps are underneath Linux containers instantiated from a Docker image.
See General availability of App Service on Linux and Web App for Containers for a great post by Microsoft on the topic.
When I started with the Linux Containers I got a time-limited App Service plan in free tier to deploy my containers to. Initially it all worked fine, but after a few days I got the following error message (see “Container settings” under “SETTINGS” to see and download logs) when I tried to deploy a new version of my image:
ERROR - Container start failed for test-myserver-devweb_0 with System.AggregateException, One or more errors occurred.
InnerException: Docker.DotNet.DockerApiException, Docker API responded with status code=InternalServerError, response={'message':'write /var/run/docker/containerd/daemon/io.containerd.runtime.v1.linux/moby/ef7a4a2eaf7511a9aed14857fe6814
a3e6ea32bfc2ad34ddfc4ea7fb5c8957f/config.json: no space left on device: unknown'}I issued a support request on the Azure portal and got great help.
The issue was that in the free tier App Service plan multiple users share a VM to run their containers on. Other users can take up the available space with their images one the VM resulting in the above error. I never had such issues with the free Windows Web Apps so I didn’t expect this to happen. I actually thought that I would get my personal free tier VM because I had to create a separate Linux based App Service plan and it is time limited! I had to turn it into a paid version anyway to keep using it — unlike how it works with the Windows Web Apps.
The solution was to scale up to a paid tier (see “Scale up (App Service plan)” under “SETTINGS”). Because I was in a dev scenario I wanted to scale up to a B1 pricing tier (1 core, 1,75GB, A-Series compute), but the “Apply” button was grayed-out when trying to select this option. As a work-around select the link “See additional options” to see additional pricing tiers and select the B2 pricing tier and select “Apply”. When this works scale down to the B1 pricing tier.
Note that when you scale up your Docker Image based Web App you actually scale up your App Service plan the Web App is running in. An App Service plan is the actual VM you are running. On this App Service plan (VM) you can run multiple containers for the same price, but beware: these containers share the same VM and its underlying resources: cores, memory, network, … If you put some thinking in building you Docker images the running containers (Web Apps) can share most layers. Only your application specific layer differs. This allows you to separate functionality over multiple containers each with their own deployment schedule. See Best practices for writing Dockerfile and Use multi-stage builds for some tips on writing optimal Docker images.
Glossary:
Azure Web App Service Plan — “The role of the App Service Plan is to specify a series of virtual machines with a set of capabilities and to size the virtual machine’s RAM and processor capacities” (see https://www.petri.com/azure-web-app-service-plans)
Docker image — “Docker images are the basis of containers. An Image is an ordered collection of root filesystem changes and the corresponding execution parameters for use within a container runtime. An image typically contains a union of layered filesystems stacked on top of each other. An image does not have state and it never changes.” (see https://docs.docker.com/glossary/?term=image)
Docker container — “A container is a runtime instance of a docker image.” (see https://docs.docker.com/glossary/?term=container)
Nice article on Design Considerations for Azure Web Apps.
