Resolving the dpkg lock contention problem in Azure DevOps self-hosted Agents
In Azure DevOps self hosted agents running Ubuntu 18.04, I recently saw the following intermittent failure when trying to install packages.
E: Could not get lock /var/lib/dpkg/lock-frontend - open (11: Resource temporarily unavailable)E: Unable to acquire the dpkg frontend lock (/var/lib/dpkg/lock-frontend), is another process using it?
This happened when trying to use apt-get to install some packages. Since the package installation fails, the rest of the commands in the pipeline also fail.
When I looked up this error, I came across [1] by Andy Roberts. After reading his blog, I learned that when a Virtual Machine Scale Set instance is starting, the Azure DevOps extension uses apt-get to get updates automatically.
Therefore, the script task running in the pipeline (that we added) runs apt-get commands without knowledge of the apt-get command executed by the Azure DevOps extension. This results in contention for the /var/lib/dpkg/lock-frontend lock and results in the above error.
This issue occurs only at the beginning. So all we have to do is to wait for the lock to be released by the automatic update process.
I used a different approach to solve this contention problem than what Andy has suggested.
I used a while loop to check if any process is using the /var/lib/dpkg/lock-frontend file. If so, I put the loop to sleep. This was coupled with a timeout of 5 minutes for the entire loop (i.e. Only wait for 5 minutes for the lock to be released)
The Azure DevOps pipeline task with this fix looks like this
As you can see in the log below, the loop waits for the lock to be released thus preventing the lock contention.
References
