Problem: App upgrade fails with messge "no space left on device"

Problem

While upgrading a docker app a new container is downloaded. You might end up in a situation where the docker pull displays a message that there is no space left on device

failed to register layer: ApplyLayer exit status 1 stdout: stderr: open /tmp/APP/PATH/TO/SOME/FILE: no space left on device

or

7722 docker  19-04-09 14:11:42 [    INFO]: failed to register layer: mkdir /var/lib/docker/overlay/\<GUID\>/tmproot\<ID\>/usr/lib/node_modules/npm/node_modules/flush-write-stream: no space left on device
7722 docker  19-04-09 14:11:42 [   ERROR]: Command docker pull docker.software-univention.de/APP:x.y.z-ucs.n failed with: x.y.z-ucs.n: Pulling from \<APP\>

Investigation

Because the failed docker pull has saved several files and probably Megabytes or even Gigabytes it’s a good idea to clean up first. In order to do so investigate the docker environment. The following commands display the used disk space, the used space that was freed by that command and a list of all image which are present

root@ucs:~# docker system df
root@ucs:~# docker system prune
root@ucs:~# docker image ls -a

When you investigate the free space you might realize there is plenty or at least enough free space left.

root@ucs:~# df -h
Dateisystem            Größe Benutzt Verf. Verw% Eingehängt auf
udev                     10M       0   10M    0% /dev
tmpfs                   401M     18M  384M    5% /run
tmpfs                  1003M       0 1003M    0% /dev/shm
tmpfs                   5,0M       0  5,0M    0% /run/lock
tmpfs                  1003M       0 1003M    0% /sys/fs/cgroup
/dev/sda1               227M    108M  104M   51% /boot
/dev/mapper/vg_ucs-var   39G     17G   20G   46% /
overlay                  39G     17G   20G   46% /var/lib/docker/overlay/\<GUID\>/merged
shm                      64M       0   64M    0% /var/lib/docker/containers/\<GUID\>/shm
tmpfs                   201M       0  201M    0% /run/user/0

Solution

If there is not enough free space you have to enlarge your partition; see this article on how to do so.

Bluespice

There was a special case with that app, see here

Mastodon