Hello Help,
I’m digging into a performance issue, that I observe since some time but didn’t have the time to do so.
On a member server I have nextcloud, onlyoffice and guacamole up and running.
My monitoring system shows me a higher load since 4 weeks or so. When I look at the processes running with top I can see a very cpu intensive task on dockerd- not permanent but repeating every? minute or so. I’ve looked for something unusual inside the containers with docker logs. There is not much access to ether of these apps and nothing unusual in the docker logs.
So I looked at the docker service and got some (actually a lot of) error messages. My internet research wasn’t helpful until now so I thought - maybe someone here can help.
root@ucs-member:~# systemctl status docker
● docker.service - Docker Application Container Engine
Loaded: loaded (/lib/systemd/system/docker.service; enabled; vendor preset: enabled)
Drop-In: /etc/systemd/system/docker.service.d
└─http-proxy.conf
Active: active (running) since Tue 2019-12-03 20:56:41 CET; 2 days ago
Docs: https://docs.docker.com
Process: 2558 ExecStartPost=/usr/sbin/service univention-firewall restart (code=exited, status=0/SUCCESS)
Main PID: 1420 (dockerd)
Tasks: 47
Memory: 2.0G
CPU: 15h 7min 32.460s
CGroup: /system.slice/docker.service
├─1420 /usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock
├─2097 /usr/bin/docker-proxy -proto tcp -host-ip 0.0.0.0 -host-port 40004 -container-ip 172.17.0.1 -container-port 8080
├─6007 /usr/bin/docker-proxy -proto tcp -host-ip 0.0.0.0 -host-port 5443 -container-ip 172.17.0.3 -container-port 443
└─6046 /usr/bin/docker-proxy -proto tcp -host-ip 0.0.0.0 -host-port 40000 -container-ip 172.17.0.4 -container-port 80
Dez 06 08:04:49 ucs-member dockerd[1420]: time="2019-12-06T08:04:49.818709884+01:00" level=error msg="stream copy error: reading from a closed fifo"
Dez 06 08:04:49 ucs-member dockerd[1420]: time="2019-12-06T08:04:49.824586768+01:00" level=error msg="stream copy error: reading from a closed fifo"
Dez 06 08:04:49 ucs-member dockerd[1420]: time="2019-12-06T08:04:49.827261662+01:00" level=error msg="stream copy error: reading from a closed fifo"
Dez 06 08:04:49 ucs-member dockerd[1420]: time="2019-12-06T08:04:49.835654150+01:00" level=error msg="stream copy error: reading from a closed fifo"
Dez 06 08:04:49 ucs-member dockerd[1420]: time="2019-12-06T08:04:49.835705916+01:00" level=error msg="stream copy error: reading from a closed fifo"
Dez 06 08:04:49 ucs-member dockerd[1420]: time="2019-12-06T08:04:49.895881376+01:00" level=error msg="Error running exec 992515ea5b7fa0d9e1f90b8b8af3a6feb2b1447aab4f8f9a483c0bcf6d354c95 in container: OCI runtime exec failed: exec failed: container_linux.go:345: starting container
Dez 06 08:04:49 ucs-member dockerd[1420]: time="2019-12-06T08:04:49.898527646+01:00" level=error msg="Error running exec 0b78cf0d95433e2f312e2feda99269fa489742b5ac5470a851b512764fb8e55e in container: OCI runtime exec failed: exec failed: container_linux.go:345: starting container
Dez 06 08:04:49 ucs-member dockerd[1420]: time="2019-12-06T08:04:49.910672228+01:00" level=error msg="Error running exec 148f39dbd4a1a4ee9a0f9bdc5e9827b4002473962c39ab62c689ad6f9efecb99 in container: OCI runtime exec failed: exec failed: container_linux.go:345: starting container
Dez 06 08:04:49 ucs-member dockerd[1420]: time="2019-12-06T08:04:49.913254359+01:00" level=error msg="Error running exec 77871280be5c840b7efad0809921926e3094c66a229987bbe17f81433c0f204f in container: OCI runtime exec failed: exec failed: container_linux.go:345: starting container
Dez 06 08:05:35 ucs-member dockerd[1420]: time="2019-12-06T08:05:35.422940296+01:00" level=warning msg="failed to retrieve runc version: unknown output format: runc version spec: 1.0.1-dev\n"