Problem: UCS consumes a lot of memory
Even though not much processes are running your UCS system consumes a lot of memory which is not useed by processes or buffers/cache.
It might look as follows:
root@ucs:~# free -h total used free shared buff/cache available Mem: 141G 60G 72G 15M 8,6G 80G Swap: 375G 0B 375G
So there are 60G of memory used but
top does not show any process using so much memory:
top - 13:07:21 up 285 days, 19:53, 2 users, load average: 1,08, 1,00, 1,00 Tasks: 282 total, 3 running, 279 sleeping, 0 stopped, 0 zombie %Cpu(s): 4,6 us, 5,9 sy, 0,0 ni, 88,8 id, 0,1 wa, 0,0 hi, 0,6 si, 0,0 st KiB Mem : 14854566+total, 76275712 free, 63281320 used, 8988636 buff/cache KiB Swap: 39389184+total, 39389184+free, 0 used. 84186600 avail Mem PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 13706 root 20 0 458032 258232 2624 S 0,0 0,2 47:34.64 rsyslogd 7562 listener 20 0 2636816 53028 27624 S 0,0 0,0 0:01.22 univention-dire 25959 root 20 0 85172 48068 5300 R 47,2 0,0 0:01.44 apt-get 19332 root 20 0 1382992 37464 29508 S 0,0 0,0 20:28.46 libvirtd 420 root 20 0 144524 36996 36400 S 1,0 0,0 665:11.79 systemd-journa
Linux kernel has some issues with releasing ressources:
The second number reported by the following command should be well below 500:
root@ucs:~ # grep '^memory' /proc/cgroups
pam_systemd is currently used by the PAM stack:
root@ucs:~ # grep pam_systemd /etc/pam.d/common-session session optional pam_systemd.so
The module is active as it isn’t commented out; it’s triggering a memory leak in the kernel.
Fixed by: updating to at least UCS 4.4 Errata 191 which deactivates the
pam_systemd module by default. You may have to run
ucr commit /etc/pam.d/common-session manually after the update. Verify that the module is commented-out with
grep pam_systemd /etc/pam.d/common-session
If the update cannot be applied at the moment, you can work around it by modifying the PAM configuration files directly:
root@ucs:~ # sed -i '/pam_systemd/s/^session/#&/' /etc/univention/templates/files/etc/pam.d/common-session.d/10univention-pam_common /etc/pam.d/common-session
Caution: this modifies a UCS template file, preventing newer versions from being installed automatically. You will have to reset that template back to its original content manually whenever you’ll be able to update to 4.4 Errata 191 or later.
Whichever route you chose, you have to reboot the server as it’s the only way to free the leaked kernel memory.