Problem
Users can not login any more to any service or through Windows clients.
Additionally you can not join any server to your UCS domain and you will see following screen:
Investigation
Check replication status:
root@ucs01:~# univention-directory-listener-ctrl status
Listener status:
finish: univention-directory-listener: (pid 7920) 6s, normally down
Current Notifier ID on "ucs.multi.intranet"
2525
Last Notifier ID processed by local Listener:
2212
Last transaction processed:
2525 cn=app_syntax,cn=udm_syntax,cn=univention,dc=multi,dc=intranet a
Modules:
3 bind /usr/lib/univention-directory-listener/system/bind.py
3 dhcp /usr/lib/univention-directory-listener/system/dhcp.py
You notice the difference between “Current Notifier ID” and “Last Notifier ID…”. This can be caused by different reasons.
This time a check of disk usage showed the /- filesystem being out of space:
root@ucs:~# df -h
Dateisystem Größe Benutzt Verf. Verw% Eingehängt auf
udev 10M 0 10M 0% /dev
tmpfs 792M 13M 779M 2% /run
/dev/dm-0 15G 15G 0G 100% /
/dev/sda1 472M 149M 300M 34% /boot
[...]
Further investigation about the used disk space there was a huge amount of space used under the /var. Expected sizes are around 10GB but this may vary.
root@ucs:~# du -hs /var /usr/
12G /var
1,5G /usr/
By narrowing down with the “du” command the directory /var/lib/univention-connector/ad
used about 11G of disk space. Inside of this directory existed millions of very small “pickle files” used to synchronize between a UCS domain and the AD through univention-ad-connector which was installed.
But in this environment no Microsoft AD existed thus the AD-Connector was installed but not configured nor active. Therefore the files for synchronization did never get cleaned up.
Solution
Step 1
Remove most of the files. As long as the AD-connector is installed there will be new files created. For the moment this is not important. Just free some space by the following command which will take a while:
rm /var/lib/univention-connector/ad/*
Note: Second option is to increase disk space by adding additional harddisks and extending the /-filesystem.
Step 2
Use Univention Management Console (UMC) to remove the AD-connector app.
On command line you could remove the app as follows: univention-app remove adconnector
Step 3
Restart replication services:
systemctl restart univention-directory-notifier
systemctl restart slapd
systemctl restart univention-directory-listener