Problem: Users Can Not Login Any More

listener
notifier
problem
ucs4

#1

Problem

Users can not login any more to any service or through Windows clients.
Additionally you can not join any server to your UCS domain and you will see following screen:
domain_join_error(1)

Investigation

Check replication status:

root@ucs01:~# univention-directory-listener-ctrl status
Listener status:
 finish: univention-directory-listener: (pid 7920) 6s, normally down

Current Notifier ID on "ucs.multi.intranet"
 2525

Last Notifier ID processed by local Listener:
 2212

Last transaction processed:
 2525 cn=app_syntax,cn=udm_syntax,cn=univention,dc=multi,dc=intranet a

Modules:
3       bind    /usr/lib/univention-directory-listener/system/bind.py
3       dhcp    /usr/lib/univention-directory-listener/system/dhcp.py

You notice the difference between “Current Notifier ID” and “Last Notifier ID…”. This can be caused by different reasons.
This time a check of disk usage showed the /- filesystem being out of space:

root@ucs:~# df -h
Dateisystem    Größe Benutzt Verf. Verw% Eingehängt auf
udev             10M       0   10M    0% /dev
tmpfs           792M     13M  779M    2% /run
/dev/dm-0        15G     15G   0G   100% /
/dev/sda1       472M    149M  300M   34% /boot
[...]

Further investigation about the used disk space there was a huge amount of space used under the /var. Expected sizes are around 10GB but this may vary.

root@ucs:~# du -hs /var /usr/
12G	/var
1,5G	/usr/

By narrowing down with the “du” command the directory /var/lib/univention-connector/ad used about 11G of disk space. Inside of this directory existed millions of very small “pickle files” used to synchronize between a UCS domain and the AD through univention-ad-connector which was installed.
But in this environment no Microsoft AD existed thus the AD-Connector was installed but not configured nor active. Therefore the files for synchronization did never get cleaned up.

Solution

Step 1

Remove most of the files. As long as the AD-connector is installed there will be new files created. For the moment this is not important. Just free some space by the following command which will take a while:
rm /var/lib/univention-connector/ad/*
Note: Second option is to increase disk space by adding additional harddisks and extending the /-filesystem.

Step 2

Use Univention Management Console (UMC) to remove the AD-connector app.
On command line you could remove the app as follows: univention-app remove adconnector

Step 3

Restart replication services:

systemctl restart univention-directory-notifier
systemctl restart slapd
systemctl restart univention-directory-listener