MDB Database Full and Stops Replication

UCS - Univention Corporate Server
, ,
#1

About a month ago, I ran into an issue outlined on step 3 of the following document: Problem: UMC Diagnostic Module Complains about Problems with UDN Replication

I changed the UCR variable ‘ldap/database/mdb/maxsize’ from 2GB to 20GB and this worked in getting replication running smoothly again. Fast forward to today and that 20GB is consumed and I had to increase the value yet again to get replication working (Still catching up and will be some time before I’m in sync again).

What should I be checking to prevent the maxsize being hit? Something seems wrong if after a month a 10x increase in database size isn’t enough and gets exhausted.

Current Version: UCS 4.4 Errata 137

Any pointers would be very much appreciated!

Thank you!

#2

Hi, yes, I agree, something seems to constantly fill up. As a first step I would recommend checking the last transaction ID. Maybe you can post the output of this command:

/usr/share/univention-directory-notifier/univention-translog stat

Do you have univention-s4-connector running in your domain? If so, I would next check for synchronization rejects: univention-s4connector-list-rejected

#3

Hey, thank you so much for the reply!

The output of those commands I get are the following:

root@ucs-master:~# /usr/share/univention-directory-notifier/univention-translog stat
Index.file: /var/lib/univention-ldap/notify/transaction.index
Index.size: 217232099
Index.count: 24136898
Index.okay: no
Translog.file: /var/lib/univention-ldap/notify/transaction
Translog.size: 1801286156
Translog.first: 4338615
Translog.last: 24136898
Translog.count: 19798284
Ldap.last: 24136898
Ldap.okay: yes



root@ucs-master:~# univention-s4connector-list-rejected

UCS rejected


S4 rejected


There may be no rejected DNs if the connector is in progress, to be
sure stop the connector before running this script.


        last synced USN: 259854

root@ucs-master:~# /usr/lib/nagios/plugins/check_univention_replication
OK: replication complete (nid=24137240 lid=24137240)
#4

I’m guessing it has something to do with the “Index.okay: no” output. How do I check the index and repair it if needed?

#5

Hi, a quick look at the code shows that the “Index.okay: no” message is printed when Index.count and Translog.count don’t match. That probably happened because the DB reached its limit. Before trying to fix that, I’d suggest to first check if there is some ongoing replication issue that keeps filling your transaction list. Please check the /var/log/univention/connector-s4.log for tracebacks (and connector.log if you have an AD-Connector installed too). Actually it may be even simpler to check the output of the following command for repetitive add (“a”) and delete (“d”) operations below cn=temporary:

grep "( PROCESS ) : updating" /var/log/univention/listener.log  | tail -50
#6

Hey,

In the connector-s4.log, I have the following tracebacks I can see:

11.08.2019 10:05:31.039 LDAP        (PROCESS): sync to ucs:   [windowscomputer] [    modify] cn=kh63-john-doe,cn=computers,dc=int,dc=exampledomain,dc=net
11.08.2019 10:05:31.230 LDAP        (ERROR  ): get_ucs_object: could not identify UDM object type: cn=kh63-john-doe,cn=computers,dc=int,dc=exampledomain,dc=net
11.08.2019 10:05:31.231 LDAP        (PROCESS): get_ucs_object: using default: users/user
11.08.2019 10:05:31.235 LDAP        (WARNING): get_ucs_object: failure was:

11.08.2019 10:05:31.325 LDAP        (WARNING): Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/univention/s4connector/__init__.py", line 999, in get_ucs_object
    ucs_object = univention.admin.objects.get(module, co=None, lo=self.lo, position='', dn=searchdn)
  File "/usr/lib/pymodules/python2.7/univention/admin/objects.py", line 113, in get
    return module.object(co, lo, position, dn, superordinate=superordinate, attributes=attributes)
  File "/usr/lib/pymodules/python2.7/univention/admin/handlers/users/user.py", line 1243, in __init__
    univention.admin.handlers.simpleLdap.__init__(self, co, lo, position, dn, superordinate, attributes=attributes)
  File "/usr/lib/pymodules/python2.7/univention/admin/handlers/__init__.py", line 232, in __init__
    raise univention.admin.uexceptions.wrongObjectType('%s is not recognized as %s.' % (self.dn, self.module))
wrongObjectType: cn=kh63-john-doe,cn=computers,dc=int,dc=exampledomain,dc=net is not recognized as users/user.


11.08.2019 23:48:15.773 LDAP        (PROCESS): sync to ucs:   [windowscomputer] [    modify] cn=desktop-24hp9tm,cn=computers,dc=int,dc=exampledomain,dc=net
11.08.2019 23:48:15.888 LDAP        (ERROR  ): get_ucs_object: could not identify UDM object type: cn=desktop-24hp9tm,cn=computers,dc=int,dc=exampledomain,dc=net
11.08.2019 23:48:15.888 LDAP        (PROCESS): get_ucs_object: using default: users/user
11.08.2019 23:48:15.896 LDAP        (WARNING): get_ucs_object: failure was:

11.08.2019 23:48:15.897 LDAP        (WARNING): Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/univention/s4connector/__init__.py", line 999, in get_ucs_object
    ucs_object = univention.admin.objects.get(module, co=None, lo=self.lo, position='', dn=searchdn)
  File "/usr/lib/pymodules/python2.7/univention/admin/objects.py", line 113, in get
    return module.object(co, lo, position, dn, superordinate=superordinate, attributes=attributes)
  File "/usr/lib/pymodules/python2.7/univention/admin/handlers/users/user.py", line 1243, in __init__
    univention.admin.handlers.simpleLdap.__init__(self, co, lo, position, dn, superordinate, attributes=attributes)
  File "/usr/lib/pymodules/python2.7/univention/admin/handlers/__init__.py", line 232, in __init__
    raise univention.admin.uexceptions.wrongObjectType('%s is not recognized as %s.' % (self.dn, self.module))
wrongObjectType: cn=desktop-24hp9tm,cn=computers,dc=int,dc=exampledomain,dc=net is not recognized as users/user.

11.08.2019 23:48:15.898 LDAP        (ERROR  ): password_sync_s4_to_ucs: couldn't get user-object from UCS


12.08.2019 06:30:34.070 LDAP        (PROCESS): sync to ucs:   [windowscomputer] [    modify] cn=ws-20,cn=computers,dc=int,dc=exampledomain,dc=net
12.08.2019 06:30:34.192 LDAP        (ERROR  ): get_ucs_object: could not identify UDM object type: cn=ws-20,cn=computers,dc=int,dc=exampledomain,dc=net
12.08.2019 06:30:34.192 LDAP        (PROCESS): get_ucs_object: using default: users/user
12.08.2019 06:30:34.195 LDAP        (WARNING): get_ucs_object: failure was:

12.08.2019 06:30:34.195 LDAP        (WARNING): Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/univention/s4connector/__init__.py", line 999, in get_ucs_object
    ucs_object = univention.admin.objects.get(module, co=None, lo=self.lo, position='', dn=searchdn)
  File "/usr/lib/pymodules/python2.7/univention/admin/objects.py", line 113, in get
    return module.object(co, lo, position, dn, superordinate=superordinate, attributes=attributes)
  File "/usr/lib/pymodules/python2.7/univention/admin/handlers/users/user.py", line 1243, in __init__
    univention.admin.handlers.simpleLdap.__init__(self, co, lo, position, dn, superordinate, attributes=attributes)
  File "/usr/lib/pymodules/python2.7/univention/admin/handlers/__init__.py", line 232, in __init__
    raise univention.admin.uexceptions.wrongObjectType('%s is not recognized as %s.' % (self.dn, self.module))
wrongObjectType: cn=ws-20,cn=computers,dc=int,dc=exampledomain,dc=net is not recognized as users/user.

12.08.2019 06:30:34.196 LDAP        (ERROR  ): password_sync_s4_to_ucs: couldn't get user-object from UCS


12.08.2019 06:55:40.460 LDAP        (PROCESS): sync to ucs:   [windowscomputer] [    modify] cn=vh-31,cn=computers,dc=int,dc=exampledomain,dc=net
12.08.2019 06:55:40.512 LDAP        (ERROR  ): get_ucs_object: could not identify UDM object type: cn=vh-31,cn=computers,dc=int,dc=exampledomain,dc=net
12.08.2019 06:55:40.513 LDAP        (PROCESS): get_ucs_object: using default: users/user
12.08.2019 06:55:40.515 LDAP        (WARNING): get_ucs_object: failure was:

12.08.2019 06:55:40.516 LDAP        (WARNING): Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/univention/s4connector/__init__.py", line 999, in get_ucs_object
    ucs_object = univention.admin.objects.get(module, co=None, lo=self.lo, position='', dn=searchdn)
  File "/usr/lib/pymodules/python2.7/univention/admin/objects.py", line 113, in get
    return module.object(co, lo, position, dn, superordinate=superordinate, attributes=attributes)
  File "/usr/lib/pymodules/python2.7/univention/admin/handlers/users/user.py", line 1243, in __init__
    univention.admin.handlers.simpleLdap.__init__(self, co, lo, position, dn, superordinate, attributes=attributes)
  File "/usr/lib/pymodules/python2.7/univention/admin/handlers/__init__.py", line 232, in __init__
    raise univention.admin.uexceptions.wrongObjectType('%s is not recognized as %s.' % (self.dn, self.module))
wrongObjectType: cn=vh-31,cn=computers,dc=int,dc=exampledomain,dc=net is not recognized as users/user.

12.08.2019 06:55:40.516 LDAP        (ERROR  ): password_sync_s4_to_ucs: couldn't get user-object from UCS


12.08.2019 08:32:50.707 LDAP        (PROCESS): sync to ucs:   [windowscomputer] [    modify] cn=ws-3,cn=computers,dc=int,dc=exampledomain,dc=net
12.08.2019 08:32:50.787 LDAP        (ERROR  ): get_ucs_object: could not identify UDM object type: cn=ws-3,cn=computers,dc=int,dc=exampledomain,dc=net
12.08.2019 08:32:50.787 LDAP        (PROCESS): get_ucs_object: using default: users/user
12.08.2019 08:32:50.790 LDAP        (WARNING): get_ucs_object: failure was:

12.08.2019 08:32:50.791 LDAP        (WARNING): Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/univention/s4connector/__init__.py", line 999, in get_ucs_object
    ucs_object = univention.admin.objects.get(module, co=None, lo=self.lo, position='', dn=searchdn)
  File "/usr/lib/pymodules/python2.7/univention/admin/objects.py", line 113, in get
    return module.object(co, lo, position, dn, superordinate=superordinate, attributes=attributes)
  File "/usr/lib/pymodules/python2.7/univention/admin/handlers/users/user.py", line 1243, in __init__
    univention.admin.handlers.simpleLdap.__init__(self, co, lo, position, dn, superordinate, attributes=attributes)
  File "/usr/lib/pymodules/python2.7/univention/admin/handlers/__init__.py", line 232, in __init__
    raise univention.admin.uexceptions.wrongObjectType('%s is not recognized as %s.' % (self.dn, self.module))
wrongObjectType: cn=ws-3,cn=computers,dc=int,dc=exampledomain,dc=net is not recognized as users/user.

12.08.2019 08:32:50.791 LDAP        (ERROR  ): password_sync_s4_to_ucs: couldn't get user-object from UCS


12.08.2019 08:43:48.763 LDAP        (PROCESS): sync to ucs:   [windowscomputer] [    modify] cn=pn41sales,cn=computers,dc=int,dc=exampledomain,dc=net
12.08.2019 08:43:48.879 LDAP        (ERROR  ): get_ucs_object: could not identify UDM object type: cn=pn41sales,cn=computers,dc=int,dc=exampledomain,dc=net
12.08.2019 08:43:48.879 LDAP        (PROCESS): get_ucs_object: using default: users/user
12.08.2019 08:43:48.885 LDAP        (WARNING): get_ucs_object: failure was:

12.08.2019 08:43:48.886 LDAP        (WARNING): Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/univention/s4connector/__init__.py", line 999, in get_ucs_object
    ucs_object = univention.admin.objects.get(module, co=None, lo=self.lo, position='', dn=searchdn)
  File "/usr/lib/pymodules/python2.7/univention/admin/objects.py", line 113, in get
    return module.object(co, lo, position, dn, superordinate=superordinate, attributes=attributes)
  File "/usr/lib/pymodules/python2.7/univention/admin/handlers/users/user.py", line 1243, in __init__
    univention.admin.handlers.simpleLdap.__init__(self, co, lo, position, dn, superordinate, attributes=attributes)
  File "/usr/lib/pymodules/python2.7/univention/admin/handlers/__init__.py", line 232, in __init__
    raise univention.admin.uexceptions.wrongObjectType('%s is not recognized as %s.' % (self.dn, self.module))
wrongObjectType: cn=pn41sales,cn=computers,dc=int,dc=exampledomain,dc=net is not recognized as users/user.

12.08.2019 08:43:48.886 LDAP        (ERROR  ): password_sync_s4_to_ucs: couldn't get user-object from UCS

And these are repetitive commands in the /var/log/univention/listener.log file

12.08.19 11:38:44.040  LISTENER    ( PROCESS ) : updating 'cn=SteveB,cn=uid,cn=temporary,cn=univention,dc=int,dc=exampledomain,dc=net' command a
12.08.19 11:38:44.100  LISTENER    ( PROCESS ) : updating 'cn=2745,cn=uidNumber,cn=temporary,cn=univention,dc=int,dc=exampledomain,dc=net' command a
12.08.19 11:38:44.165  LISTENER    ( PROCESS ) : updating 'cn=2745,cn=gidNumber,cn=temporary,cn=univention,dc=int,dc=exampledomain,dc=net' command a
12.08.19 11:38:44.239  LISTENER    ( PROCESS ) : updating 'cn=2745,cn=gidNumber,cn=temporary,cn=univention,dc=int,dc=exampledomain,dc=net' command d
12.08.19 11:38:44.372  LISTENER    ( PROCESS ) : updating 'cn=2745,cn=uidNumber,cn=temporary,cn=univention,dc=int,dc=exampledomain,dc=net' command d
12.08.19 11:38:44.457  LISTENER    ( PROCESS ) : updating 'cn=SteveB,cn=uid,cn=temporary,cn=univention,dc=int,dc=exampledomain,dc=net' command d
12.08.19 11:38:44.600  LISTENER    ( PROCESS ) : updating 'cn=EricG,cn=uid,cn=temporary,cn=univention,dc=int,dc=exampledomain,dc=net' command a
12.08.19 11:38:44.710  LISTENER    ( PROCESS ) : updating 'cn=2745,cn=uidNumber,cn=temporary,cn=univention,dc=int,dc=exampledomain,dc=net' command a
12.08.19 11:38:44.792  LISTENER    ( PROCESS ) : updating 'cn=2745,cn=gidNumber,cn=temporary,cn=univention,dc=int,dc=exampledomain,dc=net' command a
12.08.19 11:38:44.803  LISTENER    ( PROCESS ) : updating 'cn=2745,cn=gidNumber,cn=temporary,cn=univention,dc=int,dc=exampledomain,dc=net' command d
12.08.19 11:38:44.926  LISTENER    ( PROCESS ) : updating 'cn=2745,cn=uidNumber,cn=temporary,cn=univention,dc=int,dc=exampledomain,dc=net' command d
12.08.19 11:38:45.000  LISTENER    ( PROCESS ) : updating 'cn=EricG,cn=uid,cn=temporary,cn=univention,dc=int,dc=exampledomain,dc=net' command d
12.08.19 11:38:45.152  LISTENER    ( PROCESS ) : updating 'cn=JonasB,cn=uid,cn=temporary,cn=univention,dc=int,dc=exampledomain,dc=net' command a
12.08.19 11:38:45.275  LISTENER    ( PROCESS ) : updating 'cn=2745,cn=uidNumber,cn=temporary,cn=univention,dc=int,dc=exampledomain,dc=net' command a
12.08.19 11:38:45.347  LISTENER    ( PROCESS ) : updating 'cn=2745,cn=gidNumber,cn=temporary,cn=univention,dc=int,dc=exampledomain,dc=net' command a
12.08.19 11:38:45.364  LISTENER    ( PROCESS ) : updating 'cn=2745,cn=gidNumber,cn=temporary,cn=univention,dc=int,dc=exampledomain,dc=net' command d
12.08.19 11:38:45.486  LISTENER    ( PROCESS ) : updating 'cn=2745,cn=uidNumber,cn=temporary,cn=univention,dc=int,dc=exampledomain,dc=net' command d
12.08.19 11:38:45.561  LISTENER    ( PROCESS ) : updating 'cn=JonasB,cn=uid,cn=temporary,cn=univention,dc=int,dc=exampledomain,dc=net' command d
12.08.19 11:38:45.682  LISTENER    ( PROCESS ) : updating 'cn=JoshB,cn=uid,cn=temporary,cn=univention,dc=int,dc=exampledomain,dc=net' command a
12.08.19 11:38:45.749  LISTENER    ( PROCESS ) : updating 'cn=2745,cn=uidNumber,cn=temporary,cn=univention,dc=int,dc=exampledomain,dc=net' command a
12.08.19 11:38:45.824  LISTENER    ( PROCESS ) : updating 'cn=2745,cn=gidNumber,cn=temporary,cn=univention,dc=int,dc=exampledomain,dc=net' command a
12.08.19 11:38:45.898  LISTENER    ( PROCESS ) : updating 'cn=2745,cn=gidNumber,cn=temporary,cn=univention,dc=int,dc=exampledomain,dc=net' command d
12.08.19 11:38:46.020  LISTENER    ( PROCESS ) : updating 'cn=2745,cn=uidNumber,cn=temporary,cn=univention,dc=int,dc=exampledomain,dc=net' command d
12.08.19 11:38:46.095  LISTENER    ( PROCESS ) : updating 'cn=JoshB,cn=uid,cn=temporary,cn=univention,dc=int,dc=exampledomain,dc=net' command d
12.08.19 11:38:46.231  LISTENER    ( PROCESS ) : updating 'cn=RodneyS,cn=uid,cn=temporary,cn=univention,dc=int,dc=exampledomain,dc=net' command a
12.08.19 11:38:46.299  LISTENER    ( PROCESS ) : updating 'cn=2745,cn=uidNumber,cn=temporary,cn=univention,dc=int,dc=exampledomain,dc=net' command a
12.08.19 11:38:46.397  LISTENER    ( PROCESS ) : updating 'cn=2745,cn=gidNumber,cn=temporary,cn=univention,dc=int,dc=exampledomain,dc=net' command a
12.08.19 11:38:46.509  LISTENER    ( PROCESS ) : updating 'cn=2745,cn=gidNumber,cn=temporary,cn=univention,dc=int,dc=exampledomain,dc=net' command d
12.08.19 11:38:46.645  LISTENER    ( PROCESS ) : updating 'cn=2745,cn=uidNumber,cn=temporary,cn=univention,dc=int,dc=exampledomain,dc=net' command d
12.08.19 11:38:46.719  LISTENER    ( PROCESS ) : updating 'cn=RodneyS,cn=uid,cn=temporary,cn=univention,dc=int,dc=exampledomain,dc=net' command d
12.08.19 11:38:46.883  LISTENER    ( PROCESS ) : updating 'cn=RhysT,cn=uid,cn=temporary,cn=univention,dc=int,dc=exampledomain,dc=net' command a
12.08.19 11:38:47.037  LISTENER    ( PROCESS ) : updating 'cn=2745,cn=uidNumber,cn=temporary,cn=univention,dc=int,dc=exampledomain,dc=net' command a
12.08.19 11:38:47.154  LISTENER    ( PROCESS ) : updating 'cn=2745,cn=gidNumber,cn=temporary,cn=univention,dc=int,dc=exampledomain,dc=net' command a
12.08.19 11:38:47.212  LISTENER    ( PROCESS ) : updating 'cn=2745,cn=gidNumber,cn=temporary,cn=univention,dc=int,dc=exampledomain,dc=net' command d
12.08.19 11:38:47.310  LISTENER    ( PROCESS ) : updating 'cn=2745,cn=uidNumber,cn=temporary,cn=univention,dc=int,dc=exampledomain,dc=net' command d
12.08.19 11:38:47.342  LISTENER    ( PROCESS ) : updating 'cn=RhysT,cn=uid,cn=temporary,cn=univention,dc=int,dc=exampledomain,dc=net' command d
12.08.19 11:38:47.470  LISTENER    ( PROCESS ) : updating 'cn=AshttonE,cn=uid,cn=temporary,cn=univention,dc=int,dc=exampledomain,dc=net' command a
12.08.19 11:38:47.646  LISTENER    ( PROCESS ) : updating 'cn=2745,cn=uidNumber,cn=temporary,cn=univention,dc=int,dc=exampledomain,dc=net' command a
12.08.19 11:38:47.671  LISTENER    ( PROCESS ) : updating 'cn=2745,cn=gidNumber,cn=temporary,cn=univention,dc=int,dc=exampledomain,dc=net' command a
12.08.19 11:38:47.689  LISTENER    ( PROCESS ) : updating 'cn=2745,cn=gidNumber,cn=temporary,cn=univention,dc=int,dc=exampledomain,dc=net' command d
12.08.19 11:38:47.790  LISTENER    ( PROCESS ) : updating 'cn=2745,cn=uidNumber,cn=temporary,cn=univention,dc=int,dc=exampledomain,dc=net' command d
12.08.19 11:38:47.874  LISTENER    ( PROCESS ) : updating 'cn=AshttonE,cn=uid,cn=temporary,cn=univention,dc=int,dc=exampledomain,dc=net' command d
12.08.19 11:38:48.007  LISTENER    ( PROCESS ) : updating 'cn=JamesT,cn=uid,cn=temporary,cn=univention,dc=int,dc=exampledomain,dc=net' command a
12.08.19 11:38:48.074  LISTENER    ( PROCESS ) : updating 'cn=2745,cn=uidNumber,cn=temporary,cn=univention,dc=int,dc=exampledomain,dc=net' command a
12.08.19 11:38:48.149  LISTENER    ( PROCESS ) : updating 'cn=2745,cn=gidNumber,cn=temporary,cn=univention,dc=int,dc=exampledomain,dc=net' command a
12.08.19 11:38:48.231  LISTENER    ( PROCESS ) : updating 'cn=2745,cn=gidNumber,cn=temporary,cn=univention,dc=int,dc=exampledomain,dc=net' command d
12.08.19 11:38:48.364  LISTENER    ( PROCESS ) : updating 'cn=2745,cn=uidNumber,cn=temporary,cn=univention,dc=int,dc=exampledomain,dc=net' command d
12.08.19 11:38:48.432  LISTENER    ( PROCESS ) : updating 'cn=JamesT,cn=uid,cn=temporary,cn=univention,dc=int,dc=exampledomain,dc=net' command d
12.08.19 11:38:48.616  LISTENER    ( PROCESS ) : updating 'cn=MikeB,cn=uid,cn=temporary,cn=univention,dc=int,dc=exampledomain,dc=net' command a
12.08.19 11:38:48.674  LISTENER    ( PROCESS ) : updating 'cn=2745,cn=uidNumber,cn=temporary,cn=univention,dc=int,dc=exampledomain,dc=net' command a
12.08.19 11:38:48.745  LISTENER    ( PROCESS ) : updating 'cn=2745,cn=gidNumber,cn=temporary,cn=univention,dc=int,dc=exampledomain,dc=net' command a
12.08.19 11:38:48.819  LISTENER    ( PROCESS ) : updating 'cn=2745,cn=gidNumber,cn=temporary,cn=univention,dc=int,dc=exampledomain,dc=net' command d
12.08.19 11:38:48.948  LISTENER    ( PROCESS ) : updating 'cn=2745,cn=uidNumber,cn=temporary,cn=univention,dc=int,dc=exampledomain,dc=net' command d
12.08.19 11:38:49.030  LISTENER    ( PROCESS ) : updating 'cn=MikeB,cn=uid,cn=temporary,cn=univention,dc=int,dc=exampledomain,dc=net' command d
12.08.19 11:38:49.154  LISTENER    ( PROCESS ) : updating 'cn=NathanP,cn=uid,cn=temporary,cn=univention,dc=int,dc=exampledomain,dc=net' command a
12.08.19 11:38:49.217  LISTENER    ( PROCESS ) : updating 'cn=2745,cn=uidNumber,cn=temporary,cn=univention,dc=int,dc=exampledomain,dc=net' command a
12.08.19 11:38:49.291  LISTENER    ( PROCESS ) : updating 'cn=2745,cn=gidNumber,cn=temporary,cn=univention,dc=int,dc=exampledomain,dc=net' command a
12.08.19 11:38:49.362  LISTENER    ( PROCESS ) : updating 'cn=2745,cn=gidNumber,cn=temporary,cn=univention,dc=int,dc=exampledomain,dc=net' command d
#7

We also had AD-Connector running (Didn’t mean to as this was just for an initial domain sync/takeover from AD and then the AD controller was no longer used.) I just stopped the AD-Connector in the UCS web management and the repeating ‘a’ and ‘d’ operations quieted down after that.

Does this stay stopped even after reboots?

#8

Ok, great that we nailed that source of changes down. Regarding your question: No, if a service is not explicitly disabled then it gets startet during reboot. You chould run “ucr set connector/ad/autostart=no” on the host where it is installed, that would disable it from starting. Or you could uninstall the package “univention-ad-connector” if you don’t need it any longer.

Now, regarding the tracebacksin the S4-Connector, I guess this should be fixed by UCS 4.4-1 Errata Update 155 (http://errata.software-univention.de/ucs/4.4/155.html), but it could also be a separate problem. Since your are on Errata137 that may be worth a try to update.

Finally, regarding fixing the state of the translog: Do you only run a single UCS DC Master or do you run systems of role DC Backup and/or DC Slave too? It’s possible to fix and reduce the size of the translog, but since it plays a central role in the LDAP replication, care should be taken to not cut of the replicating systems (well, they can be rejoined to fix that in case it happens).

#9

Thank you, I will uninstall the connector to prevent it from starting up again.

In regards to the setup, right now it is a Single UCS Master DC with 4 Slave DCs in different locations that replicate from the master. Rejoining the slaves one of the evenings here shouldn’t be an issue if I have to stop the sync to fix the translog.

#10

Hi, more feedback from colleagues, The “Index.okay: no” is just a silly bug in the check which we will fix in the product and you can ignore it in the mean time. The important thing in the output of univention-translog stat is the ‘Ldap.okay: yes’ (i.e. LDAP.last = Translog.last).

So, now you could go on and check the replication state of all of your DCs, e.g. by running
univention-directory-listener-dump -i
(or cat /var/lib/univention-directory-listener/notifier_id)

For example let’s assume the case, that one of the Slave DCs is still 900 IDs behind the value of Ldap.last on the master and all others are already closer to the state of the master. In that case you could clean up all those transaction IDs from the cn=translog, that everybody already has replicated, and for example only keep the last 1000. You can do that by specifying this as a negative number to the univention-translog prune command:

/usr/share/univention-directory-notifier/univention-translog prune  -1000

Following this general recipe you should again have more space available in your cn=translog MDB database on your UCS DC Master.

#11

Running the prune I ran into this error now and replication is broken. Looking to repair now.

2019-08-14 10:02:27,683:INFO:Deleted reqSession=399997,cn=translog
2019-08-14 10:02:27,684:INFO:Deleted reqSession=399998,cn=translog
2019-08-14 10:02:27,685:INFO:Deleted reqSession=399999,cn=translog
2019-08-14 10:02:27,686:INFO:Deleted reqSession=400000,cn=translog
Traceback (most recent call last):
  File "/usr/share/univention-directory-notifier/univention-translog", line 1406, in <module>
    exit(main())
  File "/usr/share/univention-directory-notifier/univention-translog", line 420, in main
    return opt.func(opt) or 0
  File "/usr/share/univention-directory-notifier/univention-translog", line 1182, in prune
    prune_ldap(opt)
  File "/usr/share/univention-directory-notifier/univention-translog", line 1237, in prune_ldap
    rtype, rdata, rmsgid, serverctrls = ld.result3(response)
  File "/usr/lib/python2.7/dist-packages/ldap/ldapobject.py", line 514, in result3
    resp_ctrl_classes=resp_ctrl_classes
  File "/usr/lib/python2.7/dist-packages/ldap/ldapobject.py", line 521, in result4
    ldap_result = self._ldap_call(self._l.result4,msgid,all,timeout,add_ctrls,add_intermediates,add_extop)
  File "/usr/lib/python2.7/dist-packages/ldap/ldapobject.py", line 106, in _ldap_call
    result = func(*args,**kwargs)
ldap.SIZELIMIT_EXCEEDED: {'desc': 'Size limit exceeded'}
root@ucs-master:~# /usr/lib/nagios/plugins/check_univention_replication
CRITICAL: no change of listener transaction id for last 0 checks (nid=Error: [Errno 111] Connection refused lid=25690797)
root@ucs-master:~# /usr/lib/nagios/plugins/check_univention_replication
CRITICAL: no change of listener transaction id for last 0 checks (nid=Error: [Errno 111] Connection refused lid=25690797)
root@ucs-master:~#
#12

Restarted listener and notifier services and it seems okay now.

Thank you!

#13

Checking the MDB Database size still returns the following (Right now USR variable ‘ldap/database/mdb/maxsize’ is set to 30GB for the maxsize):

root@ucs-master:~# used_pages=$(mdb_stat -e /var/lib/univention-ldap/translog | sed -n 's/^ *Number of pages used: //p')
root@ucs-master:~# max_pages=$(mdb_stat -e /var/lib/univention-ldap/translog | sed -n 's/^ *Max pages: //p')
root@ucs-master:~# python -c "print('%.1f used' % (float($used_pages) / $max_pages * 100))"
82.4 used

Will this get cleaned up by a process at some point or is there a way to shrink the size of this again too? It’s not ballooning up like it was before so there’s no broken sync imminent at the moment but would be nice to clean that up if there’s something I have to do for that.

You’ve been a great help, Thanks again!

#14

Yes, the mdb_copy tool can be used with the -c option to create a compact copy. The following steps may be useful to do this:

/etc/init.d/slapd stop
mv /var/lib/univention-ldap/translog /var/lib/univention-ldap/translog.bak
mkdir /var/lib/univention-ldap/translog
mdb_copy -c /var/lib/univention-ldap/translog.bak /var/lib/univention-ldap/translog
chown -R openldap.openldap /var/lib/univention-ldap/translog
/etc/init.d/slapd start

Please note that I have no experience in reducing the ldap/database/mdb/maxsize UCR variable, and from what I remember reading about MDB, I think the author doesn’t support shrinking it. I think it’s ok the way you have it. Please also note that this UCR variable simultaneously controls the size of the main OpenLDAP backend database located in /var/lib/univention-ldap/ldap. If you would be so brave to experiment (which I explicitly don’t recommend), you would need take care about the health of that one too, obviously.

Mastodon