Problem: LMDB fragmentation slows down LDAP modifications

Knowledge Base Supported
, , ,
1

Problem

The LMBD key value stores used in UCS for OpenLDAP and Univention Directory Listener can get fragmented due to modification and removal of objects and attribute values, which can slow down LDAP modifications.

Investigation

UCS administrators can use the command mdb_stat -ef <database-directory> to analyze the number of Free pages in relation to the Number of pages used. The number of Free pages shows how many pages have been used but were freed again due to data changes. They could be re-used, and thus the LMDB library (used by OpenLDAP) scans this list for fitting space. If this list is excessively long, then LDAP modifications can slow down. UCS 5.2-1 ships a command line tool univention-lmdb-fragmentation which may be helpful to automate retrieval of the relevant values. The tool has some threshold values (adjustable via command line options) and returns a non-zero exit code and corresponding messages if the values are too high. This tool is used by a diagnostic UMC module to make this functionality available via web browser, too.
By default the tool checks the following three databases:

  • /var/lib/univention-ldap/ldap
  • /var/lib/univention-ldap/translog
  • /var/lib/univention-directory-listener/cache

Solution

LMDB databases can be “compacted” by using the command mdb_copy -c.

The following steps show how to use that command to defragment the main OpenLDAP database:

systemctl stop slapd
mv /var/lib/univention-ldap/ldap /var/lib/univention-ldap/ldap.backup
mkdir /var/lib/univention-ldap/ldap
mdb_copy -c \
  /var/lib/univention-ldap/ldap.backup \
  /var/lib/univention-ldap/ldap
chmod 600 /var/lib/univention-ldap/ldap/*
chown -R openldap:openldap /var/lib/univention-ldap/ldap
systemctl start slapd

The following steps show how to defragment the translog OpenLDAP database:

systemctl stop slapd
mv /var/lib/univention-ldap/translog /var/lib/univention-ldap/translog.backup
mkdir /var/lib/univention-ldap/translog
mdb_copy -c \
  /var/lib/univention-ldap/translog.backup \
  /var/lib/univention-ldap/translog
chmod 600 /var/lib/univention-ldap/ldap/translog/*
chown -R openldap:openldap /var/lib/univention-ldap/translog
systemctl start slapd

The following steps show how to defragment the univention-directory-listener cache database:

systemctl stop univention-directory-listener
mv /var/lib/univention-directory-listener/cache /var/lib/univention-directory-listener/cache.backup
install -d -m 700 -o listener -g nogroup /var/lib/univention-directory-listener/cache
mdb_copy -c \
  /var/lib/univention-directory-listener/cache.backup \
  /var/lib/univention-directory-listener/cache
chmod 600 /var/lib/univention-directory-listener/cache/*
chown -R listener:nogroup /var/lib/univention-directory-listener/cache
systemctl start univention-directory-listener
1 Like