Problem: UCS@school CSV Import Performance Issues Related to LDAP Index Configuration

Knowledge Base Community
, , , ,
1

Problem

A UCS@school environment experienced unusually long CSV user import runtimes. Imports were started shortly before midnight after CSV preparation and ordinarily took up to two hours. During the affected period, individual imports reportedly took 5–15 hours. The administrator observed multi-minute pauses when new school classes were created and suspected that creating class shares on remote UCS Replica servers might be responsible.

Comparison of annual import runs

Metric Previous year Current year
School holiday period import started 2025-07-14_22:02:21 2026-07-19_23:40:32
Import completed 2025-07-15_05:28:35 2026-07-21_15:32:05
Modified 103,925 103,292
Added 4,665 4,847
Removed 17,802 16,090
Elapsed time 7 h 26 min 14 s 39 h 51 min 33 s

The reported 5–15-hour daily import times and the nearly 40-hour comparison run refer to different observations. They should not be treated as the same measurement.

Root cause

Investigation of class and class-share creation identified an LDAP search using univentionObjectIdentifier. This attribute was missing from the configured equality index. An unindexed subtree search across a large LDAP directory can be expensive, particularly when repeated during object creation. The logs and UCR output support the missing-index diagnosis; the traces alone do not establish that every minute of the class-share delay was spent in this one LDAP search.

The LDAP trace showed:

04.09.26 15:29:00.100  LDAP        ( ALL     ) : search | filter=univentionObjectIdentifier=42f78224-5964-4b6d-a011-c77748f4a1d0 base=dc=example,dc=org scope=sub attributes=['dn'] unique=False required=False timeout=-1 sizelimit=0

A configuration check returned zero:

root@primary:~/univention-support # ucr dump | grep ^ldap/index/* | grep -c univentionObjectIdentifier
0

An environment where the attribute is present in the configured index lists returned:

root@ucs5primary:~# ucr dump | grep ^ldap/index/* | grep -c univentionObjectIdentifier
1

This check searches all matching UCR index settings, not specifically the equality index; use ucr get ldap/index/eq to verify the required equality-index configuration. The UCR value alone does not prove that the physical database index has been built.

Solution

:stop_sign: Important: Index rebuilding requires a maintenance window and an LDAP service interruption. Depending on directory size, it may take several hours. Back up the LDAP database and current configuration according to your operational recovery procedure before making changes. Run the commands on the LDAP server that hosts the affected database and verify replication and dependent services afterward.

1. Inspect the equality-index configuration

ucr get ldap/index/eq

Expected default output in the documented environment:

aRecord,automountInformation,cNAMERecord,cn,description,dhcpHWAddress,displayName,entryUUID,gidNumber,givenName,homeDirectory,krb5PrincipalName,krb5ValidStart,macAddress,mail,mailAlternativeAddress,mailPrimaryAddress,memberUid,objectClass,ou,pTRRecord,relativeDomainName,sambaAcctFlags,sambaDomainName,sambaGroupType,sambaPrimaryGroupSID,sambaSID,sambaSIDList,secretary,shadowExpire,sn,ucsschoolLegalGuardian,ucsschoolLegalWard,ucsschoolRecordUID,ucsschoolRole,ucsschoolSchool,ucsschoolSourceUID,uid,uidNumber,uniqueMember,univentionAppID,univentionCanonicalRecipientRewriteEnabled,univentionDataType,univentionInventoryNumber,univentionLicenseModule,univentionLicenseObject,univentionMailHomeServer,univentionNagiosHostname,univentionObjectFlag,univentionObjectIdentifier,univentionObjectType,univentionPolicyReference,univentionServerRole,univentionService,univentionShareGid,univentionShareSambaName,univentionShareWriteable,univentionUDMOptionModule,univentionUDMPropertyCLIName,univentionUDMPropertyCopyable,univentionUDMPropertyDefault,univentionUDMPropertyDeleteObjectClass,univentionUDMPropertyDoNotSearch,univentionUDMPropertyHook,univentionUDMPropertyLayoutOverwritePosition,univentionUDMPropertyLayoutOverwriteTab,univentionUDMPropertyLayoutPosition,univentionUDMPropertyLayoutTabAdvanced,univentionUDMPropertyLayoutTabName,univentionUDMPropertyLdapMapping,univentionUDMPropertyLongDescription,univentionUDMPropertyModule,univentionUDMPropertyMultivalue,univentionUDMPropertyObjectClass,univentionUDMPropertyOptions,univentionUDMPropertyShortDescription,univentionUDMPropertySyntax,univentionUDMPropertyTranslationLongDescription,univentionUDMPropertyTranslationShortDescription,univentionUDMPropertyTranslationTabName,univentionUDMPropertyValueMayChange,univentionUDMPropertyValueRequired,univentionUDMPropertyVersion,zoneName

Check whether univentionObjectIdentifier is present. Preserve any site-specific attributes in the existing value rather than blindly replacing the entire list with the example default.

2. Add the missing attribute

If the attribute is missing, the following command sets the complete example equality-index list. Use it only if this list is appropriate for your installation; otherwise append the missing attribute to the existing value.

ucr set ldap/index/eq="aRecord,automountInformation,cNAMERecord,cn,description,dhcpHWAddress,displayName,entryUUID,gidNumber,givenName,homeDirectory,krb5PrincipalName,krb5ValidStart,macAddress,mail,mailAlternativeAddress,mailPrimaryAddress,memberUid,objectClass,ou,pTRRecord,relativeDomainName,sambaAcctFlags,sambaDomainName,sambaGroupType,sambaPrimaryGroupSID,sambaSID,sambaSIDList,secretary,shadowExpire,sn,ucsschoolLegalGuardian,ucsschoolLegalWard,ucsschoolRecordUID,ucsschoolRole,ucsschoolSchool,ucsschoolSourceUID,uid,uidNumber,uniqueMember,univentionAppID,univentionCanonicalRecipientRewriteEnabled,univentionDataType,univentionInventoryNumber,univentionLicenseModule,univentionLicenseObject,univentionMailHomeServer,univentionNagiosHostname,univentionObjectFlag,univentionObjectIdentifier,univentionObjectType,univentionPolicyReference,univentionServerRole,univentionService,univentionShareGid,univentionShareSambaName,univentionShareWriteable,univentionUDMOptionModule,univentionUDMPropertyCLIName,univentionUDMPropertyCopyable,univentionUDMPropertyDefault,univentionUDMPropertyDeleteObjectClass,univentionUDMPropertyDoNotSearch,univentionUDMPropertyHook,univentionUDMPropertyLayoutOverwritePosition,univentionUDMPropertyLayoutOverwriteTab,univentionUDMPropertyLayoutPosition,univentionUDMPropertyLayoutTabAdvanced,univentionUDMPropertyLayoutTabName,univentionUDMPropertyLdapMapping,univentionUDMPropertyLongDescription,univentionUDMPropertyModule,univentionUDMPropertyMultivalue,univentionUDMPropertyObjectClass,univentionUDMPropertyOptions,univentionUDMPropertyShortDescription,univentionUDMPropertySyntax,univentionUDMPropertyTranslationLongDescription,univentionUDMPropertyTranslationShortDescription,univentionUDMPropertyTranslationTabName,univentionUDMPropertyValueMayChange,univentionUDMPropertyValueRequired,univentionUDMPropertyVersion,zoneName"

Verify:

ucr dump | grep ^ldap/index/* | grep -c univentionObjectIdentifier

Expected result for the example configuration:

1

3. Build the physical index

Changing the UCR value alone does not necessarily populate the database index. In the documented case, the missing attribute was indexed individually:

  • systemctl stop slapd
  • slapindex -f /etc/ldap/slapd.conf univentionObjectIdentifier
  • systemctl start slapd

Confirm that slapindex completed successfully before starting slapd; inspect the command output, service status and LDAP logs. Run slapindex as the appropriate account for the local deployment and check database ownership afterward if required. Do not run a manual rebuild concurrently with an automatic rebuild.

4. Validate the result

Confirm the equality-index setting, successful LDAP startup, expected LDAP operations:

grep -i univentionobjectidentifier /usr/share/univention-ldap/ldap_setup_index

root@ucs5primary:~# grep -i univentionobjectidentifier  /usr/share/univention-ldap/ldap_setup_index
        'univentionObjectIdentifier',

(Optional): rebuild all LDAP indexes

The investigation also documented the following full rebuild procedure:

# Stop LDAP service
service slapd stop

# Rebuild indexes
slapindex

# Start LDAP service
service slapd start

A full rebuild may take considerably longer than rebuilding a single attribute. Choose one supported rebuild procedure appropriate to the installed UCS/OpenLDAP version, not both without reason.

Investigation

Class creation workflow

  1. The user import processes a user and determines the required groups.
  2. For each group, get_or_create_group_udm_object() is called in ucs-school-lib/modules/ucsschool/lib/models/user.py:641-675.
  3. A missing class causes creation of a SchoolClass object.
  4. SchoolClass.create_without_hooks() first creates the LDAP/UDM group object (ucs-school-lib/modules/ucsschool/lib/models/group.py:228-232).
  5. By default, the class share is then created:
if self._create_share and self.exists(lo):
    success = success and self.create_share(self.get_machine_connection())
  1. create_share() constructs a ClassShare and saves it using the machine connection (ucs-school-lib/modules/ucsschool/lib/models/group.py:234-236).

Additional class-share operations

The class-share creation process retrieves the school’s file server from LDAP; may select a remote UCS Slave/Replica as the share server; sets the server FQDN, share path and Unix/Samba permissions; reads the group GID; constructs NT ACLs from the group SID; and creates a shares/share UDM object. Relevant source locations are ucs-school-lib/modules/ucsschool/lib/models/share.py:170-187, :212-253, and :279-313.

Server selection may involve LDAP queries for the OU, server object and univentionLDAPAccessWrite. Slow LDAP, DNS, Samba or remote-server operations can also contribute to latency and should be investigated if indexing does not resolve the problem.

Timed reproduction of a class creation

root@primary:~/univention-support # time python3 -m ucsschool.lib.models --debug create SchoolClass --name ge164161-1g --school ge164161
2026-09-04 16:38:20 DEBUG __main__.create:235  create: model='SchoolClass' name='ge164161-1g' school='ge164161' multi_value=() single_value=()
2026-09-04 16:38:20 DEBUG base.get_only_udm_obj:1096  Getting SchoolClass UDM object by filter: name=ge164161-1g
2026-09-04 16:38:20 DEBUG base.call_hooks:478  Starting SchoolClass.call_hooks('pre', 'create', lo('cn=admin,dc=example,dc=org')) for SchoolClass(name='ge164161-1g', school='ge164161', dn='cn=ge164161-1g,cn=klassen,cn=schueler,cn=group
hule,dc=org').
2026-09-04 16:38:20 INFO  pyhooks_loader.get_hook_classes:101  Searching for hooks of type 'Hook' in: /var/lib/ucs-school-lib/hooks...
2026-09-04 16:38:20 INFO  pyhooks_loader.get_hook_classes:127  Found hook classes: EnableKiksUser
2026-09-04 16:38:20 INFO  pyhooks_loader.get_hook_objects:185  Loaded hooks: {'post_create': ['EnableKiksUser.post_create']}.
2026-09-04 16:38:20 INFO  base.create_without_hooks:534  Creating SchoolClass(name='ge164161-1g', school='ge164161', dn='cn=ge164161-1g,cn=klassen,cn=schueler,cn=groups,ou=ge164161,dc=example,dc=org')
2026-09-04 16:38:20 DEBUG base.get_udm_object:823  Getting School UDM object by dn: ou=ge164161,dc=example,dc=org
2026-09-04 16:38:25 INFO  base.create_without_hooks:560  SchoolClass(name='ge164161-1g', school='ge164161', dn='cn=ge164161-1g,cn=klassen,cn=schueler,cn=groups,ou=ge164161,dc=example,dc=org') successfully created
2026-09-04 16:38:25 DEBUG base.get_only_udm_obj:1096  Getting SchoolClass UDM object by filter: name=ge164161-1g
2026-09-04 16:38:25 DEBUG base.get_udm_object:823  Getting ClassShare UDM object by dn: cn=ge164161-1g,cn=klassen,cn=shares,ou=ge164161,dc=example,dc=org
2026-09-04 16:38:25 DEBUG base.call_hooks:478  Starting ClassShare.call_hooks('pre', 'create', lo('cn=primary,cn=dc,cn=computers,dc=example,dc=org')) for ClassShare(name='ge164161-1g', school='ge164161', dn='cn=ge164161-1g,cn=klassen,cn
,dc=example,dc=org').
2026-09-04 16:38:25 INFO  base.create_without_hooks:534  Creating ClassShare(name='ge164161-1g', school='ge164161', dn='cn=ge164161-1g,cn=klassen,cn=shares,ou=ge164161,dc=example,dc=org')
2026-09-04 16:38:25 DEBUG base.get_udm_object:823  Getting School UDM object by dn: ou=ge164161,dc=example,dc=org
2026-09-04 16:38:25 DEBUG base.from_dn:1069  Looking up School with dn 'ou=ge164161,dc=example,dc=org'
2026-09-04 16:38:25 DEBUG base.get_only_udm_obj:1096  Getting SchoolClass UDM object by filter: name=ge164161-1g
2026-09-04 16:38:25 INFO  share.do_create:186  Creating share on "school-fileserver.example.org"
2026-09-04 16:40:27 INFO  base.create_without_hooks:560  ClassShare(name='ge164161-1g', school='ge164161', dn='cn=ge164161-1g,cn=klassen,cn=shares,ou=ge164161,dc=example,dc=org') successfully created
2026-09-04 16:40:27 DEBUG base.call_hooks:478  Starting ClassShare.call_hooks('post', 'create', lo('cn=primary,cn=dc,cn=computers,dc=example,dc=org')) for ClassShare(name='ge164161-1g', school='ge164161', dn='cn=ge164161-1g,cn=klassen,c
1,dc=example,dc=org').
2026-09-04 16:40:27 DEBUG base.call_hooks:478  Starting SchoolClass.call_hooks('post', 'create', lo('cn=admin,dc=example,dc=org')) for SchoolClass(name='ge164161-1g', school='ge164161', dn='cn=ge164161-1g,cn=klassen,cn=schueler,cn=grou
hule,dc=org').

real    2m7,809s
user    0m1,118s
sys     0m0,120s

The group object was created by 16:38:25, but the share creation started at 16:38:25 and completed at 16:40:27. The complete command took 2 min 7.809 s, with very little user or system CPU time. This establishes that the observed wait occurred during the share-creation phase, but not which underlying call accounted for all of it.

Debugging script and command

The investigation used the following script (entered as vim testshare.py):

import logging, univention.logging
univention.logging.basicConfig(
    filename="stdout", level=logging.DEBUG,
    univention_debug_categories=["ADMIN", "LDAP"],
    use_structured_logging=False)
from ucsschool.lib.models.__main__ import cli
cli(["--debug", "create", "SchoolClass", "--name", "ge164161-1d", "--school", "ge164161"], obj={})

vim testshare.py
time python3 testShare.py 2>&1 | tee -a testShare.log

Note: The supplied script filename uses testshare.py while the execution command uses testShare.py. On case-sensitive filesystems, use the same spelling for both.

Relevant LDAP debug output:

04.09.26 15:29:00.093  LDAP        ( ALL     ) : add | dn=cn=ge164161-1d,cn=groupName,cn=temporary,cn=univention,dc=example,dc=org addlist=[('objectClass', [b'top', b'lock']), ('cn', [b'ge164161-1d']), ('lockTime', [b'1788528840'])]
04.09.26 15:29:00.093  LDAP        ( ALL     ) : add | dn=cn=ge164161-1d,cn=groupName,cn=temporary,cn=univention,dc=example,dc=org
04.09.26 15:29:00.095  LDAP        ( ALL     ) : search | filter=(|(&(cn=ge164161-1d)(|(objectClass=univentionGroup)(objectClass=sambaGroupMapping)(objectClass=posixGroup)))(uid=ge164161-1d)) base=dc=example,dc=org scope=sub attributes
e required=False timeout=-1 sizelimit=0
04.09.26 15:29:00.097  LDAP        ( ALL     ) : search | filter=(|(uid=ge164161-1d)) base=cn=klassen,cn=schueler,cn=groups,ou=ge164161,dc=example,dc=org scope=one attributes=['dn'] unique=False required=False timeout=-1 sizelimit=0
04.09.26 15:29:00.099  LDAP        ( ALL     ) : add | dn=cn=42f78224-5964-4b6d-a011-c77748f4a1d0,cn=univentionObjectIdentifier,cn=temporary,cn=univention,dc=example,dc=org addlist=[('objectClass', [b'top', b'lock']), ('cn', [b'42f7822
7748f4a1d0']), ('lockTime', [b'1788528840'])]
04.09.26 15:29:00.099  LDAP        ( ALL     ) : add | dn=cn=42f78224-5964-4b6d-a011-c77748f4a1d0,cn=univentionObjectIdentifier,cn=temporary,cn=univention,dc=example,dc=org
04.09.26 15:29:00.100  LDAP        ( ALL     ) : search | filter=univentionObjectIdentifier=42f78224-5964-4b6d-a011-c77748f4a1d0 base=dc=example,dc=org scope=sub attributes=['dn'] unique=False required=False timeout=-1 sizelimit=0
04.09.26 15:29:01.337  LDAP        ( ALL     ) : add | dn=cn=84609,cn=gidNumber,cn=temporary,cn=univention,dc=example,dc=org addlist=[('objectClass', [b'top', b'lock']), ('cn', [b'84609']), ('lockTime', [b'1788528841'])]

Count the configured LDAP attributes

# ucr search --brief ldap/index/{eq,pres,sub} | cut -d: -f2 | tr ',' ' ' | sed -re 's/ +/ /' | tr " " "\n" | sort -u
[.... List of all LDAP indexes with duplicates removed ...]
# ucr search --brief ldap/index/{eq,pres,sub} | cut -d: -f2 | tr ',' ' ' | sed -re 's/ +/ /' | tr " " "\n" | sort -u  | wc -l
124

One hypothesis was that an earlier index limit of approximately 128 attributes had motivated omission of univentionObjectIdentifier. The investigation states that a later erratum increased the supported limit to 256, but gives only the placeholder Erratum XYZ. Therefore the exact release, applicability and current limit must be verified for the deployed UCS version; the hypothesis is not established as the historical cause.

Minimal class-creation test

python3 -c '  
import logging, univention.logging  
univention.logging.basicConfig(
    filename="stdout", level=logging.DEBUG,
    univention_debug_categories=["ADMIN", "LDAP"],
    use_structured_logging=False)
from ucsschool.lib.models.__main__ import cli
cli(["--debug", "create", "SchoolClass", "--name", "DEMOSCHOOL-1a", "--school", "DEMOSCHOOL"], obj={})
'

Caution: This creates real directory objects. Use an appropriate test school/class name and clean up test objects through supported UDM/UCS@school procedures.

Where the LDAP indexes are defined

1. UCR variable declarations: management/univention-ldap/debian/univention-ldap-server.univention-config-registry-variables documents ldap/index/eq, ldap/index/pres, ldap/index/sub, ldap/index/approx, ldap/index/autorebuild and ldap/index/quickmode. This declaration file does not contain the actual default attribute lists.

2. Recommended default attributes: management/univention-ldap/scripts/ldap_setup_index contains the RECOMMENDED_LDAP_INDEX dictionary with eq, pres, sub and approx entries. For example:

'approx': {
    'cn',
    'givenName',
    'mail',
    'sn',
    'uid',
}

The script reads ldap/index/<type> UCR values and can write changes through ucr_update(ucr, changes). An example invocation is:

/usr/share/univention-ldap/ldap_setup_index --add-defaults

The investigation confirmed the recommended attribute in the script:

root@primary:~/univention-support # grep -i univentionobjectidentifier  /usr/share/univention-ldap/ldap_setup_index
        'univentionObjectIdentifier',

3. OpenLDAP configuration generation: management/univention-ldap/conffiles/etc/ldap/slapd.conf.d/40univention-ldap-server_database converts UCR attribute lists into index directives:

attr2index = defaultdict(list)

for indextype in ('pres', 'eq', 'sub', 'approx'):
    value = configRegistry.get("ldap/index/" + indextype, "").strip()
    if value == "none":
        continue
    for attr in value.split(","):
        attr2index[attr].append(indextype)

for attr, indextypes in sorted(attr2index.items()):
    print("index\t%s\t%s" % (attr, ",".join(indextypes)))

For example, ldap/index/eq=cn,uid, ldap/index/pres=cn,uid and ldap/index/sub=cn produce approximately:

index cn eq,pres,sub
index uid eq,pres

The configuration path is:

UCR ldap/index/*
        |
        v
40univention-ldap-server_database
        |
        v
Generated /etc/ldap/slapd.conf
        |
        v
OpenLDAP uses the index directives

The investigation recommends preferring ldap_setup_index for index configuration because it also validates configuration and, with ldap/index/autorebuild enabled, can initiate slapindex. Review its version-specific behavior before combining it with a manual rebuild.

Conclusion

The observed slowdown coincided with class-share creation and an absent univentionObjectIdentifier LDAP equality index. Restoring the configured attribute and building its physical index addresses a plausible, concrete performance bottleneck. The material provided does not include a measured post-fix import runtime, so a quantitative improvement cannot be claimed without follow-up benchmarking.