Problem: UDM REST API returns HTTP 500 due to `shared_memory.authenticated` exception

Problem

In some OpenDesk/Nubus installations, the UDM REST API enters a broken state and starts returning HTTP 500 for authenticated requests.

Typical symptoms include:

  • ums-stack-data jobs fail during initialization.
  • Later ums-stack-data jobs fail in the wait-for-udm-rest-api init container.
  • Portal functions depending on the UDM REST API stop working.
  • User creation via the UDM REST API is no longer possible.

The issue is intermittent. It has been observed shortly after installation as well as during normal operation.

Investigation

The first indication is usually failed ums-stack-data jobs.

List the job pods:

kubectl -n <namespace> get pods | grep ums-stack-data

Inspect the failed job:

kubectl -n <namespace> logs <ums-stack-data-pod> --all-containers --timestamps

The init container initially reports that the API is reachable:

INFO:app:Checking if UDM Rest API can be reached...
INFO:app:Success, UDM Rest API is available

The main container then starts importing Stack Data and fails while creating the first UDM object:

INFO:univention.data_loader.actions:
Ensuring udm object groups/group,
cn=groups,dc=swp-ldap,dc=internal,
Domain Service Users

The request fails with HTTP 500:

univention.admin.rest.client.ServerError:
POST http://ums-udm-rest-api:9979/univention/udm/groups/group/: 500

cannot unpack non-iterable NoneType object

Next, inspect the UDM REST API pod:

kubectl -n <namespace> get pods | grep udm-rest-api
kubectl -n <namespace> logs <udm-rest-api-pod> \
  --all-containers \
  --timestamps

The server-side traceback shows the exception in authenticate():

Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/tornado/web.py", line 1712, in _execute
    result = self.prepare()

  File "/usr/lib/python3/dist-packages/univention/admin/rest/module.py", line 163, in prepare
    if authorization and self.authenticate(authorization):

  File "/usr/lib/python3/dist-packages/univention/admin/rest/module.py", line 172, in authenticate
    auth_type, username, userdn, password = shared_memory.authenticated[authorization]

TypeError: cannot unpack non-iterable NoneType object

After the first occurrence, subsequent authenticated requests fail with the same code path, but may report:

TypeError: cannot unpack non-iterable bool object

The error is no longer limited to object creation. Even simple API requests fail with HTTP 500, for example:

GET /univention/udm/
GET /univention/udm/settings/data/...

Example:

application:
Uncaught exception GET /univention/udm/

Traceback (most recent call last):
  ...
  File "/usr/lib/python3/dist-packages/univention/admin/rest/module.py", line 172, in authenticate
    auth_type, username, userdn, password = shared_memory.authenticated[authorization]

TypeError: cannot unpack non-iterable bool object

access:
500 GET /univention/udm/

This demonstrates that the failure occurs during request authentication before the requested UDM operation is executed.

Solution

A temporary workaround is to restart the UDM REST API and rerun the failed Stack Data job.

Restart the UDM REST API deployment:

kubectl -n <namespace> rollout restart deployment ums-udm-rest-api

After the API is available again, rerun the failed Stack Data job.

The issue has been observed to reoccur later.

Related information

This issue affects the UDM REST API authentication path and is characterized by server-side exceptions originating from:

/usr/lib/python3/dist-packages/univention/admin/rest/module.py

during access to:

shared_memory.authenticated[authorization]

The observed exceptions are:

TypeError: cannot unpack non-iterable NoneType object

and

TypeError: cannot unpack non-iterable bool object

See also Bug 59702