Problem: ucsschool-id-connector queue not processed

Problem

The ucsschool-id-connector fills its processing queue, but no entries are executed or forwarded to Kelvin. Although the connector appears active, no synchronization tasks are processed, and queued items remain indefinitely.

Analysis

A connectivity issue was identified between the ucsschool-id-connector container running on the Backup DC and the Primary DC (10.0.0.200). To verify outbound connectivity from the connector container, the following test was performed:

root@backup01:~ # univention-app shell ucsschool-id-connector
root@ucssc-24106813:/ucsschool-id-connector# bash -c "echo > /dev/tcp/10.0.0.200/80" && echo OK || echo FEHLER
^C
root@ucssc-24106813:/ucsschool-id-connector# bash -c "echo > /dev/tcp/10.0.0.200/8911" && echo OK || echo FEHLER
^C

The command does not return OK, and instead hangs until it is canceled manually. This confirms that the container cannot open a TCP connection to the Primary DC, neither on port 8911 (Kelvin API) or on port 80.

Since Kelvin communication and connector synchronization require the container to reach systems outside the Docker subnet, this connectivity failure explains why the queue fills but nothing is processed.

Root Cause

The issue was traced back to the UCS system upgrade from 5.0 to 5.2. During this upgrade, the package univention-firewall was removed:

Bug 58827

Starting univention-upgrade. Current UCS version is 5.1-0 errata0
[...]
Checking for release updates:                           found: UCS 5.2-0
Starting update to UCS version 5.2-0 at Sat Oct  4 17:31:59 2025...
Starting update to UCS version 5.2-0
[...]
Paketlisten werden gelesen…
Abhängigkeitsbaum wird aufgebaut…
Statusinformationen werden eingelesen…
Die folgenden Pakete werden ENTFERNT:
  bc bind9 bind9-utils libreadline7 memtest86+ nfs-kernel-server python3-gdbm
  python3-gnupg python3-pygresql python3-univention-group-membership-cache
  python3-univention-pkgdb python3-univention-portal univention-firewall
  univention-group-membership-cache univention-initrd univention-maintenance
  univention-nfs-server univention-pkgdb-tools univention-portal
  univention-role-common univention-support-info
0 aktualisiert, 0 neu installiert, 21 zu entfernen und 1 nicht aktualisiert.
[...]


root@backup01:~ # dpkg -l | grep -i firewall
rc  univention-firewall                                 13.0.3                                             all          UCS - firewall integration

This package normally ensures that the system generates the necessary iptables NAT rules used by Docker and other UCS components.

After the upgrade, the NAT table was almost empty, missing critical MASQUERADE rules for outbound Docker traffic:

iptables NAT Table on the Broken System

root@backup01:~ # iptables -vnL -t nat
Chain PREROUTING (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination

Chain INPUT (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination

Chain OUTPUT (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination

Chain POSTROUTING (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination
    0     0 MASQUERADE  6    --  *      *       172.17.0.1           172.17.0.1           tcp dpt:8911
    0     0 MASQUERADE  6    --  *      *       172.31.1.2           172.31.1.2           tcp dpt:8180
    0     0 MASQUERADE  6    --  *      *       172.31.1.2           172.31.1.2           tcp dpt:7600

Chain DOCKER (0 references)
 pkts bytes target     prot opt in     out     source               destination
    0     0 DNAT       6    --  !docker0 *       0.0.0.0/0            0.0.0.0/0            tcp dpt:40001 to:172.17.0.1:8911
    0     0 DNAT       6    --  !br-870a5db2fc46 *       0.0.0.0/0    0.0.0.0/0            tcp dpt:8180 to:172.31.1.2:8180
    0     0 DNAT       6    --  !br-870a5db2fc46 *       0.0.0.0/0    0.0.0.0/0            tcp dpt:7600 to:172.31.1.2:7600

What is missing?

There is no MASQUERADE rule for the Docker subnet 172.17.0.0/16 that would allow containers to reach external systems. Thus, containers can reach each other within Docker but cannot reach the LAN, including the Primary DC.

Expected iptables NAT Table on a Working System

root@primary:~# iptables -vnL -t nat
Chain PREROUTING (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination         
  513 37401 DOCKER     0    --  *      *       0.0.0.0/0            0.0.0.0/0            ADDRTYPE match dst-type LOCAL

Chain INPUT (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination         

Chain OUTPUT (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination         
  483 34683 DOCKER     0    --  *      *       0.0.0.0/0           !127.0.0.0/8          ADDRTYPE match dst-type LOCAL

Chain POSTROUTING (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 MASQUERADE  0    --  *      !docker0      172.17.0.0/16        0.0.0.0/0           
    0     0 MASQUERADE  0    --  *      !br-59e1aba97a74  172.16.1.0/24    0.0.0.0/0           
    0     0 MASQUERADE  6    --  *      *       172.16.1.2           172.16.1.2           tcp dpt:7600
    0     0 MASQUERADE  6    --  *      *       172.16.1.2           172.16.1.2           tcp dpt:8180
    0     0 MASQUERADE  6    --  *      *       172.17.0.1           172.17.0.1           tcp dpt:8911
    0     0 MASQUERADE  6    --  *      *       172.17.0.2           172.17.0.2           tcp dpt:8911

Chain DOCKER (2 references)
 pkts bytes target     prot opt in     out     source               destination         
  210 13284 RETURN     0    --  br-59e1aba97a74 *       0.0.0.0/0    0.0.0.0/0           
    8   516 RETURN     0    --  docker0 *       0.0.0.0/0            0.0.0.0/0           
    0     0 DNAT       6    --  !docker0 *       0.0.0.0/0            0.0.0.0/0            tcp dpt:40002 to:172.17.0.2:8911
    0     0 DNAT       6    --  !br-59e1aba97a74 *       0.0.0.0/0    0.0.0.0/0            tcp dpt:8180 to:172.16.1.2:8180
    0     0 DNAT       6    --  !br-59e1aba97a74 *       0.0.0.0/0    0.0.0.0/0            tcp dpt:7600 to:172.16.1.2:7600
    0     0 DNAT       6    --  !docker0 *       0.0.0.0/0            0.0.0.0/0            tcp dpt:40001 to:172.17.0.1:8911

The crucial difference

This working system includes:

  • MASQUERADE for 172.17.0.0/16 → required for Docker container outbound traffic
  • Fully populated DOCKER chains
  • Correct NAT handling for port forwarding and container reachability

These are automatically provided by the univention-firewall package, which installs template files in /etc/univention/templates/files/etc/security/packetfilter.d/. Those template files generate config files in /etc/security/packetfilter.d/, and those include the necessary iptables rules.

Solution

To restore full network functionality, reinstall the missing univention-firewall package, regenerate the packet filter rules (iptables), and restart required services:

univention-install univention-firewall
systemctl restart univention-appcenter-listener-converter@ucsschool-id-connector.service
ucr commit /etc/security/packetfilter.d/*
systemctl restart univention-firewall.service

What these steps accomplish

  • univention-firewall is restored
  • The package regenerates the missing iptables NAT rules
  • The ucsschool-id-connector listener-converter is restarted so it can immediately use the new rules
  • ucr commit rewrites any packetfilter rules according to UCS policies
  • The firewall service reloads all NAT and filter configurations

After these steps, Docker containers can once again reach systems outside their subnet, and the ucsschool-id-connector successfully forwards tasks to Kelvin.

This topic was automatically closed after 24 hours. New replies are no longer allowed.