Recover broken environments with early installations of OX App Suite 7.10.6-ucs4 and OX Connector 2.1.2

[jgarcia]

During the latest update of the OX App Suite (7.10.6-ucs4) a number of different problems occurred in environments where OX Users were not assigned to any OX Contexts (i.e. their oxContext attribute is set to None). As a consequence, that update had been disabled after a short period of time to prevent environments from running into those issues. Unfortunately, at that point some customers had already upgraded to the faulty version of the app.

Environments that are upgrading to any of the currently released versions will not run into these issues anymore.

The following instructions are intended to recover environments that upgraded to OX App Suite 7.10.6-ucs4 before the faulty version was disabled and before fixes to that version had been published.

1. Check if a fix is needed.

Run the following command. univention-ldapsearch -LLL '(&(isOxUser=OK)(!(oxContextIDNum=*)))' 1.1

If the output is not empty, then there are users with an empty oxContext attribute and a fix is needed.

Upgrade OX App Suite and the OX Connector to their latest versions (OX App Suite 7.10.6-ucs5 and OX Connector 2.1.3 at the time of writing). OX App Suite 7.10.6-ucs5 includes a script that will help with fixing misconfigured users.

2. Run the fix script.

Scenario 1: OX App Suite and OX Connector are installed on the same server:

Run: fix_oxContext_attribute or fix_oxContext_attribute --binddn $admin --bindpwd $pwd if you are on a non primary server.

The script will first remove all old listener files that contain misconfigured OX users from the OX Connector and then look into the OX database to check and set all oxContext attributes for those users.

Scenario 2: OX App Suite and OX Connector are installed on seperate servers:

Before you run fix_oxContext_attribute you need to manually execute the script that deletes faulty json files from the OX Connector. On the server where the ox-connector is running, execute
/var/lib/univention-appcenter/apps/ox-connector/data/resources/remove-empty-oxContext-entries

After that, on the server where the OX App Suite is running, execute fix_oxContext_attribute --removed-files or fix_oxContext_attribute --binddn $admin --bindpwd $pwd --removed-files if you are running the OX App Suite app on a non-primary server.

3. Verify the fix

After running the fix there are two things to verify.

Firstly, check that there are no users with an empty oxContext attribute anymore. Run univention-ldapsearch -LLL '(&(isOxUser=OK)(!(oxContextIDNum=*)))' 1.1

Secondly, verify that the OX Connector finished processing all the files. /var/lib/univention-appcenter/apps/ox-connector/data/listener/ should not contain files named YYYY-MM-DD-hh-mm-ss-xxxxxx.json and the logfile /var/log/univention/listener_modules/ox-connector.log should show a success message in the last line.

If that is not the case, it is likely that modifications to the misconfigured users had been made before the missing OX Context attribute was added. In that case, it may be necessary to remove the files of users that did not yet have oxContext when they were modified from /var/lib/univention-appcenter/apps/ox-connector/data/listener/old. It may be necessary to perform this process several times if many modifications were made.

[ckowarzik]

The script fix_oxContext_attribute will skip ox-users whose OX-Displayname does not consist of two words (e.g. “Juan Carlos García García” ;-).

Apply the following patch to avoid that problem for univention-ox<12.0.34
(patch only valid for unlimited ox filestore quota - which is the default in UCS):

--- /usr/sbin/fix_oxContext_attribute.old       2023-04-25 13:00:37.510165406 +0200
+++ /usr/sbin/fix_oxContext_attribute   2023-04-25 13:00:48.595153037 +0200
@@ -70,7 +70,7 @@
 
 /opt/open-xchange/sbin/listcontext -P $(< /etc/ox-secrets/master.secret) -A oxadminmaster | awk 'NR>1 { print $1; }' > ctxs.txt
 cat ctxs.txt | xargs -I{} bash -c '/opt/open-xchange/sbin/listuser -c {} -A $(if [[ {} == 10 ]]; then echo "oxadmin"; else echo "oxadmin-context{}"; fi) -P $(< /etc/ox-secrets/context{}.secret) \
-| awk '"'"'NR > 1 { print $5; }'"'"' \
+| awk '"'"'NR > 1 { print $(NF-1); }'"'"' \
 | xargs -I{} univention-ldapsearch -xLLL '"'"'(&(&(mailPrimaryAddress={})(isOXUser=OK))(!(oxContextIDNum=*)))'"'"' 1.1 \
 | sed -ne '"'"'s,dn: ,,p'"'"' >> dns_ctx{}.txt '

(basically replace print $5 with print $(NF-1) in line 73)

Apply the following patch to avoid that problem for univention-ox=12.0.34
(patch only valid for unlimited ox filestore quota - which is the default in UCS):

--- /usr/sbin/fix_oxContext_attribute.old	2023-04-25 15:02:30.600000000 +0200
+++ /usr/sbin/fix_oxContext_attribute	2023-04-25 15:02:59.864000000 +0200
@@ -99,7 +99,7 @@
 echo
 /opt/open-xchange/sbin/listcontext -P $(< /etc/ox-secrets/master.secret) -A oxadminmaster | awk 'NR>1 { print $1; }' > ctxs.txt
 cat ctxs.txt | xargs -I{} bash -c '/opt/open-xchange/sbin/listuser -c {} -A $(if [[ {} == 10 ]]; then echo "oxadmin"; else echo "oxadmin-context{}"; fi) -P $(< /etc/ox-secrets/context{}.secret) \
-| awk '"'"'NR > 1 { print $5; }'"'"' \
+| awk '"'"'NR > 1 { print $(NF-1); }'"'"' \
 | xargs -I{} univention-ldapsearch -xLLL '"'"'(&(&(mailPrimaryAddress={})(isOXUser=OK))(!(oxContextIDNum=*)))'"'"' 1.1 \
 | sed -ne '"'"'s,dn: ,,p'"'"' >> dns_ctx{}.txt '

(basically replace print $5 with print $(NF-1) in line 102)

[ckowarzik]

If OX App Suite and OX Connector are installed on the same server then before running the script fix_oxContext_attribute make sure you have installed at least ox-connector=2.1.3.

The reason for this is that the script fix_oxContext_attribute tries to execute the script /var/lib/univention-appcenter/apps/ox-connector/data/resources/remove-empty-oxContext-entries which will not be available before ox-connector=2.1.3.

[ckowarzik]

With OX App Suite 7.10.6-ucs9 there comes an updated script fix_oxContext_attribute (package univention-ox-12.0.38) that will fix the above mentioned problem with ox-users whose OX-Displayname does not consist of two words.

Unfortunately now the script will skip ox-users whose OX-Displayname contains a comma-character (e.g. “Doe, John” or “John M, Doe”).

For details and suggested patch see Bug 56622 (Bad CSV parsing leading to fix_oxContext_attribute skipping users).