OX Appsuite 7.10-6-ucs4 installation/upgrade

Apps & App Center
, , , ,
#1

With the release of OX App Suite 7.10.6-ucs4, the app has been integrated with the OX Connector app (ver. >= 2.1.2) from the UCS App Center. This means that the OX App Suite is responsible for running the OX software and the OX Connector does the provisioning of users, groups, etc. into the OX database (via OX’ SOAP API). More details on how the OX Connector works can be found here: OX Connector app documentation — OX Connector app.

You can install or upgrade OX App Suite either from UMC or the CLI. Ensure that the following sequence is used:

  1. Update the corresponding UCS systems to at least UCS 5.0.3 errata632. This is a requirement for the following steps.
  2. When upgrading from a previous OX App Suite version, note that you need to install the OX Connector app first. OX App Suite has to be upgraded immediately afterwards.
  3. When installing, the App Center will resolve that dependency automatically.
  4. Make sure that the join scripts are run successfully after installation/upgrade of both apps (OX App Suite and OX Connector). Otherwise, problems related to OX context credentials might happen.

The OX Connector can be installed on any UCS server role. But it is more convenient to install it on a UCS server that also runs the OX App Suite: The OX Connector app needs the OX Administrator’s credentials to create OX context objects. These credentials are only available after OX has already been installed. The username is oxadminmaster, the password can be found in the file /etc/ox-secrets/master.secret on the OX App Suite system. You can set these credentials in the OX Connector settings afterwards. But if you have installed both apps on the same host, the OX App Suite app will reconfigure the OX Connector for you.

This holds for Single Server installations (OX App Suite installation on only one server) and Distributed Server Installation (OX App Suite installations on multiple servers). You will only need one OX Connector in each scenario. The OX Connector app should be configured against the “active OX server installation” in case of a distributed setup.

Important notes:

  • The OX Connector ships it’s own default “access profiles”. If you adjusted the access profiles in OX, you will probably want to mirror that in the OX Connector as well. Please see the documentation 2. Usage — OX Connector app
  • The OX Connector will process all users, groups, etc in the LDAP once again to perform neccessary adjustments. This may take a lot of time, depending on your environment.
    • For example, the OX connector requires that all OX users in LDAP have the LDAP attribute oxContextIdNum set. If the attribute value is missing, it is automatically set to the default context during the update.
    • During the update the UDM user property oxBirthday will be removed and is no longer supported. Instead the UDM property birthday is used for user provisioning by the OX connector. Even if the UDM property oxBirthday is removed, the LDAP values will remain in the LDAP directory and no data is lost.
  • You will need to set the credentials for already existing contexts - unless you installed the OX App Suite and the OX Connector on the same host (then it is done for you). The credentials need to be set in /var/lib/univention-appcenter/apps/ox-connector/data/secrets/contexts.json on the server where OX Connector runs. You can find the passwords in /etc/ox-secrets/ on the server where the OX App Suite runs.

see also:

Changelog:

  • (2023-11-20) Add hint to update to UCS 5.0-3 errata 632 in advance.
  • (2023-11-20) Add hint that oxContextIdNum is added to all OX user objects.
  • (2023-11-20) Add hint to call all pending join scripts.
  • (2023-11-20) Add hint that oxBirthday is no longer supported after the update.
2 Likes
Problem:Can´t upgrade OX App Suite to version 7.10.6-ucs4
OX-Guard removed when performing latest Updates? Seriously?
OX cannot be (re)installed after upgrade to UCS 5
#2

The OX Connector was broken after update.

The joinscript 50ox-connector.inst was failing, also 65univention-ox.inst as follow-up problem.

I got following error message in join.log:
ERROR: Timeout waiting for /usr/lib/python3/dist-packages/univention/admin/handlers/oxmail/oxcontext.py.
ucs_registerLDAPExtension: registraton of /var/lib/univention-appcenter/apps/ox-connector/data/resources//udm/handlers/oxmail/oxcontext.py failed.
unknown module oxmail/oxcontext.
…50ox-connector.inst: Failed to register LDAP module.

The problem was the missing handlers which I manually downloaded from github:

git clone GitHub - univention/ox-connector
cd ox-connector/udm/handlers/
cp -a oxmail oxresources /usr/lib/python3/dist-packages/univention/admin/handlers/

After that I could add new users with OX access and found following in /var/log/univention/listener.log:

grafik

Off-Topic: The marked “b” in the screenshot looks identically to this issue, which could be a hint for the developers: UCS 5/ox-connector issue: OX global addressbook: all telephone numbers with leading "b'[number]'"

Back to topic: The test user cannot login to OX, because the user is not found in context:

/var/log/open-xchange/open-xchange.log.11: com.openexchange.login.login=testos.teron
/var/log/open-xchange/open-xchange.log.11:com.openexchange.exception.OXException: USR-0015 Categories=ERROR Message=‘Cannot find user with identifier testos.teron in context 10.’ exceptionID=-299353422-237959

After that I checked the folder /var/lib/univention-appcenter/listener/ox-connector/ which is empty.
I checked the container and found persistent folder e.g. /var/lib/univention-appcenter/listener/ox-connector/ with 651 JSON files in it.
This means the SOAP requests are not working.
But log file looks ok:
grafik

The OX SOAP service is up and running:

/var/log/open-xchange/open-xchange.log.0:CXF SOAP service is up and running

I could not find anything interesting in the open-xchange logs related to the SOAP API which means the ox-connector does not connect to it.

Then I started a shell in the ox-connector container and executed /usr/bin/python3 /tmp/univention-ox-connector.listener_trigger

This resulted in following error message:

Error while processing /var/lib/univention-appcenter/apps/ox-connector/data/listener/2023-04-16-00-22-58-029382.json
Traceback (most recent call last):

  • File “/usr/lib/python3.9/site-packages/urllib3/connectionpool.py”, line 699, in urlopen*
  • httplib_response = self._make_request(*
  • File “/usr/lib/python3.9/site-packages/urllib3/connectionpool.py”, line 382, in _make_request*
  • self._validate_conn(conn)*
  • File “/usr/lib/python3.9/site-packages/urllib3/connectionpool.py”, line 1010, in _validate_conn*
  • conn.connect()*
  • File “/usr/lib/python3.9/site-packages/urllib3/connection.py”, line 411, in connect*
  • self.sock = ssl_wrap_socket(*
  • File “/usr/lib/python3.9/site-packages/urllib3/util/ssl_.py”, line 449, in ssl_wrap_socket*
  • ssl_sock = _ssl_wrap_socket_impl(*
  • File “/usr/lib/python3.9/site-packages/urllib3/util/ssl_.py”, line 493, in _ssl_wrap_socket_impl*
  • return ssl_context.wrap_socket(sock, server_hostname=server_hostname)*
  • File “/usr/lib/python3.9/ssl.py”, line 501, in wrap_socket*
  • return self.sslsocket_class._create(*
  • File “/usr/lib/python3.9/ssl.py”, line 1041, in _create*
  • self.do_handshake()*
  • File “/usr/lib/python3.9/ssl.py”, line 1310, in do_handshake*
  • self._sslobj.do_handshake()*
    ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1129)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):

  • File “/usr/lib/python3.9/site-packages/requests/adapters.py”, line 489, in send*
  • resp = conn.urlopen(*
  • File “/usr/lib/python3.9/site-packages/urllib3/connectionpool.py”, line 755, in urlopen*
  • retries = retries.increment(*
  • File “/usr/lib/python3.9/site-packages/urllib3/util/retry.py”, line 574, in increment*
  • raise MaxRetryError(_pool, url, error or ResponseError(cause))*
    urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host=’********’, port=443): Max retries exceeded with url: /webservices/OXContextService?wsdl (Caused by SSLError(SSLCertVerificationError(1, ‘[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1129)’)))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):

  • File “/tmp/univention-ox-connector.listener_trigger”, line 324, in run_on_files*
  • f(obj)*
  • File “/usr/lib/python3.9/site-packages/univention/ox/provisioning/init.py”, line 69, in run*
  • create_context(obj)*
  • File “/usr/lib/python3.9/site-packages/univention/ox/provisioning/contexts.py”, line 64, in create_context*
  • if context_exists(obj):*
  • File “/usr/lib/python3.9/site-packages/univention/ox/provisioning/contexts.py”, line 59, in context_exists*
  • return bool(get_obj_by_name_from_ox(Context, context_id, None))*
  • File “/usr/lib/python3.9/site-packages/univention/ox/provisioning/helpers.py”, line 41, in get_obj_by_name_from_ox*
  • return klass.from_ox(context_id, name=name)*
  • File “/usr/lib/python3.9/site-packages/univention/ox/soap/backend.py”, line 280, in from_ox*
  • context_service = cls.service(DEFAULT_CONTEXT)*
  • File “/usr/lib/python3.9/site-packages/univention/ox/soap/backend.py”, line 105, in service*
  • cls._service_objs[cls._object_type][context_id] = OxSoapServiceClass(cls.get_client_credentials(context_id))*
  • File “/usr/lib/python3.9/site-packages/univention/ox/soap/services.py”, line 129, in init*
  • self.Type = getattr(client_credentials.types, self._type_name)*
  • File “/usr/lib/python3.9/site-packages/univention/ox/soap/credentials.py”, line 84, in types*
  • self.class._types = Types(self.server)*
  • File “/usr/lib/python3.9/site-packages/univention/ox/soap/types.py”, line 71, in init*
  • self.class.wsdl_context = get_wsdl(server, ‘Context’)*
  • File “/usr/lib/python3.9/site-packages/univention/ox/soap/services.py”, line 94, in get_wsdl*
  • return ZeepClient(WS_URLS[object_type].format(server=server)).wsdl*
  • File “/usr/lib/python3.9/site-packages/zeep/client.py”, line 76, in init*
  • self.wsdl = Document(wsdl, self.transport, settings=self.settings)*
  • File “/usr/lib/python3.9/site-packages/zeep/wsdl/wsdl.py”, line 92, in init*
  • self.load(location)*
  • File “/usr/lib/python3.9/site-packages/zeep/wsdl/wsdl.py”, line 95, in load*
  • document = self._get_xml_document(location)*
  • File “/usr/lib/python3.9/site-packages/zeep/wsdl/wsdl.py”, line 155, in _get_xml_document*
  • return load_external(*
  • File “/usr/lib/python3.9/site-packages/zeep/loader.py”, line 89, in load_external*
  • content = transport.load(url)*
  • File “/usr/lib/python3.9/site-packages/zeep/transports.py”, line 123, in load*
  • content = self._load_remote_data(url)*
  • File “/usr/lib/python3.9/site-packages/zeep/transports.py”, line 135, in _load_remote_data*
  • response = self.session.get(url, timeout=self.load_timeout)*
  • File “/usr/lib/python3.9/site-packages/requests/sessions.py”, line 600, in get*
  • return self.request(“GET”, url, *kwargs)
  • File “/usr/lib/python3.9/site-packages/requests/sessions.py”, line 587, in request*
  • resp = self.send(prep, *send_kwargs)
  • File “/usr/lib/python3.9/site-packages/requests/sessions.py”, line 701, in send*
  • r = adapter.send(request, *kwargs)
  • File “/usr/lib/python3.9/site-packages/requests/adapters.py”, line 563, in send*
  • raise SSLError(e, request=request)*
    requests.exceptions.SSLError: HTTPSConnectionPool(host=’**********’, port=443): Max retries exceeded with url: /webservices/OXContextService?wsdl (Caused by SSLError(SSLCertVerificationError(1, ‘[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1129)’)))
    This is consecutive error #810
    Sleeping for 0 sec
    Successfully processed 0 files during this run

This means the ox-connector & with OX installed on UCS internally (error message sounds like self-signed certificate?) is not working at the moment for the UCS server itself?
The configuration of the ox-connector, for the UCS server where OX is installed was: “https://hostname.subdomain.domain.tld”. But the webserver of the UCS server is using a Letsencrypt certificate. We have no problems connecting to it, but the ox-connector has…

Next test:
Then I setup OX to listen also on ethernet:
ucr set ox/cfg/server.properties/com.openexchange.connector.networkListenerHost=*

Setup another apache webserver on a different (Non-UCS) Linux VM, added a proxy setup and a vhost ox.domain.tld, added a DNS A record, added a similar Letsencrypt certificate, copied the OX frontend files into document root, reloaded apache and modified then the server URL in the configuration of the ox-connector.
Smoke test: OX login possible with my previously existing user…OX is accessible.

Some seconds later:…Bamm, working now…all JSON files were processed and moved to a subdirectory “old”. Why the same OX server with similar Letsencrypt certificate on the UCS webserver is not working but a different apache which is proxying to it is?
This is now something for Univention developers to clarify this topic. For me the workaround is Ok. for now.

UCS 5/ox-connector issue: OX global addressbook: all telephone numbers with leading "b'[number]'"
#3

Additionally some questions to Univention related to this issues:

In the very early stage where I wanted to find out what is this oxmail/oxcontext for and which deb package provides it, I tried to use apt-file search oxmail. The problem was that I always got the answer that I should refresh the cache which I tried to do with apt-file update, but no success. Always the same message.
Was the cache moved to another location? If yes, why? Is there an Univention successor of apt-file search?

Next is why it is possible, that a deb package build has missing folders? Was there another deb package missing which I have overseen?

Why I could not find anything in the logs related to the certificate issue?
I saw 651 JSON files for ~50 users.

With e.g. round about 20.000 users what will happen with the file system if something goes wrong? Why not using a rabbitmq or similar with cluster functionality plus a consumer as a cluster ressource to get HA? If one single JSON file could not be executed, the complete user deployment will stop. Why not splitting this into user or group related jobs which could be executed in parallel? Additionally why is there no complete or delta sync available?

The leading system is UCS, the secondary system is the OX database. Why not comparing the both systems and creating delta jobs for each users/groups for which any deviation exists? I assume that the overall effort will be reduced to a fraction.

What will happen if an admin is deleting the old directory, which is the current memory for which jobs are already done? Is it a problem if something is missing here? Is there any cleanup job? Normally all interesting information should be in the log file. Why not deleting the JSONs after a successful processing?

In my view, the architecture of the OX Connector in its current form leads to avoidable errors.

#4

Reason for the Letsencrypt issue:
In file /etc/univention/templates/files/etc/apache2/sites-available/ssl.d/00start
Uncommented SSLCACertificateFile & SSLCertificateChainFile:
if configRegistry.get(‘apache2/ssl/ca’):
print(’# SSLCACertificateFile %s’ % configRegistry.get(‘apache2/ssl/ca’))
else:
print(’# SSLCACertificateFile /etc/univention/ssl/ucsCA/CAcert.pem’)
if configRegistry.get(‘apache2/ssl/certificatechain’):
print(’# SSLCertificateChainFile %s’ % configRegistry.get(‘apache2/ssl/certificatechain’))

There is still one join script which runs into an error:
…
Updating uid=dns-ucs,cn=users,dc=dom,dc=domain,dc=tld
Failed to modify uid=dns-ucs,cn=users,dc=dom,dc=domain,dc=tld: (‘Tuple_to_LDAPMod(): expected a byte string in the list’, ‘’)
Done.
Converting OX mail domains to UCS mail domains…
Done.
*93univention-ox.inst: *
EXITCODE=1

This seems to be a python update issue. No clue how to fix that at the moment.

Another topic is the ox-connector. New users does not exist in OX context even the json file was processed and moved to old. The complete ox-connector construction is dubious. My preferred solution would compare the LDAP and OX current state and if there is a diff it will then update it. The state in old maybe is not the current state in OX which is a big problem.

#6

Hi @audiolinux ,
the letsencrypt certificate is a valid certificate.
In this case a SSLCACertificateFile and a SSLCertificateChainFile is not needed and not working.
You need a fullchain certificate and a private key, nothing else.

#7

Since both, the ox-connector and ox running on the same host, I just configured the ox-connector to establish a non-encrypted connection via port 80. In my test env, this did the trick. Not to mention, that such a workaround isn’t an option in productive environments at all.

It is documented as well: 3. Configuration — OX Connector app

#8

Hi audiolinux

Thx for your hint, now my OX will sync my new users too. It was a long way after the last update :slight_smile: hope it will be fixed soon.

#9

Hey @BenSommer,

glad to hear, that it worked for you :slight_smile:

Have you checked, if functional_accounts working as expected? In our environment, the ox-connector creates the dovecot mailboxes. In the OX-UI one cannot access these mailboxes due to insufficient access rights.

Cheers

Sebastian

#10

Hi Sebastian

I have full access and no problems.

Best Ben

Mastodon