Upgrade to 5.2 fails (log attached)

[toko42]

Good afternoon,

I followed the descriptions in the release notes (Release notes for the installation and update of Univention Corporate Server (UCS) 5.2-0 — UCS 5.2-0 Release Notes).

Pre-Upgrade system was on 5.0-10 errata1323.

There were no errors in the pre-upgrade-check script(s) (tried 5.1, 5.2-0, 5.2-1, 5.2-2, 5.2-3) and I checked LDAP migration as well as Keycloak migration status.

I had univention-dhcp uninstalled but the PKGDB error persisted (installed version 2.28 but should be 2.32) ('database "pkgdb" has a collation version mismatch' while upgrading to 5.2).

Maybe someone can get me up to speed with the attached updater.log in which I renamed my domain name to MYDOMAIN.

Thank you very much.

updater.log (2.1 MB)

[dzidek23]

Sorry @toko42 I can’t provide you with much more than I already did.

Your log is showing “pkgdb” issues in quite a few places, but I assume you have already run the database scan, didn’t you?

I would be tempted to remove the ‘univention-bind’ package as this is exactly where the install failed. Other than that, I’m out of ideas…

[toko42]

Yes, I ran the DB commands but the system said that no changes were made.

What is ‘univention-bind’ used for?
Is it safe to remove and re-install afterwards?

Thank you very much.

[dzidek23]

It looks like it is a DNS server package…

Is it safe to remove? Well, UCS 5.1-0 doesn’t seem to offer much of a functionality so you might as well remove the DNS (I guess).

In my case, normal apt remove univention-dhcp got rid of the app but not the database entries. I suspect it might be the same with this package. On reinstall, all entries should still be there*.

• treat with caution, test before running on live system

[msi]

TL;DR: Could it be that you are - or at some point used to back up UCS with Veeam? That could be one culprit, see my digging below.

univention-bind is (or at least used to be) package that contains the glue for the integration of the BIND DNS server in UCS. So yes, it’s rather a central piece, depending on roles installed on it I wouldn’t remove it light-hearted.

However there is another error the you can see before (starting at line 27531) that I sumbled upon:

warning: rule setup/* already exists
dpkg: Abhängigkeitsprobleme verhindern Konfiguration von univention-bind:
 univention-bind hängt ab von univention-role-server-common (>= 12.0.0-8) | univention-container-role-server-common; aber:
  Paket univention-role-server-common ist noch nicht konfiguriert.
  Paket univention-container-role-server-common ist nicht installiert.

dpkg: Fehler beim Bearbeiten des Paketes univention-bind (--configure):
 Abhängigkeitsprobleme - verbleibt unkonfiguriert

These are also pretty central meta-packages for UCS if I remember correctly.

So when following that bit, on line 27601 you can see that there is an issue when configuring univention-role-server-common:

dpkg: Abhängigkeitsprobleme verhindern Konfiguration von univention-role-server-common:
 univention-role-server-common hängt ab von linux-image-amd64; aber:
  Paket linux-image-amd64 ist noch nicht konfiguriert.

dpkg: Fehler beim Bearbeiten des Paketes univention-role-server-common (--configure):
 Abhängigkeitsprobleme - verbleibt unkonfiguriert

And when we dig further, we see at 27531 that linux-image-amd64 couldn’t be configured:

dpkg: Abhängigkeitsprobleme verhindern Konfiguration von linux-image-amd64:
 linux-image-amd64 hängt ab von linux-image-6.1.0-28-amd64 (= 6.1.119-1); aber:
  Paket linux-image-6.1.0-28-amd64 ist noch nicht konfiguriert.

That’s the meta package for the Linux kernel and it isually depends on an actual kernel version. So we end up at line 27329 where we can see that an issue happens with DKMS: and a module veeamsnap:

linux-image-6.1.0-28-amd64 (6.1.119-1) wird eingerichtet ...
/etc/kernel-img.conf:2: W: ignoring unknown parameter do_bootfloppy
/etc/kernel-img.conf:3: W: ignoring unknown parameter silent_loader
I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.10.0-30-amd64
I: /initrd.img.old is now a symlink to boot/initrd.img-5.10.0-30-amd64
I: /vmlinuz is now a symlink to boot/vmlinuz-6.1.0-28-amd64
I: /initrd.img is now a symlink to boot/initrd.img-6.1.0-28-amd64
/etc/kernel/postinst.d/dkms:
dkms: running auto installation service for kernel 6.1.0-28-amd64.
Deprecated feature: REMAKE_INITRD (/var/lib/dkms/veeamsnap/5.0.2.4567/source/dkms.conf)
[...]
Deprecated feature: REMAKE_INITRD (/etc/dkms/framework.conf)
Sign command: /usr/lib/linux-kbuild-6.1/scripts/sign-file
Signing key: /var/lib/dkms/mok.key
Public certificate (MOK): /var/lib/dkms/mok.pub
Certificate or key are missing, generating self signed certificate for MOK...

Building module:
Cleaning build area...
make -j8 KERNELRELEASE=6.1.0-28-amd64 VM_UNAME=6.1.0-28-amd64 MODULEBUILDDIR=/var/lib/dkms/open-vm-tools/10.1.5/build -C vmxnet...(bad exit status: 2)
Error! Bad return status for module build on kernel: 6.1.0-28-amd64 (x86_64)
Consult /var/lib/dkms/open-vm-tools/10.1.5/build/make.log for more information.
Deprecated feature: REMAKE_INITRD (/etc/dkms/framework.conf)
Sign command: /usr/lib/linux-kbuild-6.1/scripts/sign-file
Signing key: /var/lib/dkms/mok.key
Public certificate (MOK): /var/lib/dkms/mok.pub
Deprecated feature: REMAKE_INITRD (/var/lib/dkms/veeamsnap/5.0.2.4567/source/dkms.conf)

Running the pre_build script:
Fake System.map was found in '/boot/System.map-6.1.0-28-amd64'
Generate "/var/lib/dkms/veeamsnap/5.0.2.4567/build/config.h" for kernel "6.1.0-28-amd64".

Building module:
Cleaning build area...
make -j8 KERNELRELEASE=6.1.0-28-amd64 -C /lib/modules/6.1.0-28-amd64/build M=/var/lib/dkms/veeamsnap/5.0.2.4567/build modules...(bad exit status: 2)
Error! Bad return status for module build on kernel: 6.1.0-28-amd64 (x86_64)
Consult /var/lib/dkms/veeamsnap/5.0.2.4567/build/make.log for more information.
Error! One or more modules failed to install during autoinstall.
Refer to previous errors for more information.
dkms: autoinstall for kernel: 6.1.0-28-amd64 failed!
run-parts: /etc/kernel/postinst.d/dkms exited with return code 11
dpkg: Fehler beim Bearbeiten des Paketes linux-image-6.1.0-28-amd64 (--configure):
 »installiertes post-installation-Skript des Paketes linux-image-6.1.0-28-amd64«-Unterprozess gab den Fehlerwert 1 zurück

So my current guess would be that you are using Veeam as backup solution (or used to at some point) and have installed the veeamsnap module is having issues with being rebuilt for the newer kernel.

While I’m not using Veeam, it looks like you will have to update or at least temporarily remove these components so that you can fix the configuration step of the Linux kernel 6.1.0-28.

This KB article from Veeam cam up pretty quickly, this snippet might be helpful: “Veeamsnap module is supported up to Linux kernel 5.18” (See: KB2804: Veeam Agent for Linux - veeamsnap and blksnap Extended Linux Distribution Support)

So yes, veeamsnap will not work with this newer kernel as per their documentation and you’d have to replace it by blksnap. But at this point it might just be easier to at least temporarily uninstall the Veeam agent and then attempt configuring the kernel package.

Keep in mind that there are other issues that I spotted due errors in config files of PHP and FreeRADIUS, but these error don’t seem to block the upgrade process (yet).

[toko42]

Wonderful! Thank you very much!

I am indeed using Veeam (Agent) which I have installed manually but as every other upgrade worked fine, I did not connect this.

I am not aware of having tempered with univention-bind/univention-role-server-common, though.

I’ll make a new (last) backup with Veeam, uninstall it and try again from there but probably not before the next weekend.

[msi]

Good to hear, it made sense to you. I decided to share the full stream of thought instead of just pointing out the culprit with the idea to provide how I drilled down.

Initially it was some guessing based on the initial thread but once I had some thing to “follow after” it was “quite easy”.

I think it’s actually quite unlikely, that you tempered with univention-bind or univention-role-server-common. The core issue is rather that there is a long chain of dependencies from the individual kernel package that lead far up to pretty big meta packages that end up failing because the kernel package fails to configure.

Depending on how you installed the veeamsnap module, you might just be able to uninstall some Veeam-specific packages (i.e. using apt search veeam and then purge them using apt purge veeamxyz, or you might have to fiddle with DKMS directly if it was installed completely manually.

[toko42]

Hello again,

uninstallation of Veeam has not been sufficient for the upgrade to succeed.

Right now, I think it would be easier to just set up a new server and join it to the domain as backup DC.

As far as I understand it, this would result in all users and also Nextcloud to be available on that server, also.

If this is successfull, I can upgrade the new installation, retire the old server and make the new one primary DC.

If my train of thoughts is correct: How can this be achieved?
Until now the one VM has been sufficient for my home operation.

Thank you very much for your support.

[dzidek23]

I suspect this won’t work (might be wrong, tho)

You join a server to the domain, you get a copy of problematic database (pkgdb).
If you promote this server you still will have problems upgrading.

Have you tried to remove the univention-bind package? At this stage you have little to lose if you do it

[toko42]

I have removed univention-bind, also but the upgrade failed again.

If a new server instance with Nextcloud move and domain takeover does not work, I will have to set up completely new.

I have found some documentation on how to do the first option but would appreciate if someone could have a look if I am missing something when following those:

How To: backup2master

This article is written for UCS 4. It uses the old UCS Role Names (Master, Backup, Slave, Member). The general steps should be the same when used for UCS 5, but they might differ in some places. There might come a time when you want to replace the UCS Master server for another one. It might be that the old Master is still using the 32bit architecture or it might broke in some way. Whatever the reason, we can setup a UCS Backup to take over its role in the domain. The most distinguishing featur…

How to migrate Nextcloud & OpenProject to new server?

Hello, We have just purchased a new dedicated box, we would like to migrate our whole setup to the new server but I have not been able to find any guides on doing this? So far I have just taken a snapshot of the old server and installed it on the new server, but that kind of defeats the purpose of having this new server since we wanted to try and set up a real domain instead of just having everything on one server… So to reiterate my question: How do I migrate app with all their data to a ne…

Thank you very much for your support and have a nice weekend.

[msi]

I wasn’t able to keep up with this thread but: If you’d provide the new logs, we might be able to provide some pointers. It might be that some packages still remain in unconfigured state.

Sometimes packages that are left behind because if previous issues can be fixed with dpkg --configure --pending (or dpkg --configure -a which is the same).

[toko42]

I would have to roll back the VM to an earlier production point.

As I have fostered this installation from UCS v2.0 on, I guess this is just the point where remnants of older/deprecated packages (open-vm-tools, veeam 5 etc etc) are just too much of a burden and would take much time to figure out what else there might be to removed.

I’ll try to set up the 2nd/backup server in the course of the next weeks and follow the linked docs.

Thank you very much.

[toko42]

I’ve done the comparison as described in the links above.
The total lists are in the attached PDF:

UCS-Diff_Packages.pdf (438.5 KB)

The main differences are the following, I’ve listed the packages missing on the new/backup server and would be grateful if someone could tell me which ones are necessary before going further.

Thank you very much!

There were older versions of some packages in the original server → ignored.

binutils
binutils-common:amd64
binutils-x86-64-linux-gnu
bridge-utils
bsd-mailx
build-essential
clamav-base
clamav-daemon
clamav-freshclam
dkms
emacs
ethtool
fakeroot
firmware-amd-graphics
firmware-linux
firmware-linux-nonfree
firmware-misc-nonfree
fuse
g++
g+±8
galera-3
gcc
gcc-6
gcc-6-base:amd64
gcc-7-base:amd64
gcc-8
gconf2-common
gdisk
gnupg-agent
gnupg1
gnupg1-1curl
gnupg1-l10n
gnutls-bin

kopano* (not necessary anymore)

??? lib* can be installed when needed?

libalgorithm-diff-perl
libalgorithm-diff-xs-perl
libalgorithm-merge-perl
libapt-inst1.5:amd64
libapt-pkg4.12:amd64
libatomic1:amd64

lots of librte* in the new system which are not in the old one

linux-headers*
linux-image-4.x (installed are 5.1*)

manpages*

maridadb-client* → would have thought that this would be installed if needed
mariadb-server* → would have thought that this would be installed if needed

mrt
multiarch-support
nftables
openbox
openssl-blacklist
opsi-server-full
opsi-tfptd-hpa
opsi-utils
opsiconfd
opsipxeconfd
per-openssl-defualts:amd64

php5*
php7.0*

python* - > various packages, can be installed if needed?

qrencode
redis-server
redis-tools
runit
runit-helper
runit-systemd
sa-compile
telnet

univention-antivir-mail
univention-ftp
univention-ifplugd
univention-management-console-module-mrtg
univention-management-console-module-welcome

univention-mariadb → would have thought that this would be installed if needed
univention-mysql → would have thought that this would be installed if needed

univention-runit
univention-sasl
univention-skel
univention-spamassassin
univention-virtual-maschine-manager-schema

vlan
w3m
zerofree

[msi]

So that’s a comparison between a newly-installed system (right side) and your old one (left side)?

You can basically ignore all packages with the status rc as it means that they are uninstalled but have residual cconfig files left on the file system. But they are properly uninstalled.

If you want to get right of them, you can refer to this page here and use this to get the list of these packages

# Get the list of 'rc' package and print it
pkg_list=$(dpkg --list | awk '$1 == "rc" {print $2}')
echo $pkg_list

# .. and finally purge them
sudo apt --purge remove $pkg_list

[toko42]

I just want to make sure that all necessary packages are installed on the new server which I will make master following the linked documentation above.

So the old server will just retire (unless something goes wrong, then I’ll just turn of the VM of the new one).

So next step is

• UCR variables comparison
• Nextcloud backup and restore on the new VM.

Thank you very much.

[toko42]

I have compared the UCR variables and amended some.

The remaining are imho ok as they are either obsolete (Kopano, OPSI, Jitsi Meet…) or should have the correct values as this is a new installation.

UCR-Diff.pdf (239.8 KB)

Next step according to doc is the LDAP check.

[toko42]

So I had time to do the LDAP schema check which has not turned out promisingly as all of the univention-ldap schemas are missing on the new/backup machine

Unfortunately the WinMerge printout is capped in width, so I copied the missing lines below:

UCS-Diff_LDAP.pdf (293.5 KB)

include /etc/ldap/schema/core.schema
include /etc/ldap/schema/cosine.schema
include /etc/ldap/schema/nis.schema
include /etc/ldap/schema/inetorgperson.schema
include /etc/ldap/schema/ppolicy.schema
include /usr/share/univention-ldap/schema/samba.schema
include /usr/share/univention-ldap/schema/mail.schema
include /usr/share/univention-ldap/schema/user.schema
include /usr/share/univention-ldap/schema/directory.schema
include /usr/share/univention-ldap/schema/policy.schema
include /usr/share/univention-ldap/schema/dnszone.schema
include /usr/share/univention-ldap/schema/univention.schema
include /usr/share/univention-ldap/schema/lock.schema
include /usr/share/univention-ldap/schema/custom-attribute.schema
include /usr/share/univention-ldap/schema/krb5-kdc.schema
include /usr/share/univention-ldap/schema/dhcp.schema
include /usr/share/univention-ldap/schema/portal.schema
include /usr/share/univention-ldap/schema/univention-dhcp.schema
include /usr/share/univention-ldap/schema/univention-default.schema
include /usr/share/univention-ldap/schema/license.schema
include /usr/share/univention-ldap/schema/share.schema
include /usr/share/univention-ldap/schema/printer.schema
include /usr/share/univention-ldap/schema/automount.schema
include /usr/share/univention-ldap/schema/network.schema
include /usr/share/univention-ldap/schema/solaris.schema
include /usr/share/univention-ldap/schema/courier.schema
include /usr/share/univention-ldap/schema/univention-scalix.schema
include /usr/share/univention-ldap/schema/univention-syntax.schema
include /usr/share/univention-ldap/schema/admin-settings.schema
include /usr/share/univention-ldap/schema/template.schema
include /usr/share/univention-ldap/schema/univention-ldap-acl.schema
include /usr/share/univention-ldap/schema/nagios.schema
include /usr/share/univention-ldap/schema/univention-directory.schema
include /usr/share/univention-ldap/schema/univention-objecttype.schema
include /usr/share/univention-ldap/schema/msgpo.schema
include /usr/share/univention-ldap/schema/univention-object-metadata.schema
include /usr/share/univention-ldap/schema/univention-ldap-extension.schema
include /usr/share/univention-ldap/schema/udm-extension.schema
include /usr/share/univention-ldap/schema/univention-data.schema
include /usr/share/univention-ldap/schema/blocklist.schema

SAML schema

include /usr/share/univention-saml/schema/univention-saml.schema

include /var/lib/univention-ldap/local-schema/jitsimeet.schema
include /var/lib/univention-ldap/local-schema/kopano4ucs.schema
include /var/lib/univention-ldap/local-schema/msgpipsec.schema
include /var/lib/univention-ldap/local-schema/msgpsi.schema
include /var/lib/univention-ldap/local-schema/msgpwl.schema
include /var/lib/univention-ldap/local-schema/msprintconnectionpolicy.schema
include /var/lib/univention-ldap/local-schema/mswmi.schema
include /var/lib/univention-ldap/local-schema/networkaccess.schema
include /var/lib/univention-ldap/local-schema/nextcloud.schema
include /var/lib/univention-ldap/local-schema/openid-connect-provider.schema
include /var/lib/univention-ldap/local-schema/self-service-passwordreset.schema
include /var/lib/univention-ldap/local-schema/univention-app.schema
include /var/lib/univention-ldap/local-schema/univention-monitoring.schema
include /var/lib/univention-ldap/local-schema/univention-portal.schema
include /var/lib/univention-ldap/local-schema/univention-virtual-machine-manager.schema

How can I backup/restore this?

I’ve found a guide here:

Backup und Wiederherstellung eines UCS-Systems

Dieser Artikel beschreibt, welche Daten eines UCS-Systems gesichert werden müssen. Im zweiten Teil ist die Wiederherstellung eines UCS-Systems dokumentiert. Sicherung eines UCS-Systems UCS folgt wie Debian dem Filesystem Hierarchy Standard, einer Richtlinie, die die Aufteilung der Daten in der Unix-Dateisystemhierarchie beschreibt. Dies erlaubt eine effiziente Trennung zwischen lokal erzeugten Dateien und Daten, die im Rahmen der Installation erstellt werden. Sicherung der LDAP-Daten Die Sic…

but that seems to include some major changes to the system.

All users and computers are present on the backup system, so in essence it would suffice for me to import my encrypted Nextcloud data (as there are some non domain users and their data) and promote the new system to master (since Kopano was dropped, I use the system only as AD domain controller).

Thanks for your support.