The following write-up is based on notes taken during a Barcamp session at the Univention Summit 2018.
Thanks to all contributors, especially @Moritz_Bunkus
Step 1: Prepare and clean up!
RTFRN (Read The Fine Release Notes) at https://docs.software-univention.de/ !
This does not just mean read but also try to understand and follow the instructions. In case of doubt ask here or - if you have a support contract - the friendly support team at Univention.
Recent Release have a separate Chapter for the preparation of an upgrade by using the script the updater process will run. It is highly recommended to run the script (pre-update-checks-*) in preparation of the upgrade and check for pitfalls.
Check for the availability of the installed Apps in your domain and how an upgrade of a domain controller could be affected even if the application is installed an another system.
Take a backup, check the backup, make sure that it is possible to restore the complete system from the backup in case you need it.
When running UCS as a virtual machine you can consider taking a snapshot, but be aware that snapshotting a running system with multiple Samba 4 systems in the environment is not recommended as it can easily lead into a situation where an older state of the AD-database is restored on one system. The result could be a so called “USN rollback” with a stuck replication (https://wiki.samba.org/index.php/Directory_Replication_Service_(DRS)) or even worse stuff.
To avoid this, consider to snapshot the environment while the Samba-DCs are shut down.
You may have already seen that UCS is taking a backup of the LDAP-database and all UCR-variables into /var/univention-backup/ every night. This is useful but may not reflect changes in between. It might be a good idea to do an actual dump of the LDAP and the UCR immediately before starting the backup.
Example:
slapcat > ldapdump-$(date +"%Y%m%d%H%M").ldif
ucr dump > ucrdump-$(date +"%Y%m%d%H%M").txt
In addition you could copy the tree below/etc to another location. This will enable you to compare and review configuration files from before and after the upgrade.
Watch out for *.dpkg-new files in /etc and especially the output of univention-check-templates. If the latter shows any modified templates try to clean up before the upgrade.
Check the list of packages for removed packages with leftover configuration files and remove them
Example:
# list
dpkg -l | grep ^rc
# show what would be done during purge
dpkg -l | grep ^rc | awk ' { print $2 } ' | xargs dpkg --no-act -P
# purge
dpkg -l | grep ^rc | awk ' { print $2 } ' | xargs dpkg -P
Check /etc/apt/sources.list* for active 3rd-party repositories. Make sure those repositories available for newer UCS versions or see, if you can disable them.
Consider to reboot the system directly before the upgrade.
Step 2: do the upgrade
When running the upgrade remotely though ssh with univention-upgrade it is highly recommended to use a terminal multiplexer like screen (https://en.wikipedia.org/wiki/GNU_Screen) or tmux to prevent a stuck upgrade because of network outages.
screen et al. will also allow you to follow the log (less +F /var/log/univention/updater.log) and check whats currently happening (e.g. pstree) by using multiple terminal windows.
Please note the some errors shown in the log may be temporary ones.
During the upgrade you should not try to join other UCS systems.
The command-line tool univention-upgrade may also offer to upgrade installed apps if applicable. There is no rule of thumb whether to upgrade them directly after an UCS-upgrade. If possible try to split upgrade tasks and do the app-upgrade afterwards in a separate step.
Once the upgrade is successfully completed, reboot the system.
Step 3: Check
When accessing the UMC for the first time after an upgrade it might be a good idea to do a complete reload the page in your browser or clear the cache.
Use the “system diagnostics” UMC module for the first check.
On the command line use univention-check-join-status and eventually also univention-check-templates
Once again check the package status by using dpkg -C.
There is also a command-line version of the system check available which - of course - is useful in other situations too:
curl -OOs https://updates.software-univention.de/download/univention-system-check/univention-system-check.tar.gz{,.gpg}
gpgv \
--keyring /usr/share/keyrings/univention-archive-key-ucs-4x.gpg \
univention-system-check.tar.gz.gpg \
univention-system-check.tar.gz \
&& tar -xzf univention-system-check.tar.gz \
&& python ./univention-system-check
Have Fun