Unmaintained Repo - how can i undertand the new packages sources in UCS5?

Hello all,

i have upgraded my UCS Servers to 5. Now I would like to clean up the systems as well. First task: Remove all installed packages from the Unmaintained repository, if it is possible. Now I have seen that this repository has a new handling in UCS5.

https://docs.software-univention.de/release-notes-5.0-0-de.html#changlog:updater

The bug is fixed according to the bug tracker. But now I do not quite understand the whole thing. According to the documentation this variable is used:
ucr get repository/online/component/.*/unmaintained
But no matter if you turn it off or on here or in the web interface, nothing changes.
So I installed a new UCS5 and had a look. The fact is when you enter this command:

root@dc1:~# univention-list-installed-unmaintained-packages
The following packages are unmaintained:
 aspell aspell-en ca-certificates-java default-jre-headless i965-va-driver ibverbs-providers intel-cmt-cat intel-media-va-driver
 java-common libaom0 libaspell15 libatasmart4 libavcodec58 libavutil56 libboost-atomic1.67.0 libboost-iostreams1.67.0
 libboost-regex1.67.0 libboost-system1.67.0 libboost-thread1.67.0 libc-ares2 libcephfs2 libcodec2-0.8.1 libconfuse-common libconfuse2
 libcrystalhd3 libdata-dump-perl libencode-locale-perl libesmtp6 libfile-listing-perl libfont-afm-perl libftdi1-2 libganglia1 libgfapi0
 libgfrpc0 libgfxdr0 libglusterfs0 libgoogle-perftools4 libgps23 libgrpc++1 libgrpc6 libgsm1 libhiredis0.14 libhtml-form-perl
 libhtml-format-perl libhtml-tree-perl libhttp-cookies-perl libhttp-daemon-perl libhttp-message-perl libhttp-negotiate-perl libi2c0
 libibverbs1 libigdgmm5 libio-html-perl liblwp-mediatypes-perl liblwp-protocol-https-perl libmicrohttpd12 libmodbus5 libmosquitto1
 libmp3lame0 libnet-http-perl libnotify4 libopenipmi0 liboping0 libow-3.2-3 libowcapi-3.2-3 libprotobuf17 librabbitmq4 librados2
 librdkafka1 libriemann-client0 librrd8 librte-acl18.11 librte-bbdev18.11 librte-bitratestats18.11 librte-bpf18.11 librte-cfgfile18.11
 librte-cmdline18.11 librte-compressdev18.11 librte-cryptodev18.11 librte-distributor18.11 librte-eal18.11 librte-efd18.11
 librte-ethdev18.11 librte-eventdev18.11 librte-flow-classify18.11 librte-gro18.11 librte-gso18.11 librte-hash18.11 librte-ip-frag18.11
 librte-jobstats18.11 librte-kni18.11 librte-kvargs18.11 librte-latencystats18.11 librte-lpm18.11 librte-mbuf18.11 librte-member18.11
 librte-mempool18.11 librte-meter18.11 librte-metrics18.11 librte-net18.11 librte-pci18.11 librte-pdump18.11 librte-pipeline18.11
 librte-port18.11 librte-power18.11 librte-rawdev18.11 librte-reorder18.11 librte-ring18.11 librte-sched18.11 librte-security18.11
 librte-table18.11 librte-telemetry18.11 librte-timer18.11 librte-vhost18.11 libshine3 libsnappy1v5 libsoxr0 libspeex1 libswresample3
 libtcmalloc-minimal4 libtheora0 libtokyocabinet9 libtokyotyrant3 libtry-tiny-perl libtwolame0 libupsclient4 libva-drm2 libva-x11-2
 libva2 libvarnishapi2 libvdpau-va-gl1 libvdpau1 libvirt0 libvpx5 libwavpack1 libwww-perl libwww-robotrules-perl libx264-155
 libxencall1 libxendevicemodel1 libxenevtchn1 libxenforeignmemory1 libxengnttab1 libxenmisc4.11 libxenstore3.0 libxentoolcore1
 libxentoollog1 libxvidcore4 libyajl2 libzvbi-common libzvbi0 notification-daemon openjdk-11-jre-headless owfs-common python3-markdown
 rrdtool sysstat va-driver-all vdpau-driver-all wipe

So many unmaintained packages are installed on the system by default? How can I understand that?
It will be exciting on an updated system. If you remove all these packages, the system will not boot anymore, I have tested it.

root@app:~# univention-list-installed-unmaintained-packages

The following packages are unmaintained:
 dh-python galera-3 gcc-6-base git git-man irqbalance libapt-inst1.5
 libapt-pkg4.12 libavcodec57 libavutil55 libbind9-140 libboost-iostreams1.49.0 libboost-iostreams1.55.0 libcgi-fast-perl
 libconfig-inifiles-perl libcryptsetup4 libcrystalhd3 libdaemon0 libdbd-mysql-perl libdns-export162 libdns162 libenca0
 libencode-locale-perl libexempi3 libexif12 libfcgi-perl libgdbm3 libgsm1 libhtml-template-perl libhttp-message-perl libicu57
 libio-html-perl libiptcdata0 libisc-export160 libisc160 libisccc140 libisccfg140 libjson-c2 liblogging-stdlog0 liblognorm1
 liblwp-mediatypes-perl liblwres141 libmp3lame0 libnftables0 libnss-mdns libprocps0 libprocps3 libprocps6 libpython3.5-minimal
 libpython3.5-stdlib libqrencode4 libreadline6 libshine3 libsnappy1v5 libsoxr0 libspeex1 libssl1.0.0 libssl1.0.2 libswresample2
 libtheora0 libtracker-sparql-1.0-0 libtwolame0 libudev0 libunistring0 libustr-1.0-1 libva-drm1 libva-x11-1 libva1 libvdpau1 libvpx4
 libwavpack1 libwebpmux2 libx264-148 libx265-95 libxapian30 libxtables10 libxvidcore4 libzvbi-common libzvbi0
 linux-image-4.19.0-20-amd64-signed linux-image-4.19.0-21-amd64-signed linux-image-4.9.0-8-amd64 nftables openssl-blacklist
 perl-modules-5.24 pigz postgresql-client-9.6 python-imaging python-olefile python-repoze.lru
 python-routes python-support python-talloc python3-bs4 python3-html5lib python3-olefile python3-repoze.lru python3-routes
 python3-soupsieve python3-webencodings python3.5 python3.5-minimal qrencode runit-helper socat thin-provisioning-tools
 univention-mariadb

Here you can see e.g. also the postgresql-client in an old version. This was (for whatever reason) also not updated to v11.
Please enlighten me on how to properly deal with this now and also clean up my servers.

Very Thanks :slight_smile:

The .* is a regular expression pattern, which actually matches many UCR variables: For each component you will have multiple variables all with a common prefix repository/online/component/XXX/… where XXX is the component name. Each component has an individual unmaintained setting, which would be repository/online/component/XXX/unmaintained.

Before there also was UCRV repository/online/unmaintained, which no longer is used since UCS-5. With UCS-5 all packages — both maintained and unmaintained — live in the same repository. The distinctions if a package is maintained or unmaintained only happens after the fact: The file /usr/share/univention-errata-level/maintained-packages.txt list all names of binary package names we consider maintained; all other are unmaintained.

With UCS-5 we reset our internal list and started from scratch as the previous list contained to many packages for App, which are no longer relevant for us or names packages, which are now deprecated. As such several packages might be missing.
There is especially on unsolved problem: The list of maintained packages ignores several Recommended packages: In contrast to Depends they can be removed, but by default are also installed. When such packages are installed later on when repository/online=yes is already enabled, they might get picked up while they are not, when only the packages on the ISO image at available.
Another difference comes from the fact, that for the installation an X11 environment is needed to run the graphical Firefox Web browser. After the installation several of those packages belonging to the X11-server can be removed: For some reason some of them stay installed but are considered unmaintained.

The new mechanism is more flexible and it is easier for us to make additional packages maintained, as we just have to add them to the list and do another release on the package univention-errata-level containing that list. Just today we added several packages and that extended list will probably be shipped latest next Wednesday.

Looking at the list of packages you provided: Do not blindly remove all of them:

  • Start with apt autoremove and remove the now no longer required (library) packages: those which no required package any longer depends on.
  • Run univention-prune-kernels to remove old Linux kernel packages, which are no longer used.
  • If you still see unmaintained packaged try to remove library packages first, those that start with lib: Pick some of them and remove them via apt remove. Look closely at the list of extra packages, will will also be removed. If you see any important package in them — iptbales, grub , univention-…. — abort and try a smaller set or another set.
  • if still unmaintained packages remain please feel free to post that list here, so that we can investigate it and add them to our maintained list if that is the right option.
2 Likes

Hello Philipp and very thanks for your detailed answer. I must admit a few things are still clear to me, but it will be fine. I still have a few questions…

I have tested the whole thing because we are not quite clear. I did a md5sum of /etc before and after enabling “unmaintained” in the web interface.

find /etc -type f -exec md5sum {} + > md5sumetc.txt
# activate unmaintained in the webinterface
find /etc -type f -exec md5sum {} + > md5sumetc_after.txt

After that I compared files and noticed that the variable “repository/online/unmaintained” continues to be used.

root@dc1:~# diff md5sumetc_after.txt  md5sumetc.txt
289c289
< 58676099cace562fce415f620f99e33f  /etc/univention/base.conf
---
> 53e34feed01823c904b76ffd6ef24329  /etc/univention/base.conf
735c735
< 58676099cace562fce415f620f99e33f  /etc/univention/base.conf.bak
---
> 53e34feed01823c904b76ffd6ef24329  /etc/univention/base.conf.bak
root@dc1:~# nano /etc/univention/base.conf
root@dc1:~# diff  /etc/univention/base.conf etc-backup/univention/base.conf
307d306
< repository/online/unmaintained: on

And:

root@dc1:~# ucr get repository/online/unmaintained 
on

So the variable is still used after all?

Thanks for the list.

Noted. These packages are already all uninstalled after the installation of the Windows compatible domain controller.

Thank you

First remove line for unavailable and/or unused packages:
PHP 7.0
apt remove php7.0*
and others:
apt remove libnss-mdns avahi-* libavahi-core7 libdaemon0 openssl-blacklist nftables libssl1.0.2 libssl1.0.0 libavcodec57 libapt-inst1.5 libapt-pkg4.12 libboost-iostreams1.49.0 libboost-iostreams1.55.0 libcryptsetup4 libicu57 libisc-export160 libisc160 libisccc140 libjson-c2 liblogging-stdlog0 liblognorm1 liblwres141 libprocps0 libprocps3 libprocps6 libpython3.5-stdlib libtracker-sparql-1.0-0 libudev0 libunistring0 libwebpmux2 libxtables10 perl-modules-5.24 pigz python-support python-talloc python3.5 python3.5-minimal tcpdump wipe libpython3.5-minimal gcc-6-base

Just because the variable can (still) be defined does not make it used: Basically you can invent and define any variable you want as variables names need not be registered first before they can be used or set.
Your commands just show that the variable was changed and stored within the files /etc/univention/base*.conf*, but nothing else. Especially /etc/apt/sources.list.d/15_ucs-online-version.list was not updated and stayed unchanged - with UCS-4 it would change.

The toggle switch for unmaintained in UMC was just left behind by Bug #51316 because some external components still use the variable and we kept the button in the UI for that. It will be removed next time someone touches the code.

Mastodon