Bug 52834 - univention-updater: univention-list-installed-unmaintained-packages finds "unmaintained" packages on new UCS5 installations
univention-updater: univention-list-installed-unmaintained-packages finds "un...
Status: CLOSED FIXED
Product: UCS
Classification: Unclassified
Component: Update - univention-updater
UCS 5.0
Other Linux
: P5 normal (vote)
: UCS 5.0-1
Assigned To: Philipp Hahn
Erik Damrose
:
: 53241 (view as bug list)
Depends on: 52715
Blocks: 54263
  Show dependency treegraph
 
Reported: 2021-02-23 10:30 CET by Christian Castens
Modified: 2021-12-20 09:35 CET (History)
8 users (show)

See Also:
What kind of report is it?: Bug Report
What type of bug is this?: 3: Simply Wrong: The implementation doesn't match the docu
Who will be affected by this bug?: 5: Will affect all installed domains
How will those affected feel about the bug?: 2: A Pain – users won’t like this once they notice it
User Pain: 0.171
Enterprise Customer affected?:
School Customer affected?:
ISV affected?:
Waiting Support:
Flags outvoted (downgraded) after PO Review:
Ticket number:
Bug group (optional):
Max CVSS v3 score:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Christian Castens univentionstaff 2021-02-23 10:30:00 CET
In its current state the tool "univention-list-unmaintained-installed-packages" finds unmaintained packages on new UCS5 installations. The tool has to be adjusted so it does not have that false behaviour anymore.
Comment 2 Felix Botner univentionstaff 2021-02-23 10:40:17 CET
(In reply to Christian Castens from comment #0)
> In its current state the tool
> "univention-list-unmaintained-installed-packages" finds unmaintained
> packages on new UCS5 installations. The tool has to be adjusted so it does
> not have that false behaviour anymore.

No, not the tool,

(a) update the maintained list

and/or

(b) prevent the installation of these packages

see also https://jenkins.knut.univention.de:8181/job/UCS-5.0/job/UCS-5.0-0/job/CheckMaintainedList/

currently the tools find these packages

 libcephfs2 rrdtool ifplugd notification-daemon libradcli4
  liblouisutdml-bin python-sqlalchemy librte-gro18.11
  librte-bbdev18.11 librte-rawdev18.11 libgconf-2-4
  librte-power18.11 gsfonts libgrpc++1 librte-ring18.11
  librte-mempool18.11 libgfxdr0 libhiredis0.14 libdbi1
  libboost-thread1.67.0 liblouisutdml8 libc-ares2
  librte-kvargs18.11 librte-table18.11 librte-acl18.11
  dpt-i2o-raidutils libatasmart4 librte-ip-frag18.11
  librte-security18.11 libxencall1 libieee1284-3
  libnet-snmp-perl librte-efd18.11 libopenipmi0
  default-jre-headless libibverbs1 librte-latencystats18.11
  libboost-regex1.67.0 librte-eal18.11 librte-cryptodev18.11
  libsnmp-session-perl python3-sqlalchemy
  univention-admingrp-user-passwordreset libglusterfs0
  univention-nagios-squid libyajl2 policycoreutils libftdi1-2
  memtest86+ libprotobuf17 libxengnttab1 librte-reorder18.11
  owfs-common libesmtp6 librte-member18.11 libowcapi-3.2-3
  librte-compressdev18.11 libxentoollog1 libxenevtchn1
  libxenforeignmemory1 librte-lpm18.11 libboost-atomic1.67.0
  librabbitmq4 libfile-copy-recursive-perl librte-hash18.11
  libtokyocabinet9 libow-3.2-3 libxentoolcore1
  ibverbs-providers libvarnishapi2 librte-pipeline18.11
  univention-nagios-s4-connector libmosquitto1 java-common
  librte-bpf18.11 libgphoto2-6 librte-pdump18.11 libnss-mdns
  libsane libvirt0 univention-nagios-samba
  librte-flow-classify18.11 librte-bitratestats18.11
  intel-cmt-cat librrd8 librte-kni18.11 libconfuse2
  python-sqlalchemy-ext python3-markdown libnotify4
  gconf2-common librte-ethdev18.11 python3-sqlalchemy-ext
  libupsclient4 libxenstore3.0 libgusb2 librte-timer18.11
  libgphoto2-l10n librte-sched18.11 librte-port18.11 sysstat
  libxenmisc4.11 libexif12 librte-gso18.11
  univention-nagios-cups libgrpc6 libdaemon0 librte-pci18.11
  libconfuse-common openjdk-11-jre-headless libgfrpc0
  monitoring-plugins-common univention-nagios-client liboping0
  libtokyotyrant3 nagios-nrpe-server python-egenix-mxdatetime
  libgphoto2-port12 libtcmalloc-minimal4 libmodbus5
  librdkafka1 sane-utils python-egenix-mxtools
  monitoring-plugins-standard libgoogle-perftools4 librados2
  libsane-common libboost-iostreams1.67.0 ca-certificates-java
  univention-nagios-common librte-distributor18.11
  libmicrohttpd12 libboost-system1.67.0 librte-vhost18.11
  librte-metrics18.11 monitoring-plugins librte-eventdev18.11
  libriemann-client0 librte-meter18.11 libmcrypt4
  librte-jobstats18.11 librte-telemetry18.11 libganglia1
  libi2c0 librte-net18.11 librte-cmdline18.11
  libxendevicemodel1 librte-cfgfile18.11 libgps23
  monitoring-plugins-basic liblouisutdml-data librte-mbuf18.11
  libgfapi0

There are some package (e.g. univention-nagios-common) that should no longer be installed, once we removed these package we can decide what to do with the rest
Comment 4 Felix Botner univentionstaff 2021-04-15 13:29:39 CEST
There are still some "unmaintained" package on a new installation

-> univention-list-installed-unmaintained-packages 
The following packages are unmaintained:
 gconf2-common gsfonts ifplugd libdaemon0 libdbi1 libfile-copy-recursive-perl libgconf-2-4 libmcrypt4 libnet-snmp-perl libradcli4 libsnmp-session-perl
 linux-image-4.19.0-14-amd64 memtest86+
Comment 5 Philipp Hahn univentionstaff 2021-04-17 19:44:54 CEST
(In reply to Felix Botner from comment #4)
> There are still some "unmaintained" package on a new installation
> 
> -> univention-list-installed-unmaintained-packages 
> The following packages are unmaintained:
> gconf2-common gsfonts ifplugd libdaemon0 libdbi1
> libfile-copy-recursive-perl libgconf-2-4 libmcrypt4 libnet-snmp-perl
> libradcli4 libsnmp-session-perl
> memtest86+

Not yet checked.

>  linux-image-4.19.0-14-amd64

This will happen with each kernel update with bumps the API counter. The ISO will ever contain the latest image. Depending on if a newer ISO has been built with a newer Linux kernel it will either list the OLD or NEW kernel as then unmaintained.
Comment 6 Philipp Hahn univentionstaff 2021-04-21 13:20:02 CEST
(In reply to Philipp Hahn from comment #5)
> (In reply to Felix Botner from comment #4)
> > There are still some "unmaintained" package on a new installation
> > 
> > -> univention-list-installed-unmaintained-packages 
> > The following packages are unmaintained:
> > gconf2-common gsfonts ifplugd libdaemon0 libdbi1
> > libfile-copy-recursive-perl libgconf-2-4 libmcrypt4 libnet-snmp-perl
> > libradcli4 libsnmp-session-perl
> > memtest86+

This happens because there exist some packages, which "Recommends" those packages. Those "Recommends" are NOT part of dependency resolution, which only follows "[Pre-]Depends" (the historical reason for this to minimize the ISO size).

As USS sometimes runs AFTER the online repository has already been enabled (this is the case for our internal KVM images) if any of those "recommending" packages are installed they will pull in those "Recommended" packages too as they are now available from our online repository: There we no longer distinguish between maintained and unmaintained, so all packages from Debian are available from there.

All of those packages can be removed later one without breaking any UCS functionality:
  univention-list-installed-unmaintained-packages |
  tail -n +2 |
  xargs apt purge --assume-yes
Comment 7 Philipp Hahn univentionstaff 2021-06-03 12:48:50 CEST
*** Bug 53241 has been marked as a duplicate of this bug. ***
Comment 8 Philipp Hahn univentionstaff 2021-06-03 15:03:57 CEST
base/univention-server/debian/control: Package:univention-role-common only RECOMMENDS the following packages:
...
203 Recommends:
204  cifs-utils,                 base/univention-dvd/tasks/ucs500/task-ucs500:133
205  eject,                      base/univention-dvd/tasks/ucs500/task-ucs500:44
206  elinks,                     base/univention-dvd/tasks/ucs500/task-ucs500:45
207  emacs-gtk,                  base/univention-dvd/tasks/ucs500/task-ucs500:46
208  less,                       base/univention-dvd/tasks/ucs500/task-ucs500:47
209  locate,                     base/univention-dvd/tasks/ucs500/task-ucs500:48
210  man-db,                     base/univention-dvd/tasks/ucs500/task-ucs500:58
211  nmap,                       base/univention-dvd/tasks/ucs500/task-ucs500:49
212  univention-network-common,  base/univention-dvd/tasks/ucs500/task-ucs500:50
213  univention-welcome-screen,  base/univention-dvd/tasks/ucs500/task-ucs500:18
214  unzip,                      base/univention-dvd/tasks/ucs500/task-ucs500:51
215  vim,                        base/univention-dvd/tasks/ucs500/task-ucs500:52
216  wget,                       base/univention-dvd/tasks/ucs500/task-ucs500:53
217  zip,                        base/univention-dvd/tasks/ucs500/task-ucs500:54
218  memtest86+,
219  lvm2,

Similar for gsfonts, which is indirectly pulled in when univention-server-master is installed:
gsfonts
+<-D- python[3]-renderpm
      +<-R- python[3]-reportlab
            +<-D- python[3]-trml2pdf
                  +<-D- python[3]-univention-directory-reports
                  |     +<-D- univention-directory-manager-rest
                  |           +<-D- univention-server-master
                  +<-D- univention-directory-reports
                        +<-D- univention-management-console-module-udm
                              +<-D- univention-server-master

base/univention-dvd/ucs500.conf:268 export NORECOMMENDS=1 prevents Recommends from being pulled on to the ISO image.
repo-apt-dependencies also does not pull in Recommends by default, see https://hutten.knut.univention.de/blog/wip-verandertes-verfahren-fur-maintained-paketei/

-> ADDED to base/univention-dvd/tasks/ucs500/task-ucs500


The following packages are installed by test/utils/base_appliance.sh:650 appliance_preinstall_non_univention_packages ONLY in our internal KVM images - not when installing from ISO:
- libsnmp-session-perl
- libnet-snmp-perl
- libradcli4
- libmcrypt4
- libgconf-2-4 → gconf2-common
- libdbi1
- libfile-copy-recursive-perl
- libdaemon0
- ifplugd
- emacs24 → [ emcas-gtk → libmagickcore-6.q16-6 ] → gsfonts

-> I have NO idea for what the function is / packages are needed and this only affects our INTERNAL KVM images.

@fbotner: Can you please have a look at:
  test/scenarios/appliances/generic-kvm-template.cfg:31  . base_appliance.sh && setup_appliance
  test/utils/base_appliance.sh: 799 »·······appliance_preinstall_non_univention_packages
< > Is appliance_preinstall_non_univention_packages() still needed or can we remove it
< > Or move the call to appliance_preinstall_non_univention_packages() from setup_appliance() into
      test/scenarios/app-appliance.cfg
      test/scenarios/appliances/ec2-appliance.cfg
      test/scenarios/appliances/ucs-appliance.cfg
    but NOT
      test/scenarios/appliances/generic-kvm-template.cfg


We shipped an out-dated univention-errata-level package with UCS-5.0-0+0 missing the following packages:
- python-unidecode
- python3-unidecode
- univention-management-console-module-welcome

-> NOTHING-MORE-TO-DO: The already shipped univention-errata-level update packages already include them


[5.0-0] 6b74662181 Bug #52124: Update DVD task list
 base/univention-dvd/debian/changelog         | 6 ++++++
 base/univention-dvd/tasks/ucs500/task-ucs500 | 8 +++-----
 2 files changed, 9 insertions(+), 5 deletions(-)

Package: univention-dvd
Version: 5.0.1-3A~5.0.0.202106031452
Branch: ucs_5.0-0
Scope: errata5.0-0

Nothing to release as univention-dvd is only pulled in internally for the next DVD build.
Comment 9 Felix Botner univentionstaff 2021-06-03 21:43:12 CEST
(In reply to Philipp Hahn from comment #8)
> @fbotner: Can you please have a look at:
>   test/scenarios/appliances/generic-kvm-template.cfg:31  . base_appliance.sh
> && setup_appliance
>   test/utils/base_appliance.sh: 799
> »·······appliance_preinstall_non_univention_packages
> < > Is appliance_preinstall_non_univention_packages() still needed or can we
> remove it
> < > Or move the call to appliance_preinstall_non_univention_packages() from
> setup_appliance() into
>       test/scenarios/app-appliance.cfg
>       test/scenarios/appliances/ec2-appliance.cfg
>       test/scenarios/appliances/ucs-appliance.cfg
>     but NOT
>       test/scenarios/appliances/generic-kvm-template.cfg
i am not completely sure, but i think this appliance_preinstall_non_univention_packages stuff is a historic relict (the idea was to speed up the setup process by providing packages that otherwise would be downloaded)
can be removed in my opinion, but we should ask eric
Comment 10 Erik Damrose univentionstaff 2021-06-04 09:13:53 CEST
Yes appliance_preinstall_non_univention_packages was introduced to speed up the install process, but the package list is outdated for UCS 5 anyway. Its okay to remove it for now.
Comment 11 Philipp Hahn univentionstaff 2021-06-04 14:13:49 CEST
[5.0-0] 3b0a9bee61 test: Skip appliance_preinstall_non_univention_packages
 test/utils/base_appliance.sh | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

To get rid of those packages listed as unmaintained we must build and release
- a new ISO image
- new KVM images
- new VirtualBox images
- new EC2 AMI

This will only happen earliest with the next point release UCS 5.0-1.

A new ISO already was built from the current errata scope:
<https://jenkins.knut.univention.de:8181/job/UCS-5.0/job/UCS-5.0-0/job/BuildDVD/198/>

That ISO already includes the missing packages `gsfonts` and `memtest86+`:
vimdiff <(isoinfo -f -R -i mirror/ftp/download/ucs-cds/ucs5.0-0/UCS_5.0-0-amd64.iso) <(isoinfo -f -R -i isotests/ucs_5.0-0-20210603-224506-dvd-amd64.iso)
Comment 12 Philipp Hahn univentionstaff 2021-12-10 18:09:02 CET
From @edamrose: After upgrading from 5.0-0 to 5.0-1 the following packages where shown as unmaintained:
 gconf2-common
 ifplugd
 libdaemon0
 libdbi1
 libfile-copy-recursive-perl
 libgconf-2-4
 libmcrypt4
 libnet-snmp-perl
 libradcli4
 libsnmp-session-perl

See my comment 6: our internal KVM images are dirty, e.g. 5.0-0 so any update will be dirty, too.
My new installation from UCS 5.0-1.iso was clean.
Comment 13 Erik Damrose univentionstaff 2021-12-13 19:18:21 CET
(In reply to Philipp Hahn from comment #12)
> See my comment 6: our internal KVM images are dirty, e.g. 5.0-0 so any
> update will be dirty, too.
> My new installation from UCS 5.0-1.iso was clean.

My mistake. It works as described.
-> Verified
Comment 14 Erik Damrose univentionstaff 2021-12-15 16:47:40 CET
UCS 5.0-1 has been released.

https://docs.software-univention.de/release-notes-5.0-1-en.html

If this error occurs again, please clone this bug.
Comment 15 Philipp Hahn univentionstaff 2021-12-20 08:17:17 CET
bin:libradcli1 from src:radcli and bin:libdbi1 from src:libdbi are still listed as unmaintained as triggers/ucs_5.0-0-ucs5.0-1.txt listed them by *source* package name instead of *binary* package name.

<https://jenkins.knut.univention.de:8181/job/Mitarbeiter/job/phahn/job/CalculateMaintained/191/console>

The next erratum upload for 5.0-1 will include this updated list.
Comment 16 Philipp Hahn univentionstaff 2021-12-20 08:22:40 CET
(In reply to Philipp Hahn from comment #15)
> bin:libradcli1 from src:radcli and bin:libdbi1 from src:libdbi are still
> listed as unmaintained as triggers/ucs_5.0-0-ucs5.0-1.txt listed them by
> *source* package name instead of *binary* package name.

r83167 | Bug #52834: UCS 5.0-1 more maintained packages

> <https://jenkins.knut.univention.de:8181/job/Mitarbeiter/job/phahn/job/
> CalculateMaintained/191/console>
> 
> The next erratum upload for 5.0-1 will include this updated list.