Bug 47192 - very old online repositories still included in 15_ucs-online-version.list
very old online repositories still included in 15_ucs-online-version.list
Status: CLOSED FIXED
Product: UCS
Classification: Unclassified
Component: Update - univention-updater
UCS 4.4
Other Linux
: P5 normal (vote)
: UCS 5.0
Assigned To: Philipp Hahn
Jürn Brodersen
:
: 48832 49279 (view as bug list)
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2018-06-14 13:25 CEST by Moritz Bunkus
Modified: 2021-05-25 15:59 CEST (History)
6 users (show)

See Also:
What kind of report is it?: Bug Report
What type of bug is this?: 5: Major Usability: Impairs usability in key scenarios
Who will be affected by this bug?: 5: Will affect all installed domains
How will those affected feel about the bug?: 2: A Pain – users won’t like this once they notice it
User Pain: 0.286
Enterprise Customer affected?: Yes
School Customer affected?:
ISV affected?:
Waiting Support:
Flags outvoted (downgraded) after PO Review:
Ticket number: 2018020121000903
Bug group (optional): API change, Cleanup, Roadmap discussion, Security, UCS Performance
Max CVSS v3 score:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Moritz Bunkus 2018-06-14 13:25:38 CEST
If the online repositories are enabled, the file 15_ucs-online-version.list is generated from all available repositories for all version numbers that share the same major version number but aren't higher than the system's current version. For example, on a UCS 4.3-0 system you'll find the repositories for the 4.0-0, 4.0-1, … 4.1-0, … 4.2-0, … 4.3-0 but not 4.3-1.

To me this seems rather wasteful: it wastes bandwidth, and more importantly a lot of time as all repos are updated each time you want to install something via the officially supported methods ("univention-install …").

UCS 4.0 and 4.1 are now two Debian releases removed from the current release, 4.3. Therefore it's more than doubtful those repositories will ever be used again on a system running 4.3.

The old method might have made sense back in the day when switching Debian releases also meant that UCS's major version number is increased, to (e.g. from 2.x to 3.0). However, nowadays that isn't the case anymore.

The reason this happens is:

• The template for 15_ucs-online-version.list calls "updater.print_version_repositories()"
• That function takes the current major version with minor 0 and patchlevel 0 as the default start (from tools.py):

        if not start:
            start = UCS_Version((self.version_major, 0, 0))

I suggest this be changed to either:

        if not start:
            start = UCS_Version((self.version_major, self.version_minor, 0))

which would only include repositories for the current major/minor combination, or maybe the following if one minor back should be included as well:

        if not start:
            minor_start = 0 if self.version_minor == 0 else self.version_minor - 1
            start = UCS_Version((self.version_major, minor_start, 0))

See e.g. the following Discourse thread:

https://help.univention.com/t/the-list-of-apt-repositories/8921

(not sure which component this might fit in)
Comment 1 Philipp Hahn univentionstaff 2018-06-14 13:33:32 CEST
We cannot change this within a major release, only for the next major release.
Actually this issue is currently discussed internally within Univention.

(In reply to Moritz Bunkus from comment #0)
> UCS 4.0 and 4.1 are now two Debian releases removed from the current
> release, 4.3. Therefore it's more than doubtful those repositories will ever
> be used again on a system running 4.3.

This is wrong as UCS-4.3 is not self-contained; it still uses packages from older minor releases.
Comment 2 Moritz Bunkus 2018-06-14 13:47:27 CEST
I see. Thanks for the clarification. I'll add a note to the aforementioned Discourse thread.
Comment 3 Philipp Hahn univentionstaff 2019-06-05 08:53:04 CEST
The Updater has to perform many HTTP requests, which already were rate-limited by certain providers (Hetzner, Amazon) as they were considered a Denial-of-Service attack.
This also just happened again at at customer size, where the HTTP-Proxy times out for certain requests leading to broken APT list. This even happened for the local mirror leading to all subsequent local system not getting errata updates.
Comment 4 Philipp Hahn univentionstaff 2019-06-05 08:53:08 CEST
*** Bug 48832 has been marked as a duplicate of this bug. ***
Comment 5 Philipp Hahn univentionstaff 2019-06-05 08:55:29 CEST
*** Bug 49279 has been marked as a duplicate of this bug. ***
Comment 6 Philipp Hahn univentionstaff 2019-08-07 09:55:03 CEST
Again:
 ucr search --brief --non-empty ^repository/online/component/4.[0-3]-[0-9]-errata|cut -d: -f1|xargs -r ucr unset
Comment 7 Ingo Steuwer univentionstaff 2020-07-06 16:29:50 CEST
We'll change the repository structure in UCS 5.0, should be fixed then.
Comment 8 Philipp Hahn univentionstaff 2021-03-11 14:49:19 CET
[phahn/52847ucr] ddf4620997 feat[up]: Remove deprecated errata components
 base/univention-updater/debian/changelog | 6 ++++++
 base/univention-updater/script/postup.sh | 6 ++++++
 doc/changelog/changelog-5.0-0.xml        | 3 +++
 3 files changed, 15 insertions(+)

Package: univention-updater
Version: 15.0.3-25A~5.0.0.202103111329
Comment 9 Jürn Brodersen univentionstaff 2021-03-23 19:17:04 CET
Old key=value are logged to updater.log -> OK
Tested 4.4-7-errata, 4.0-1-errata, 3.1-0-errata -> OK
Tested non ucs errata component -> not removed -> OK
changelog -> OK

-> verified
Comment 10 Florian Best univentionstaff 2021-05-25 15:59:07 CEST
UCS 5.0 has been released:
 https://docs.software-univention.de/release-notes-5.0-0-en.html
 https://docs.software-univention.de/release-notes-5.0-0-de.html

If this error occurs again, please use "Clone This Bug".