Univention Bugzilla – Bug 42450
Empty sources.list may result in unwanted package operations
Last modified: 2016-09-23 13:53:18 CEST
We found in /etc/apt/sources.list.d/15_ucs-online-version.list # An error occurred during the repository check. The error message: # Traceback (most recent call last): # File "<stdin>", line 13, in <module> # File "/usr/lib/pymodules/python2.7/univention/updater/tools.py", line 579, in __init__ # self.ucr_reinit() # File "/usr/lib/pymodules/python2.7/univention/updater/tools.py", line 660, in ucr_reinit # assert self.server.access(None, '') # File "/usr/lib/pymodules/python2.7/univention/updater/tools.py", line 488, in access # raise ConfigurationError(uri, reason) # ConfigurationError: Configuration error: host is unresolvable Only 4.1-3-errata was included in 20_ucs-online-component.list This resulted in the following apt-get operation set: Statusinformationen werden eingelesen.... Die folgenden Pakete werden ENTFERNT: univention-management-console univention-management-console-module-setup univention-server-master Die folgenden NEUEN Pakete werden installiert: univention-nagios-samba Die folgenden Pakete sind zurückgehalten worden: python-univention-appcenter Die folgenden Pakete werden aktualisiert (Upgrade): libidn11 libpython2.7 libxslt1.1 python-univention-connector-s4 [...] The problem came from added dependencies in univention-management-console-module-setup and python-univention-appcenter in the latest 4.1-3 errata updates. These dependencies could not be found and thus the packages were not installed / had to be removed. The updater should not operate under these conditions. Also, the sources.list should rewrite itself. We got feedback that later on, the error messages disappeared.
Some of 37146,26432,28853,38373,38810 might be duplicates.
While the updater may stop operating when the file is (more or less) empty, one could also just write the sources.list without ever hitting the server. Just iterate over ["4.0", "4.1"] and write it down. Would also greatly reduce the amount of time it takes to commit these files. Also applies to 20_ucs-online-component.list. The App Center sometimes struggles with this file as it takes very long. There seems to be no benefit in hitting the server and telling that the host is unresolvable. Only if there was some kind of rollback to the last "correct" version of the file (or even a fallback), it would make sense to test the configuration.
It seems to affect a lot of testers.
(In reply to Stefan Gohmann from comment #3) > It seems to affect a lot of testers.
(In reply to Dirk Wiesenthal from comment #2) > While the updater may stop operating when the file is (more or less) empty, > one could also just write the sources.list without ever hitting the server. Wrong: The updater (currently) has now knowledge, which previous releases exist, that is 4.1-3 needs 4.1-2 4.1-1 4.1-0 4.0-5 \ 4.0-4 | 4.0-3 + those are found only by iterating through all existing releases 4.0-2 | 4.0-1 / 4.0-0 I see two options: 1. we could hard-code the previous releases, as they are known at the point of releasing 2. we should create a single /index.json with all UCS release information (including expiry date of standard support). That way a single fetch would be sufficient and the UMC frontend could warn about expired releases. (there alredy exists <http://updates.software-univention.de/download/ucs-maintenance/>, but 1) YAML is not browser friendly, 2) it's one file per patch-level
The UMC warning about releases is planned, see the technical concept here: https://mail.univention.de/appsuite/ui#!!&app=io.ox/office/text&folder=1117&id=1117/1025 We have already implemented the repository restrictions in our updater lib. I guess we will lost some functionality if we add a static entry. And we need to change the version if we release a patchlevel release after the next minor release, see 4.0-5 and 4.1-0. What about using the old sources.list entry if a traceback happened? Philipp, you said something like that today? I think that would avoid this issue.
(In reply to Philipp Hahn from comment #5) > (In reply to Dirk Wiesenthal from comment #2) > > While the updater may stop operating when the file is (more or less) empty, > > one could also just write the sources.list without ever hitting the server. > > Wrong: > The updater (currently) has now knowledge, which previous releases exist, > that is 4.1-3 needs > 4.1-2 > 4.1-1 > 4.1-0 > 4.0-5 \ > 4.0-4 | > 4.0-3 + those are found only by iterating through all existing releases > 4.0-2 | > 4.0-1 / > 4.0-0 > Ah, right. > 2. we should create a single /index.json with all UCS release information > (including expiry date of standard support). That way a single fetch would > be sufficient and the UMC frontend could warn about expired releases. > (there alredy exists > <http://updates.software-univention.de/download/ucs-maintenance/>, but 1) > YAML is not browser friendly, 2) it's one file per patch-level Not necessarily. In fact, we could add one (extensive) file at http://updates.software-univention.de/index.json and put every release in there. 4.1-3 would know about 4.0-5 (and eventually even 4.2-0 - but skip it during the ucr commit). One would ship one initial file in univention-updater locally, so that an internet connection is not necessary during the initial setup. (In reply to Stefan Gohmann from comment #6) > We have already implemented the repository restrictions in our updater lib. > I guess we will lost some functionality if we add a static entry. Restriction functionality? So it will result in a lot of error messages if 4.0-6 is added without permission? This would indeed be a problem. Depending on the complexity of the authentication / authorization mechanism, a simple "extended_support_only": true in the json could be enough.
(In reply to Dirk Wiesenthal from comment #7) > (In reply to Stefan Gohmann from comment #6) > > We have already implemented the repository restrictions in our updater lib. > > I guess we will lost some functionality if we add a static entry. > > Restriction functionality? So it will result in a lot of error messages if > 4.0-6 is added without permission? This would indeed be a problem. Depending > on the complexity of the authentication / authorization mechanism, a simple > "extended_support_only": true in the json could be enough. No, the updater has to check every release separately if username and password are required. I don't think that we talk about the root cause of the error. I guess the same issue would have been occurred if we use a json file. The 15sources.list is generated during the system setup. It looks like bind or the forwarder wasn't ready at that point. That would have been the same issue if we use a json file. The 20sources.list is re-created every time the updater checks for an update. This is at least once a day but I guess it happens more often. My suggestion is to re-create the 15sources.list also every time if it is empty or it contains only a traceback and an existing 15sources.list should not be overwritten with an empty file or a file which contains only a traceback.
r72760 | Bug #42450 up: Assert valid /etc/apt/sources.list.d/15_ucs-online-version.list Add Pre-Invoke checks Package: univention-updater Version: 11.0.11-9.1491.201609221623 Branch: ucs_4.1-0 Scope: errata4.1-3 r72761 | Bug #42450 up: Assert valid /etc/apt/sources.list.d/15_ucs-online-version.list YAML univention-updater.yaml QA: repo=$(dig +short $(ucr get repository/online/server | sed -re 's,https?://,,;s,/.*,,')|tail -n1) ip route add prohibit "$repo" ucr commit /etc/apt/sources.list.d/15_ucs-online-version.list apt-get update # will fail ip route del "$repo" apt-get update # will succeed ip route add prohibit "$repo" ucr commit /etc/apt/sources.list.d/15_ucs-online-version.list apt-get remove vim # will fail ip route del "$repo" apt-get remove vim # will fail ucr commit /etc/apt/sources.list.d/15_ucs-online-version.list apt-get remove vim # will succeed
Code review: OK Merge to UCS 4.2: Fail, I've cloned it into Bug #42481 Tests: OK YAML: OK ucs-test: OK
<http://errata.software-univention.de/ucs/4.1/277.html>