Bug 47271 - Feature request: Disable UCR for certain templates
Feature request: Disable UCR for certain templates
Status: NEW
Product: UCS
Classification: Unclassified
Component: UCR
UCS 4.4
Other other
: P5 normal with 2 votes (vote)
: ---
Assigned To: UCS maintainers
UCS maintainers
:
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2018-06-29 12:45 CEST by Michael Grandjean
Modified: 2023-10-26 12:03 CEST (History)
5 users (show)

See Also:
What kind of report is it?: Feature Request
What type of bug is this?: 4: Minor Usability: Impairs usability in secondary scenarios
Who will be affected by this bug?: 2: Will only affect a few installed domains
How will those affected feel about the bug?: 5: Blocking further progress on the daily work
User Pain: 0.229
Enterprise Customer affected?:
School Customer affected?: Yes
ISV affected?:
Waiting Support:
Flags outvoted (downgraded) after PO Review: Yes
Ticket number:
Bug group (optional):
Max CVSS v3 score:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Michael Grandjean univentionstaff 2018-06-29 12:45:29 CEST
It should be possible to disable UCR (commit) for certain templates.

Scenario:
A customer uses a configuration management system (Ansible, Puppet, Chef ...) and wants to deploy e.g. /etc/ntp.conf for all their machines (not only UCS, maybe also Debian, Ubuntu ...) across the entire fleet. In the case of UCS, the UCR mechanism would overwrite such a configuration file sooner or later. Therefore it should be possible to exclude certain UCR templates from being committed.
Comment 1 Florian Best univentionstaff 2019-03-15 21:51:06 CET
Hmm, can't they overwrite the template itself with the static content?
Comment 2 Philipp Hahn univentionstaff 2019-03-28 08:20:15 CET
(In reply to Florian Best from comment #1)
> Hmm, can't they overwrite the template itself with the static content?

Which is a bad idea as this might break future updates: files in /etc/ are Debian conffiles and are handed specially by `dpkg` during upgrades.
Also `univention-check-templates` will complain for modified files.

A "global kill switch" also would help during installation as there we have the problem with D-I, that as soon as 'univention-base-files' is installed, several configuration files are diverted by UCR, even before all UCRV are properly setup. For example this leads to a network restart, which is especially bad when doing a PXE installation.
Another case is /etc/resolv.conf, which is normally written by '/sbin/dhclient-script', then parsed to set the UCRV, which then overwrites that file. (Bug #44462) So for a short period the file is not generated by UCR. Also with (Docker) containers that file is normally provided by the container environment and not supposed to be managed by the container itself.
Comment 3 Michel Smidt 2019-03-28 08:34:22 CET
Multiple times requested by customer.
Comment 4 Moritz Bunkus 2019-05-13 09:45:15 CEST
Not saying that such a feature might not be useful, but from the standpoint of a Univention partner supporting a lot of systems:

1. I'm using Salt for managing heterogeneous fleets of systems that include a lot of Univention machines. I'm simply special-casing Univention systems to do the right thing in Salt: for NTP the corresponding UCR variables are set, for other systems /etc/ntp.conf is written directly. I already have to special-case a of lot of things based on the OS (e.g. CentOS vs. Ubuntu vs. certain OS versions), so special-casing Univention isn't that much of a hassle.

2. Allowing to deactivate the templating mechanism will lead to increased support requests and be more confusing for the supporters, especially for big upgrades (e.g. the next change of the underlying Debian version). If the templating mechanism is disabled for certain templates, the command `univention-check-templates` should list those, too, in order to be able to analyze quickly where problems might come from.

A better solution might be to offer UCS-specific management modules for those management platforms in order to make handling UCR variables easier. I have code for that for Salt that I'd be willing to share/offer.
Comment 5 Philipp Hahn univentionstaff 2020-05-26 07:02:08 CEST
This is no longer a feature request, but a blocking bug: While re-building packages for UCS-5 in Docker and Pbuilder some packages pull in "univention-base-files", which then replaces "/etc/resolv.conf" (and other files) in the middle of the build process, thus breaking the network setup inside the Docker container. The build then fails with

1631 $ apt-get -qq --no-install-recommends install curl || ( cat /etc/resolv.conf ; getent hosts updates.software-univention.de ; ping -c 3 updates.software-univention.de ; false )
...
1638 E: Failed to fetch http://updates.knut.univention.de/4.4/maintained/4.4-4/amd64/curl_7.52.1-5+deb9u10_amd64.deb  Temporary failure resolving 'updates.knut.univention.de'
1639 E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing?
...
1647 # 	/etc/univention/templates/files/etc/resolv.conf
1648 # 
1649 options timeout:2

As the environment is setup by Docker here we need a switch to disable certain UCS substitutions in those scenarios.
This also is a problem during the initial setup from Debian Installer:  As soon as u-b-f is installed the network breaks and we have to go great length to work-around that behavior there.
Comment 7 Philipp Hahn univentionstaff 2020-05-28 17:51:36 CEST
I just stumbled over over this hidden feature in `python/univention-config-registry` to disable UCR completely:

> if __name__ == '__main__' and not os.path.exists('/etc/univention/disable_baseconfig') and not os.path.exists('/etc/univention/disable_config_registry'):
Comment 8 Ingo Steuwer univentionstaff 2020-06-03 10:00:34 CEST
(In reply to Philipp Hahn from comment #7)
> I just stumbled over over this hidden feature in
> `python/univention-config-registry` to disable UCR completely:
> 
> > if __name__ == '__main__' and not os.path.exists('/etc/univention/disable_baseconfig') and not os.path.exists('/etc/univention/disable_config_registry'):

I set this bug back to "feature request". The bug requests a way to deactivate individual templates.

My understanding of comment #5 is a need to deactivate UCR completely, which is a different scenario. If there is still a need after this founding please file a new bug.
Comment 9 Philipp Hahn univentionstaff 2020-06-08 15:00:25 CEST
(In reply to Ingo Steuwer from comment #8)
> (In reply to Philipp Hahn from comment #7)
> > I just stumbled over over this hidden feature in
> > `python/univention-config-registry` to disable UCR completely:
> > 
> > > if __name__ == '__main__' and not os.path.exists('/etc/univention/disable_baseconfig') and not os.path.exists('/etc/univention/disable_config_registry'):
> 
> I set this bug back to "feature request". The bug requests a way to
> deactivate individual templates.
> 
> My understanding of comment #5 is a need to deactivate UCR completely, which
> is a different scenario. If there is still a need after this founding please
> file a new bug.

This is not a "need" to "implement" deactivating UCR completely; this was merely documenting that fact, that there already is this undocumented feature to disable UCR completely. But disabling UCR completely by creating the above files that way breaks UCR completely: Even then all UCR templates files remain un-registered and the original files are not `dpkg-divert`ed until `ucr update` is performed later on, but also neither `ucr get` nor `ucr set` nor `ucr search` not anything works; you cannot even any longer provision UCS as many script use `ucr get` internally and then work with unset values.
So this all-or-nothing approach is completely useless as many `ucr set` calls from postinst or join scripts are simply lost.

My current work-around is to replace /usr/sbin/ucr with the following wrapper:

#!/bin/sh
case "$1" in
set|get|unset|dump|search|info|shell|filter) exec /usr/sbin/univention-config-registry.docker "$@" ;;
commit|update|register|unregister) exit 0 ;;
*) echo "$0" "$@" >&2 ;;
esac

So we still need a way to disable UCR for individual files! This is what this bus initially was about and still is about.
Comment 10 Ingo Steuwer univentionstaff 2020-06-09 08:13:25 CEST
(In reply to Philipp Hahn from comment #9)

> So we still need a way to disable UCR for individual files! This is what
> this bus initially was about and still is about.

Call me pedantic, but "having a need" for a new configuration option is a Feature Request. It would be a Bug if a change in UCR has caused the problems in package builds, but that's not the case.
Comment 11 Philipp Hahn univentionstaff 2020-11-19 15:42:11 CET
Requested during TT 2020-11-19/20: Needed for custom NRPE configuration
Comment 12 Philipp Hahn univentionstaff 2020-12-16 14:21:32 CET
My work-around is again broken -  wasting my time to find the next work-around.
https://git.knut.univention.de/univention/internal/repo-ng/-/issues/56#note_6459
Comment 13 Philipp Hahn univentionstaff 2020-12-17 16:46:14 CET
(In reply to Philipp Hahn from comment #12)
> My work-around is again broken -  wasting my time to find the next
> work-around.
> https://git.knut.univention.de/univention/internal/repo-ng/-/issues/
> 56#note_6459

FYI: At least for our docker based package builder I disable the UCR template mechanism completely by doing this:
  echo 'path-exclude /etc/univention/templates/info/*.info' >/etc/dpkg/dpkg.cfg.d/no-ucr
Comment 14 Philipp Hahn univentionstaff 2021-12-16 13:30:19 CET
Again a partner was hit by UCR overwriting /etc/resolv.conf and breaking his network setup: https://help.univention.com/t/network-installations-hangs/19051
Comment 15 Ingo Steuwer univentionstaff 2021-12-16 13:56:55 CET
(In reply to Philipp Hahn from comment #14)
> Again a partner was hit by UCR overwriting /etc/resolv.conf and breaking his
> network setup:
> https://help.univention.com/t/network-installations-hangs/19051

I had a look at the same artice - for me that is not an issue which can be solved by fixing this bug. The partner tries to deploy an UCS as close as possible to our standard deployment using a netboot image - changing the core configuration of UCS would be the opposite of what he intends.
Comment 16 Philipp Hahn univentionstaff 2021-12-16 16:32:58 CET
(In reply to Ingo Steuwer from comment #15)
> (In reply to Philipp Hahn from comment #14)
> > Again a partner was hit by UCR overwriting /etc/resolv.conf and breaking his
> > network setup:
> > https://help.univention.com/t/network-installations-hangs/19051
> 
> I had a look at the same article - for me that is not an issue which can be
> solved by fixing this bug. The partner tries to deploy an UCS as close as
> possible to our standard deployment using a netboot image - changing the
> core configuration of UCS would be the opposite of what he intends.

PXE is *very* fragile: I regularly experienced these kind of problems while I still was TAM for 26. I have stopped counting the times where UCR took over some file before all UCRV were initialized and thus UCR broke the previously working service.

There are many cases this UCS attitude of "one size fits all and UCS always knows better" is just annoying: PXE is one and another is when using containers, where it's the job of the container runtime to setup the infrastructure for network/logging/etc, but UCR breaks it by force-overwriting everything.

For those cases a switch to *temporarily* disable UCR templates would really help as you can always fix them later on by re-enabling template and committing the file.