Bug 43689 - UCS-4.2 with systemd: convert bind9
UCS-4.2 with systemd: convert bind9
Status: CLOSED FIXED
Product: UCS
Classification: Unclassified
Component: DNS
UCS 4.2
Other Linux
: P5 normal (vote)
: UCS 4.3-1-errata
Assigned To: Jürn Brodersen
Philipp Hahn
: systemd
Depends on: 43685 43470 43690
Blocks: 43691 43330 47714
  Show dependency treegraph
 
Reported: 2017-03-01 13:44 CET by Philipp Hahn
Modified: 2021-04-21 10:28 CEST (History)
1 user (show)

See Also:
What kind of report is it?: Development Internal
What type of bug is this?: ---
Who will be affected by this bug?: ---
How will those affected feel about the bug?: ---
User Pain:
Enterprise Customer affected?:
School Customer affected?:
ISV affected?:
Waiting Support:
Flags outvoted (downgraded) after PO Review:
Ticket number:
Bug group (optional):
Max CVSS v3 score:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Philipp Hahn univentionstaff 2017-03-01 13:44:57 CET
We should convert /etc/init.d/univention-bind to a bind9.service
This would allow us to remove runit.

- Consider handling of autostart UCRV (Bug #43470)
- We need to split /etc/init.d/univention-bind into multiple services files for systemd to allow it to manage the services on their own. The tricky part is to EITHER start the S4 backed OR both the LDAP backend and the proxy.
- The really hard part is that lots of scripts still call /etc/init.d/univention-bind directly, so a backward compatibility layers seems to be required.
Comment 1 Philipp Hahn univentionstaff 2017-03-01 16:50:11 CET
When replacing runit by systemd, some join scripts/infrastructure must be updated to only start the service after joining:

# find / -xdev \( -name log -o -name dpkg -o -name init.d -o -name cache \) -prune -o -type f -exec grep -n --color /etc/runit/univention {} +

/usr/lib/univention-install/25univention-dhcp.inst:110:ln -s /etc/runit/univention-dhcp /etc/runit/univention/univention-dhcp >/dev/null 2>&1
/usr/lib/univention-install/90univention-bind-post.inst:88:if [ ! -e /etc/runit/univention/univention-bind-proxy ]; then
/usr/lib/univention-install/90univention-bind-post.inst:89:     ln -s /etc/runit/univention-bind-proxy /etc/runit/univention/univention-bind-proxy
/usr/lib/univention-install/90univention-bind-post.inst:92:if [ ! -e /etc/runit/univention/univention-bind-samba4 ]; then
/usr/lib/univention-install/90univention-bind-post.inst:93:     ln -s /etc/runit/univention-bind-samba4 /etc/runit/univention/univention-bind-samba4
/usr/lib/univention-install/02univention-directory-notifier.inst:38:ln -sf /etc/runit/univention-directory-notifier /etc/runit/univention/univention-directory-notifier
/usr/lib/univention-install/05univention-bind.inst:102:ln -s /etc/runit/univention-bind /etc/runit/univention/univention-bind >/dev/null 2>&1
/usr/lib/univention-install/03univention-directory-listener.inst:38:ln -sf /etc/runit/univention-directory-listener /etc/runit/univention/univention-directory-listener

/etc/logrotate.d/univention-directory-notifier:21: test -x /usr/bin/sv && test -e /etc/runit/univention/univention-directory-notifier && sv term univention-directory-notifier || true
/etc/logrotate.d/univention-dhcp:21: test -x /usr/bin/sv && test -e /etc/runit/univention/univention-dhcp && sv term univention-dhcp || true
/etc/logrotate.d/univention-directory-listener:21: test -x /usr/bin/sv && test -e /etc/runit/univention/univention-directory-listener && sv term univention-directory-listener || true

univention-join also has some internal knowledge of runit:

/usr/share/univention-join/univention-join:665: if [ -e "/etc/runit/univention/$service" ]
/usr/share/univention-join/univention-join:926:if [ -d /etc/runit/univention-directory-notifier ]; then
/usr/share/univention-join/univention-join:931:if [ -d /etc/runit/univention-directory-listener ]; then
Comment 2 Jürn Brodersen univentionstaff 2018-05-18 12:14:22 CEST
Commit on feature branch: juern/43689_bind_systemd
https://git.knut.univention.de/univention/ucs/commit/192144cc7847d0d668c4f0766c03de3ad7bb78d4
Comment 3 Philipp Hahn univentionstaff 2018-05-30 17:30:21 CEST
FAIL: (use `rm --force`)
 dpkg: Warnung: Altes Verzeichnis »/etc/univention/templates/files/etc/runit/univention-bind-samba4« kann nicht gelöscht werden: Das Verzeichnis ist nicht leer
 dpkg: Warnung: Altes Verzeichnis »/etc/univention/templates/files/etc/runit/univention-bind-proxy« kann nicht gelöscht werden: Das Verzeichnis ist nicht leer
 dpkg: Warnung: Altes Verzeichnis »/etc/univention/templates/files/etc/runit/univention-bind« kann nicht gelöscht werden: Das Verzeichnis ist nicht leer
 dpkg: Warnung: Altes Verzeichnis »/etc/runit/univention-bind-samba4« kann nicht gelöscht werden: Das Verzeichnis ist nicht leer
 dpkg: Warnung: Altes Verzeichnis »/etc/runit/univention-bind-proxy« kann nicht gelöscht werden: Das Verzeichnis ist nicht leer
 dpkg: Warnung: Altes Verzeichnis »/etc/runit/univention-bind« kann nicht gelöscht werden: Das Verzeichnis ist nicht leer

RFA: Please remove the /usr/sbin/ prefix from ucr

FAIL: dpkg-query -W -f '${Conffiles}\n' univention-bind | grep /down
 /etc/runit/univention-bind-samba4/down d41d8cd98f00b204e9800998ecf8427e obsolete
 /etc/runit/univention-bind-proxy/down d41d8cd98f00b204e9800998ecf8427e obsolete
 /etc/runit/univention-bind/down d41d8cd98f00b204e9800998ecf8427e obsolete

OK: service univention-bind-ldap restart
OK: service univention-bind-proxy status
OK: service univention-bind-samba4 restart
OK: service bind9 reload
OK: systemctl status univention-bind-ldap.service
OK: systemctl status univention-bind-proxy.service
??: /etc/init.d/bind9 crestart
 Are you sure this is not used anymore? (systemctl try-reload-or-restart)
??: Please lookt at /lib/systemd/system/bind9.service.debian and merge *=nss-lookup.target ?
??: Please spell LDAP as LDAP (all capital)
Comment 4 Jürn Brodersen univentionstaff 2018-05-31 15:26:21 CEST
Branch is updated

"/etc/init.d/bind9 crestart" is now using "systemctl try-restart bind9".

I don't think I can get rid of these: "dpkg: Warnung: Altes Verzeichnis". But the old folders and conf files should now all deleted in the postinst.

/usr/sbin/ prefix -> removed

nss-lookup.target -> Added

ldap -> LDAP in service descriptions

I also added a ExecStartPost to check if bind is actually ready. That way starting the service blocks until bind is ready to serve queries. I hope that will make the systemsetup more stable. But if the ExecStartPost fails the service is stopped and that might be to much? (Adding "-" to the command or remove it again?)
Comment 5 Jürn Brodersen univentionstaff 2018-06-15 14:36:48 CEST
Merged and build
[4.3-1 f827061465] Bug #43689: YAML

Lets see how the test go.
Comment 6 Jürn Brodersen univentionstaff 2018-06-17 19:13:03 CEST
Tests failed
Comment 7 Jürn Brodersen univentionstaff 2018-06-25 10:56:21 CEST
The tests seem to have stabilized. Some udm tests with samba are still failing  from time to time but I don't think that is directly related.

I had to remove the alias and instead use a drop in file. An alias can't be enabled, which is a problem for ucr autostart.

Lessons learned (so far)...:
Don't use an alias if you want to enable the service using that name.
A working init script is needed for the setup inside a chroot (setup from dvd). 
Only use "Requires" if the service really doesn't work at all without that dependency otherwise use "Wants".

Some remaining questions:
Should the service always restart? (Currently set to on-failure)

Should the service be working without a drop in file? The bind9.service is currently not working without a drop in file to make it clear that it gets further configured.

Should the "ExecStartPost" stop the service if it fails?
Comment 8 Quality Assurance univentionstaff 2018-07-04 16:05:12 CEST
--- mirror/ftp/4.3/unmaintained/4.3-1/source/univention-bind_12.0.1-4A~4.3.0.201805301225.dsc
+++ apt/ucs_4.3-0-errata4.3-1/source/univention-bind_12.0.2-7A~4.3.0.201806251024.dsc
@@ -1,6 +1,38 @@
-12.0.1-4A~4.3.0.201805301225 [Wed, 30 May 2018 12:25:26 +0200] Univention builddaemon <buildd@univention.de>:
+12.0.2-7A~4.3.0.201806251024 [Mon, 25 Jun 2018 10:24:44 +0200] Univention builddaemon <buildd@univention.de>:
 
   * UCS auto build. No patches were applied to the original source package
+
+12.0.2-7 [Mon, 25 Jun 2018 10:20:28 +0200] Jürn Brodersen <brodersen@univention.de>:
+
+  * Bug #43689: fix init script warnings
+
+12.0.2-6 [Mon, 18 Jun 2018 22:22:09 +0200] Jürn Brodersen <brodersen@univention.de>:
+
+  * Bug #43689: change requires dependency to wants
+
+12.0.2-5 [Mon, 18 Jun 2018 18:17:27 +0200] Jürn Brodersen <brodersen@univention.de>:
+
+  * Bug #43689: use start-stop-daemon
+
+12.0.2-4 [Mon, 18 Jun 2018 16:40:53 +0200] Jürn Brodersen <brodersen@univention.de>:
+
+  * Bug #43689: fork from the init script
+
+12.0.2-3 [Mon, 18 Jun 2018 13:34:20 +0200] Jürn Brodersen <brodersen@univention.de>:
+
+  * Bug #43689: Add simple init script for setup inside chroot
+
+12.0.2-2 [Sun, 17 Jun 2018 19:04:57 +0200] Jürn Brodersen <brodersen@univention.de>:
+
+  * Bug #43689: Fix samab -> samba4; Cleanup
+
+12.0.2-1 [Sat, 16 Jun 2018 19:16:40 +0200] Jürn Brodersen <brodersen@univention.de>:
+
+  * Bug #43689: use drop in file to configure bind9.service
+
+12.0.1-5 [Fri, 15 Jun 2018 12:58:54 +0200] Jürn Brodersen <brodersen@univention.de>:
+
+  * Bug #43689: use systemd services instead of runit
 
 12.0.1-4 [Wed, 30 May 2018 12:22:04 +0200] Philipp Hahn <hahn@univention.de>:
 

<http://10.200.17.11/4.3-1/#3957169505830358521>
Comment 9 Philipp Hahn univentionstaff 2018-07-04 18:48:16 CEST
(In reply to Jürn Brodersen from comment #7)
> The tests seem to have stabilized. Some udm tests with samba are still
> failing  from time to time but I don't think that is directly related.

1049724bfecc | Bug #43689: Merge branch 'juern/43689_bind_systemd' into 4.3-1
1aad49a974c9 | Bug #43689: Remove old config files and dirs
4b9075fb63fb | Bug #43689: use drop in file to configure bind9.service
4d36c6af2c95 | Bug #43689: use systemd services instead of runit
9303b73a247b | Bug #43689: Fix samab -> samba4; Cleanup
952e11fe0c8b | Bug #43689: fix init script warnings
b1b54aab3b34 | Bug #43689: Add nss-lookup.target to services
beefee0c576f | Bug #43689: Add ExecStartPost to services
c2797acb0c5e | Bug #43689: use start-stop-daemon
cdfe089b1f17 | Bug #43689: Retry resolveDnsEntry in case the zone was not yet transfered
d3913fa1261c | Bug #43689: change requires dependency to wants
d5d989db1cfe | Bug #43689: Add simple init script for setup inside chroot
e5e8fa1f9950 | Bug #43689: fork from the init script
f47ab5bdddc3 | Bug #43689: yaml
f82706146529 | Bug #43689: YAML

> I had to remove the alias and instead use a drop in file. An alias can't be
> enabled, which is a problem for ucr autostart.

OK

> Lessons learned (so far)...:
> Don't use an alias if you want to enable the service using that name.
> A working init script is needed for the setup inside a chroot (setup from
> dvd). 
> Only use "Requires" if the service really doesn't work at all without that
> dependency otherwise use "Wants".
> 
> Some remaining questions:
> Should the service always restart? (Currently set to on-failure)

No, as there are cases like "rndc stop" where the daemon should NOT restart.

> Should the service be working without a drop in file? The bind9.service is
> currently not working without a drop in file to make it clear that it gets
> further configured.

I think this is okay for now.

> Should the "ExecStartPost" stop the service if it fails?

AFAIK no: If ExecStart fails this is because the service couldn't be started, so there is nothing to stop. Even if an other error occurs before the final exec, there is nothing to do.

OK: dns/backend=ldap
OK: dns/backend=samba4
OK: systemctl status bind9.service
OK: systemctl restart bind9.service
OK: kill + restart
OK: systemctl status univention-bind-ldap.service
~ok: rndc stop # restarts both services, but okay
OK: UMC service stop / start
OK: autostart

OK: <http://jenkins.knut.univention.de:8080/job/UCS-4.3/job/UCS-4.3-1/job/Installation%20Tests/>
OK: <http://jenkins.knut.univention.de:8080/job/UCS-4.3/job/UCS-4.3-1/job/AutotestJoin/>

OK: errata-announce -V --only univention-bind.yaml
OK: univention-bind.yaml
Comment 10 Arvid Requate univentionstaff 2018-07-11 15:09:04 CEST
<http://errata.software-univention.de/ucs/4.3/146.html>