Bug 47452 - DC Master and Backup DVD installation hangs - slapd not running at end of installation before reboot
DC Master and Backup DVD installation hangs - slapd not running at end of ins...
Status: CLOSED FIXED
Product: UCS
Classification: Unclassified
Component: LDAP
UCS 4.3
Other Linux
: P5 normal (vote)
: UCS 4.3-1-errata
Assigned To: Philipp Hahn
Arvid Requate
:
Depends on: 39683 47196
Blocks:
  Show dependency treegraph
 
Reported: 2018-08-03 18:23 CEST by Philipp Hahn
Modified: 2018-08-08 14:22 CEST (History)
1 user (show)

See Also:
What kind of report is it?: Bug Report
What type of bug is this?: 6: Setup Problem: Issue for the setup process
Who will be affected by this bug?: 3: Will affect average number of installed domains
How will those affected feel about the bug?: 5: Blocking further progress on the daily work
User Pain: 0.514
Enterprise Customer affected?:
School Customer affected?:
ISV affected?:
Waiting Support:
Flags outvoted (downgraded) after PO Review:
Ticket number:
Bug group (optional):
Max CVSS v3 score:


Attachments
screendump 4 (2.26 KB, text/plain)
2018-08-03 18:23 CEST, Philipp Hahn
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Philipp Hahn univentionstaff 2018-08-03 18:23:11 CEST
Created attachment 9618 [details]
screendump 4

On all 8 trainee PCs a DC Backup was installed as a VM: Selected components:
- Mail server
- Nagios
- UVMM
- Samba4 AD Domain Controller
- DHCP

The installation seems to run fine until the last screen to reboot the VM; clicking OK then shows the infinite progress bar, but the VM does not reboot.
Switching to console 4 shows the attached errors repeating.

management-console-server.log:
03.08.18 18:08:24.510  MAIN        ( PROCESS ) : Connection timed out.
03.08.18 18:08:24.510  MAIN        ( PROCESS ) : Processor: dying
03.08.18 18:08:24.510  MAIN        ( PROCESS ) : Processor: dying
03.08.18 18:08:27.707  AUTH        ( WARN    ) : Canonicalization of username was not possible: {'desc': "Can't contact LDAP server"}
03.08.18 18:08:27.752  MODULE      ( PROCESS ) : Setting auth type to None
03.08.18 18:08:37.810  MODULE      ( PROCESS ) : Konnte nicht zum LDAP-Dienst verbinden.
Die folgenden Schritte können helfen, das Problem zu beheben:
 * Stellen Sie sicher, dass der Domänencontroller Master läuft und von dc2.schulung8.ucs aus erreichbar ist
 * Überprüfen Sie, ob auf diesem Server und auf dem Domänencontroller Master genügend Festplattenspeicher und Arbeitsspeicher (RAM) verfügbar sind
 * Starten Sie den LDAP-Dienst entweder über "service slapd restart" per Kommandozeile oder mit dem UMC Modul "Systemdienste" auf dem Domänencontroller Master neu

management-console-web-server.log:
03.08.18 18:09:18.390  MAIN        ( PROCESS ) : SessionClient(0x7fd0546b30d0): _authenticated: success=False  status=503  message=Konnte nicht zum LDAP-Dienst verbinden.
Die folgenden Schritte können helfen, das Problem zu beheben:
 * Stellen Sie sicher, dass der Domänencontroller Master läuft und von dc2.schulung8.ucs aus erreichbar ist
 * Überprüfen Sie, ob auf diesem Server und auf dem Domänencontroller Master genügend Festplattenspeicher und Arbeitsspeicher (RAM) verfügbar sind
 * Starten Sie den LDAP-Dienst entweder über "service slapd restart" per Kommandozeile oder mit dem UMC Modul "Systemdienste" auf dem Domänencontroller Master neu
03.08.18 18:09:18.391  MAIN        ( PROCESS ) : CPAuth (::1:59696) response status code: 503
03.08.18 18:09:18.391  MAIN        ( PROCESS ) : CPAuth (::1:59696) response message: Konnte nicht zum LDAP-Dienst verbinden.
Die folgenden Schritte können helfen, das Problem zu beheben:
 * Stellen Sie sicher, dass der Domänencontroller Master läuft und von dc2.schulung8.ucs aus erreichbar ist
 * Überprüfen Sie, ob auf diesem Server und auf dem Domänencontroller Master genügend Festplattenspeicher und Arbeitsspeicher (RAM) verfügbar sind
 * Starten Sie den LDAP-Dienst entweder über "service slapd restart" per Kommandozeile oder mit dem UMC Modul "Systemdienste" auf dem Domänencontroller Master neu
03.08.18 18:09:18.391  MAIN        ( PROCESS ) : CPAuth (::1:59696) response result: None

listener.log:
03.08.18 18:10:12.401  DEBUG_INIT
UNIVENTION_DEBUG_BEGIN  : uldap.__open host=dc0.schulung8.ucs port=7389 base=dc=schulung8,dc=ucs
UNIVENTION_DEBUG_END    : uldap.__open host=dc0.schulung8.ucs port=7389 base=dc=schulung8,dc=ucs
03.08.18 18:10:13.608  LISTENER    ( PROCESS ) : updating 'cn=dns-dc2,cn=uid,cn=temporary,cn=univention,dc=schulung8,dc=ucs' command a
03.08.18 18:10:13.609  LISTENER    ( ERROR   ) : 'failed.ldif' exists. Check for /var/lib/univention-directory-replication/failed.ldif


/var/lib/univention-directory-replication/failed.ldif:
dn: cn=dns-dc2,cn=uid,cn=temporary,cn=univention,dc=schulung8,dc=ucs
changetype: add
entryCSN: 20180803154246.963948Z#000000#000#000000
cn: dns-dc2
objectClass: lock
objectClass: top
creatorsName: cn=admin,dc=schulung8,dc=ucs
entryUUID: 9d3cb6a0-2b7f-1038-9e40-2db42601fe87
modifiersName: cn=admin,dc=schulung8,dc=ucs
createTimestamp: 20180803154246Z
structuralObjectClass: lock
lockTime: 1533311266
modifyTimestamp: 20180803154246Z

dn: cn=dns-dc2,cn=uid,cn=temporary,cn=univention,dc=schulung8,dc=ucs
changetype: delete


# systemctl status slapd
Running in chroot, ignoring request.


Manually starting the slapd fixes the problem:
# /etc/init.d/slapd start
after that the VM finally reboots.


UCS Technical Training Task #10201
Comment 1 Erik Damrose univentionstaff 2018-08-06 16:28:07 CEST
I can reproduce the issue. Installation from UCS 4.3-1 DVD fails for at least DC master and backup roles.

System Setup running in Firefox is not shutting down after clicking on the 'finished' button. A setup/closebrowser umcp command is send, but returns with HTTP 401. System setup then tries to re-authenticate with UMC, which fails, see comment 0.

I suspect the openldap update (4.3-1 errata 155). While installing errata updates, updater.log contains

slapd (2.4.45+dfsg-1~bpo9+1A~4.3.0.201807101905) wird eingerichtet ...
File: /etc/init.d/slapd
Multifile: /etc/ldap/slapd.conf
  Backing up /etc/ldap/slapd.conf in /var/backups/slapd-2.4.45+dfsg-1~bpo9+1A~4.3.0.201801091316... done.
Stopping ldap server(s): slapd ...done.
Running in chroot, ignoring request.

which corresponds to the slapd postinst:
/etc/init.d/slapd stop
# Automatically added by dh_installinit
if [ "$1" = "configure" ] || [ "$1" = "abort-upgrade" ]; then
        if [ -x "/etc/init.d/slapd" ]; then
                update-rc.d slapd defaults 14 80 >/dev/null
                if [ -n "$2" ]; then
                        _dh_action=restart
                else
                        _dh_action=start
                fi
                invoke-rc.d slapd $_dh_action || exit $?
        fi
fi
Comment 2 Philipp Hahn univentionstaff 2018-08-06 17:34:06 CEST
Bug #47196 was release as errata155, which triggers this issue for the first time.
The original change was from Bug #39683: 86_postinst_slapd_stop.patch

r18224 | Bug #47452: Really restart slapd during USS

Package: openldap
Version: 2.4.45+dfsg-1~bpo9+1A~4.3.0.201808061728
Branch: ucs_4.3-0
Scope: errata4.3-1

[4.3-1] 8c6587bbda Bug #47452 OpenLDAP: Really restart slapd during USS YAML
 doc/errata/staging/openldap.yaml | 11 +++++++++++
 1 file changed, 11 insertions(+)

TODO: tests
Comment 3 Philipp Hahn univentionstaff 2018-08-07 12:21:10 CEST
(In reply to Philipp Hahn from comment #2)
> TODO: tests

OK: During a running DVD installation I hacked /usr/lib/univention-system-setup/scripts/90_postjoin/20upgrade:47 to delay and then changed UCRV repo/online/server="http://apt.knut.univention.de/" and renamed to component to "4.3-1-errata-test". The new package version was installed. The VM restarted without problems after the installation. slapd was running before and after the reboot.
OK: errata-announce -V --only openldap.yaml
Comment 4 Arvid Requate univentionstaff 2018-08-07 18:16:47 CEST
Ok, verified:

* Code review ok
* A/B test successful
* Advisory Ok
* Normal package update: Ok
Comment 5 Arvid Requate univentionstaff 2018-08-08 14:22:55 CEST
<http://errata.software-univention.de/ucs/4.3/161.html>