Bug 49475 - Join of UCS@school edu slave fails
Join of UCS@school edu slave fails
Status: CLOSED FIXED
Product: UCS@school
Classification: Unclassified
Component: General
UCS@school 4.4
Other All
: P5 normal (vote)
: UCS@school 4.4 v2-errata
Assigned To: Sönke Schwardt-Krummrich
Jürn Brodersen
:
Depends on:
Blocks: 49497
  Show dependency treegraph
 
Reported: 2019-05-14 20:28 CEST by Michael Grandjean
Modified: 2019-05-20 14:15 CEST (History)
6 users (show)

See Also:
What kind of report is it?: Bug Report
What type of bug is this?: 6: Setup Problem: Issue for the setup process
Who will be affected by this bug?: 3: Will affect average number of installed domains
How will those affected feel about the bug?: 3: A User would likely not purchase the product
User Pain: 0.309
Enterprise Customer affected?:
School Customer affected?: Yes
ISV affected?:
Waiting Support:
Flags outvoted (downgraded) after PO Review:
Ticket number:
Bug group (optional):
Max CVSS v3 score:


Attachments
USI (612.79 KB, application/x-bzip)
2019-05-14 20:28 CEST, Michael Grandjean
Details
join.log slave (10.51 KB, text/plain)
2019-05-14 20:30 CEST, Michael Grandjean
Details
setup.log slave (251.54 KB, text/plain)
2019-05-14 20:30 CEST, Michael Grandjean
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Michael Grandjean univentionstaff 2019-05-14 20:28:56 CEST
Created attachment 10026 [details]
USI

UCS@school Multi-Master
Master: UCS 4.4-0 errata91, nagios=4.3 samba4=4.10 self-service=4.0 ucsschool=4.4 v2
Edu School Slave: UCS 4.4-0 errata91

How to reproduce:

1. Install UCS 4.4-0 Master with Samba AD, update at end of installation
2. Install and configure UCS@school on Master
3. Create a school (including a school server)
4. Install a UCS 4.4-0 Slave with the Hostname of the school server in step 3
5. Keep the checkboxes to join the domain and to update the system after installation checked

Expectation:
The UCS Slave joins as UCS@school Edu Slave

Observed behaviour:
The UCS Slave join fails with:

/var/log/univention/join.log
> [...]
> E: Paket ucs-school-umc-installer kann nicht gefunden werden.
> Failed to uninstall the App
> Aborting...
> [...]
> ERROR:__main__:univention-app install ucsschool=4.4 v2 --skip-check must_have_valid_license --do-not-call-join-scripts failed with exit code 1!
> ERROR: join/pre-joinscripts hook /tmp/tmppqIClB/tmp0ESg7C failed.

I will attach the join.log, setup.log and a complete USI of the Slave.

There's also a UCS@school Slave in our KVM test environment that can be used to investigate this further:
VM: mgrandje_schule-gymn-36-250
IP: 10.200.36.250

The corresponding Master is:
VM: mgrandje_schule-master-10
IP: 10.200.30.10
Comment 1 Michael Grandjean univentionstaff 2019-05-14 20:30:05 CEST
Created attachment 10027 [details]
join.log slave
Comment 2 Michael Grandjean univentionstaff 2019-05-14 20:30:32 CEST
Created attachment 10028 [details]
setup.log slave
Comment 3 Sönke Schwardt-Krummrich univentionstaff 2019-05-14 21:39:36 CEST
First debugging:
I was able to reproduce the issue when joining directly in system setup. During system setup, repository/online=false is set. 
Therefore /etc/apt/sources.list.d/20_ucs-online-components.list is empty and the join hook is unable to install the app (→ the package ucs-school-umc-installer).
The failed join hook also aborts the join process.
Comment 4 Sönke Schwardt-Krummrich univentionstaff 2019-05-14 21:59:43 CEST
Workaround:
if a join error occurred during the system setup of the installation DVD, just boot the system and call "univention-join" again manually (or via UMC).

Better workaround:
*Do not* join during installation and do this later after rebooting the slave via UMC or CLI.

We have to investigate further on how to modify the join hook, so the join will succeed during DVD installation (or KVM template).
Comment 5 Sönke Schwardt-Krummrich univentionstaff 2019-05-14 22:04:49 CEST
(In reply to Sönke Schwardt-Krummrich from comment #4)
> We have to investigate further on how to modify the join hook, so the join
> will succeed during DVD installation (or KVM template).

Blind guess:
the hook could set repository/online=true on its own.
Comment 6 Sönke Schwardt-Krummrich univentionstaff 2019-05-15 14:16:40 CEST
The join hook has been adapted:
if the UCS@school app is not installed, the join hook checks the UCR variable "repository/online". If the online repo is deactivated, it (re)activates the online repository and calls 
univention-upgrade \
   --update-to=$version_version-$version_patchlevel --noninteractive
to install all available errata/package updates. The update is performed to prevent dependency errors if only some errata packages are automatically installed during UCS@school installation and others aren't.

The join will not fail if univention-upgrade returns with a non-zero exitcode.

The VERSION of the master's join scripts for single server and multi server environments has been increased, so the new join hook is installed automatically upon package installation/update.

[4.4] 306e07c80 Bug #49475: update advisory
[4.4] 253afbb82 Bug #49475: Merge branch 'sschwardt/49475/4.4/joinhook' into 4.4
[4.4] 49a816937 Bug #49475: add advisory
[4.4] e7baf6da9 Bug #49475: add changelog entry
[4.4] 9d93aa144 Bug #49475: increase join script version so the new hook is registered
[4.4] 4d05b9142 Bug #49475: activate repository in join hook and update to latest errata
[4.4] f322ad236 Bug #49475: add additional debug output
[4.4] d002530d8 Bug #49475: remove __main__ from log level and add timestamp

Package: ucs-school-metapackage
Version: 12.0.2-1A~4.4.0.201905151414
Branch: ucs_4.4-0
Scope: ucs-school-4.4
Comment 7 Jürn Brodersen univentionstaff 2019-05-15 18:12:19 CEST
The frontend now reports to early that the system setup has finished.


I think the the problem is the univention-upgrade in the hook. One of the mechanism for the frontend JavaScript code to decide if the setup is finished is checking if the apache is reachable. Due to the upgrade that was not the case for some time.

Do we really need to upgrade? And in case we do, the upgrade is not performed if repository/online=yes is already set.

(In case we need to prevent the apache from restarting: for example the schoolinstaller has code to prevent the apache from restarting)
Comment 8 Sönke Schwardt-Krummrich univentionstaff 2019-05-16 17:11:17 CEST
(In reply to Jürn Brodersen from comment #7)
> The frontend now reports to early that the system setup has finished.
> 
> I think the the problem is the univention-upgrade in the hook. One of the
> mechanism for the frontend JavaScript code to decide if the setup is
> finished is checking if the apache is reachable. Due to the upgrade that was
> not the case for some time.

As discussed with all parties involved, no package update will be performed now. However, a package list update will be done via "apt-get update" so that they are always up-to-date. 

[4.4] bb4e7d10b Bug #49475: do not perform package update but only a package list update

Package: ucs-school-metapackage
Version: 12.0.2-2A~4.4.0.201905161706
Branch: ucs_4.4-0
Scope: ucs-school-4.4

It has been tested with a 4.4-0e0 KVM image and a 4.4-0e0 install DVD. In both cases the installation was possible again. See join.log:

16.05.19 16:29:08.168  DEBUG_INIT
univention-join-hooks: looking for hook type "join/pre-joinscripts" on master150.nstx150.ucs
Found hooks:
  cn=ucsschool-join-hook.py,cn=data,cn=univention,dc=nstx,dc=local
Running: ucsschool-join-hook.py (cn=ucsschool-join-hook.py,cn=data,cn=univention,dc=nstx,dc=local) in /tmp/tmpF_jFOM/tmph1HdiE
2019-05-16 16:29:08,889 ucsschool-join-hook: [INFO] ucsschool-join-hook.py has been started
2019-05-16 16:29:08,889 ucsschool-join-hook: [INFO] Connecting to LDAP as 'uid=Administrator,cn=users,dc=nstx,dc=local' ...
2019-05-16 16:29:08,946 ucsschool-join-hook: [INFO] Host is member of following groups: ['cn=DC Slave Hosts,cn=groups,dc=nstx,dc=local', 'cn=DC-Edukativnetz,cn=ucsschool,cn=groups,dc=nstx,dc=local', 'cn=OUschule180-DC-Edukativnetz,cn=ucsschool,cn=groups,dc=nstx,dc=local']
2019-05-16 16:29:08,947 ucsschool-join-hook: [INFO] Determined role packages: ['ucs-school-slave']
2019-05-16 16:29:08,947 ucsschool-join-hook: [INFO] Calling ['univention-app', 'info', '--as-json'] ...
2019-05-16 16:29:10,403 ucsschool-join-hook: [INFO] Installed packages: []
2019-05-16 16:29:10,403 ucsschool-join-hook: [INFO] Is ucsschool already installed? False
2019-05-16 16:29:10,403 ucsschool-join-hook: [INFO] Calling ['univention-ssh', '/etc/machine.secret', 'slave180$@master150.nstx150.ucs', '/usr/sbin/ucr get version/version'] ...
2019-05-16 16:29:11,055 ucsschool-join-hook: [INFO] Calling ['ucr', 'get', 'version/version'] ...
2019-05-16 16:29:11,105 ucsschool-join-hook: [INFO] Master version: '4.4'
2019-05-16 16:29:11,105 ucsschool-join-hook: [INFO] Local version: '4.4'
2019-05-16 16:29:11,105 ucsschool-join-hook: [INFO] Calling ['univention-ssh', '/etc/machine.secret', 'slave180$@master150.nstx150.ucs', '/usr/bin/univention-app info --as-json'] ...
2019-05-16 16:29:13,403 ucsschool-join-hook: [INFO] repository/online: 'false'
2019-05-16 16:29:13,404 ucsschool-join-hook: [WARNING] The online repository is deactivated. Reactivating it.
2019-05-16 16:29:14,080 ucsschool-join-hook: [INFO] Calling ['/usr/bin/apt-get', 'update'] ...
Holen:1 file:/var/cache/univention-system-setup/packages ./ InRelease
Ign:1 file:/var/cache/univention-system-setup/packages ./ InRelease
Holen:2 file:/var/cache/univention-system-setup/packages ./ Release
Ign:2 file:/var/cache/univention-system-setup/packages ./ Release
[...]
Holen:203 http://univention-repository.knut.univention.de/4.4/maintained/component 4.4-0-errata/amd64/ Packages [187 kB]
Es wurden 10,9 MB in 6 s geholt (1.706 kB/s).
Paketlisten werden gelesen...
2019-05-16 16:29:21,076 ucsschool-join-hook: [INFO] Updating app center information...
[...]
2019-05-16 16:29:23,055 ucsschool-join-hook: [INFO] Installing ucsschool=4.4 v2 ...
Going to install UCS@school (4.4 v2)
[...]
2019-05-16 16:29:41,718 ucsschool-join-hook: [INFO] Not all required packages installed - calling univention-install...
2019-05-16 16:33:17,799 ucsschool-join-hook: [INFO] ucsschool-join-hook.py is done
Comment 9 Jürn Brodersen univentionstaff 2019-05-17 17:15:31 CEST
What I tested:

Master update -> Hook is updated -> OK

Edu Slave from iso -> OK
Edu Slave from applicance -> OK

The rest was mixed between iso and appliance
Backup -> OK
Member -> OK
Central slave -> OK
admin slave -> OK

I removed the sentence about the package upgrade from the yaml, since that is no longer the case.

-> Verified
Comment 10 Sönke Schwardt-Krummrich univentionstaff 2019-05-20 14:15:51 CEST
UCS@school 4.4 v2 has been released.

https://docs.software-univention.de/changelog-ucsschool-4.4v2-de.html

If this error occurs again, please clone this bug.