Bug 41007 - replace ssh and ldap connection to master with HTTP UMC request in schoolinstaller
replace ssh and ldap connection to master with HTTP UMC request in schoolinst...
Status: CLOSED FIXED
Product: UCS@school
Classification: Unclassified
Component: UMC - Installer
UCS@school 4.1
Other Linux
: P5 normal (vote)
: UCS@school 4.1 R2 vXXX
Assigned To: Florian Best
Daniel Tröder
: interim-3
: 38683 (view as bug list)
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2016-04-06 13:56 CEST by Florian Best
Modified: 2016-12-12 13:10 CET (History)
1 user (show)

See Also:
What kind of report is it?: Development Internal
What type of bug is this?: ---
Who will be affected by this bug?: ---
How will those affected feel about the bug?: ---
User Pain: 0.006
Enterprise Customer affected?:
School Customer affected?:
ISV affected?:
Waiting Support:
Flags outvoted (downgraded) after PO Review:
Ticket number:
Bug group (optional): Cleanup, Error handling, Troubleshooting
Max CVSS v3 score:


Attachments
patch (2.12 KB, patch)
2016-08-19 15:33 CEST, Florian Best
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Florian Best univentionstaff 2016-04-06 13:56:00 CEST
The function get_user_dn(username) in the ucs-school-umc-installer might return a wrong DN. It makes a UMC call to udm/query which searches with a substring search for the username. If one puts the user "foo" into the input box the username "foobar" will be found, too.

Can we replace the function to make a LDAP query which uses uid=%s or ldap_whoami_s()?
Comment 1 Sönke Schwardt-Krummrich univentionstaff 2016-04-11 12:17:59 CEST
(In reply to Florian Best from comment #0)
> The function get_user_dn(username) in the ucs-school-umc-installer might
> return a wrong DN. It makes a UMC call to udm/query which searches with a
> substring search for the username. If one puts the user "foo" into the input
> box the username "foobar" will be found, too.

But it will be no problem since wrong matches are filtered out:

	result = umc(username, password, master, 'udm/query', 
                        {"objectProperty": "username", 
                         "objectPropertyValue": username}, 'users/user')
	result = [ientry.get('$dn$') for ientry in result \
                                   if ientry.get('username') == username]
	try:
		return result[0]
	except IndexError:
		pass
Comment 2 Florian Best univentionstaff 2016-04-11 12:29:28 CEST
Oh yes. Only problem would be differences in lower/uppercase.
Comment 3 Florian Best univentionstaff 2016-04-12 14:06:13 CEST
(In reply to Florian Best from comment #2)
> Oh yes. Only problem would be differences in lower/uppercase.
As discussed: we should look closer on this.
Comment 4 Florian Best univentionstaff 2016-08-19 15:33:40 CEST
Created attachment 7905 [details]
patch

Would it be an option to switch to ssh? (I am thinking about API/securtiy changes in the UDM module which would require changes here.)
Comment 5 Florian Best univentionstaff 2016-11-15 19:19:24 CET
The internal logic of the UCS@school installer has been changed to not anymore make any SSH or LDAP connection to the DC Master.
Instead all connections are done via UMC. Therefore no LDAP-bind is needed anymore.

ucs-school-umc-installer.yaml:
r74459 | YAML Bug #41007

ucs-school-umc-installer (4.0.1-1):
r74458 | Bug #41007: fix jshint issues
r74457 | Bug #41007: only increase progressbar percentage if a new script is executed
r74455 | Bug #41007: show the currently executed join script
r74454 | Bug #41007: use UMC instead of SSH when receiving information about schools

I tested the following scenarios:
* Install DC Master Multiserver
* Install DC Master Singleserver
* Join DC Slave into Multiserver
* Join Administrative DC Slave into Multiserver

Invalid combinations are still detected, e.g. Installing a DC Slave with a singleserver DC Master.

I tested the error handling:
If the master is not reachable UMC shows:
'master120.school.local: Authentication failed while contacting: [Errno -2] Der Name oder der Dienst ist nicht bekannt'
If apache doesn't run UMC shows:
'master119.school.local: Authentication failed while contacting: [Errno 111] Verbindungsaufbau abgelehnt'
If the UMC-Server doesn't run UMC shows:
'master119.school.local: Authentication failed: {"status": 503, "message": "The Univention Management Console Server is currently not running. \\nIf you have root permissions on the system you can restart it by executing the following command:\\n * invoke-rc.d univention-management-console-server restart\\nThe following logfile may contain information why the server is not running:\\n * /var/log/univention/management-console-server.log\\nOtherwise please contact an administrator or try again later."}'
If the UMC-Webserver doesn't run UMC shows:
'master119.school.local: Authentication failed: {"status": 503, "message": "The Univention Management Console Web Server could not be reached. Please restart it or try again later."}'
If LDAP doesn't run UMC shows:
'master119.school.local: Authentication failed: {"status": 401, "message": "The authentication has failed, please login again."}'
If the password was entered wrong UMC shows:
'master119.school.local: Authentication failed: {"status": 401, "message": "The authentication has failed, please login again."}'

Should we change those messages? currently the UMCConnection class doesn't give us much information (Bug #34490).
→ For the messages there also exists Bug #38720
Comment 6 Florian Best univentionstaff 2016-11-16 18:04:32 CET
Why is the user pain not set automatically here?
Comment 7 Florian Best univentionstaff 2016-12-01 16:18:58 CET
In Jenkins a DC Backup installation failed:

Domänenbeitritt - Ausführung des Join-Skriptes 98univention-pkgdb-tools
Domänenbeitritt - 
Domänenbeitritt - beendet...
UMC and Apache will be restarted on the system. Waiting for 20 seconds.
Execute: echo -n "univention" >/tmp/univention
Execute: /root/schoolinstaller.py -uAdministrator -p univention -M
Traceback (most recent call last):
  File "/root/schoolinstaller.py", line 129, in <module>
    result = connection.request('schoolinstaller/install', params)
  File "/usr/lib/pymodules/python2.7/univention/lib/umc_connection.py", line 143, in request
    raise HTTPException(error_message)
httplib.HTTPException: 400 on localhost (schoolinstaller/install): {"status": 400, "message": "Konnte nicht mit dem DC Master master300.autotest300.local verbinden: 422 on master300.autotest300.local (schoolinstaller/get/schoolinfo): {\"status\": 422, \"message\": \"Eine Option f\\u00fcr get_schoolinfo hat den falschen Typ: 1 Fehler aufgetreten\", \"result\": {\"schoolOU\": \"Wert ist ung\\u00fcltig\"}}"}
*** Failed 1: /root/schoolinstaller.py -uAdministrator -p univention -M
Comment 8 Daniel Tröder univentionstaff 2016-12-01 17:58:48 CET
On another VM there was:

30.11.16 08:28:53.540  MODULE      ( PROCESS ) : Konnte nicht mit dem DC Master master300.autotest300.local verbinden: master300.autotest300.local: Authentication failed: {"status": 503, "message": "The Univention Management Console Server is currently not running. \nIf you have root permissions on the system you can restart it by executing the following command:\n * invoke-rc.d univention-management-console-server restart\nThe following logfile may contain information why the server is not running:\n * /var/log/univention/management-console-server.log\nOtherwise please contact an administrator or try again later."}


A retry after 10s/30s would be good, as those ldap-/umc-down situation tend to be temporary.
Comment 9 Florian Best univentionstaff 2016-12-01 20:57:20 CET
*** Bug 38683 has been marked as a duplicate of this bug. ***
Comment 10 Florian Best univentionstaff 2016-12-01 20:59:20 CET
(In reply to Florian Best from comment #7)
> In Jenkins a DC Backup installation failed:
fixed.

ucs-school-umc-installer (4.0.5-1):
r74916 | Bug #41007: fix backup installation; Bug #38683: skip environment type selection page on DC Master


(In reply to Daniel Tröder from comment #8)
> On another VM there was:
> 
> 30.11.16 08:28:53.540  MODULE      ( PROCESS ) : Konnte nicht mit dem DC
> Master master300.autotest300.local verbinden: master300.autotest300.local:
> Authentication failed: {"status": 503, "message": "The Univention Management
> Console Server is currently not running. \nIf you have root permissions on
> the system you can restart it by executing the following command:\n *
> invoke-rc.d univention-management-console-server restart\nThe following
> logfile may contain information why the server is not running:\n *
> /var/log/univention/management-console-server.log\nOtherwise please contact
> an administrator or try again later."}
> 
> 
> A retry after 10s/30s would be good, as those ldap-/umc-down situation tend
> to be temporary.
Well, this problem exists before, too. Did it happen in Jenkins?
@Sönke: Please decide, should I fix this now? Or create a new bug?
Comment 11 Sönke Schwardt-Krummrich univentionstaff 2016-12-05 13:49:46 CET
(In reply to Florian Best from comment #10)
> > A retry after 10s/30s would be good, as those ldap-/umc-down situation tend
> > to be temporary.
> Well, this problem exists before, too. Did it happen in Jenkins?
> @Sönke: Please decide, should I fix this now? Or create a new bug?

Please create a new bug → UCS@school 4.1R2 interim-4
Comment 12 Florian Best univentionstaff 2016-12-05 14:53:45 CET
(In reply to Sönke Schwardt-Krummrich from comment #11)
> (In reply to Florian Best from comment #10)
> > > A retry after 10s/30s would be good, as those ldap-/umc-down situation tend
> > > to be temporary.
> > Well, this problem exists before, too. Did it happen in Jenkins?
> > @Sönke: Please decide, should I fix this now? Or create a new bug?
> 
> Please create a new bug → UCS@school 4.1R2 interim-4Bug #43116
Comment 13 Daniel Tröder univentionstaff 2016-12-07 13:36:57 CET
reopen:

* on a backup the installer restarts after successfully installing packages and running join scripts, which leads to "ValueError: The installation was started twice..."

* on a backup single/multi setup is now autodetected. This leaves a page page in the background of the "All necessary information..." window. That looks broken to the user (although it isn't). Please add some text there.

Instead of fixing both points, the code could also be reverted, as this feature was not part of this bug anyway.
Comment 14 Florian Best univentionstaff 2016-12-07 19:10:01 CET
ucs-school-umc-installer (4.0.6-1):
r75099 | Bug #41007: Add special page for DC Backup installation
(In reply to Daniel Tröder from comment #13)
> * on a backup the installer restarts after successfully installing packages
> and running join scripts, which leads to "ValueError: The installation was
> started twice..."
Oh yes, this was because the progressbar did a request after the installation was already finished.
Also locking in the frontend has been added (if one clicks enter during installation and start a parallel installation).

> * on a backup single/multi setup is now autodetected. This leaves a page
> page in the background of the "All necessary information..." window. That
> looks broken to the user (although it isn't). Please add some text there.
I added this page. Is the text okay? If not, please make suggestments.
Should the installation start immediately after clicking on next? Or is the pop up okay?

> Instead of fixing both points, the code could also be reverted, as this
> feature was not part of this bug anyway.
~Well, no this would allow to make 2 parallel installations silently which causes corrupt package states and other side effects.
Comment 15 Sönke Schwardt-Krummrich univentionstaff 2016-12-07 20:49:01 CET
(In reply to Florian Best from comment #14)
> > * on a backup single/multi setup is now autodetected. This leaves a page
> > page in the background of the "All necessary information..." window. That
> > looks broken to the user (although it isn't). Please add some text there.
> I added this page. Is the text okay? If not, please make suggestments.
> Should the installation start immediately after clicking on next? Or is the
> pop up okay?

Can you attach a screenshot?
I would prefer keeping the popup dialog. Does the dialog show the same message as  the wizard page?
Comment 16 Daniel Tröder univentionstaff 2016-12-08 09:33:26 CET
Screenshots: https://bepasty.knut.univention.de/LWKMADzM

OK: r75099 fixes remaining points "installation was started twice" and "empty text page"
OK: advisory
OK: (repeated) manual test: installed on master, backup and edu-slave
Comment 17 Florian Best univentionstaff 2016-12-08 16:28:44 CET
I made some more changes to make the API better for future changes, as we now have to take care of API changes.

ucs-school-umc-installer (4.0.7-2):
r75130 | Bug #41007: fiy typo
r75129 | Bug #41007: enhance the API / make it future proof
Comment 18 Daniel Tröder univentionstaff 2016-12-09 10:26:50 CET
OK: manual test: installed on master, backup and edu-slave
Comment 19 Sönke Schwardt-Krummrich univentionstaff 2016-12-12 13:10:11 CET
UCS@school 4.1 R2 v9 has been released.

http://docs.software-univention.de/changelog-ucsschool-4.1R2v9-de.html