Univention Bugzilla – Bug 40046
Univention App setup takes too long
Last modified: 2016-10-24 08:42:16 CEST
During a Univention App setup, the UCS and the App packages are installed and configured which takes a lot of time. It would be really nice if the system is preconfigured and the setup only changes the preinstalled and preconfigured system. This should reduce the setup time enormously.
One thing i noticed in my tests of 4.1 appliances is that 1GiB of RAM is just enough for the UCS installation. A normal appliance mode install still uses swap for a small amount. Add an App into the mix and 200MiB+ are easily used during system-setup. I think we should use at least 1.5GiB in the App Appliances by default.
We could add another option in the setup wizard on the role configuration page which allows a "demo mode", i.e., the master system has already been configured and joined with host and domain name. This would be very fast.
In a UCS appliance test installation firefox consumed more than 1min30sec cpu time during system-setup. Is there potential to reduce this to lower the time system setup requires?
(In reply to Erik Damrose from comment #3) > In a UCS appliance test installation firefox consumed more than 1min30sec > cpu time during system-setup. Is there potential to reduce this to lower the > time system setup requires? See also Bug 38866, I am not sure whether we can do something here... surely, not in the scope of this bug.
I have implemented a "demo mode", i.e., instead of executing setup-join.sh along with all join scripts, the appliance as a pre-configured system can be put into operation. For this, all setup scripts (along with the parameters --force-recreate and --appliance-mode as in setup-join.sh) and all appliance hooks/cleanup scripts are called. I needed to adapt some logic which relied on checking for /var/univention-join/joined and which is obsolete now. Therefore, I tried to simplify and clarify the process of putting a VM image into the appliance mode. Basically, a VM image can be changed to an appliance by installing univention-system-setup-boot. All necessary configuration settings via UCR (startsite, system/setup/boot/start, umc/module/setup/wizard/disabled) etc. have been transfered to the script /usr/sbin/univention-system-setup-boot [start|stop] which is called via the postinst/prerm of univention-system-setup-boot. The local Firefox instance within the VM is now called via JavaScript (window.close()). With this change, there is no need anymore for suspending the execution of cleanup scripts and thus holding the UMC session open until the user clicks on the "Finish" button. In turn, a join action is now always atomar, i.e., cleanup scripts are called (as at jobs) directly from the setup-join.sh script and from the save action (for a pre-configured system). Package yet needs to be built. univention-system-setup (9.1.4-2): r67635 | Bug #40046: Make the option for a pre-configured system blacklistable r67633 | Bug #40046: Update text in setup wizard for preconfigured system r67632 | Bug #40046: Update App Center cache via cleanup script r67613 | Bug #40046: Run setup scripts with options as in setup-join.sh + some JS cosmetics r67611 | Bug #40046: Adjust checks for /var/univention-join/joined and refactor config logic r67557 | Bug #40046: Adjust summary page for preconfigured test system r67556 | Bug #40046: make sure that cleanup scripts via at job log into setup.log r67555 | Bug #40046: Also change Administrator password in 10_basis/18root_password r67551 | Bug #40046: remove watch handler when polling progress information r67546 | Bug #40046: refine setup process for IP changes and request timeouts r67509 | Bug #40046: take into account IP address changes + bump package version r67505 | Bug #40046: clean up within join process + allow for preconfigured test sytem r67504 | Bug #40046: pre-compute city-data JSON files to improve build time r67503 | Bug #40046: run create-dh-parameter-files.sh in the background (from Bug #37459) r67502 | Bug #40046: allow setup of pre-configured system + close browser via JS univention-bootsplash (7.1.2-13): r67506 | Bug #40046: remove cleanup script for setup process
A "ucr commit /etc/apt/sources.list.d/*" needs to be executed during the fast setup at some point, otherwise the user may end up with empty sources.list files.
(In reply to Alexander Kläser from comment #6) > A "ucr commit /etc/apt/sources.list.d/*" needs to be executed during the > fast setup at some point, otherwise the user may end up with empty > sources.list files. This is implicitely done by fixing Bug 40710, isn't it?
It would be good to hide the new option for a fast setup process on an unjoined appliance system.
(In reply to Erik Damrose from comment #7) > (In reply to Alexander Kläser from comment #6) > > A "ucr commit /etc/apt/sources.list.d/*" needs to be executed during the > > fast setup at some point, otherwise the user may end up with empty > > sources.list files. > > This is implicitely done by fixing Bug 40710, isn't it? Yes, true :) . (In reply to Alexander Kläser from comment #8) > It would be good to hide the new option for a fast setup process on an > unjoined appliance system. Done. univention-system-setup (9.1.4-3): r68257 | Bug #40046: ensure that fast setup is only available on a joined system
On VirtualBox, I observed that the request for the status file were cached such that the end of the setup process could not be detected and the progress bar stayed visible indefinitely. univention-system-setup (9.1.4-4): r68298 | Bug #40046: make sure that requests for the status file are not cached
*** Bug 36807 has been marked as a duplicate of this bug. ***
The system UUID also needs to be regenerated for the fast demo setup mode. → REOPEN
(In reply to Alexander Kläser from comment #12) > The system UUID also needs to be regenerated for the fast demo setup mode. > > → REOPEN Will be handled via Bug 41140.
Reopen: If fast demo mode is chosen, the given nameserver value is not configured as dns/forwarder, but as nameserver1. However, nameserver1 should always be the system ip address itself.
If i select the system role as 'DC slave', i get the error 'The system role may not change on a system that has already joined to domain'. The error itself and the sentence should be fixed ;)
Created attachment 7687 [details] setup.log: failed ad join AD Join fails in 90_postjoin/10admember with the following traceback, setup.log is attached: Traceback (most recent call last): File "/usr/lib/pymodules/python2.7/univention/management/console/modules/setup/setup_script.py", line 304, in run success = self.inner_run() File "/usr/lib/univention-system-setup/scripts/90_postjoin/10admember", line 111, in inner_run admember.rename_well_known_sid_objects(username, password) File "/usr/lib/pymodules/python2.7/univention/lib/admember.py", line 865, in rename_well_known_sid_objects close_fds=True) File "/usr/lib/python2.7/subprocess.py", line 679, in __init__ errread, errwrite) File "/usr/lib/python2.7/subprocess.py", line 1259, in _execute_child raise child_exception OSError: [Errno 2] No such file or directory
TODO: building a DVD with the updated system-setup packages is currently only possible if the changes are merged and build in a -errata4.1-x scope. Tests with an updated DVD has to be done afterwards.
(In reply to Erik Damrose from comment #14) > Reopen: If fast demo mode is chosen, the given nameserver value is not > configured as dns/forwarder, but as nameserver1. However, nameserver1 should > always be the system ip address itself. Fixed. (In reply to Erik Damrose from comment #15) > If i select the system role as 'DC slave', i get the error 'The system role > may not change on a system that has already joined to domain'. > > The error itself and the sentence should be fixed ;) As discussed, I removed the role check in the python code. (In reply to Erik Damrose from comment #16) > Created attachment 7687 [details] > setup.log: failed ad join > > AD Join fails in 90_postjoin/10admember with the following traceback, > setup.log is attached: > Traceback (most recent call last): > File > "/usr/lib/pymodules/python2.7/univention/management/console/modules/setup/ > setup_script.py", line 304, in run > success = self.inner_run() > File "/usr/lib/univention-system-setup/scripts/90_postjoin/10admember", > line 111, in inner_run > admember.rename_well_known_sid_objects(username, password) > File "/usr/lib/pymodules/python2.7/univention/lib/admember.py", line 865, > in rename_well_known_sid_objects > close_fds=True) > File "/usr/lib/python2.7/subprocess.py", line 679, in __init__ > errread, errwrite) > File "/usr/lib/python2.7/subprocess.py", line 1259, in _execute_child > raise child_exception > OSError: [Errno 2] No such file or directory → Bug 37333, comment 3 ... I do not see that this case should be part of this bug. univention-system-setup (9.1.4-15): r70277 | Bug #40046: Set nameserver and forwader correctly for fast setup
(In reply to Erik Damrose from comment #16) > Created attachment 7687 [details] > setup.log: failed ad join > [...] See Bug 37333, comment 5: > Ok, as discussed with Alex, this is not a ucs-test case issue but one of the > system setup software selection: > > /usr/lib/univention-system-setup/scripts/90_postjoin/10admember is part of > univention-system-setup and runs into this problem if "ad/member" is enabled > *and* the AD-Connector has not been installed. Probably system-setup should > automatically select the AD-connector component (and make it so that the > user cannot de-select it in the software selection dialogue. So you would need to install AD Connector as component when doing this.
All changes from the app-appliance component have been merged over to 4.1-2.
REOPEN: world readable logfile http://jenkins.knut.univention.de:8080/job/UCS-4.1/job/UCS-4.1-2/job/AutotestJoin/lastCompletedBuild/SambaVersion=s4,Systemrolle=master/testReport/01_base/27check_logfiles_univention/test/
(In reply to Florian Best from comment #21) > REOPEN: world readable logfile > http://jenkins.knut.univention.de:8080/job/UCS-4.1/job/UCS-4.1-2/job/ > AutotestJoin/lastCompletedBuild/SambaVersion=s4,Systemrolle=master/ > testReport/01_base/27check_logfiles_univention/test/ TODO: touch && chmod 640
Should be fine now. Package is building... univention-system-setup (9.0.4-30): r70980 | Bug #40046: fix rights for log file dh-parameter-files-creation.log
(In reply to Alexander Kläser from comment #22) > TODO: touch && chmod 640 chown root:adm ? (not sure)
(In reply to Florian Best from comment #24) > (In reply to Alexander Kläser from comment #22) > > TODO: touch && chmod 640 > > chown root:adm ? (not sure) Possible, but not really important.
Jenkins says "repaired" :) .
OK: nameserver and slave join issues. OK: yaml BAD: DVD builds currently not working, skipping here. Will be done for ucs 4.1-3 Verified
FYI: svn r70330 introduced: Extra comma. (umc/js/setup/ApplianceWizard.js:2273:51) > password: password || null, (don't know if this causes issues in IE)
<http://errata.software-univention.de/ucs/4.1/233.html> <http://errata.software-univention.de/ucs/4.1/234.html>