Bug 29714 - Definierte Reihenfolge der Setup-Skripte
Definierte Reihenfolge der Setup-Skripte
Status: CLOSED FIXED
Product: UCS
Classification: Unclassified
Component: System setup
UCS 3.0
Other Linux
: P3 minor (vote)
: UCS 3.2
Assigned To: Arvid Requate
Alexander Kläser
: interim-1
Depends on:
Blocks: 32153
  Show dependency treegraph
 
Reported: 2012-12-11 09:17 CET by Alexander Kläser
Modified: 2013-11-19 06:44 CET (History)
4 users (show)

See Also:
What kind of report is it?: ---
What type of bug is this?: ---
Who will be affected by this bug?: ---
How will those affected feel about the bug?: ---
User Pain:
Enterprise Customer affected?:
School Customer affected?:
ISV affected?:
Waiting Support:
Flags outvoted after Product Owner Review:
Ticket number:
Bug group (optional):
Max CVSS v3 score:


Attachments
Print tracebacks in UMC setup module. (1.58 KB, patch)
2013-08-02 11:31 CEST, Alexander Kläser
Details | Diff
/var/log/univention/setup.log (40.52 KB, text/plain)
2013-08-02 13:54 CEST, Alexander Kläser
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Alexander Kläser univentionstaff 2012-12-11 09:17:31 CET
Derzeite ist die Reihenfolge der Setup-Skripte bzgl. ihrer Ordner nicht fix:

> for root, dirs, files in os.walk('/usr/lib/univention-system-setup/scripts/'): print root, dirs, files
> ... 
> /usr/lib/univention-system-setup/scripts/ ['basis', 'software', 'timezone', 'modules', 'net', 'join', 'defaultlocale', 'system_settings', 'role', 'ssl', 'language', 'keyboard'] ['ldif_rewrite.py', 'setup-join.sh', 'ldif_rewrite.pyc', 'setup_utils.sh', 'ldap_available.sh']
> /usr/lib/univention-system-setup/scripts/basis [] ['16windows_domain', '12domainname', '18root_password', '14ldap_basis', '10hostname']
> /usr/lib/univention-system-setup/scripts/software [] ['10software']
> /usr/lib/univention-system-setup/scripts/timezone [] ['10timezone']
> /usr/lib/univention-system-setup/scripts/modules [] []
> /usr/lib/univention-system-setup/scripts/net [] ['13ipv6gateway', '11ipv6interfaces', '16forwarder', '14nameserver', '18proxy', '12gateway', '10interfaces']
> /usr/lib/univention-system-setup/scripts/join [] []
> /usr/lib/univention-system-setup/scripts/defaultlocale [] []
> /usr/lib/univention-system-setup/scripts/system_settings [] []
> /usr/lib/univention-system-setup/scripts/role [] ['10role']
> /usr/lib/univention-system-setup/scripts/ssl [] ['10ssl']
> /usr/lib/univention-system-setup/scripts/language [] ['11default_locale', '10language']
> /usr/lib/univention-system-setup/scripts/keyboard [] ['10keyboard']

Sinnvoll wäre es wahrscheinlich, wenn die Order ebenfalls mit einem numerischen Präfix versehen und sortiert werden (wie die Skript-Dateien selber).
Comment 1 Stefan Gohmann univentionstaff 2012-12-12 08:40:09 CET
Wo werden die Skripte denn in einer beliebigen Reihenfolge ausgeführt?
Comment 2 Florian Best univentionstaff 2012-12-12 08:46:50 CET
(In reply to comment #1)
> Wo werden die Skripte denn in einer beliebigen Reihenfolge ausgeführt?
Bei mir wurde z.B. erst 10software ausgeführt und danach 10interfaces → Dadurch war noch kein Netzwerk vorhanden und es konnte keine Installation gestartet werden.
Comment 3 Stefan Gohmann univentionstaff 2012-12-12 08:56:58 CET
(In reply to comment #2)
> (In reply to comment #1)
> > Wo werden die Skripte denn in einer beliebigen Reihenfolge ausgeführt?
> Bei mir wurde z.B. erst 10software ausgeführt und danach 10interfaces → Dadurch
> war noch kein Netzwerk vorhanden und es konnte keine Installation gestartet
> werden.

Im Appliance Modus oder später?
Comment 4 Florian Best univentionstaff 2012-12-12 09:00:10 CET
(In reply to comment #3)
> (In reply to comment #2)
> > (In reply to comment #1)
> > > Wo werden die Skripte denn in einer beliebigen Reihenfolge ausgeführt?
> > Bei mir wurde z.B. erst 10software ausgeführt und danach 10interfaces → Dadurch
> > war noch kein Netzwerk vorhanden und es konnte keine Installation gestartet
> > werden.
> 
> Im Appliance Modus oder später?
Ich glaube es war auf einem gejointem system im system-setup-boot.
Comment 5 Moritz Muehlenhoff univentionstaff 2013-05-31 10:45:13 CEST
We will not ship a UCS 3.1-2 release; the next UCS release will be UCS 3.2.

As such, this bug is moved to the new target milestone.
Comment 6 Sönke Schwardt-Krummrich univentionstaff 2013-07-22 16:42:28 CEST
Please also fix setup-join.sh. The script does the following steps (among others) in this order:
1) SSL
1.1) run rdate 
1.2) create new SSL certificate
2) timezone
2.1) set timezone
2.2) /etc/init.d/ntp-server stop; /etc/init.d/ntpdate restart; 
     /etc/init.d/ntp-server start
3) power up the network

Problems:
→ rdate is run before the network is up → rdate works only if the systems starts via DHCP with valid route to the internet
→ ntp-server does not exist anymore → new script name seems to be "ntp"
→ ntpdate seems not to provide a init script anymore (ntpdate binary was 
  installed)
Comment 7 Alexander Kläser univentionstaff 2013-07-29 16:42:50 CEST
I observed this traceback:

> 29.07.13 10:11:45.416  MODULE      ( WARN    ) : Exception during saving the settings: global name 'stat' is not defined
>   File "/usr/lib/pymodules/python2.6/notifier/threads.py", line 82, in _run
>     tmp = self._function()
>   File "/usr/lib/pymodules/python2.6/notifier/__init__.py", line 104, in __call__
>     return self._function( *tmp, **self._kwargs )
>   File "/usr/lib/pymodules/python2.6/univention/management/console/modules/setup/__init__.py", line 172, in _thread
>     util.run_scripts( self._progressParser, restart )
>   File "/usr/lib/pymodules/python2.6/univention/management/console/modules/setup/util.py", line 348, in run_scripts
>     for scriptpath in sorted_files_in_subdirs( PATH_SETUP_SCRIPTS ):
>   File "/usr/lib/pymodules/python2.6/univention/management/console/modules/setup/util.py", line 335, in sorted_files_in_subdirs
>     if stat.S_ISDIR(os.stat(path).st_mode):
> 
> NameError: global name 'stat' is not defined

ps: I needed to print the traceback explicitely in order to see the traceback. Maybe adding a line to the debug such as 
> msg = '%s\n%s: %s\n' % (''.join(traceback.format_tb(thread.exc_info[2])), thread.exc_info[0].__name__, str(thread.exc_info[1]))
> MODULE.warn( 'Exception during saving the settings: %s\n%s' % (result, msg) )
would be helpful in the _finished() functions for future problems?
Comment 8 Arvid Requate univentionstaff 2013-07-29 19:02:10 CEST
* The directories are now ordered by number prefixes:

 05_role
 10_basis
 15_keyboard
 20_language
 25_defaultlocale
 30_net
 35_timezone
 40_ssl
 45_modules
 50_software
 55_system_settings
 60_join

* So, net is called before ntpdate (in 35_timezone) and before ssl.
* The python run_scripts function employed by the UMC command "setup/save" now iterates over these directories in the proper order.
* setup-join.sh also calls (a selection of) them "manually" in this order.
Comment 9 Alexander Kläser univentionstaff 2013-08-02 11:31:37 CEST
Created attachment 5348 [details]
Print tracebacks in UMC setup module.

The except is wrong, except is evil:

> +           except:
> +               msg = '%s\n%s: %s\n' % (''.join(traceback.format_tb(thread.exc_info[2])), thread.exc_info[0].__name__, str(thread.exc_info[1]))
> +               MODULE.warn( 'Exception during saving the settings: %s\n%s' % (result, msg) )

The check should be done in the finish callback function (see attached patch).
Comment 10 Alexander Kläser univentionstaff 2013-08-02 13:54:34 CEST
Created attachment 5351 [details]
/var/log/univention/setup.log

Changelog → OK
Appliance setup → OK

Could you add a MODULE.info() line prompting the currently executed script before calling subprocess.Popen() (that wasn't there before, either, but would be helpful for debugging) ?

The progress bar seems to be incorrect... in utils.py: the dict ProgressParser:FRACTIONS needs to be adapted to the new names.

Even then the progress is for the removal/installation of software components incorrect → I opened Bug 32148 for this.

Also, I get somewhat confused that some output related to the certificate generation is located before any script header (e.g., "=== 05_role/10role (2013-08-02 13:14:28) ===", see the attached log file). Maybe there is a problem related to stdout and stderr? Maybe a flush is missing?
Comment 11 Arvid Requate univentionstaff 2013-08-06 12:31:06 CEST
> MODULE.info() line prompting the currently executed script before calling 

Added.

> The progress bar seems to be incorrect... in utils.py:
> the dict ProgressParser:FRACTIONS needs to be adapted to the new names.

Adjusted.

> Also, I get somewhat confused that some output related to the certificate
> generation is located before any script header (e.g., "=== 05_role/10role
> (2013-08-02 13:14:28) ===", see the attached log file). Maybe there is a
> problem related to stdout and stderr? Maybe a flush is missing?

Popen call adjusted.
Comment 12 Alexander Kläser univentionstaff 2013-08-06 14:12:20 CEST
Looks good! → VERIFIED
Comment 13 Stefan Gohmann univentionstaff 2013-11-19 06:44:20 CET
UCS 3.2 has been released:
 http://docs.univention.de/release-notes-3.2-en.html
 http://docs.univention.de/release-notes-3.2-de.html

If this error occurs again, please use "Clone This Bug".