Univention Bugzilla – Bug 34243
Not-restarted UMC raises ImportError when opening modules of uninstalled apps
Last modified: 2014-12-05 11:31:14 CET
We received the following traceback: Traceback: Failed to import module agorum: No module named agorum Traceback (most recent call last): File "/usr/lib/pymodules/python2.6/univention/management/console/protocol/modserver.py", line 93, in _load_module self.__module = __import__(file_, [], [], modname) ImportError: No module named agorum I could reproduce this by 1. uninstalling agorum-app 2. not restarting the UMC-server 3. opening the still visible agorum module I think we can set this as WONTFIX because the user is warned and prompted to restart the UMC-Server after successful app removal. Or is here something to do in the view of usability?
We received also another traceback which was caused by ignoring the UMC-Server-Restart-Hint. Traceback: The init function of the module has failed: LDAP_ConnectionError: Opening LDAP connection failed: {'desc': "Can't contact LDAP server"} Traceback (most recent call last): File "/usr/lib/pymodules/python2.6/univention/management/console/protocol/modserver.py", line 228, in handle self.__handler.init() File "/usr/lib/pymodules/python2.6/univention/management/console/modules/udm/__init__.py", line 85, in init self.settings = UDM_Settings() File "/usr/lib/pymodules/python2.6/univention/management/console/modules/udm/udm_ldap.py", line 842, in __init__ self.read() File "/usr/lib/pymodules/python2.6/univention/management/console/modules/udm/udm_ldap.py", line 845, in read self._read_directories() File "/usr/lib/pymodules/python2.6/univention/management/console/modules/udm/udm_ldap.py", line 150, in wrapper_func raise LDAP_ConnectionError( 'Opening LDAP connection failed: %s' % str( e ) ) LDAP_ConnectionError: Opening LDAP connection failed: {'desc': "Can't contact LDAP server"} Remark: kurzt vorher das Root-Zertifikat aktualisiert (E-Mail, Bundesland, Ort) We should think about revising the concept. Some possibilities: * force restart of UMC with browser reload * pop up the message every minute and warn about possibly failures The most stable possibility would be: * store the credentials in the frontend * restart the UMC server in background * relogin * set language * rerender the module-overview → this destroys server state (the state is most often not needed within the modules).
(In reply to Florian Best from comment #1) > We received also another traceback which was caused by ignoring the > UMC-Server-Restart-Hint. → sorry, I meant: which could be caused by This was the way I reproduced it.
(In reply to Florian Best from comment #0) > ... > I think we can set this as WONTFIX because the user is warned and prompted > to restart the UMC-Server after successful app removal. > > Or is here something to do in the view of usability? Currently, we offer a restart after any operation that _could_ require a restart. It would be better to detect when a restart really is _necessary_ (e.g. via a UCR-Variable that is set) and then enforce this restart (in some way).
See also Bug 28081.
(In reply to Alexander Kläser from comment #3) > Currently, we offer a restart after any operation that _could_ require a > restart. It would be better to detect when a restart really is _necessary_ > (e.g. via a UCR-Variable that is set) and then enforce this restart (in some > way). Setting a UCR variable has the advantage that consequent errors/tracebacks could be commented with a message like "Please restart the UMC server and see if the problem persists".
For the agorum problem: See Bug#33202
A more feasible approach would be: catch the import error and send in the JSON error answer a flag that indicates that a UMC restart is recommended. Along with the error message UMC can then offer to the user to restart the UMC server.
(In reply to Alexander Kläser from comment #7) > A more feasible approach would be: catch the import error and send in the > JSON error answer a flag that indicates that a UMC restart is recommended. > Along with the error message UMC can then offer to the user to restart the > UMC server. Another idea: A UMC-Server restart is not necessary, a reload is enough. This can be triggered on ImportError.?!
(In reply to Florian Best from comment #1) > We should think about revising the concept. > Some possibilities: > * force restart of UMC with browser reload This is not a so good idea because maybe in the future we don't need to restart UMC anymore. It also requires user interaction. > * pop up the message every minute and warn about possibly failures This would probably disturbing for the user. We already have such dialog which e.g. pops up for every new opened module (instead of a time delay). > The most stable possibility would be: > * store the credentials in the frontend > * restart the UMC server in background > * relogin > * set language > * rerender the module-overview > → this destroys server state (the state is most often not needed within the > modules). This change is currently not worth it because it's too complex. (In reply to Alexander Kläser from comment #7) > A more feasible approach would be: catch the import error and send in the > JSON error answer a flag that indicates that a UMC restart is recommended. > Along with the error message UMC can then offer to the user to restart the > UMC server. A traceback / error message would still show up in that case. The user is already prompted to restart the UMC, and will be again on any operation like opening a module (because frontend-js-hash changed). (In reply to Florian Best from comment #8) > Another idea: A UMC-Server restart is not necessary, a reload is enough. > This can be triggered on ImportError.?! Triggering the reload on ImportError is too late, so the error message that the module doesn't exists would not show up. A UMC-Server restart isn't very often required afaik. We have the following cases: * A change in the list of modules (relogin (new session) required) * A new license was imported (→ currently the UDM moduleprocess have to die) * A new SSL certificate was created (→ Apache needs a restart) * new/changed extended attributes (don't know why, UDM module process have to die) * change of the own password (new password is not passed to the modules) * change of UMC-policies or UMC ACL's (?) any more? IMHO we should fix those points instead of workarounding by making a UMC-Server restart necessary. The fix for this bug addresses the first point: A function umc.app.reloadModules() has been implemented which reloads the module metainformation from the server. These information are reloaded before the modules/list command is executed. (which is literally Bug #33202). univention-management-console 6.0.24-4.777.201405301355 univention-management-console-frontend 3.0.152-27.826.201405301357 This function is used after installing/uninstalling/… a App in the AppCenter. univention-management-console-module-appcenter 3.0.51-15.249.201405301403 2014-05-26-univention-management-console-frontend.yaml 2014-05-30-univention-management-console-module-appcenter.yaml 2014-05-30-univention-management-console.yaml
I am able to crash the UMC server by logging in with FF and Chrome, uninstalling pkgdb in FF and opening it in Chrome 05.06.14 16:08:47.754 MAIN ( ERROR ) : Traceback (most recent call last): File "/usr/sbin/univention-management-console-server", line 209, in <module> umc_daemon.do_action() File "/usr/lib/pymodules/python2.6/daemon/runner.py", line 186, in do_action func(self) File "/usr/sbin/univention-management-console-server", line 142, in _restart self._start() File "/usr/lib/pymodules/python2.6/daemon/runner.py", line 131, in _start self.app.run() File "/usr/sbin/univention-management-console-server", line 192, in run notifier.loop() File "/usr/lib/pymodules/python2.6/notifier/nf_generic.py", line 284, in loop step() File "/usr/lib/pymodules/python2.6/notifier/nf_generic.py", line 271, in step not __sockets[ cond ][ fd ]( sock_obj ): File "/usr/lib/pymodules/python2.6/univention/management/console/protocol/server.py", line 165, in _receive self._handle( state, msg ) File "/usr/lib/pymodules/python2.6/univention/management/console/protocol/server.py", line 282, in _handle state.processor.request( msg ) File "/usr/lib/pymodules/python2.6/univention/management/console/protocol/session.py", line 276, in request self.handle_request_command( msg ) File "/usr/lib/pymodules/python2.6/univention/management/console/protocol/session.py", line 667, in handle_request_command mod_proc = ModuleProcess( module_name, debug = MODULE_DEBUG_LEVEL, locale = self.i18n.locale ) File "/usr/lib/pymodules/python2.6/univention/management/console/protocol/session.py", line 138, in __init__ modxmllist = moduleManager[ module ] KeyError: 'pkgdb'
I am not so sure how I like the patch. You do not really catch the error but hide the overview page. This may improve the usability, but there are too many issues left that should all be addressed. * Overview page is removed and then added again. Strange, if you are scrolling the overview at this moment. Can you operate on the store instead? * New Installed Apps do not show up in "Favorites" * There is a HEAD request before each module opening. This prevents using UMC after installing an application, although all modules are up to date now * Do we really need to restart the UMC now that the modules are all up to date? Maybe we can stop requesting a restart after App installation/uninstallation? * We need the reloadModules also after "normal" package management (also done in App Center) and possibly Updater (but this may be dangerous during release update) If we can get rid of the UMC restart in most cases, we still need to track a forced restart during package_manager.no_umc_restart(). See Bug#28081
(In reply to Dirk Wiesenthal from comment #10) > I am able to crash the UMC server by logging in with FF and Chrome, > uninstalling pkgdb in FF and opening it in Chrome > > 05.06.14 16:08:47.754 MAIN ( ERROR ) : Traceback (most recent call > last): > "/usr/lib/pymodules/python2.6/univention/management/console/protocol/session. > py", line 138, in __init__ > modxmllist = moduleManager[ module ] > KeyError: 'pkgdb' This error isn't new, it exists since the beginning of UMC. The error was that 1. the XML definition of a module has been removed (e.g. uninstallation) 2. moduleManager.load() (e.g. someone new logged in | "invoke-rc.d univention-management-console-server reload") 3. the ACL's/permitted commands weren't reloaded, so access to the module is still allowed. → a active session opened the not-anymore-existing-module caused the UMC server crash It has been fixed in SVN 50989.
sorry, not RESOLVED, the other comments are left ;)
(In reply to Dirk Wiesenthal from comment #11) > I am not so sure how I like the patch. You do not really catch the error but > hide the overview page. yep, that was untidy. Changed ;) > * Overview page is removed and then added again. Strange, if you are > scrolling the overview at this moment. Can you operate on the store instead? Done > * New Installed Apps do not show up in "Favorites" Fixed > * There is a HEAD request before each module opening. This prevents using > UMC after installing an application, although all modules are up to date now The umc_frontend_new_hash and invoke umc-server reload is not necessary anymore due to the changes. They have been removed from dh-umc-module-install. When packages with UMC modules are rebuild the dialogue doesn't occur anymore then. umc_frontend_new_hash is of course triggered on release updates in the UMC postinst.
> * Do we really need to restart the UMC now that the modules are all up to > date? Maybe we can stop requesting a restart after App > installation/uninstallation? A restart is only required in the cases listed above (Comment 9) plus also we cannot be sure that the appcenter may update the UMC server. So the restart dialog keeps existing. > * We need the reloadModules also after "normal" package management (also > done in App Center) and possibly Updater (but this may be dangerous during > release update) Yes, the reloading of modules is now done in lib.server.askRestart() if the user cancels the dialog. > If we can get rid of the UMC restart in most cases, we still need to track a > forced restart during package_manager.no_umc_restart(). See Bug#28081 → keeps being Bug#28081, the restart dialog still exists. Package: univention-management-console-module-lib Version: 3.0.7-2.23.201406121114 Package: univention-management-console-frontend Version: 3.0.152-29.828.201406121117 Package: univention-management-console-module-appcenter Version: 3.0.51-19.253.201406121123
No Apps installed => No "Installed Apps" category. Install App, cancel dialog: "All" and "Favorites" are empty, "Installed Apps" is not present.
(In reply to Dirk Wiesenthal from comment #16) > No Apps installed => No "Installed Apps" category. > > Install App, cancel dialog: "All" and "Favorites" are empty, "Installed > Apps" is not present. oh yes, the grid didn't have the new categories which caused this.
(In reply to Florian Best from comment #14) > ... > > * There is a HEAD request before each module opening. This prevents using > > UMC after installing an application, although all modules are up to date now > The umc_frontend_new_hash and invoke umc-server reload is not necessary > anymore due to the changes. They have been removed from > dh-umc-module-install. When packages with UMC modules are rebuild the > dialogue doesn't occur anymore then. > umc_frontend_new_hash is of course triggered on release updates in the UMC > postinst. removing umc_frontend_new_has from the postinst will mean that images (e.g., icons) that are updated with a new module version will probably not be updated in the browser itself due to caching problems? Not sure, but I fear that might be a problem. Could you please check?
(In reply to Alexander Kläser from comment #18) > removing umc_frontend_new_has from the postinst will mean that images (e.g., > icons) that are updated with a new module version will probably not be > updated in the browser itself due to caching problems? Not sure, but I fear > that might be a problem. Could you please check? yes, you are right. In IE8-11 the icons are cached. I reverted the removal of umc_frontend_new_hash. Also it has to be checked what happens in large environments, if the modules/list request takes too long due to the reload of ACL's.
(In reply to Florian Best from comment #19) > yes, you are right. In IE8-11 the icons are cached. > I reverted the removal of umc_frontend_new_hash. The same caching problem might apply to .js files. Instead, the handling of the reload hint (when the HEAD request failed) could be adapted (e.g., only prompt it once per session).
The reload of ACL's now only happens to existing sessions, not after a login because there isn't a session open so the ACL's are fresh. The 'invoke-rc.d umc reload' has been reintegrated → It's probably not required anymore but at least it doesn't matter if it is in there. The reload-UMC hint is now a singleton. Package: univention-management-console-frontend Version: 3.0.152-36.835.201407031523 Package: univention-management-console Version: 6.0.24-15.790.201407031531
Error introduced by revision 51504: Index: umc/tools.js =================================================================== --- umc/tools.js (Revision 51503) +++ umc/tools.js (Revision 51504) @@ -207,6 +207,8 @@ } }] }); + } else { + return; // show this dialog only once per session } if (!this._reloadDialog.open) { // check if UMC needs a browser reload and prompt the user to reload This patch has as consequence that no hint is displayed after calling umc_frontend_new_hash on the server side.
(In reply to Alexander Kläser from comment #22) > Error introduced by revision 51504: > > This patch has as consequence that no hint is displayed after calling > umc_frontend_new_hash on the server side. yes, fixed in univention-management-console-frontend 3.0.152-38.837.20140709084
Okay, looks fine. But the actual problem was not resolved, was it? This is still possible in a different session. Failed to import module pkgdb: No module named pkgdb Traceback (most recent call last): File "/usr/lib/pymodules/python2.6/univention/management/console/protocol/modserver.py", line 94, in _load_module self.__module = __import__(file_, [], [], modname) ImportError: No module named pkgdb
(In reply to Dirk Wiesenthal from comment #24) > Okay, looks fine. But the actual problem was not resolved, was it? This is > still possible in a different session. yes, in a different session the error is still possible. It is very unlikely to occur. Nevertheless i am able to catch the import error, too and check if the module doesn't exists to represent a human readable error message. The trade off here is that the error message can not be translated.
Implemented your wish! The error message is currently: > The requested module "modulid" does not exists. > This could be caused by a current removal of the module. > Please relogin to the Univention Management Console to see if the error persists. Happy with that string?
Package: univention-management-console Version: 6.0.24-17.792.201407101206
Well I fixed also the translation of strings before the SET locale requests comes in by changing NullTranslation → Translation and using the locale (not language) from what is given to the process with argument -l because the translation domain is a multiton. Some missing (unused before) translations have also been added.
The last sentence is not translated: Further information can be found in the logfile /var/log/univention/management-console-module-pkgdb. The filename is also lacking a ".log".
Question: Is it possible that the locale is also set when doing modules/list through lib.askRestart()? Currently, a newly installed app adds a module in "Installed apps" with the english locale until logged out and logged in again.
(In reply to Dirk Wiesenthal from comment #30) > Question: Is it possible that the locale is also set when doing modules/list > through lib.askRestart()? > > Currently, a newly installed app adds a module in "Installed apps" with the > english locale until logged out and logged in again. oh yes, the translation .mo file was modified while installing an app. The gettext instance is now reloaded. (In reply to Dirk Wiesenthal from comment #29) > The last sentence is not translated: > > Further information can be found in the logfile > /var/log/univention/management-console-module-pkgdb. > > The filename is also lacking a ".log". fixed.
Looks fine. Big change, but improves UMC's abilities. Minor flaws, but overall modules/list {reload: true} und modules/list {reload: false} can be used one day to not restart UMC completely (except UMC-updates, of course). Error message okay. YAML Ok
*** Bug 33202 has been marked as a duplicate of this bug. ***
http://errata.univention.de/ucs/3.2/145.html
http://errata.univention.de/ucs/3.2/149.html
reset status to VERIFIED until the package univention-management-console is also released as erratum.
http://errata.univention.de/ucs/3.2/159.html