Bug 32978 - It takes too long to open a user
It takes too long to open a user
Status: CLOSED FIXED
Product: UCS
Classification: Unclassified
Component: UMC - Domain management (Generic)
UCS 3.1
Other Linux
: P5 normal (vote)
: UCS 3.2
Assigned To: Alexander Kläser
Dirk Wiesenthal
: interim-3
Depends on:
Blocks: 33148
  Show dependency treegraph
 
Reported: 2013-10-24 16:45 CEST by Stefan Gohmann
Modified: 2013-11-19 06:43 CET (History)
1 user (show)

See Also:
What kind of report is it?: ---
What type of bug is this?: ---
Who will be affected by this bug?: ---
How will those affected feel about the bug?: ---
User Pain:
Enterprise Customer affected?:
School Customer affected?:
ISV affected?:
Waiting Support:
Flags outvoted (downgraded) after PO Review:
Ticket number:
Bug group (optional):
Max CVSS v3 score:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Stefan Gohmann univentionstaff 2013-10-24 16:45:38 CEST
As discussed, the time for opening a user or a computer in UMC takes to much time. I think the progress bar suggests a long wait.
Comment 1 Alexander Kläser univentionstaff 2013-10-29 18:32:00 CET
Average loading time (including loading of dynamic values) has been optimized down to on average 1.66 sec in FF. It used to be on average 2.66 sec.

What has been optimized?
* UDM module related static information (i.e., layout, properties, policies) are preloaded when the UMC module is loaded. When editing/removing extended attributes via UMC, this cache is emptied as the static information may change through this modification.
* Policy information are loaded on demand when clicking onto the policy tab. A standby animation is then shown. (The standby animation does not seem to work with IE8, though.)
* Object LDAP values are loaded in parallel to the rendering of the object's detail page. (This data used to be loaded after rendering the full page.)
* The form is shown as soon as it is rendered. Until all dynamic values (i.e., combo boxes) have been loaded, the submit button is disabled.
* The rendering of the UDM detail pages has been refactorized and made asynchronously. This reduces the browser's DOM computation load a bit and seems to speed up the total process a bit. Overall, the browser is also more responsive when opening a UDM object.
* Dynamic widget values (i.e., combo boxes) are now loaded a bit earlier in the postCreate() method instead of the startup() method (→ change in UMC frontend).


 univention-management-console-module-udm (4.0.71-1) unstable; urgency=low
 .
   * Bug #32978: clean cache for UDM objects if extended attributes are modified
Comment 2 Alexander Kläser univentionstaff 2013-10-30 12:31:55 CET
QA: Please also check the correct functioning for non-admins that do have access to only one UDM module flavor.
Comment 3 Alexander Kläser univentionstaff 2013-10-30 13:13:54 CET
I added caching of more types. This speeds up opening the module search page. This was also done to speed up the dialogue for Bug 23214.

 univention-management-console-module-udm (4.0.72-1) unstable; urgency=low
 .
   * Bug #32978: cache more UDM values to speed up opening a UMC module
Comment 4 Alexander Kläser univentionstaff 2013-10-30 15:38:25 CET
(In reply to Alexander Kläser from comment #3)
> I added caching of more types. This speeds up opening the module search
> page. This was also done to speed up the dialogue for Bug 23214.
> 
>  univention-management-console-module-udm (4.0.72-1) unstable; urgency=low
>  .
>    * Bug #32978: cache more UDM values to speed up opening a UMC module

Two detail fixes:
* Cache for users/user module is reset when user templates are modified.
* Problems with superordinates (when adding/removing superordinates) have been fixed.

 univention-management-console-module-udm (4.0.73-1) unstable; urgency=low
 .
   * Bug #32978: fix cache reload with superordinates, reset users/user cache
     when editing or removing user templates
Comment 5 Alexander Kläser univentionstaff 2013-10-30 15:47:39 CET
Currently, there exist two scenarios when the caching masks possible changes in the backend:

* UCR variables are set to change module property syntaxes
* New UDM modules are installed (e.g., when installing UCC)

I opened Bug 33021 to handle theses issues. At the moment, this problem is not as important as a fast user interface. Additionally, problems will occur only in rare cases as after installing new software that extends UMC/UDM a reload of UMC is suggested.
Comment 6 Florian Best univentionstaff 2013-10-31 09:17:46 CET
When opening a user for editing which e.g. only have posix,person option all default options are checked and all tabs / titlepanes are shown.
Comment 7 Alexander Kläser univentionstaff 2013-10-31 10:42:58 CET
Noticed a static array reference that got fixed along the way.

 univention-management-console-frontend (3.0.139-1) unstable; urgency=low
 .
   * Bug #32978: fixed static array reference in umc/store
Comment 8 Florian Best univentionstaff 2013-10-31 11:00:56 CET
(In reply to Alexander Kläser from comment #2)
> QA: Please also check the correct functioning for non-admins that do have
> access to only one UDM module flavor.
Functionality seems OK, but:

On a very large system (>= 5000 Users)
udm/self UMCP command takes very long:

/umcp/command/udm/properties
22.38 s
/umcp/command/udm/layout
22.39 s
/umcp/command/udm/policies
22.40 s
Comment 9 Florian Best univentionstaff 2013-10-31 12:04:19 CET
The dynamicValues() function (e.g. in a Wizard._form) is now triggered before buildRendering() is executed. This causes the ucs-school-umc-installer module to crash when loading.
Comment 10 Florian Best univentionstaff 2013-10-31 12:17:45 CET
The password field in users module is not reseted/cleared after unsuccessful saving.
Comment 11 Florian Best univentionstaff 2013-10-31 12:23:21 CET
I only set "override PW history" on a user object and the following data is send along with the /udm/put request:

CtxBrokenSession: "0000"
CtxReconnectSession: "0000"
overridePWHistory: true
Comment 12 Florian Best univentionstaff 2013-10-31 12:26:22 CET
(In reply to Florian Best from comment #11)
> I only set "override PW history" on a user object and the following data is
> send along with the /udm/put request:
> 
> CtxBrokenSession: "0000"
> CtxReconnectSession: "0000"
> overridePWHistory: true
→ Changes are detected even if no changes have been done. Every object can be saved and these 2 values are send.
I had an request where these 2 values weren't send but:
CtxRASDialin: "E"
(when i changed password, overridePWHistory and overridePWLength)
Comment 13 Alexander Kläser univentionstaff 2013-10-31 12:50:09 CET
(In reply to Florian Best from comment #8)
> (In reply to Alexander Kläser from comment #2)
> > QA: Please also check the correct functioning for non-admins that do have
> > access to only one UDM module flavor.
> Functionality seems OK, but:
> 
> On a very large system (>= 5000 Users)
> udm/self UMCP command takes very long:
> 
> /umcp/command/udm/properties
> 22.38 s
> /umcp/command/udm/layout
> 22.39 s
> /umcp/command/udm/policies
> 22.40 s

This seems to be a problem with your system as S4 connector is still syncing:
>   PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
> 25010 root      20   0  515m 104m 3464 R 38.0  5.2 378:59.85 /usr/bin/python2.6 -W ignore /usr/lib/pymodules/python2.6/univention/s4connector/s4/main.py
Comment 14 Alexander Kläser univentionstaff 2013-10-31 12:56:22 CET
(In reply to Alexander Kläser from comment #13)
> This seems to be a problem with your system as S4 connector is still syncing:
> >   PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
> > 25010 root      20   0  515m 104m 3464 R 38.0  5.2 378:59.85 /usr/bin/python2.6 -W ignore /usr/lib/pymodules/python2.6/univention/s4connector/s4/main.py

Otherwise the main time is used for license evaluation:

> 27.10.13 15:04:42.022  MODULE      ( INFO    ) : Opening LDAP connection for user uid=u5800,cn=users,dc=ucs,dc=school
> 27.10.13 15:04:42.077  ADMIN       ( INFO    ) : LICENSE: None
> 27.10.13 15:04:42.077  ADMIN       ( INFO    ) : LICENSE: Version = 1
> 27.10.13 15:04:42.077  ADMIN       ( INFO    ) : LICENSE: Univention Accounts allowed 99999
> 27.10.13 15:04:42.077  ADMIN       ( INFO    ) : LICENSE: Accounts = 99999
> 27.10.13 15:04:42.078  ADMIN       ( INFO    ) : LICENSE: Univention Clients allowed 99999
> 27.10.13 15:04:42.078  ADMIN       ( INFO    ) : LICENSE: Clients = 99999
> 27.10.13 15:04:42.078  ADMIN       ( INFO    ) : LICENSE: Univention Desktops allowed 99999
> 27.10.13 15:04:42.079  ADMIN       ( INFO    ) : LICENSE: Desktops = 99999
> 27.10.13 15:04:42.079  ADMIN       ( INFO    ) : LICENSE: Univention Groupware Accounts allowed 99999
> 27.10.13 15:04:42.079  ADMIN       ( INFO    ) : LICENSE: Groupware Accounts = 99999
> 27.10.13 15:04:42.080  ADMIN       ( INFO    ) : LICENSE: Univention License Type allowed ('UCS', 'UGS')
> 27.10.13 15:04:42.080  ADMIN       ( INFO    ) : LICENSE: License Type = ('UCS', 'UGS')
> 27.10.13 15:04:42.080  ADMIN       ( INFO    ) : LICENSE: univentionLicenseOEMProduct attribute not found
> 27.10.13 15:04:42.081  ADMIN       ( INFO    ) : LICENSE: License Type = []
> 27.10.13 15:04:42.081  ADMIN       ( INFO    ) : LICENSE: Univention License end date allowed unlimited
> 27.10.13 15:04:42.081  ADMIN       ( INFO    ) : LICENSE: License end date = unlimited
> 27.10.13 15:04:42.114  ADMIN       ( INFO    ) : LICENSE: Univention sysAccountsFound: 5

→ 22sec
 
> 27.10.13 15:05:05.828  ADMIN       ( INFO    ) : LICENSE: Univention Accounts real 5015

→ 6sec

> 27.10.13 15:05:11.720  ADMIN       ( INFO    ) : LICENSE: Univention Clients real 3755
> 27.10.13 15:05:11.747  ADMIN       ( INFO    ) : LICENSE: Univention Desktops real 19
> 27.10.13 15:05:11.748  ADMIN       ( INFO    ) : LICENSE: Univention Groupware Accounts real 0
> 27.10.13 15:05:11.748  ADMIN       ( INFO    ) : LICENSE: Univention  allowed dc=ucs,dc=school
Comment 15 Alexander Kläser univentionstaff 2013-10-31 12:57:36 CET
(In reply to Florian Best from comment #11)
> I only set "override PW history" on a user object and the following data is
> send along with the /udm/put request:
> 
> CtxBrokenSession: "0000"
> CtxReconnectSession: "0000"
> overridePWHistory: true

This is another bug → Bug 29884
Comment 16 Alexander Kläser univentionstaff 2013-10-31 14:02:14 CET
(In reply to Florian Best from comment #9)
> The dynamicValues() function (e.g. in a Wizard._form) is now triggered
> before buildRendering() is executed. This causes the
> ucs-school-umc-installer module to crash when loading.

I reverted that change. In speed this change should not be noticeable.


 univention-management-console-frontend (3.0.140-1) unstable; urgency=low
 .
   * Bug #32978: revert changes in _SelectMixin from version 3.0.134-1
Comment 17 Alexander Kläser univentionstaff 2013-10-31 16:50:29 CET
(In reply to Florian Best from comment #6)
> When opening a user for editing which e.g. only have posix,person option all
> default options are checked and all tabs / titlepanes are shown.

Yes, this was my error. Fixed.

 univention-management-console-module-udm (4.0.84-1) unstable; urgency=low
 .
   * Bug #32978: fixed problems with option handling

QA: Please check also multi-editing.

(In reply to Florian Best from comment #10)
> The password field in users module is not reseted/cleared after unsuccessful
> saving.

Not scope of this bug.
Comment 18 Dirk Wiesenthal univentionstaff 2013-10-31 21:26:45 CET
Objects without options cannot be opened anymore.
Comment 19 Dirk Wiesenthal univentionstaff 2013-10-31 21:40:47 CET
Maybe just

_registerOptionWatchHandler: function() {
  var optionsWidget = this._form.widgets.$options$;
+ if (!optionsWidget) {
+   return;
+ }
  this.own(optionsWidget.watch('value', lang.hitch(this, function(attr, oldVal, newVal) {
    this.onOptionsChanged(newVal);
  })));
  ...


At least it worked for me. But I do not want to mess around. Please also see Bug#32381, Comment 17 - maybe caching problem? Or maybe just timing... I don't know.
Comment 20 Alexander Kläser univentionstaff 2013-11-01 10:23:37 CET
(In reply to Dirk Wiesenthal from comment #19)
> Maybe just
> 
> _registerOptionWatchHandler: function() {
>   var optionsWidget = this._form.widgets.$options$;
> + if (!optionsWidget) {
> +   return;
> + }
>   this.own(optionsWidget.watch('value', lang.hitch(this, function(attr,
> oldVal, newVal) {
>     this.onOptionsChanged(newVal);
>   })));
>   ...
> 
> At least it worked for me. But I do not want to mess around.

Yep, good point. I added the lines, thanks.


 univention-management-console-module-udm (4.0.85-1) unstable; urgency=low
 .
   * Bug #32978: handle UDM objects without options properly
Comment 21 Dirk Wiesenthal univentionstaff 2013-11-01 10:49:23 CET
moduleCache.getChildModules(options.superordinate) - as done in udm.js' widgets.objectType.dynamicValues does not return a proper {id: x, label:y} array. This leads to non human-readable (okay, it is readable...) values in DNS flavor.

I had to use the third parameter ("asIdLabelPair") in the wizards. Please check all cases where getChildModules is used.
Comment 22 Alexander Kläser univentionstaff 2013-11-01 12:01:19 CET
(In reply to Dirk Wiesenthal from comment #21)
> moduleCache.getChildModules(options.superordinate) - as done in udm.js'
> widgets.objectType.dynamicValues does not return a proper {id: x, label:y}
> array. This leads to non human-readable (okay, it is readable...) values in
> DNS flavor.
> 
> I had to use the third parameter ("asIdLabelPair") in the wizards. Please
> check all cases where getChildModules is used.

Done. I also removed the unneeded Deferred for the NewObjectDialog. Even when emptying the cache (require("umc/modules/udm/cache").reset()) and launching the wizard, there are no visibility problems.

 univention-management-console-module-udm (4.0.86-1) unstable; urgency=low
 .
   * Bug #23214: removed unneeded Deferred for NewObjectDialog, added autoHide
     for template combo box
   * Bug #32978: adjusted calls to getChildModules() to return id-label pairs
Comment 23 Dirk Wiesenthal univentionstaff 2013-11-01 14:18:26 CET
Open DHCP. Select a DHCP/Service in the tree. Verify that the grid is updated. Close the module. Open DHCP. Try to verify that the grid shows all superordinates and discover that the grid shows all objects below the previously selected DHCP/Service.
Comment 24 Dirk Wiesenthal univentionstaff 2013-11-01 14:22:39 CET
No, more generic: First opening of DHCP shows all "containers", further openings show all objects under first container.
Comment 25 Alexander Kläser univentionstaff 2013-11-01 14:30:42 CET
Adapted cache resetting when changing user templates or extended attributes.

 univention-management-console-module-udm (4.0.89-1) unstable; urgency=low
 .
   * Bug #32978: adapt handling of cache resetting
Comment 26 Alexander Kläser univentionstaff 2013-11-01 15:23:09 CET
 univention-management-console-module-udm (4.0.90-1) unstable; urgency=low
 .
   * Bug #32978: added cache resetting when objects are moved
Comment 27 Alexander Kläser univentionstaff 2013-11-01 15:50:46 CET
(In reply to Dirk Wiesenthal from comment #24)
> No, more generic: First opening of DHCP shows all "containers", further
> openings show all objects under first container.

Wrong guess → see Bug 32381, comment 19 (I fixed it there).
Comment 28 Dirk Wiesenthal univentionstaff 2013-11-04 10:33:08 CET
While not ready(), the form cannot be submitted but is already shown. This improves "felt" performance.

But: While not ready() you may edit fields. When the form finished loading the DetailPage does not recognize any changes done during that phase.
Comment 29 Dirk Wiesenthal univentionstaff 2013-11-04 10:37:45 CET
Open a printer driver list: univention/cups/HP -> opens immediately, MultiInputWidget is empty and growing... See Bug#31376
Comment 30 Dirk Wiesenthal univentionstaff 2013-11-04 10:45:15 CET
(In reply to Dirk Wiesenthal from comment #29)
> univention/cups/HP

Advice: Do not open HP. Better open Fujitsu with FF and Firebug (takes less than a second but you still get my point). HP takes ages. Also note that you may "Save changes". It does not seem to work but it is not grayed out.
Comment 31 Alexander Kläser univentionstaff 2013-11-04 14:26:36 CET
(In reply to Dirk Wiesenthal from comment #28)
> While not ready(), the form cannot be submitted but is already shown. This
> improves "felt" performance.
> 
> But: While not ready() you may edit fields. When the form finished loading
> the DetailPage does not recognize any changes done during that phase.

(In reply to Dirk Wiesenthal from comment #29)
> Open a printer driver list: univention/cups/HP -> opens immediately,
> MultiInputWidget is empty and growing... See Bug#31376

Both points have been addressed. The progress is now displayed on the disabled submit button. The initial values are now stored per widget as soon as the widget is ready.

 univention-management-console-module-udm (4.0.92-1) unstable; urgency=low
 .
   * Bug #32978: save initial form value as soon as each widget is ready,
     display progress of form loading
Comment 32 Dirk Wiesenthal univentionstaff 2013-11-04 16:22:39 CET
(In reply to Alexander Kläser from comment #17)
> QA: Please check also multi-editing.

Okay. Does not work! No "Overwrite" checkboxes
Comment 33 Alexander Kläser univentionstaff 2013-11-04 16:52:16 CET
(In reply to Dirk Wiesenthal from comment #32)
> (In reply to Alexander Kläser from comment #17)
> > QA: Please check also multi-editing.
> 
> Okay. Does not work! No "Overwrite" checkboxes

Fixed. That was an easy one ;) .

 univention-management-console-module-udm (4.0.93-1) unstable; urgency=low
 .
   * Bug #32978: fixed multi edit mode
Comment 34 Dirk Wiesenthal univentionstaff 2013-11-04 17:13:57 CET
(In reply to Alexander Kläser from comment #33)
> Fixed. That was an easy one ;) .

Okay. What about this one:

If editing an object (e.g. Printer, they take a bit longer if CUPS is installed). the submit button is greened out. But not when adding a new Printer: You may even click on the button and it validates the input before printer model and printer driver is loaded
Comment 35 Alexander Kläser univentionstaff 2013-11-04 17:36:28 CET
(In reply to Dirk Wiesenthal from comment #34)
> Okay. What about this one:
> 
> If editing an object (e.g. Printer, they take a bit longer if CUPS is
> installed). the submit button is greened out. But not when adding a new
> Printer: You may even click on the button and it validates the input before
> printer model and printer driver is loaded

Much better, but still easy enough. This hint led me to removing an unnecessary method.

 univention-management-console-module-udm (4.0.94-1) unstable; urgency=low
 .
   * Bug #32978: disable submit button when creating a new UDM object
Comment 36 Dirk Wiesenthal univentionstaff 2013-11-06 12:53:37 CET
(In reply to Alexander Kläser from comment #35)
> Much better, but still easy enough.

I have one more issue, but I put it in Bug#33148. Lucky you.

VERIFIED
Comment 37 Stefan Gohmann univentionstaff 2013-11-19 06:43:44 CET
UCS 3.2 has been released:
 http://docs.univention.de/release-notes-3.2-en.html
 http://docs.univention.de/release-notes-3.2-de.html

If this error occurs again, please use "Clone This Bug".