Bug 44422 - Improve performance of udm/query command
Improve performance of udm/query command
Status: RESOLVED WONTFIX
Product: UCS
Classification: Unclassified
Component: UMC - Users
UCS 4.2
Other Linux
: P5 normal (vote)
: ---
Assigned To: UMC maintainers
UMC maintainers
:
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2017-04-20 18:01 CEST by Jürn Brodersen
Modified: 2020-07-03 20:55 CEST (History)
1 user (show)

See Also:
What kind of report is it?: Development Internal
What type of bug is this?: ---
Who will be affected by this bug?: ---
How will those affected feel about the bug?: ---
User Pain:
Enterprise Customer affected?:
School Customer affected?:
ISV affected?:
Waiting Support:
Flags outvoted (downgraded) after PO Review:
Ticket number:
Bug group (optional): UCS Performance
Max CVSS v3 score:


Attachments
proposed patch (4.00 KB, patch)
2017-04-20 18:01 CEST, Jürn Brodersen
Details | Diff
proposed patch (4.07 KB, patch)
2017-06-27 12:14 CEST, Jürn Brodersen
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Jürn Brodersen univentionstaff 2017-04-20 18:01:10 CEST
Created attachment 8785 [details]
proposed patch

On my system with 600 users, 400 with jpegPhoto set (around 300MB in total), the "udm/query" command takes 42sec.

With the attached patch I can get it down to 3sec.

The patch removes: "module = get_module(object_type, obj.dn)" in management/univention-management-console-module-udm/umc/python/udm/__init__.py

The "get_module" call checks if the object still exists before adding it to the results. If that happened often in the past I would suggest to replace that call with a less expensive ldapsearch.

With the patch the fields options is passed to the lookup function in order to retrieve only the needed values from the ldap. Instead of retrieving all values and throwing the unneeded ones away.
I think that might also help with complicated acls, because the ldap server needs to check less attributes.
Comment 1 Florian Best univentionstaff 2017-04-25 16:07:51 CEST
The patch looks too invasive because it extends the UDM API as uses try-except TypeError for checking if the API change was implemented. Also UDM properties aren't LDAP attributes, so the query returns wrong/no results.
Comment 2 Jürn Brodersen univentionstaff 2017-06-27 12:14:04 CEST
Created attachment 8970 [details]
proposed patch

Before this gets moved even further down my TODO list ;)
I forget a line in users/user.py:
attributes = [ mapping.mapName(attribute) for attribute in attributes ]

A problem is that some attributes aren't set through mappings but instead during the init process of an object. E.g. "firstname" and "lastname". These would be missing. The "groups" attribute might also be a problem.

What would be a less invasive variant to extend the UDM API? Something like a capabilities variable that could be checked instead of that try except block?

TLDR: As long as there are attributes that are set/modified without a mapping, this patch would not be a good idea :(
Comment 3 Florian Best univentionstaff 2017-06-27 16:47:54 CEST
(In reply to Jürn Brodersen from comment #2)
> TLDR: As long as there are attributes that are set/modified without a
> mapping, this patch would not be a good idea :(
This won't change because there are also attributes which map to multiple properties.
Comment 4 Ingo Steuwer univentionstaff 2020-07-03 20:55:27 CEST
This issue has been filed against UCS 4.2.

UCS 4.2 is out of maintenance and many UCS components have changed in later releases. Thus, this issue is now being closed.

If this issue still occurs in newer UCS versions, please use "Clone this bug" or reopen it and update the UCS version. In this case please provide detailed information on how this issue is affecting you.