Bug 52175 - Use "sortvals" for uniqueMember and memberUid to improve performance of group membership changes
Summary: Use "sortvals" for uniqueMember and memberUid to improve performance of group...
Status: CLOSED FIXED
Alias: None
Product: UCS
Classification: Unclassified
Component: LDAP
Version: UCS 5.0
Hardware: Other Linux
: P5 normal
Target Milestone: UCS 5.0-9-errata
Assignee: Iván.Delgado
QA Contact: Felix Botner
URL: https://git.knut.univention.de/univen...
Keywords:
Depends on:
Blocks: 57065
  Show dependency tree / graph
 
Reported: 2020-10-02 10:07 CEST by Arvid Requate
Modified: 2025-04-14 22:19 CEST (History)
8 users (show)

See Also:
What kind of report is it?: Bug Report
What type of bug is this?: 5: Major Usability: Impairs usability in key scenarios
Who will be affected by this bug?: 1: Will affect a very few installed domains
How will those affected feel about the bug?: 5: Blocking further progress on the daily work
User Pain: 0.143
Enterprise Customer affected?:
School Customer affected?:
ISV affected?:
Waiting Support:
Flags outvoted (downgraded) after PO Review:
Ticket number: 2024100921000238
Bug group (optional): Large environments, UCS Performance
Customer ID:
Max CVSS v3 score:

Attachments
Time required for adding one uniqueMember entry to a group depending on the group size (100.19 KB, image/png)
2025-01-31 16:52 CET, Sönke Schwardt-Krummrich 		Details
Time required for removing one random uniqueMember entry from a group depending on the group size (both test series used the same randomized sequence → same seed) (106.99 KB, image/png)
2025-01-31 16:53 CET, Sönke Schwardt-Krummrich 		Details
View All Add an attachment (proposed patch, testcase, etc.)

Note You need to log in before you can comment on or make changes to this bug.
Description Arvid Requate univentionstaff 2020-10-02 10:07:02 CEST 
We should check, if we can improve the performance of modifications to large groups by using the sortvals parameter on uniqueMember and memberUid. The comment https://bugs.openldap.org/show_bug.cgi?id=9349#c6 recommends this.

       sortvals <attr> [...]
              Specify  a  list of multi-valued attributes whose values will always be maintained in sorted order. Using this option will allow Modify, Compare, and filter
              evaluations on these attributes to be performed more efficiently. The resulting sort order depends on the attributes' syntax and matching rules and may  not
              correspond to lexical order or any other recognizable order.
Comment 1 Dirk Wiesenthal univentionstaff 2025-01-30 17:42:27 CET 
We found the performance decreases exponentially when the groups get big (multiple thousands). We should give this option a try in a errata update.
Comment 2 Dirk Wiesenthal univentionstaff 2025-01-30 17:43:19 CET 
Performance can become so bad that requests get cancelled and the database is inconsistent.
Comment 3 Jürn Brodersen univentionstaff 2025-01-30 22:42:52 CET 
small typo, but broke the ldap server:

[5.0-9 c3d59f4502] Bug #52175: fix typo

already rebuild
Comment 4 Iván.Delgado univentionstaff 2025-01-31 15:07:23 CET 
Successful build
Package: univention-ldap
Version: 16.0.17-5
Branch: 5.0-0
Scope: errata5.0-9

univention-ldap.yaml
9ccad473e4de | Bug #52175: make sortvals configurable via UCR variable via ldap/server/sortvals
e646d6d03a53 | Bug #52175: Use sortvals in slapd.conf for uniqueMember and memberUid

univention-ldap (16.0.17-5)
9ccad473e4de | Bug #52175: make sortvals configurable via UCR variable via ldap/server/sortvals

univention-ldap (16.0.17-4)
c3d59f4502e7 | Bug #52175: fix typo

univention-ldap (16.0.17-3)
d1300226af13 | Bug #52175: Found a nicer place for sortvals

univention-ldap (16.0.17-2)
cd2d9aaa1e3a | Bug #52175: Use sortvals in slapd.conf for uniqueMember and memberUid
Comment 5 Sönke Schwardt-Krummrich univentionstaff 2025-01-31 16:52:25 CET 
Created attachment 11280 [details]
Time required for adding one uniqueMember entry to a group depending on the group size
Comment 6 Sönke Schwardt-Krummrich univentionstaff 2025-01-31 16:53:06 CET 
Created attachment 11281 [details]
Time required for removing one random uniqueMember entry from a group depending on the group size (both test series used the same randomized sequence → same seed)
Comment 7 Sönke Schwardt-Krummrich univentionstaff 2025-01-31 16:55:10 CET 
We should update the UCR variable description since it has a very limited or no effect when adding entries to an attribute (see attached graphs).
Comment 8 Iván.Delgado univentionstaff 2025-02-03 11:58:25 CET 
Successful build
Package: univention-ldap
Version: 16.0.17-6
Branch: 5.0-0
Scope: errata5.0-9


univention-ldap (16.0.17-6)
6c6f095edf2c | Bug #52175: Update description for `ldap/server/sortvals`
Comment 9 Daniel Tröder univentionstaff 2025-02-03 13:19:28 CET 
Sönke: Can you run those performance tests also for group sizes > 5000? Say 5, 10, 15, 20k members? IMHO that is where the pain is really big.