Univention Bugzilla – Bug 34877
replace ldapsearch with "pagedResults" ldapsearch in the listener
Last modified: 2023-02-01 17:24:15 CET
Currently the listener's ldapsearch is limited to "ldap/sizelimit" (from the master) results (default 400000). The listener can't handle setups with more than 400000 ldap objects (i guess). We need to: (1) configure limits for "pagedResults" ldapsearch in slapd.conf (e.g. add "size.pr=5000 size.prtotal=unlimited" to limits in slapd.conf, max 5000 results per page, unlimited pagedResults search), see bug #34873 (2) replace normale ldapsearch in the listener with "pagedResults" ldapsearch
It is possible to define multiple limits in slapd.conf and to overwrite the default sizelimit for special dns/groups with e.g. "unlimited". So maybe it is enough to add the special limit "size.soft=unlimited size.hard=unlimited" for cn=admin and slave hosts.
(In reply to Felix Botner from comment #1) > It is possible to define multiple limits in slapd.conf and to overwrite the > default sizelimit for special dns/groups with e.g. "unlimited". So maybe it > is enough to add the special limit "size.soft=unlimited size.hard=unlimited" > for cn=admin and slave hosts. sizelimit 4000 limits group/posixGroup/uniqueMember="cn=DC Backup Hosts,cn=groups,dc=fff,dc=ggg" size=unlimited time.soft=-1 time.hard=-1 limits dn="cn=admin,dc=fff,dc=ggg" size=unlimited time.soft=-1 time.hard=-1 limits users time.soft=-1 time.hard=-1 => sizelimit 4000 for everybody except cn=admin and members of DC Backup Hosts
*** Bug 40230 has been marked as a duplicate of this bug. ***
There is a Customer ID set so I set the flag "Enterprise Customer affected".
25.10.22 11:33:17.367 LISTENER ( WARN ) : initializing module replication File: /var/lib/univention-ldap/ldap/DB_CONFIG slapd: Kein Prozess gefunden File: /var/lib/univention-ldap/ldap/DB_CONFIG Starting slapd (via systemctl): slapd.service. 25.10.22 11:33:33.991 LISTENER ( ERROR ) : could not get DNs when initializing replication: Size limit exceeded
root@dc0:~# univention-ldapsearch -s sub -b "$(ucr get ldap/base)" -z max -l max -A -E pr=100/noprompt '(objectClass=*)' 1.1 |tail # requesting: 1.1 # with pagedResults control: size=100 # # search result search: 4003 result: 4 Size limit exceeded # numResponses: 404001 # numEntries: 400000
$ slapcat | grep -c ^dn: Customer has 1.5 M entries, but UCRV "ldap/sizelimit=400k" Using paged-results does not help to mitigate the limit. This becomes a BIG problem when joining a new Backup into a large environment or doing a re-join: In that case the replication.py module is not initialized at all: $ /usr/sbin/univention-directory-listener-dump|grep '^listenerModule: replication' Maybe also add a UMC diagnostics module to warn if "number of LDAP entries" exceeds UCRV "ldap/sizelimit". (Sadly I know of now simple way to just count the number of entries except the above mentioned command.)
Funny thing is, that we have got 25.10.22 11:33:33.991 LISTENER ( ERROR ) : could not get DNs when initializing replication: Size limit exceeded during listener cache recreation but starting the listener the normal way it was fine: 26.10.22 09:11:13.232 LISTENER ( WARN ) : initializing module replication File: /var/lib/univention-ldap/ldap/DB_CONFIG slapd: no process found File: /var/lib/univention-ldap/ldap/DB_CONFIG Starting slapd (via systemctl): slapd.service. 26.10.22 12:13:03.992 LISTENER ( WARN ) : finished initializing module replication with rv=0 3 replication /usr/lib/univention-directory-listener/system/replication.py
Workaround described here: https://help.univention.com/t/howto-fix-re-join-in-big-environments/20662
QA: - Code review - installed univention-ldap package - diff-ed /etc/ldap/slapd.conf before and after - tested size limit exceeded behaviour before and after.
Package: univention-ldap Version: 16.0.7-26A~5.0.0.202301201745 Branch: ucs_5.0-0 Scope: errata5.0-2 [5.0-2] 1087b64d44 Bug #34877: univention-ldap 16.0.7-26A~5.0.0.202301201745 doc/errata/staging/univention-ldap.yaml | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-)
QA: - Check YAML file for completeness and validity
<https://errata.software-univention.de/#/?erratum=5.0x568>