Univention Bugzilla – Bug 32213
Endless change check while searching for S4 changes
Last modified: 2016-12-21 16:27:56 CET
Discovered while testing with 65.000 users. The S4 connector seems to search for changes again and again without syncing an object: 11.08.2013 06:45:01,656 LDAP (PROCESS): sync to ucs: [ user] [ modify] uid=tim13-176,cn=schueler,cn=users,ou=176,dc=school65000,dc=local 11.08.2013 06:45:26,855 LDAP (PROCESS): S4 search continues, already found 2003 objects 11.08.2013 06:45:27,122 LDAP (PROCESS): S4 search continues, already found 3003 objects [...] 11.08.2013 06:48:04,691 LDAP (PROCESS): S4 search continues, already found 70003 objects 11.08.2013 06:48:04,944 LDAP (PROCESS): S4 search continues, already found 71003 objects 11.08.2013 07:03:19,233 LDAP (PROCESS): sync to ucs: [ user] [ modify] uid=tim13-176,cn=schueler,cn=users,ou=176,dc=school65000,dc=local 11.08.2013 07:03:45,507 LDAP (PROCESS): S4 search continues, already found 2003 objects 11.08.2013 07:03:46,721 LDAP (PROCESS): S4 search continues, already found 3003 objects [...] It seems the initial search is running into a timeout. +++ This bug was initially created as a clone of Bug #31695 +++ UCS should be tested with different environments with more than 5000 users, specially UMC and Samba 4. The tests should be modeled on existing UCS@school environments. We should test these two scenarios: 1. nearly 30.000 users with all users in Domain Users 2. nearly 65.000 users split into different schools Both scenarios should be tested with a central Samba 4 + S4 connector.
OK, the following changes have been done: r43103 * Search for uSNCreated and uSNChanged in one search to prevent a slow list comparison (Bug #32213) r43145 * Use again two searches for the change list but a faster comparison. The order is important first the created objects are needed otherwise the child object is added before the parent object. An alternative would be to sort the one search result by the dn length (Bug #32213) r43169 * Limit the changes from UCS to S4 for one interval to 50.000 changes. Otherwise it took too much time to generate the list of duplicated DNs. In my performance tests it too nearly one hour to check all 600.000 pickle files / 8 GB (Bug #32213) Changelog: r43523
This is tested by the automated jenkins tests. A setup with 1000 users and 30 groups (each group has all users as members) was successfully synchronized. Changelog OK
UCS 3.2 has been released: http://docs.univention.de/release-notes-3.2-en.html http://docs.univention.de/release-notes-3.2-de.html If this error occurs again, please use "Clone This Bug".