Bug 40514 - Initialize ldap handler without listener (a simple slapadd instead of replication.py)
Initialize ldap handler without listener (a simple slapadd instead of replica...
Status: NEW
Product: UCS
Classification: Unclassified
Component: Listener (univention-directory-listener)
UCS 4.4
Other Linux
: P5 normal (vote)
: ---
Assigned To: UCS maintainers
:
: 35170 (view as bug list)
Depends on: 35165 35170 38823
Blocks:
  Show dependency treegraph
 
Reported: 2016-01-27 14:17 CET by Michel Smidt
Modified: 2019-04-24 14:23 CEST (History)
5 users (show)

See Also:
What kind of report is it?: Feature Request
What type of bug is this?: ---
Who will be affected by this bug?: ---
How will those affected feel about the bug?: ---
User Pain:
Enterprise Customer affected?: Yes
School Customer affected?:
ISV affected?:
Waiting Support:
Ticket number:
Bug group (optional): Forked for project
Max CVSS v3 score:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Michel Smidt univentionstaff 2016-01-27 14:17:47 CET
The univention-directory-listener was forked for customer.
Changes:
* Initialize handlers only. Issue #570
* "fake" listener handler initialization. Issue #570
* Added database copy support (From the changelog).
* Performance in univention-get-ldif-from-master.py (From the changelog)
* Prevent certain LDAP objects from being stored in the local Listener cache. Issue #2387

Goal is to create an API or merge the fork to the product.
Comment 1 Philipp Hahn univentionstaff 2016-01-27 16:40:30 CET
(In reply to Michel Smidt from comment #0)
> The univention-directory-listener was forked for customer.
> Changes:
> * Initialize handlers only. Issue #570
> * "fake" listener handler initialization. Issue #570
Bug #35165

> * Added database copy support (From the changelog).
> * Performance in univention-get-ldif-from-master.py (From the changelog)
Bug #35170

> * Prevent certain LDAP objects from being stored in the local Listener
> cache. Issue #2387
Already implemented via Bug #38823
Comment 2 Stefan Gohmann univentionstaff 2016-04-27 16:58:21 CEST
Please provide more information about the goal and split these issues into separate bugs.
Comment 3 Michel Smidt univentionstaff 2016-05-31 11:26:04 CEST
(In reply to Stefan Gohmann from comment #2)
> Please provide more information about the goal and split these issues into
> separate bugs.

Only one change left:
* Performance in univention-get-ldif-from-master.py (From the changelog)

Asked author for more information.
Comment 4 Felix Botner univentionstaff 2016-06-03 12:34:20 CEST
(In reply to Michel Smidt from comment #3)
> (In reply to Stefan Gohmann from comment #2)
> > Please provide more information about the goal and split these issues into
> > separate bugs.
> 
> Only one change left:
> * Performance in univention-get-ldif-from-master.py (From the changelog)
> 
> Asked author for more information.

There is a new script python/univention-get-ldif-from-master.py in the univention-directory-listener source package, This script is used during the fake ldap initialization (if no ldif or database backend file is provided) to create a local ldif of the masters ldap database.

The commits 

 a8376edbdc566aea4ccd919f32b9a0662034f3fb
 ce122e862d78ffc03a1e1c4ba42f625ca2e23221

improved the performance of the ldap search script (univention-get-ldif-from-master.py) and the import of the ldif in 03univention-directory-listener.inst.
Comment 5 Michel Smidt univentionstaff 2016-06-14 12:25:39 CEST
Intention of this changes and the related bugs #35165, #35170 and #38823 was to initialize handlers without listeners. The initialization of the listener handlers will be skipped and in some cases initialized differently or just don't initialized.

For the LDAP the script python/univention-get-ldif-from-master.py was introduced.
This script basically read the ldap from the DC master, create a ldif file and update the local schema.

This bug is about this "fake replication" and the optimizations.
Comment 6 Ingo Steuwer univentionstaff 2016-06-23 09:40:34 CEST
In addition: the main goal here is to join in an environment that has more entries in LDAP than the sizelimit/timelimit allows to replicate during join. By initializing the replication listener plugin "manually" and deploy the LDAP based on an LDIF we avoid to run into timeouts. Similar processing is needed for other listener plugins.
Comment 7 Arvid Requate univentionstaff 2016-06-23 11:45:22 CEST
Generally we should adjust the listener to use paging during the initial search but I understand there's still a performance issue with very large data sets.
Also Felix reminded me that the listener sorts the returned DNs, but I doubt that that's still necessary with current OpenLDAP versions.
Comment 8 Philipp Hahn univentionstaff 2016-06-23 12:12:57 CEST
(In reply to Arvid Requate from comment #7)
> Also Felix reminded me that the listener sorts the returned DNs, but I doubt
> that that's still necessary with current OpenLDAP versions.

The "sorting" is used to guarantee that parent nodes are pulled before their children, e.g. "pre-order".
AFAIK ldapsearch doesn't guarantee any ordering by default, so "post-order" is allowed by the spec:
<https://tools.ietf.org/html/rfc4511#section-4.5.2>
> The SearchResultEntry and SearchResultReference messages may come in any order.

Sorting by »len(dn)« would be enough.
Comment 9 Philipp Hahn univentionstaff 2016-07-19 09:15:39 CEST
*** Bug 35170 has been marked as a duplicate of this bug. ***
Comment 10 Stefan Gohmann univentionstaff 2017-06-16 20:36:09 CEST
This issue has been filed against UCS 3. UCS 3 is out of the normal maintenance and many UCS components have vastly changed in UCS 4.

If this issue is still valid, please change the version to a newer UCS version otherwise this issue will be automatically closed in the next weeks.
Comment 11 Florian Best univentionstaff 2017-06-28 14:52:23 CEST
There is a Customer ID set so I set the flag "Enterprise Customer affected".