Bug 38823

Summary: Add cache filter mechanism
Product: UCS Reporter: Philipp Hahn <hahn>
Component: Listener (univention-directory-listener)Assignee: Philipp Hahn <hahn>
Status: CLOSED FIXED QA Contact: Arvid Requate <requate>
Severity: normal    
Priority: P5 CC: gohmann, walkenhorst
Version: UCS 4.0   
Target Milestone: UCS 4.0-3-errata   
Hardware: Other   
OS: Linux   
See Also: https://forge.univention.org/bugzilla/show_bug.cgi?id=39439
https://forge.univention.org/bugzilla/show_bug.cgi?id=42662
What kind of report is it?: --- What type of bug is this?: ---
Who will be affected by this bug?: --- How will those affected feel about the bug?: ---
User Pain: Enterprise Customer affected?:
School Customer affected?: ISV affected?:
Waiting Support: Flags outvoted (downgraded) after PO Review:
Ticket number: Bug group (optional):
Max CVSS v3 score:
Bug Depends on:    
Bug Blocks: 40514    
Attachments: Issue #2387: Add Listener cache filter
ucs-test/tests/10_ldap/52listener-filter
qa.patch

Description Philipp Hahn univentionstaff 2015-07-02 16:06:58 CEST 
Created attachment 6995 [details]
Issue #2387: Add Listener cache filter

Add ability to Lisener to prevent certain objects being cached locally.

Implemented for customer <https://univention.plan.io/issues/2387> for UCS-3.2.
Needs to be merged into current UCS-4
Comment 1 Philipp Hahn univentionstaff 2015-07-15 12:00:52 CEST 
Created attachment 7027 [details]
ucs-test/tests/10_ldap/52listener-filter

The test shows a fundamental problem in the implementation: the handler is not called when the object is deleted!

This is caused by the fact, that on delete the listener neither knows about the object from the cache not is able to retrieve the data from the (remote) LDAP server, as the object is already deleted there. So the listener can't decide, if the module should be called.
Normally src/handlers.c:880 calls cache_entry_module_present() to check, if the module is registered for the object.

There already is a special-handling for the "replication" module in handlers_delete(). Maybe make that dynamic by using a UCRV to configure a list of modules to always call on delete?
Comment 2 Philipp Hahn univentionstaff 2015-09-03 13:25:33 CEST 
r63433 | Bug #38823 test: 10/52listener-filter
r63432 | Bug #38823 listener: Fix build issues
r63431 | Bug #38823 listener: Add cache filter mechanism
r63430 | Bug #38823 test: 10/52listener-filter
r63429 | Bug #38823 listener: Fix build issues
r63428 | Bug #38823 listener: Add cache filter mechanism

Package: univention-directory-listener
Version: 9.0.2-6.278.201509031256
Branch: ucs_4.0-0
Scope: errata4.0-3

Package: univention-directory-listener
Version: 10.0.0-1.279.201509031315
Branch: ucs_4.1-0

Package: ucs-test
Version: 5.0.169-6.1194.201509031319
Branch: ucs_4.0-0
Scope: errata4.0-3

Package: ucs-test
Version: 6.0.4-10.1193.201509031318
Branch: ucs_4.1-0

r63444 | Bug #34763 Listener: timeout, filter
 2015-09-32-univention-directory-listener.yaml
Comment 3 Arvid Requate univentionstaff 2015-09-23 13:36:10 CEST 
Created attachment 7182 [details]
qa.patch

Small suggestions.
Comment 4 Arvid Requate univentionstaff 2015-09-23 13:46:13 CEST 
Maybe also ad a debug message when the cache filter hits, sth like:

univention_debug(UV_DEBUG_LISTENER, UV_DEBUG_ALL, "Not caching %s, filtered out.", dn);
Comment 5 Philipp Hahn univentionstaff 2015-09-23 14:04:40 CEST 
The (In reply to Arvid Requate from comment #3)
> Created attachment 7182 [details]
> qa.patch
> 
> Small suggestions.
Rejected / modified:

1. Calling free(NULL) is valid; no need to nest it in an if.
2. univention_config_get_string() returns a dynamically allocated string; if setup_cache_filter() would be called a 2nd time, that would leak the memory.
3. Until we have a coding style, I'll follow linux/Documentation/CodingStyle:156
   > Do not unnecessarily use braces where a single statement will do.

(In reply to Arvid Requate from comment #4)
> Maybe also ad a debug message when the cache filter hits, sth like:
> 
> univention_debug(UV_DEBUG_LISTENER, UV_DEBUG_ALL, "Not caching %s, filtered
> out.", dn);

Added.

r63924 | Bug #38823,Bug #34763 Listener: LDAP timeout,filter
r63923 | Bug #38823,Bug #34763 Listener: LDAP timeout,filter

Package: univention-directory-listener
Version: 9.0.2-7.283.201509231400
Branch: ucs_4.0-0
Scope: errata4.0-3

Package: univention-directory-listener
Version: 10.0.0-2.282.201509231400
Branch: ucs_4.1-0

r63925 | Bug #38823,Bug #34763 Listener: LDAP timeout,filter YAML
 2015-09-32-univention-directory-listener.yaml
Comment 6 Arvid Requate univentionstaff 2015-09-23 15:43:29 CEST 
Currently the test 52listener-filter is skipped:

=========================================
Skip Message

E: Missing executable: /usr/bin/python2.6
=========================================

After fixing this I get this error in a UCS 4.0-3 Samba4 Slave:

E: Found other: ('ou=Domain Controllers,dc=ar40i1,dc=qa', True, False, 'n')
I: Found add: ('ou=ucs-test38823,dc=ar40i1,dc=qa', True, False, 'a')
I: Found modify: ('ou=ucs-test38823,dc=ar40i1,dc=qa', True, False, 'm')
Comment 7 Philipp Hahn univentionstaff 2015-09-23 16:29:14 CEST 
(In reply to Arvid Requate from comment #6)
> Currently the test 52listener-filter is skipped:
> E: Found other: ('ou=Domain Controllers,dc=ar40i1,dc=qa', True, False, 'n')

r63948 | Bug #38823 test: Fix 10/52listener-filter
r63947 | Bug #38823 test: Fix 10/52listener-filter

Package: ucs-test
Version: 5.0.171-10.1211.201509231622
Branch: ucs_4.0-0
Scope: errata4.0-3

Package: ucs-test
Version: 6.0.4-21.1210.201509231623
Branch: ucs_4.1-0
Comment 8 Arvid Requate univentionstaff 2015-09-24 13:15:56 CEST 
Ok.
Comment 9 Janek Walkenhorst univentionstaff 2015-09-30 12:56:02 CEST 
<http://errata.software-univention.de/ucs/4.0/336.html>
Comment 10 Philipp Hahn univentionstaff 2015-10-02 12:12:16 CEST 
Fixed ucs-test:
r64161 | Bug #38823 test: Wait for pending transactions
r64160 | Bug #38823 test: Wait for pending transactions