First Last Prev Next    This bug is not in your last search results.
Bug 40600 - /var/lib/univention-ldap/listener/listener isn't needed on member servers
/var/lib/univention-ldap/listener/listener isn't needed on member servers
Status: CLOSED FIXED
Product: UCS
Classification: Unclassified
Component: Listener (univention-directory-listener)
UCS 4.0
Other Linux
: P5 normal (vote)
: UCS 4.1-2-errata
Assigned To: Philipp Hahn
Arvid Requate
:
Depends on:
Blocks:
  Show dependency tree / graph
 
Reported: 2016-02-08 14:10 CET by Stefan Gohmann
Modified: 2016-07-21 15:16 CEST (History)
3 users (show)

See Also:
What kind of report is it?: ---
What type of bug is this?: ---
Who will be affected by this bug?: ---
How will those affected feel about the bug?: ---
User Pain:
Enterprise Customer affected?:
School Customer affected?:
ISV affected?:
Waiting Support:
Flags outvoted (downgraded) after PO Review:
Ticket number:
Bug group (optional):
Max CVSS v3 score:

Attachments
Add an attachment (proposed patch, testcase, etc.)

Note You need to log in before you can comment on or make changes to this bug.
Description Stefan Gohmann univentionstaff 2016-02-08 14:10:07 CET 
The file /var/lib/univention-ldap/listener/listener is written on member servers and DC slaves. As far as I can see. the file is only needed on a DC master or on other systems which have the notifier installed:
http://docs.software-univention.de/developer-reference-4.1.html#listener:procedure
Comment 1 Christina Scheinig univentionstaff 2016-02-08 14:16:10 CET 
Happened at ticket:2016020821000501
Comment 2 Philipp Hahn univentionstaff 2016-06-07 18:06:40 CEST 
r69909 | Bug #40600 UDL: Fix ucslint
r69904 | Bug #40600 UDL: Stop writing listener transaction log

Package: univention-directory-listener
Version: 10.0.0-11.315.201606071717
Branch: ucs_4.1-0
Scope: errata4.1-2

r69910 | Bug #22383,Bug #30227,Bug #30263,Bug #34324,Bug #34507,Bug #34738,Bug #3490,Bug #38696,Bug #39509,Bug #40600,Bug #41261: UDL YAML
 univention-directory-listener.yaml
Comment 3 Stefan Gohmann univentionstaff 2016-07-13 15:10:33 CEST 
One result is, that the listener doesn't stop anymore on a DC slave if a failed.ldif exists. Previously, the following message was written:

-------------------------------------------------------------------------------
18.11.15 20:40:34.647  LISTENER    ( ERROR   ) : Could not write to transaction file /var/lib/univention-ldap/listener/listener. Check for /var/lib/univention-directory-replication/failed.ldif

18.11.15 20:41:36.113  LISTENER    ( ERROR   ) : listener: -1
-------------------------------------------------------------------------------

I think we should stop the replication if a failed.ldif exists.
Comment 4 Stefan Gohmann univentionstaff 2016-07-13 17:17:51 CEST 
In case of a failed.ldif I get this log output:

--------------------------------------------------------------------------------
18.11.15 12:59:26.800  LISTENER    ( PROCESS ) : replication: rename phase I: univentionAppID=h4e3nbawcu_6.1.0,cn=h4e3nbawcu,cn=apps,cn=univention,dc=deadlock41,dc=intranet (entryUUID=1f376fba-2237-1035-8af8-f5de2d47ac32)
18.11.15 12:59:26.806  LISTENER    ( PROCESS ) : replication: rename phase I: univentionAppID=h4e3nbawcu_6.1.0,cn=h4e3nbawcu,cn=apps,cn=univention,dc=deadlock41,dc=intranet (entryUUID=1f376fba-2237-1035-8af8-f5de2d47ac32)
18.11.15 13:03:23.485  LISTENER    ( PROCESS ) : replication: the current modrdn points to a different entryUUID: /var/lib/univention-directory-replication/1f376fba-2237-1035-8af8-f5de2d47ac32
18.11.15 13:03:23.485  LISTENER    ( PROCESS ) : replication: the DN univentionAppID=h4e3nbawcu_6.1.0,cn=h4e3nbawcu,cn=apps,cn=univention,dc=deadlock41,dc=intranet from the current_modrdn_link has to be backuped and removed
18.11.15 13:03:23.490  LISTENER    ( PROCESS ) : replication: dump univentionAppID=h4e3nbawcu_6.1.0,cn=h4e3nbawcu,cn=apps,cn=univention,dc=deadlock41,dc=intranet to /var/univention-backup/replication/1447848203.49
18.11.15 13:03:23.492  LISTENER    ( ERROR   ) : dn=cn=w6wfrj9cuo,cn=apps,cn=univention,dc=deadlock41,dc=intranet: No such object
18.11.15 13:03:23.492  LISTENER    ( ERROR   ) :        machted dn: cn=h4e3nbawcu,cn=apps,cn=univention,dc=deadlock41,dc=intranet
18.11.15 13:03:23.493  LISTENER    ( PROCESS ) : replication: the current modrdn points to a different entryUUID: /var/lib/univention-directory-replication/1f376fba-2237-1035-8af8-f5de2d47ac32
18.11.15 13:03:23.493  LISTENER    ( PROCESS ) : replication: the DN univentionAppID=h4e3nbawcu_6.1.0,cn=h4e3nbawcu,cn=apps,cn=univention,dc=deadlock41,dc=intranet from the current_modrdn_link has to be backuped and removed
18.11.15 13:03:23.493  LISTENER    ( PROCESS ) : replication: dump univentionAppID=h4e3nbawcu_6.1.0,cn=h4e3nbawcu,cn=apps,cn=univention,dc=deadlock41,dc=intranet to /var/univention-backup/replication/1447848203.49
18.11.15 13:03:23.522  LISTENER    ( ERROR   ) : Could not write to transaction file /var/lib/univention-ldap/listener/listener. Check for /var/lib/univention-directory-replication/failed.ldif

18.11.15 13:03:23.522  LISTENER    ( ERROR   ) : listener: -1
Multifile: /etc/ldap/slapd.conf
18.11.15 13:03:30.307  DEBUG_INIT
--------------------------------------------------------------------------------

So, it is clear that a failed.ldif exists. With the updated listener and replication module I get:
--------------------------------------------------------------------------------
root@slave413:~# ls -la /var/lib/univention-directory-replication/
insgesamt 12
drwx------  2 listener root 4096 Nov 18 12:57 .
drwxr-xr-x 58 root     root 4096 Nov 18 12:55 ..
-rw-------  1 listener root  419 Nov 18 12:15 replayed.ldif_2015-11-18-12:19:02
root@slave413:~# tail -f /var/log/univention/listener.log
0: backup412.deadlock41.intranet
1: master411.deadlock41.intranet
UNIVENTION_DEBUG_BEGIN  : uldap.__open host=backup412.deadlock41.intranet port=7389 base=dc=deadlock41,dc=intranet
UNIVENTION_DEBUG_END    : uldap.__open host=backup412.deadlock41.intranet port=7389 base=dc=deadlock41,dc=intranet
18.11.15 12:57:10.347  LISTENER    ( PROCESS ) : move_same_dn(univentionAppID=jw8oy3ibcc_7.3.7,cn=jw8oy3ibcc,cn=apps,cn=univention,dc=deadlock41,dc=intranet)
18.11.15 12:57:10.348  LISTENER    ( PROCESS ) : replication: rename phase I: univentionAppID=jw8oy3ibcc_7.3.7,cn=jw8oy3ibcc,cn=apps,cn=univention,dc=deadlock41,dc=intranet (entryUUID=5864107c-2237-1035-8af8-f5de2d47ac32)
18.11.15 12:57:10.349  LISTENER    ( PROCESS ) : replication: rename phase II: univentionAppID=jw8oy3ibcc_7.3.7,cn=jw8oy3ibcc,cn=apps,cn=univention,dc=deadlock41,dc=intranet (entryUUID=5864107c-2237-1035-8af8-f5de2d47ac32)
18.11.15 12:57:10.349  LISTENER    ( PROCESS ) : replication: the rename target already exists in the local LDAP, backup and remove the dn: univentionAppID=jw8oy3ibcc_7.3.7,cn=jw8oy3ibcc,cn=apps,cn=univention,dc=deadlock41,dc=intranet
18.11.15 12:57:10.349  LISTENER    ( PROCESS ) : replication: dump univentionAppID=jw8oy3ibcc_7.3.7,cn=jw8oy3ibcc,cn=apps,cn=univention,dc=deadlock41,dc=intranet to /var/univention-backup/replication/1447847830.35
18.11.15 12:57:12.964  LISTENER    ( PROCESS ) : samba4-idmap: added entry for S-1-4-2015
18.11.15 12:57:40.882  LISTENER    ( PROCESS ) : samba4-idmap: renaming entry for S-1-4-2015 to S-1-5-21-3869528978-413072183-2970949430-1114
2387
18.11.15 13:00:45.818  LISTENER    ( PROCESS ) : ldap_extension: Reloading LDAP server.
Initiating graceful reload of ldap server(s).
Sending HUP to ldap server(s): slapd ...done.
Starting ldap server(s): slapd ...done.
18.11.15 13:00:58.252  LISTENER    ( WARN    ) : replication: Can't contact LDAP server: retrying
18.11.15 13:00:58.429  LISTENER    ( PROCESS ) : samba4-idmap: removing entry for S-1-5-21-3869528978-413072183-2970949430-1114
18.11.15 13:01:18.070  LISTENER    ( PROCESS ) : replication: rename phase I: univentionAppID=jw8oy3ibcc_7.3.7,cn=jw8oy3ibcc,cn=apps,cn=univention,dc=deadlock41,dc=intranet (entryUUID=5864107c-2237-1035-8af8-f5de2d47ac32)
18.11.15 13:01:18.077  LISTENER    ( PROCESS ) : replication: rename phase I: univentionAppID=jw8oy3ibcc_7.3.7,cn=jw8oy3ibcc,cn=apps,cn=univention,dc=deadlock41,dc=intranet (entryUUID=5864107c-2237-1035-8af8-f5de2d47ac32)
18.11.15 13:01:50.900  LISTENER    ( PROCESS ) : replication: the current modrdn points to a different entryUUID: /var/lib/univention-directory-replication/5864107c-2237-1035-8af8-f5de2d47ac32
18.11.15 13:01:50.900  LISTENER    ( PROCESS ) : replication: the DN univentionAppID=jw8oy3ibcc_7.3.7,cn=jw8oy3ibcc,cn=apps,cn=univention,dc=deadlock41,dc=intranet from the /var/lib/univention-directory-replication/current_modrdn has to be backuped and removed
18.11.15 13:01:50.900  LISTENER    ( PROCESS ) : replication: dump univentionAppID=jw8oy3ibcc_7.3.7,cn=jw8oy3ibcc,cn=apps,cn=univention,dc=deadlock41,dc=intranet to /var/univention-backup/replication/1447848110.9
18.11.15 13:01:50.910  LISTENER    ( ERROR   ) : replication: No such object; dn="cn=leuox3tyvj,cn=apps,cn=univention,dc=deadlock41,dc=intranet": Error
18.11.15 13:01:50.910  LISTENER    ( ERROR   ) :        machted dn: cn=jw8oy3ibcc,cn=apps,cn=univention,dc=deadlock41,dc=intranet
18.11.15 13:01:50.910  LISTENER    ( PROCESS ) : replication: the current modrdn points to a different entryUUID: /var/lib/univention-directory-replication/5864107c-2237-1035-8af8-f5de2d47ac32
18.11.15 13:01:50.910  LISTENER    ( PROCESS ) : replication: the DN univentionAppID=jw8oy3ibcc_7.3.7,cn=jw8oy3ibcc,cn=apps,cn=univention,dc=deadlock41,dc=intranet from the /var/lib/univention-directory-replication/current_modrdn has to be backuped and removed
18.11.15 13:01:50.921  LISTENER    ( PROCESS ) : change_update_entry for parent_dn: cn=leuox3tyvj,cn=apps,cn=univention,dc=deadlock41,dc=intranet
18.11.15 13:01:50.946  LISTENER    ( PROCESS ) : change_update_entry for parent_dn: cn=leuox3tyvj,cn=apps,cn=univention,dc=deadlock41,dc=intranet
18.11.15 13:01:50.948  LISTENER    ( PROCESS ) : move_same_dn(univentionAppID=leuox3tyvj_2.8.2,cn=leuox3tyvj,cn=apps,cn=univention,dc=deadlock41,dc=intranet)
18.11.15 13:01:50.949  LISTENER    ( PROCESS ) : replication: rename phase I: univentionAppID=leuox3tyvj_2.8.2,cn=leuox3tyvj,cn=apps,cn=univention,dc=deadlock41,dc=intranet (entryUUID=fffc7f0e-2237-1035-939f-157a1097949c)
18.11.15 13:01:50.949  LISTENER    ( PROCESS ) : replication: rename phase II: univentionAppID=leuox3tyvj_2.8.2,cn=leuox3tyvj,cn=apps,cn=univention,dc=deadlock41,dc=intranet (entryUUID=fffc7f0e-2237-1035-939f-157a1097949c)
18.11.15 13:01:50.962  LISTENER    ( PROCESS ) : change_update_entry for parent_dn: cn=leuox3tyvj,cn=apps,cn=univention,dc=deadlock41,dc=intranet
18.11.15 13:01:52.941  LISTENER    ( PROCESS ) : samba4-idmap: added entry for S-1-4-2016
18.11.15 13:01:57.113  LISTENER    ( PROCESS ) : samba4-idmap: renaming entry for S-1-4-2016 to S-1-5-21-3869528978-413072183-2970949430-1115
^C
root@slave413:~# ls -la /var/lib/univention-directory-replication/
insgesamt 28
drwx------  2 listener root     4096 Nov 18 13:01 .
drwxr-xr-x 58 root     root     4096 Nov 18 12:55 ..
-rw-------  1 listener nogroup    94 Nov 18 13:01 5864107c-2237-1035-8af8-f5de2d47ac32
-rw-------  1 listener nogroup 10282 Nov 18 13:01 failed.ldif
-rw-------  1 listener root      419 Nov 18 12:15 replayed.ldif_2015-11-18-12:19:02
root@slave413:~#
--------------------------------------------------------------------------------

After reading the logs, I have no idea that a failed.ldif has been generated.
Comment 5 Arvid Requate univentionstaff 2016-07-13 18:27:42 CEST 
Ok, to recap, the following two points should be implemented:

1. stop the replication if a failed.ldif exists
2. output a corresponding log message
Comment 6 Philipp Hahn univentionstaff 2016-07-18 17:53:43 CEST 
(In reply to Arvid Requate from comment #5)
> Ok, to recap, the following two points should be implemented:
> 
> 1. stop the replication if a failed.ldif exists
> 2. output a corresponding log message

r71071 | Bug #40600 UDL: Check for failed.ldif

Package: univention-directory-listener
Version: 10.0.0-14.325.201607181745
Branch: ucs_4.1-0
Scope: errata4.1-2

r71075 | Bug #40600,Bug #41261,Bug #34324,Bug #3490 UDL: YAML
 univention-directory-listener.yaml
Comment 7 Arvid Requate univentionstaff 2016-07-18 18:43:36 CEST 
The indentation in src/transfile.c is messed up now:

Index: src/transfile.c
===================================================================
--- src/transfile.c     (Revision 71070)
+++ src/transfile.c     (Revision 71071)
[...]
-       if (stat(failed_ldif_file, &stat_buf) != 0) {
+       assert(!notifier_has_failed_ldif());
                if ((file = fopen_lock(transaction_file, "a+", &l_file)) == NULL) {
===================================================================

Just curious: What's the reason for this change?
===================================================================
--- src/notifier.c      (Revision 71070)
+++ src/notifier.c      (Revision 71071)
@@ -164,7 +164,7 @@
                /* Try to do the change. If the LDAP server is down, try
                   to reconnect */
                while ((rv = change_update_dn(&trans)) != LDAP_SUCCESS) {
-                       univention_debug(UV_DEBUG_LISTENER, UV_DEBUG_ERROR, "change_update_dn: %s", ldap_err2string(rv));
+                       univention_debug(UV_DEBUG_LISTENER, UV_DEBUG_ERROR, "change_update_dn failed: %d", rv);
                        if (rv == LDAP_SERVER_DOWN)
                                if ((rv = connect_to_ldap(trans.lp, kp)) == 0)
                                        continue
===================================================================
Comment 8 Philipp Hahn univentionstaff 2016-07-18 19:26:39 CEST 
(In reply to Arvid Requate from comment #7)
> The indentation in src/transfile.c is messed up now:

So what? It C-code, which doesn't care about and re.indenting makes the patch less readable. But if you persist, I can add a patch on top to fix the white space.


> Just curious: What's the reason for this change?
...
> -                       univention_debug(UV_DEBUG_LISTENER, UV_DEBUG_ERROR,
> "change_update_dn: %s", ldap_err2string(rv));
> +                       univention_debug(UV_DEBUG_LISTENER, UV_DEBUG_ERROR,
> "change_update_dn failed: %d", rv);

See our discussion in Bug #32475 comment 3: change_update_dn() returns values which look like LDAP values, but more often are not related to actual LDAP Problems at all. Calling ldap_err2str() on those values is more confusing than helping (IMHO)
Comment 9 Arvid Requate univentionstaff 2016-07-18 19:40:49 CEST 
Ok, whatever, I'm not into whitespace issues anyway, I just don't get the logic:
The usual argument is that code is more often read than written.

According to your argument it's pointless to trim trailing whitespace either, isn't it? Which is fine with me.

Maybe we can keep the tone at a professional level though: As QA I just pointed out a deficit, according to your usual standards, and I don't expect to be scolded for that. I happen to understand that it is C-Code and I happen to understand that the C-Compiler doesn't care, just as I don't. And I think you know that. You could e.g. just say: "Yeah, I think that's ok, if you don't insist."


Code review: Ok
Functional test: Ok
Advisory: Ok
Comment 10 Janek Walkenhorst univentionstaff 2016-07-21 15:16:13 CEST 
<http://errata.software-univention.de/ucs/4.1/215.html>
First Last Prev Next    This bug is not in your last search results.