Bug 51061 - replication.py: IndexError while logging LDAP exceeption
replication.py: IndexError while logging LDAP exceeption
Status: CLOSED FIXED
Product: UCS
Classification: Unclassified
Component: LDAP
UCS 4.4
amd64 Linux
: P5 normal (vote)
: UCS 4.4-4-errata
Assigned To: Philipp Hahn
Arvid Requate
:
Depends on:
Blocks: 48545 51062
  Show dependency treegraph
 
Reported: 2020-04-03 14:51 CEST by Philipp Hahn
Modified: 2020-04-15 17:41 CEST (History)
4 users (show)

See Also:
What kind of report is it?: Bug Report
What type of bug is this?: 7: Crash: Bug causes crash or data loss
Who will be affected by this bug?: 2: Will only affect a few installed domains
How will those affected feel about the bug?: 5: Blocking further progress on the daily work
User Pain: 0.400
Enterprise Customer affected?: Yes
School Customer affected?:
ISV affected?:
Waiting Support:
Flags outvoted (downgraded) after PO Review:
Ticket number: 2020040321000295, 2020032321000375
Bug group (optional):
Max CVSS v3 score:
hahn: Patch_Available+


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Philipp Hahn univentionstaff 2020-04-03 14:51:07 CEST
Traceback (most recent call last):
  File "/usr/lib/univention-directory-listener/system/replication.py", line 946, in handler
    except ldap.LDAPError as ex:
      log_ldap(ud.ERROR, 'Error', ex, dn=dn)
  File "/usr/lib/univention-directory-listener/system/replication.py", line 957, in log_ldap
    ud.debug(ud.LISTENER, severity, 'replication: %s%s: %s' % (ex[0]['desc'], '; dn="%s"' % (dn,) if dn else '', msg))
IndexError: tuple index out of range

"ex" is an ldap.LDAPError with "ex.args == ()"

The original exception thus is not logged.

Even worse: neither the "write LDIF file and abort UDL"-case nor the "reconnect and re-try"-case is taken;
thus "replication.handler()" returns with an exception and UDL continues with the next transaction as UDL has no extra code to handle "replication.py" errors as fatal itself.

This happend on *multiple* slaves which are now all inconsistent with the Master!


1. Fix logging the exception: Should never throw an exception
2. Make replication.py abort UDL if anything goes wrong and is not handled explicitly.
3. Fix Bug #35707 to get a sane level of messages by default
Comment 1 Philipp Hahn univentionstaff 2020-04-03 17:01:09 CEST
Looking at ~/REPOS/PYTHON/python-ldap/Modules/constants.c there are (at least) 4 cases:

1. For not LDAP related errors python-ldap uses PyErr_SetFromErrno() and returns an LDAPError with `args=(errno, msg)`.

>> try: ldap.initialize('xxx')
... except ldap.LDAPError as ex: pass
>>> ex.args
(2, 'No such file or directory')


2. For *known* LDAP errors in the range -17 <= errnum <= 0x7B returns a specific Python exception. This can breaks when OpenLDAP introduces new LDAP_ERROR_* values and python-ldap is not re-compiled:

|             | Debian 9 | UCS-4.4 |
| OpenLDAP    | 2.4.44   | 2.4.45  |
| python-ldap | 2.4.28   | 2.4.28  |

2.2.45 did not add new LDAP_ERROR_* constants, so this is okay.
`args=(dict(desc=optional, matched=optional, info=optional),)`


3. For *unknown* LDAP errors python-ldap returns `args=(dict(errnum=….),)` (not 100% sure)


4. ldap.dn.str2dn() can return a DECODING_ERROR() with `args=()`:

>>> import ldap.dn
>>> try: ldap.dn.str2dn('x')
... except ldap.LDAPError as ex: pass
>>> ex
DECODING_ERROR()
>>> ex.args
()

The last case is not handled correctly by "replication.py" and leads to the observed exception: str2dn() is called multiple times from the handler() method.
Comment 2 Philipp Hahn univentionstaff 2020-04-03 18:36:26 CEST
[4.4-4] a7a0c790d4 Bug #51061 replication: Fix exception handling
 .../debian/changelog                               |  6 +++
 .../replication.py                                 | 46 ++++++++++++++++++++--
 2 files changed, 49 insertions(+), 3 deletions(-)

Package: univention-directory-replication
Version: 12.0.0-5A~4.4.0.202004031830
Branch: ucs_4.4-0
Scope: errata4.4-4

[4.4-4] 1f1589dcc9 Bug #51061: univention-directory-replication 12.0.0-5A~4.4.0.202004031830
 doc/errata/staging/univention-directory-replication.yaml | 10 ++++++++++
 1 file changed, 10 insertions(+)

FYI: You can run the test using
  python -m doctest replication.py
TODO: Backport for UCs-4.3-5
Comment 3 Arvid Requate univentionstaff 2020-04-04 14:56:55 CEST
Verified:
* Code review
* Package update
* Advisory
Comment 4 Erik Damrose univentionstaff 2020-04-08 17:23:41 CEST
<http://errata.software-univention.de/ucs/4.4/522.html>