Bug 44291 - Rejects for non-ascii accounts are not handled properly due to sqlite encoding problem
Rejects for non-ascii accounts are not handled properly due to sqlite encodin...
Status: CLOSED FIXED
Product: UCS
Classification: Unclassified
Component: S4 Connector
UCS 4.1
Other Linux
: P5 normal (vote)
: UCS 4.1-4-errata
Assigned To: Arvid Requate
Felix Botner
:
Depends on: 44369 45226
Blocks:
  Show dependency treegraph
 
Reported: 2017-04-04 19:26 CEST by Arvid Requate
Modified: 2017-08-21 12:46 CEST (History)
3 users (show)

See Also:
What kind of report is it?: Bug Report
What type of bug is this?: 3: Simply Wrong: The implementation doesn't match the docu
Who will be affected by this bug?: 2: Will only affect a few installed domains
How will those affected feel about the bug?: 2: A Pain – users won’t like this once they notice it
User Pain: 0.069
Enterprise Customer affected?:
School Customer affected?:
ISV affected?:
Waiting Support:
Flags outvoted (downgraded) after PO Review:
Ticket number:
Bug group (optional):
Max CVSS v3 score:
requate: Patch_Available+


Attachments
save_reject_for_nonascii_dn.patch (641 bytes, patch)
2017-04-04 19:28 CEST, Arvid Requate
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Arvid Requate univentionstaff 2017-04-04 19:26:10 CEST
If there is a reject for a DN containing non-ascii characters, then the reject cannot be saved into the corresponding sqlite database table. As a result it's not shown as reject and the S4-Connector is contantly trying to sync that object, spamming the s4-cconector.log:


========================================================================
03.05.2016 01:33:13,727 LDAP        (PROCESS): sync from ucs: [         group] [       add] cn=Domänen Gruppe,cn=groups,DC=ar41i2,DC=qa
03.05.2016 01:33:13,754 LDAP        (ERROR  ): sync_from_ucs: traceback during add object: cn=Domänen Gruppe,cn=groups,DC=ar41i2,DC=qa
03.05.2016 01:33:13,754 LDAP        (ERROR  ): sync_from_ucs: traceback due to addlist: [('objectClass', ['top', 'group']), ('groupType', [u'-2147483646']), ('sAMAccountName', [u'Dom\xe4nen Gruppe'])]
03.05.2016 01:33:13,754 LDAP        (WARNING): sqlite: You must not use 8-bit bytestrings unless you use a text_factory that can interpret 8-bit bytestrings (like text_factory = str). It is highly recommended that you instead just switch your application to Unicode strings.
03.05.2016 01:33:13,755 LDAP        (WARNING): sqlite: You must not use 8-bit bytestrings unless you use a text_factory that can interpret 8-bit bytestrings (like text_factory = str). It is highly recommended that you instead just switch your application to Unicode strings.
03.05.2016 01:33:13,756 LDAP        (WARNING): sync failed, saved as rejected
        /var/lib/univention-connector/s4/1462231951.138939
========================================================================


+++ This bug was initially created as a clone of Bug #44276 +++
Comment 1 Arvid Requate univentionstaff 2017-04-04 19:28:47 CEST
Created attachment 8755 [details]
save_reject_for_nonascii_dn.patch

I guess that would fix the issue.
Comment 2 Arvid Requate univentionstaff 2017-04-18 12:29:47 CEST
Package rebuilt with patch.

Advisory: univention-s4-connector.yaml
Comment 3 Stefan Gohmann univentionstaff 2017-05-02 20:32:22 CEST
It leads to the following traceback:

    s4.initialize_ucs()
  File "/usr/lib/pymodules/python2.7/univention/s4connector/__init__.py", line 963, in initialize_ucs
    self.poll_ucs()
  File "/usr/lib/pymodules/python2.7/univention/s4connector/__init__.py", line 1055, in poll_ucs
    self._save_rejected_ucs(filename, dn)
  File "/usr/lib/pymodules/python2.7/univention/s4connector/__init__.py", line 573, in _save_rejected_ucs
    self._set_config_option('UCS rejected', filename, modstring_dn)
NameError: global name 'modstring_dn' is not defined
Comment 5 Arvid Requate univentionstaff 2017-05-02 21:29:36 CEST
Package rebuilt with fixed patch, advisory updated.
Comment 6 Felix Botner univentionstaff 2017-05-12 14:26:30 CEST
still get

12.05.2017 14:22:55,910 LDAP        (WARNING): sqlite: You must not use 8-bit bytestrings unless you use a text_factory that can interpret 8-bit bytestrings (like text_factory = str). It is highly recommended that you instead just switch your application to Unicode strings.
12.05.2017 14:22:55,911 LDAP        (WARNING): sqlite: You must not use 8-bit bytestrings unless you use a text_factory that can interpret 8-bit bytestrings (like text_factory = str). It is highly recommended that you instead just switch your application to Unicode strings.
12.05.2017 14:22:55,911 LDAP        (WARNING): sync failed, saved as rejected
        /var/lib/univention-connector/s4/1494586750.163802
12.05.2017 14:22:55,914 LDAP        (WARNING): Traceback (most recent call last):
  File "/usr/lib/pymodules/python2.7/univention/s4connector/__init__.py", line 842, in __sync_file_from_ucs
    if ((old_dn and not self.sync_from_ucs(key, object, premapped_ucs_dn, unicode(old_dn, 'utf8'), old, new)) or (not old_dn and not self.sync_from_ucs(key, object, premapped_ucs_dn, old_dn, old, new))):
  File "/usr/lib/pymodules/python2.7/univention/s4connector/s4/__init__.py", line 2620, in sync_from_ucs
    self.lo_s4.lo.modify_ext_s(compatible_modstring(object['dn']), compatible_modlist(modlist), serverctrls=self.serverctrls_for_add_and_modify)
  File "/usr/lib/python2.7/dist-packages/ldap/ldapobject.py", line 336, in mod
...

and no rejects.

* created a group grüppe in UCS
* stopped connector
* changed group -> attributes -> description -> con_attribute to "descriptions"
  in /etc/univention/connector/s4/mapping
* started connector
* add description for group grüppe in UCS

to force this bug
Comment 7 Arvid Requate univentionstaff 2017-06-01 21:22:51 CEST
Ok, fixed and advisory updated.
Comment 8 Felix Botner univentionstaff 2017-06-02 13:20:52 CEST
OK - rejects for non ascii objects
OK - univention-s4-connector.yaml
Comment 9 Janek Walkenhorst univentionstaff 2017-07-05 13:32:20 CEST
<http://errata.software-univention.de/ucs/4.1/439.html>