Univention Bugzilla – Bug 56189
UDM REST does not handle username with umlaut | UnicodeEncodeError: 'latin-1' codec can't encode character
Last modified: 2023-07-12 13:57:22 CEST
On the ID Broker system a user exists with username "<anonymized>-inęs". GET /udm/users/user/uid=<anonymized>-in%C4%99s,cn=schueler,cn=users,ou=<anonymized>,dc=ucs,dc=production,dc=univention-id-broker,dc=com 21.06.23 17:30:16 ERROR ( 5288) : Uncaught exception 0a65f771-7: GET /udm/users/user/uid=<anonymized>-in%C4%99s,cn=schueler,cn=users,ou=<anonymized>,dc=ucs,dc=production,dc=univention-id-broker,dc=com (0.0.0.0) HTTPServerRequest(protocol='http', host='<anonymized>', method='GET', uri='/udm/users/user/uid=<anonymized>-in%C4%99s,cn=schueler,cn=users,ou=<anonymized>,dc=ucs,dc=production,dc=univention-id-broker,dc=com', version='HTTP/1.1', remote_ip='0.0.0.0') Traceback (most recent call last): File "/usr/lib/python3/dist-packages/tornado/web.py", line 1592, in _execute result = yield result File "/usr/lib/python3/dist-packages/tornado/gen.py", line 1133, in run value = future.result() File "/usr/lib/python3/dist-packages/univention/admin/rest/module.py", line 3107, in get self.content_negotiation(props) File "/usr/lib/python3/dist-packages/univention/admin/rest/module.py", line 615, in content_negotiation self.finish(codec(formatter(response))) File "/usr/lib/python3/dist-packages/tornado/web.py", line 1052, in finish future = self.flush(include_footers=True) File "/usr/lib/python3/dist-packages/tornado/web.py", line 994, in flush start_line, self._headers, chunk, callback=callback) File "/usr/lib/python3/dist-packages/tornado/http1connection.py", line 392, in write_headers lines.extend(l.encode('latin1') for l in header_lines) File "/usr/lib/python3/dist-packages/tornado/http1connection.py", line 392, in <genexpr> lines.extend(l.encode('latin1') for l in header_lines) UnicodeEncodeError: 'latin-1' codec can't encode character '\u0119' in position 265: ordinal not in range(256) 21.06.23 17:30:16 ERROR ( 5288) : Cannot send error response after headers written 21.06.23 17:30:16 ERROR ( 5288) : Failed to flush partial response Traceback (most recent call last): File "/usr/lib/python3/dist-packages/tornado/web.py", line 1592, in _execute result = yield result File "/usr/lib/python3/dist-packages/tornado/gen.py", line 1133, in run value = future.result() File "/usr/lib/python3/dist-packages/univention/admin/rest/module.py", line 3107, in get self.content_negotiation(props) File "/usr/lib/python3/dist-packages/univention/admin/rest/module.py", line 615, in content_negotiation self.finish(codec(formatter(response))) File "/usr/lib/python3/dist-packages/tornado/web.py", line 1052, in finish future = self.flush(include_footers=True) File "/usr/lib/python3/dist-packages/tornado/web.py", line 994, in flush start_line, self._headers, chunk, callback=callback) File "/usr/lib/python3/dist-packages/tornado/http1connection.py", line 392, in write_headers lines.extend(l.encode('latin1') for l in header_lines) File "/usr/lib/python3/dist-packages/tornado/http1connection.py", line 392, in <genexpr> lines.extend(l.encode('latin1') for l in header_lines) UnicodeEncodeError: 'latin-1' codec can't encode character '\u0119' in position 265: ordinal not in range(256) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/lib/python3/dist-packages/tornado/web.py", line 1098, in send_error self.finish() File "/usr/lib/python3/dist-packages/tornado/web.py", line 1053, in finish self.request.connection.finish() File "/usr/lib/python3/dist-packages/tornado/http1connection.py", line 463, in finish self._expected_content_remaining) tornado.httputil.HTTPOutputError: Tried to write 14553 bytes less than Content-Length 21.06.23 17:30:16.927 MAIN ( WARN ) : Reaching service failed: HTTP 599: Empty reply from server 21.06.23 17:30:16 ERROR ( 5100) : 503 GET /udm/users/user/uid=<anonymized>-in%C4%99s,cn=schueler,cn=users,ou=<anonymized>,dc=ucs,dc=production,dc=univention-id-broker,dc=com (127.0.0.1) 88.12ms ---------------------------------------------------------------------------------------------------- UDM can search and find the user when using a filter with umlauts as UTF-8: univention-ldapsearch -LLL uid=<anonymized>-inęs dn uid dn:: <anonymized>YXZlbi1iLWluxJlzLGNuPXNjaHVlbGVyLGNuPXVzZXJzLG91PXNjaHVsZW5icmVtZXJoYXZlbi00NTEsZGM9dWNzLGRjPXByb2R1Y3Rpb24sZGM9dW5pdmVudGlvbi1pZC1icm9rZXIsZGM9Y29t uid:: <anonymized>hhdmVuLWItaW7EmXM= udm users/user list --filter uid=<anonymized>-inęs DN: uid=<anonymized>-inęs,cn=schueler,cn=users,ou=<anonymized>,dc=ucs,dc=production,dc=univention-id-broker,dc=com ... username: <anonymized>-inęs ...
Can you tell where did you get the URL from? Was it provided by the server or generated in the client?
The URL was generated by the client in this case. But the server does generate a URL on its own, that also leads to the crash: http://<anonymized>/univention/udm/users/user/?filter=entryUUID=bd2bf89c-b2b0-103c-978f-<anonymized> finds the user object: -------------------------------------------------------------- { "dn": "uid=<anonymized>-in\u0119s,cn=schueler,cn=users,ou=<anonymized>,dc=ucs,dc=production,dc=univention-id-broker,dc=com", "objectType": "users/user", "id": "<anonymized>-in\u0119s", ... "_links": { "self": [ { "name": "uid=<anonymized>-in\u0119s,cn=schueler,cn=users,ou=<anonymized>,dc=ucs,dc=production,dc=univention-id-broker,dc=com", "title": "<anonymized>-in\u0119s", "href": "http://<anonymized>/univention/udm/users/user/uid%3D<anonymized>-in%C4%99s%2Ccn%3Dschueler%2Ccn%3Dusers%2Cou%3D<anonymized>%2Cdc%3Ducs%2Cdc%3Dproduction%2Cdc%3Dunivention-id-broker%2Cdc%3Dcom" } ] } } -------------------------------------------------------------- If I open the URL from _links→self→href the above traceback happens.
Ok, thanks. HTTP specifies it's protocol elements to be ISO8859-1. So UTF-8 is problematic. I will see what I can come up with.
This was the "title" attribute of a Link header - non-compatible chars are now just replaced in there. univention-directory-manager-rest.yaml 7e6b6faf958c | Bug #56189: fix accessing objects with UTF-8 character in its DN univention-directory-manager-rest (10.0.6-6) 7e6b6faf958c | Bug #56189: fix accessing objects with UTF-8 character in its DN
QA: reproduce via: udm users/user create --set username=foo-inęs --set lastname=foobar --set password=univention curl "http://Administrator:univention@localhost/univention/udm/users/user/uid%3Dfoo-in%C4%99s%2C$(ucr get ldap/base)"
QA: Accessing objects that contain UTF-8 characters in their LDAP DN possible (tested as described in Comment 6): OK advisories: OK no related tracebacks or errors in logfiles: OK
<https://errata.software-univention.de/#/?erratum=5.0x740>