Univention Bugzilla – Bug 46970
[4.3] ucs-school-user-import does not trim leading/trailing whitespaces
Last modified: 2018-07-04 18:09:00 CEST
A customer imported some users a while ago, e.g.: dn: uid=mary.gomme,cn=schueler,cn=users,ou=999,dc=edu,dc=example,dc=org sn: Gommer givenName: Mary Fairfax For the second import run, a trailing whitespace was unintentionally added to the lastname, resulting in "Gommer " (OpenLDAP will save this in base64, then): dn: uid=mary.gomme,cn=schueler,cn=users,ou=999,dc=edu,dc=example,dc=org sn:: R29tbWVyIA== givenName: Mary Fairfax This is especially nasty if this hits the recordUID, because then a completely new user will be created and the old one will be deleted. IMHO ucs-school-user-import should trim leading and trailing whitespaces for all (most?) attributes. Maybe we could add this as a hook, so customers can deactivate it if needed?
(In reply to Michael Grandjean from comment #0) > IMHO ucs-school-user-import should trim leading and trailing whitespaces for > all (most?) attributes. Maybe we could add this as a hook, so customers can > deactivate it if needed? I suggest to make trimming the default, because it almost always is wanted. If a user has a scenario in which the white space is required, a pre-* hook can re-add it. I suggest to trim leading & trailing white space on all input fields.
(In reply to Daniel Tröder from comment #1) > (In reply to Michael Grandjean from comment #0) > > IMHO ucs-school-user-import should trim leading and trailing whitespaces for > > all (most?) attributes. Maybe we could add this as a hook, so customers can > > deactivate it if needed? > I suggest to make trimming the default, because it almost always is wanted. > If a user has a scenario in which the white space is required, a pre-* hook > can re-add it. > > I suggest to trim leading & trailing white space on all input fields. +1 for make trimming the default
Fixed in branch dtroeder/46970_strip_whitespace in commit 59f6b5c7. The original input with whitespace is preserved, so hooks can readd it, if required. The test 90_ucsschool/201_import-users_basic was adapted to test for stripping whitespace from CSV input. Logline from test run (notice fields "Vor" and "Nach"): 2018-05-16 20:43:09 DEBUG base_reader.next:74 Input 5: ['teacher_and_staff', 'hcheyoatuy ', 'recordUID-pi7sxi4wz7', ' dt0loxmdkp', 'tx71qen12zw-75vc', '', 'tx71qen12zw', 'cgdngljfgp@uni.dtr'] -> {u'Gruppen': u'tx71qen12zw-75vc', u'recordUID': u'recordUID-pi7sxi4wz7', u'Nach': u'dt0loxmdkp', u'Beschreibung': u'', u'E-Mail': u'cgdngljfgp@uni.dtr', u'role': u'teacher_and_staff', u'Vor': u'hcheyoatuy', u'OUs': u'tx71qen12zw'} The list (after "Input 5: ") with the unmodified CSV-fields will be attached to the ImportUser object as attribute "input_data" (in base_reader.next:78).
(In reply to Daniel Tröder from comment #3) > 2018-05-16 20:43:09 DEBUG base_reader.next:74 Input 5: > ['teacher_and_staff', 'hcheyoatuy ', 'recordUID-pi7sxi4wz7', ' dt0loxmdkp', > 'tx71qen12zw-75vc', '', 'tx71qen12zw', 'cgdngljfgp@uni.dtr'] -> {u'Gruppen': > u'tx71qen12zw-75vc', u'recordUID': u'recordUID-pi7sxi4wz7', u'Nach': > u'dt0loxmdkp', u'Beschreibung': u'', u'E-Mail': u'cgdngljfgp@uni.dtr', > u'role': u'teacher_and_staff', u'Vor': u'hcheyoatuy', u'OUs': u'tx71qen12zw'} > > The list (after "Input 5: ") with the unmodified CSV-fields will be attached > to the ImportUser object as attribute "input_data" (in base_reader.next:78). So hooks have to redo the trimming again, if they have to use the input_data?
(In reply to Sönke Schwardt-Krummrich from comment #4) > (In reply to Daniel Tröder from comment #3) > > The list (after "Input 5: ") with the unmodified CSV-fields will be attached > > to the ImportUser object as attribute "input_data" (in base_reader.next:78). > So hooks have to redo the trimming again, if they have to use the input_data? Yes. If a hook requires access to unmapped input data, it'll get the untouched input data.
Merged branch 'dtroeder/46970_strip_whitespace' into 4.3 (ed4cf3a4). Code was added to the test 201_import-users_basic to check if whitespace is removed. [4.3] 7921a0f6 Bug #46970: strip leading/trailing whitespace from CSV input [4.3] 94409baa Bug #46970: add deepcopy operation support to all ucsschool.lib objects [4.3] 11b809ba Bug #46970: use deepcopy in import [4.3] 5b0e85fc Bug #46970: changelog [4.3] 58d79c11 Bug #46970: advisories ucs-school-lib (11.0.1-10) ucs-school-import (16.0.1-30) ucs-test-ucsschool (4.0.4-93)
[4.3] d3177332 Bug #46970: fix changelog [4.3] 348ecd81 Bug #46970: add missing data to advisory ucs-test-ucsschool (5.0.2-54)
leading and trailing whitespaces were removed upon import
UCS@school 4.3 v4 has been released. https://docs.software-univention.de/changelog-ucsschool-4.3v4-de.html If this error occurs again, please clone this bug.