Univention Bugzilla – Bug 42465
(4.1r2) --dry-run increases username counter
Last modified: 2017-10-16 21:33:07 CEST
I started an import with --dry-run which increased ucsschoolUsernameNextNumber: 7 to ucsschoolUsernameNextNumber: 9 Which should only happen in a real import situation. In general I think this is not good to make the ldap modify call everytime but only once after the full import and increase a internal python variable instead. This makes the function UsernameHandler.format_username() also very slow and changing this could improve the performance very much! As one can see calling the function 100 times takes 10 seconds: # python -m timeit -s 'from ucsschool.importer.utils.username_handler import UsernameHandler; u = UsernameHandler(20)' -n 100 'u.format_username("foo[ALWAYSCOUNTER]")' 100 loops, best of 3: 92.2 msec per loop Also if importing of the users fails the counter gets increased.
(In reply to Florian Best from comment #0) > I started an import with --dry-run which increased > ucsschoolUsernameNextNumber: 7 > to > ucsschoolUsernameNextNumber: 9 > > Which should only happen in a real import situation. Yes, this is a problem. > In general I think this is not good to make the ldap modify call everytime > but only once after the full import and increase a internal python variable > instead. > > This makes the function UsernameHandler.format_username() also very slow and > changing this could improve the performance very much! But it makes the import in general more unreliable, because the window for conflict gets even larger. If a manual user import happens during a larger multi-hour-long import, the chance of a conflict is extremely high if the values are only changed after the import is finished. > As one can see calling the function 100 times takes 10 seconds: > > # python -m timeit -s 'from ucsschool.importer.utils.username_handler import > UsernameHandler; u = UsernameHandler(20)' -n 100 > 'u.format_username("foo[ALWAYSCOUNTER]")' > 100 loops, best of 3: 92.2 msec per loop
(In reply to Sönke Schwardt-Krummrich from comment #1) > (In reply to Florian Best from comment #0) > > I started an import with --dry-run which increased > > ucsschoolUsernameNextNumber: 7 > > to > > ucsschoolUsernameNextNumber: 9 > > > > Which should only happen in a real import situation. > > Yes, this is a problem. This is a real bad problem because no customer understand why the usernames of the initial import all start with e.g. username3 Happened today at the startup workshop.
This bug is scheduled to be fixed in the next 4 weeks.
Created attachment 8952 [details] 42465_username_counter_dry-run.patch * when used with --dry-run, use a dict() instead of LDAP as counter storage * initialize values in dict from LDAP (using a r/o connection) * adapt code and documentation to changed signature M doc/manual/ucsschool-import-handbuch-4.1r2.xml M ucs-school-import/debian/changelog M ucs-school-import/modules/ucsschool/importer/default_user_import_factory.py M ucs-school-import/modules/ucsschool/importer/models/import_user.py M ucs-school-import/modules/ucsschool/importer/utils/ldap_connection.py M ucs-school-import/modules/ucsschool/importer/utils/username_handler.py
Created attachment 8953 [details] 42465_username_counter_dry-run_ucs-test.patch Add test: 1) import user for real 2) delete user 3) import same user with --dry-run → username counter should not rise A + ucs-test-ucsschool/90_ucsschool/215_import-users_username_dry-run M ucs-test-ucsschool/debian/changelog
r80496: UsernameHandler does not increase counter in LDAP with dry_run anymore r80497: merge to UCS@school 4.2 r80498+r80499: advisories "dry_run" has to be set in __init__(), to prevent a dependency on the configuration singleton. Using the configuration object requires a lot of initialization that is not needed for this utility class. Overwriting UsernameHandler.get_storage_backend() allows to change the storage backend. Package: ucs-school-import Version: 14.0.16-44.335.201706261615 Branch: ucs_4.1-0 Scope: ucs-school-4.1r2 Package: ucs-school-import Version: 15.0.0-5A~4.2.0.201706261618 Branch: ucs_4.2-0 Scope: ucs-school-4.2
r80504: don't create LDAP connection at import time r80506: advisories ucs-school-import (14.0.16-45.336.201706271132)
Fixed: 80536, 80537, 80538
OK: --dry-run causes the counter to not increased OK: for modifications the counter is not increased OK: import of multiple users / single users OK: YAML
UCS@school 4.1 R2 v14 has been released. http://docs.software-univention.de/changelog-ucsschool-4.1R2v14-de.html If this error occurs again, please clone this bug.