Bug 46837 - [4.2] school-import should check most common UDM/ucsschool.lib.models exceptions in dry-run
[4.2] school-import should check most common UDM/ucsschool.lib.models excepti...
Status: CLOSED FIXED
Product: UCS@school
Classification: Unclassified
Component: Import scripts
UCS@school 4.2
Other Linux
: P5 normal (vote)
: UCS@school 4.2 v11
Assigned To: Daniel Tröder
Jürn Brodersen
:
Depends on: 45715
Blocks: 45019
  Show dependency treegraph
 
Reported: 2018-04-17 13:35 CEST by Daniel Tröder
Modified: 2018-09-11 11:36 CEST (History)
4 users (show)

See Also:
What kind of report is it?: Feature Request
What type of bug is this?: ---
Who will be affected by this bug?: ---
How will those affected feel about the bug?: ---
User Pain:
Enterprise Customer affected?:
School Customer affected?:
ISV affected?:
Waiting Support:
Flags outvoted (downgraded) after PO Review:
Ticket number:
Bug group (optional):
Max CVSS v3 score:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Daniel Tröder univentionstaff 2018-04-17 13:35:00 CEST
+++ This bug was initially created as a clone of Bug #45715 +++

Some errors do not show up during a dry-run but when doing the real import. They are caught by either ucsschool.lib.models.*.modify() or udm-object.modify(), which is not done in the dry-run.

Test for the 5 most common errors, that show up in the real import, but not in dry-run.
Comment 1 Daniel Tröder univentionstaff 2018-06-18 16:49:28 CEST
Merged from 4.3 (Bug #45715):

[dtroeder/46837_4.2_checks_in_dry-run] b195c4b74 Bug #46837: consistent wording: dry-run with dash
[dtroeder/46837_4.2_checks_in_dry-run] 367c9db20 Bug #46837: use admin connection in a real run, use machine connection during a dry-run
[dtroeder/46837_4.2_checks_in_dry-run] 5133fb222 Bug #46837: add support for a dry-run mode in pyhooks
[dtroeder/46837_4.2_checks_in_dry-run] 56900d144 Bug #46837: add support for a filter function to pyhooks-loader
[dtroeder/46837_4.2_checks_in_dry-run] f25bd91f8 Bug #46837: during dry-run use only PyHook classes that support it
[dtroeder/46837_4.2_checks_in_dry-run] 72ca4b5a1 Bug #46837: don't reload user from LDAP after a dry-run create
[dtroeder/46837_4.2_checks_in_dry-run] 69f48a1d1 Bug #46837: run hooks in dry-run
[dtroeder/46837_4.2_checks_in_dry-run] 537f5b51b Bug #46837: don't run legacy hooks in dry-run
[dtroeder/46837_4.2_checks_in_dry-run] be46caccb Bug #46837: make import tests always verbose
[dtroeder/46837_4.2_checks_in_dry-run] c433710e2 Bug #46837: lower logging serverity for running hook announcement
[dtroeder/46837_4.2_checks_in_dry-run] f5e30f370 Bug #46837: fix section links
[dtroeder/46837_4.2_checks_in_dry-run] fa4442b3c Bug #46837: remove unused exception
[dtroeder/46837_4.2_checks_in_dry-run] 0e08b2e0c Bug #46837: remove useless try-except block, leftover from commit 1906c8c (Bug #41882: lower memory footprint of import)
[dtroeder/46837_4.2_checks_in_dry-run] a1077e7c9 Bug #46837: use ucsschool.lib exception instead of uldap exception
[dtroeder/46837_4.2_checks_in_dry-run] ff0e28f9a Bug #46837: add missing configuration object for when classmethod is used before initialization
[dtroeder/46837_4.2_checks_in_dry-run] 3a2cce191 Bug #46837: handle empty birthday column
[dtroeder/46837_4.2_checks_in_dry-run] 9341eb658 Bug #46837: detect role change and prevent creation of additional user
[dtroeder/46837_4.2_checks_in_dry-run] ee323bcb1 Bug #46837: check username not taken
[dtroeder/46837_4.2_checks_in_dry-run] 008eb34d9 Bug #46837: warn if school name in class name
[dtroeder/46837_4.2_checks_in_dry-run] 8d5ccb53b Bug #46837: use ucsschool.lib supported validate() instead of run_checks(), sort checks by how expensive they are and if in_hook
[dtroeder/46837_4.2_checks_in_dry-run] 0af56295b Bug #46837: add test for dry-run support in pyhooks
[dtroeder/46837_4.2_checks_in_dry-run] 5e09b9a64 Bug #46837: fix tag
Comment 2 Daniel Tröder univentionstaff 2018-07-31 08:32:45 CEST
Code was merged to 4.2 and built.

[4.2] 1d8aec8c0 Bug #46837 Bug #47204: Merge branch 'dtroeder/46837_4.2_checks_in_dry-run' into 4.2
[4.2] 5f3148805 Bug #46837 Bug #47204: changelog
[4.2] 5dac9f3ca Bug #46837 Bug #47204: advisory

ucs-school-import (15.0.3-64)
ucs-school-lib (10.0.2-17)
ucs-test-ucsschool (4.0.4-99)
Comment 3 Jürn Brodersen univentionstaff 2018-09-07 15:12:49 CEST
Reopen: You added some commits in bug 45715 do these need to be merged?
Comment 4 Sönke Schwardt-Krummrich univentionstaff 2018-09-07 17:55:01 CEST
Yes, they do. 34_import-users_legacy_migration also fails in UCS@school 4.2
Comment 5 Daniel Tröder univentionstaff 2018-09-10 13:58:48 CEST
A lot of commits were not merged from 4.3.
To get a less different code base between 4.3 and 4.2 I also cherry-picked  42fe58f404 and e2cf0a412c.

[4.2] 87aef06e6 Bug #46837: use readonly admin connection during dry run
[4.2] fce6ace05 Bug #46837: add test for "username not in use"
[4.2] 484129466 Bug #46837: check for ValidationErrors after running validate() in dry-run
[4.2] 9530e2cc3 Bug #46837: use string_types instead of basestring
[4.2] d6d918440 Bug #46837: add mypy annotations
[4.2] ca2af58d9 Bug #46837: handle ImportError while ucs-test from 4.3-1errata is not in 4.3-2
[4.2] 88317153c Bug #46837: fix mypy annotations
[4.2] cde9bd601 Bug #46837: new-legacy import must handle users from old-legacy import that don't have record_uid and source_uid
[4.2] a237d8afe Bug #46837: add/improve static type annotations
[4.2] cf7efabe7 Bug #46837: string_types and type annotations
[4.2] be73d3e43 Bug #46837: revert merge errors
[4.2] f03f16f57 Bug #46837: move module docstring below copyright comment
[4.2] d7e0a8620 Bug #47311: fix class names, but also keep old names
[4.2] 4929850c0 Bug #46837: advisory update

ucs-school-import (15.0.3-69)
Comment 6 Daniel Tröder univentionstaff 2018-09-10 17:23:17 CEST
[4.2] d5810f4eb Bug #46837: add missing import
[4.2] a5fc105bc Bug #46837: advisory update

ucs-school-import (15.0.3-70)
Comment 7 Jürn Brodersen univentionstaff 2018-09-10 23:30:26 CEST
[4.2 4e9aac92a] Bug #47451: fix expected attributes for legacy import
[4.2 d92f68e55] Bug #47451: fix 34_import-users_legacy_migration

Sorry for the wrong bug number...

34_import-users_legacy_migration will fail tomorrow! I was to late with my commit. But it's only a test change, so that should be fine.

What I tested:
Imported User through cli and http api -> OK
Login with imported users -> OK
Tests (at least the ones I checked) -> OK
YAML -> OK

-> Verified
Comment 8 Sönke Schwardt-Krummrich univentionstaff 2018-09-11 11:34:17 CEST
UCS@school 4.3 v5 has been released.

https://docs.software-univention.de/changelog-ucsschool-4.3v5-de.html

If this error occurs again, please clone this bug.
Comment 9 Sönke Schwardt-Krummrich univentionstaff 2018-09-11 11:36:47 CEST
UCS@school 4.2 v11 has been released.

https://docs.software-univention.de/changelog-ucsschool-4.2v11-de.html

If this error occurs again, please clone this bug.