Univention Bugzilla – Bug 45467
user_deletion:delete and user_deletion:expiration do not work as expected
Last modified: 2018-10-30 13:41:03 CET
While working on another bug it became clear that the import options user_deletion:delete and user_deletion:expiration do not work as expected, documented or specified. Status of current implementation: 1) delete=True, expiration=0 → the user object is removed immediately from LDAP 2) delete=True, expiration=30 → the user object is not removed, remains active (not deactivated) and the user_expiry date (Kontoablaufdatum) is set to $NOW+30 3) delete=False, expiration=0 → an exception is raised on purpose 4) delete=False, expiration=30 → the user object is not removed, immediately deactivated and the user_expiry date (Kontoablaufdatum) is set to $NOW+30 Additional notes: - Without manual configuration, case 1) is used by default. - Case 2) requires an additional script that removes users some time in the future; this script does not exist yet! - Case 4) is broken since deactivation and user_expiry date use (partially) the same LDAP attribute(s), therefore the user is partially deactivated - The description of these 2 options in the UCS@school admin manual does not match the behavior described above. During discussion with professional services it became clear that 7-8 school customers are using these flags or are about to use them: - the actual removal of users is important (→ missing script/cron job) - case 2 and 4 are both (about to be) in use The following solution is the result of the discussion: Instead of the two options above, two new options are implemented: → user_deletion:grace_period:deactivation (Integer) (value range >= 0) → user_deletion:grace_period:deletion (Integer) (value range >= 0) Both values specify a period in the unit "days". By default, both options default to "0" if not set manually in import config file. New behaviour: ("user_deletion:grace_period" == "UD:GP") ============== UD:GP:deactivation=???, UD:GP:deletion=0 → the user object is removed immediately from LDAP → the value of "UD:GP:deactivation" does not matter in this case UD:GP:deactivation=0, UD:GP:deletion=90 → the user object is not removed → the user object is immediately deactivated → the new property "purge_timestamp" is set to $NOW+90 (see below) → the user object is deleted automatically after 90 days via a cron job script UD:GP:deactivation=30, UD:GP:deletion=90 → the user object is not removed → the "user_expiry" date (Kontoablaufdatum) is set to $NOW+30, so the user object remains active/valid for 30 days → the new property "purge_timestamp" is set to $NOW+90 (see below) → the user object is deleted automatically after 90 days via a cron job script Implementation: =============== A) extend the LDAP schema of UCS@school and add a new attribute for UCS@school users that stores a date. Schema suggestion: attributetype ( .........OID............. NAME 'ucsschoolPurgeTimestamp' DESC 'Specifies the point in time after which the object is deleted automatically' EQUALITY generalizedTimeMatch ORDERING generalizedTimeOrderingMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.24 SINGLE-VALUE ) (derived from krb5PasswordEnd) The equality and ordering options are important to perform quick lookups of expired user accounts in LDAP. B) Create an extended attribute (EA) for UCS@school user objects The EA should use a date syntax: date picker in UMC, ISO8601 date on CLI ("20170929000000Z" ?). Suggestion for EA's UDM-CLI name: "purge_timestamp" C) Update at least do_delete() in ucs-school-import/modules/ucsschool/importer/mass_import/user_import.py do_delete() should implement the behavior described above and use "ucsschoolPurgeTimestamp" to store the date, when the user object shall be removed. Please make sure, that the calculated purge timestamp is correctly converted to Z-time before the value is stored in LDAP. Please also make sure that if a previously "deleted" user with given ucsschoolPurgeTimestamp is recreated again via ucs-school-import, the user object has to be reactivated! That means that the account deactivation is reverted (depending on the import data!), the expiration date is removed and the purge timestamp is also removed. D) Create a purge script The script should use a proper LDAP filter for searching UCS@school user accounts with ucsschoolPurgeTimestamp<=$NOW, so the LDAP search only returns relevant user objects (and not all!). After receiving a list of user DNs, the script should use the ucs-school-import lib to open() the user objects and call user.remove() on each object, so hooks and other import logic is also automatically considered. The script should have a dry run mode. And it should be possible to call the script manually on the command line. E) cron job A new cron job has to be implemented that calls the script from D) regulary. By default, I would suggest that the cron job is executed every night at 4:50 AM. The cron job interval should be configurable via UCR (see e.g. update/check/cron/entry). F) test script The 3 cases described in "New behaviour" shall be covered by one (or more) ucs-test script(s). User reactivation should also be covered by ucs-test. Also the cronjob/script has to be covered by ucs-test. G) manual update The import-CLI manual has to be updated accordingly. Please note: implement everything in a feature branch and DO NOT MERGE it to origin/4.2 for now!
Everything was implemented in branch dtroeder/45467_user_deletion. 8ca8bdd1 A), B) add schema, ext attribute and udm_hook for storing a sortable timestamp in LDAP 36043689 B) raise join script version 9d5b7396 G) adapt documentation 92b4841c B) handle unset and already converted timestamps more safely 358b3a89 C) new deactivation, deletion and expiration code 98bbeae7 D) E) script to remove expired user accounts bbec0edb F) add compatibility for cmdline and ucs-test name for teacher_and_staff role 69c11ce4 F) add purge script test and adapt tests to config changes, 216_import-users_delete_variants now tests the new behavior No changelog, no advisory, no build. Please reopen when this should be merged and the above will be created.
During QA I noticed, that "purge_timestamp" for the CLI name of the extended attribute did not match the names of all other extended attributes. I renamed it to ucsschoolPurgeTimestamp, so LDAP attribute and CLI name are identical (similar to all other ext attrs). I did some manual tests by calling the following command several times, that creates 2 new student accounts and removed the 2 accounts of the previous run: # /usr/share/ucs-school-import/scripts/ucs-school-testuser-import \ --staff 0 --students 2 --teachers 0 --staffteachers 0 --classes=1 \ --inclasses 1 --schools 1 --create-email-addresses gsmitte Systemdate at time of tests was "Fr 13. Okt 13:00:17 CEST 2017". Each run has been performed with different settings of deletion_grace_period::deactivation and deletion_grace_period::deletion: DGP::deactivation=0, DGP::deletion=0 ==================================== → user accounts have been removed immediately DGP::deactivation=10, DGP::deletion=0 ===================================== → user accounts have been removed immediately DGP::deactivation=0, DGP::deletion=10 ===================================== → user accounts have been deactivated immediately and marked for deletion dn: uid=neithard.bar,cn=schueler,cn=users,ou=gsmitte,dc=nstx,dc=local +ucsschoolPurgeTimestamp: 20171023000000Z +shadowExpire: 1 +sambaAcctFlags: [UD ] -sambaAcctFlags: [U ] +krb5KDCFlags: 254 -krb5KDCFlags: 126 DGP::deactivation=5, DGP::deletion=10 ===================================== → user accounts have been marked for deactivation in 5 days and marked for deletion in 10 days dn: uid=soeren.vette,cn=schueler,cn=users,ou=gsmitte,dc=nstx,dc=local +ucsschoolPurgeTimestamp: 20171023000000Z +shadowExpire: 17457 +sambaKickoffTime: 1508277600 +krb5ValidEnd: 20171018000000Z → Please note: already deactivated user accounts (see previous test runs) remain deactivated and marked for deactivation. DGP::deactivation=7, DGP::deletion=12345 ======================================== → user accounts have been marked for deactivation in 7 days and marked for deletion in 12345 days dn: uid=elina.haske,cn=schueler,cn=users,ou=gsmitte,dc=nstx,dc=local +ucsschoolPurgeTimestamp: 20510801000000Z +shadowExpire: 17459 +sambaKickoffTime: 1508450400 +krb5ValidEnd: 20171020000000Z DGP::deactivation=0, DGP::deletion=12345 ======================================== → user accounts already marked for deactivation in 7 days (see previous test runs) have been immediately deactivated → I think, this is ok because an immediate deactivation is requested. dn: uid=elina.haske,cn=schueler,cn=users,ou=gsmitte,dc=nstx,dc=local -shadowExpire: 17459 +shadowExpire: 1 +sambaAcctFlags: [UD ] -sambaAcctFlags: [U ] +krb5KDCFlags: 254 -krb5KDCFlags: 126 Tested the actual deletion of user accounts with different system dates. ucsschoolPurgeTimestamp was "20171023000000Z". Systemtime and result: 2017-10-20 21:51:48 INFO ucs-school-purge-expired-users.main:200 Found 0 expired accounts. 2017-10-22 21:24:56 INFO ucs-school-purge-expired-users.main:200 Found 0 expired accounts. 2017-10-22 23:50:44 INFO ucs-school-purge-expired-users.main:200 Found 0 expired accounts. 2017-10-22 23:58:44 INFO ucs-school-purge-expired-users.main:200 Found 0 expired accounts. 2017-10-23 00:00:03 INFO ucs-school-purge-expired-users.main:200 Found 4 expired accounts. → OK I did some extra changes and merged the branch to 4.2: ucs-school-import (15.0.2-1): ad27cba39e1c | Bug #45467: Merge branch 'sschwardt/45467/42/import-user-deletion' into 4.2 e16579afdf71 | Bug #45467: add changelog entry 92db62d181dd | Bug #45467: cronjob creates two logfiles per day - use a custom logdir 17742816b5b5 | add compatibility for cmdline and ucs-test name for teacher_and_staff role (Bug #45467) 0f1542675839 | Bug #45467: use a custom UCR cron.d template instead of the customer UCR variables 5a3abd5fd88b | script to remove expired user accounts (Bug #45467) 513b1408e9e2 | Bug #45467: do not reactivate user accounts if already disabled ff3d8a99a3c3 | new deactivation, deletion and expiration code (Bug #45467) 9122bbe12907 | handle unset and already converted timestamps more safely (Bug #45467) ad1c9a3c9550 | adapt documentation (Bug #45467) 972975b84644 | Bug #45467: raise join script version 8c7045fd08ed | add schema, ext attribute and udm_hook for storing a sortable timestamp in LDAP (Bug #45467) ucs-test-ucsschool (4.0.4-31.1): ad27cba39e1c | Bug #45467: Merge branch 'sschwardt/45467/42/import-user-deletion' into 4.2 fc6a117f80ea | add purge script test and adapt tests to config changes (Bug #45467) ucs-school-import.yaml: ad27cba39e1c | Bug #45467: Merge branch 'sschwardt/45467/42/import-user-deletion' into 4.2 3d664bf26164 | Bug #45467: add advisory Package: ucs-school-import Version: 15.0.2-1A~4.2.0.201710142334 Branch: ucs_4.2-0 Scope: ucs-school-4.2
If the next automatic test is also successful, consider this bug as verified :-)
(In reply to Sönke Schwardt-Krummrich from comment #3) > If the next automatic test is also successful, consider this bug as verified > :-) Jenkins tests showed no errors.
UCS@school 4.2 v4 has been released. http://docs.software-univention.de/changelog-ucsschool-4.2v4-de.html If this error occurs again, please clone this bug.