Univention Bugzilla – Bug 45379
provide role attribute for all UCS@school objects
Last modified: 2018-11-30 12:19:05 CET
It would be very useful to provide all UCS@school objects with a unique object class / attribute (e.g.: ucsschoolObjectType=...). Based on already existing object class on the user objects: Users: ucsschoolStudent / ucsschoolTeacher / ucsschoolAdministrator / ucsschoolStaff I would like to see this also for groups: Groups: ucsschoolWorkingGroup / ucsschoolClass / ucsschoolCompuerRoom Why is this necessary? At the moment it is really hard to find and identify all classes or all working groups. At the end they are all ldap groups. There are no difference between the objects, only the location in the ldap make a group to a class or a working group. Also, the objects do not differ from the default built-in groups. Our claim is that the LDAP is the leading data source, so it must also be easy to find the data (e.g. for sync to external services). I would suggest, that these unique object classes / attributes should be set during the import. Also, there must be an option to edit these options (like the user options).
*** Bug 44110 has been marked as a duplicate of this bug. ***
As the Bug will be worked on at a later date, here is my WIP in branch dtroeder/45379_ucsschoolObjectType: [dtroeder/45379_ucsschoolObjectType fc89bcf7] Bug #45379: add object type attribute to all UCS@school objects * create specific objectClasses * set attribute for some classes * create udm hook to add objectClasses TODO: * add OID to https://hutten.knut.univention.de/mediawiki/index.php?oldid=20518 * write listener module to add OC "ucsschoolAdministratorGroup" to user, when the option is set on the UDM object (UDM hooks do not react to changed options) * write migration script that adds the LDAP OC and attribute value to all existing (and relevant) LDAP objects * write ucs-test(s) that check if it works as expected * write system diagnose script to check for objects without OC or unset attribute
All objects of type SchoolClass, ClassShare, ComputerRoom, WorkGroup, ExamStudent, Student, Staff, Teacher, TeachersAndStaff and OU admin groups now receive a objectClass and attribute upon creation. (In reply to Daniel Tröder from comment #2) > TODO: > * add OID to > https://hutten.knut.univention.de/mediawiki/index.php I created an entry for 1.3.6.1.4.1.10176.4000.2.* as there are just to many OIDs to enter. Diff: https://hutten.knut.univention.de/mediawiki/index.php?title=Univention-OIDs&type=revision&diff=60783&oldid=59954 > * write listener module to add OC "ucsschoolAdministratorGroup" to user, > when the option is set on the UDM object (UDM hooks do not react to changed > options) A listener was added (ucs-school-import/listener/ucsschool_admin_object_type.py) which is deactivated by default (UCR listener/module/ucsschool_admin_object_type/deactivate?yes). It must be manually activated by the administrator after running the migration script. > * write migration script that adds the LDAP OC and attribute value to all > existing (and relevant) LDAP objects ucs-school-import/usr/share/ucs-school-import/scripts/migrate_ucsschool_object_types > * write ucs-test(s) that check if it works as expected Multiple tests and the test suites lib itself were modified to verify the object_type property. Additionally a test 20_object_type_migration_script was written to test the migration script. > * write system diagnose script to check for objects without OC or unset > attribute Split to Bug #46738. [4.3] e36e25f8 Bug #45379: add object type attribute to all UCS@school objects * create specific objectClasses * set attribute for some classes * create udm hook to add objectClasses [4.3] 7ec15bb0 Bug #45379: add objectClass only when creating an object [4.3] 9310c38f Bug #45379: mark ImportUser objects, update legacy import script, finish udm hook, add listener, add migration script [4.3] bc4b3711 Bug #45379: changelog [4.3] cb42a958 Bug #45379: mark administrator group, add object_type to OC mapping [4.3] 7d43c1c1 Bug #45379: changelog [4.3] 064efdcc Bug #45379: verify setting of ucsschoolObjectType attribute [4.3] eec30c5d Bug #45379: changelog [4.3] 227fd03c Bug #45379: advisories ucs-school-lib (11.0.1-7) ucs-school-import (16.0.1-11) ucs-test-ucsschool (5.0.2-29) To discuss: remove ucsschoolObjectType entry, when an Option is deactivated? That can theoretically happen with User objects, as the options can be deactivated in UMC/UDM.
Added verification of the setting of unique objectClasses to all tests. [4.3 fff1ebad] Bug #45379: verify objectClass ucs-test-ucsschool (5.0.2-32)
Please mention that existing objects are not modified and how/why to use migration script in the advisory.
(In reply to Jürn Brodersen from comment #5) > Please mention that existing objects are not modified and how/why to use > migration script in the advisory. Sorry I looked in the wrong yaml...
The listener is not restarted: /var/lib/dpkg/info/ucs-school-import.postinst: line 133: ucr_is_true: command not found And if I remember this correctly you need to add the ".service" file extension if you use deb-systemd-invoke. Don't we need a listener for staff and teachers as well?
(In reply to Jürn Brodersen from comment #7) > The listener is not restarted: > > /var/lib/dpkg/info/ucs-school-import.postinst: line 133: ucr_is_true: > command not found Fixed: [4.3 0b5400be] Bug #45379: fix typo [4.3 cb28fcb8] Bug #45379: advisory update ucs-school-import (16.0.1-15) > And if I remember this correctly you need to add the ".service" file > extension if you use deb-systemd-invoke. When installing a new service (daemon) yes, but this just installs a listener module for the existing listener service. > Don't we need a listener for staff and teachers as well? The listener is active for both: ------------------------------------------------------------------------ class UcsschoolAdminObjectType(ListenerModuleHandler): class Configuration: # admin user must be teacher and/or staff ldap_filter = '(|(objectClass=ucsschoolTeacher)(objectClass=ucsschoolStaff))' ------------------------------------------------------------------------
Turns out the options change can be handled in the UDM hook. So the listener module was removed. As the UDM hook is distributed in the domain via LDAP, but the ucsschool.lib may not have been updated on all machines yet, the UDM hook includes a static copy of the object-type to objectClass mapping. [4.3 745876f0] Bug #45379: handle options change in hook, handle hook import error, remove listener [4.3 d8a47a2c] Bug #45379: changelog [4.3 3a02e0a6] Bug #45379: advisory update ucs-school-import (16.0.1-16)
→ REOPEN: http://jenkins.knut.univention.de:8080/job/UCSschool-4.3/job/Upgrade%20Singleserver/lastCompletedBuild/Config=s4-all-components,TestGroup=base1/testReport/90_ucsschool/73_CSV_import_module/test/ (2018-04-05 23:48:54.819759) Traceback (most recent call last): (2018-04-05 23:48:54.819793) File "73_CSV_import_module", line 80, in <module> (2018-04-05 23:48:54.819846) main() (2018-04-05 23:48:54.819870) File "73_CSV_import_module", line 34, in main (2018-04-05 23:48:54.819891) verify_persons(persons_list) (2018-04-05 23:48:54.819912) File "/usr/share/ucs-test/90_ucsschool/essential/importcsv.py", line 255, in verify_persons (2018-04-05 23:48:54.819960) person.verify() (2018-04-05 23:48:54.819985) File "/usr/share/ucs-test/90_ucsschool/essential/importusers.py", line 351, in verify (2018-04-05 23:48:54.820006) utils.verify_ldap_object(self.dn, expected_attr=exp_attrs, strict=True, should_exist=True) (2018-04-05 23:48:54.820031) File "/usr/lib/pymodules/python2.7/univention/testing/decorators.py", line 39, in __call__ (2018-04-05 23:48:54.820111) self.func(*args, **kwargs) (2018-04-05 23:48:54.820139) File "/usr/lib/pymodules/python2.7/univention/testing/utils.py", line 176, in verify_ldap_object (2018-04-05 23:48:54.820160) raise LDAPObjectValueMissing(msg) (2018-04-05 23:48:54.820183) univention.testing.utils.LDAPObjectValueMissing: DN: uid=qoj7yz0qxt,cn=schueler,cn=users,ou=grzdxxuz4,dc=autotest206,dc=local (2018-04-05 23:48:54.820207) ucsschoolObjectType: None, missing: 'student'
→ REOPEN http://jenkins.knut.univention.de:8080/job/UCSschool-4.3/job/Upgrade%20Singleserver/lastCompletedBuild/Config=s4,TestGroup=import4/testReport/90_ucsschool/30_import-create_ou_via_python_api/test/ (2018-04-05 21:03:15.751892) Traceback (most recent call last): (2018-04-05 21:03:15.752012) File "30_import-create_ou_via_python_api", line 14, in <module> (2018-04-05 21:03:15.752159) eio.import_ou_basics(use_cli_api=False, use_python_api=True) (2018-04-05 21:03:15.752249) File "/usr/share/ucs-test/90_ucsschool/essential/importou.py", line 590, in import_ou_basics (2018-04-05 21:03:15.752494) use_python_api=use_python_api, (2018-04-05 21:03:15.752586) File "/usr/share/ucs-test/90_ucsschool/essential/importou.py", line 286, in create_and_verify_ou (2018-04-05 21:03:15.752812) create_ou_python_api(ou, dc, dc_administrative, sharefileserver, ou_displayname) (2018-04-05 21:03:15.752905) File "/usr/share/ucs-test/90_ucsschool/essential/importou.py", line 146, in create_ou_python_api (2018-04-05 21:03:15.753065) School(**kwargs).create(lo) (2018-04-05 21:03:15.753150) File "/usr/lib/pymodules/python2.7/ucsschool/lib/models/base.py", line 427, in create (2018-04-05 21:03:15.753359) success = self.create_without_hooks(lo, validate) (2018-04-05 21:03:15.753445) File "/usr/lib/pymodules/python2.7/ucsschool/lib/models/school.py", line 376, in create_without_hooks (2018-04-05 21:03:15.753647) if not self.add_domain_controllers(lo): (2018-04-05 21:03:15.753767) File "/usr/lib/pymodules/python2.7/ucsschool/lib/models/school.py", line 331, in add_domain_controllers (2018-04-05 21:03:15.753961) self.create_dc_slave(lo, dc_name, administrative=administrative) (2018-04-05 21:03:15.754046) File "/usr/lib/pymodules/python2.7/ucsschool/lib/models/school.py", line 304, in create_dc_slave (2018-04-05 21:03:15.754231) return dc.create(lo) (2018-04-05 21:03:15.754317) File "/usr/lib/pymodules/python2.7/ucsschool/lib/models/base.py", line 427, in create (2018-04-05 21:03:15.754517) success = self.create_without_hooks(lo, validate) (2018-04-05 21:03:15.754605) File "/usr/lib/pymodules/python2.7/ucsschool/lib/models/base.py", line 453, in create_without_hooks (2018-04-05 21:03:15.754801) self.do_create(udm_obj, lo) (2018-04-05 21:03:15.754886) File "/usr/lib/pymodules/python2.7/ucsschool/lib/models/computer.py", line 86, in do_create (2018-04-05 21:03:15.755042) return super(SchoolDCSlave, self).do_create(udm_obj, lo) (2018-04-05 21:03:15.755128) File "/usr/lib/pymodules/python2.7/ucsschool/lib/models/base.py", line 474, in do_create (2018-04-05 21:03:15.755336) udm_obj.create() (2018-04-05 21:03:15.755423) File "/usr/lib/pymodules/python2.7/univention/admin/handlers/__init__.py", line 483, in create (2018-04-05 21:03:15.755641) dn = self._create(response=response, serverctrls=serverctrls) (2018-04-05 21:03:15.755729) File "/usr/lib/pymodules/python2.7/univention/admin/handlers/__init__.py", line 1032, in _create (2018-04-05 21:03:15.756038) self._ldap_post_create() (2018-04-05 21:03:15.756127) File "/usr/lib/pymodules/python2.7/univention/admin/handlers/computers/domaincontroller_slave.py", line 515, in _ldap_post_create (2018-04-05 21:03:15.756339) univention.admin.handlers.simpleComputer.update_groups(self) (2018-04-05 21:03:15.756428) File "/usr/lib/pymodules/python2.7/univention/admin/handlers/__init__.py", line 2761, in update_groups (2018-04-05 21:03:15.757035) groupObject.modify(ignore_license=1) (2018-04-05 21:03:15.757132) File "/usr/lib/pymodules/python2.7/univention/admin/handlers/__init__.py", line 526, in modify (2018-04-05 21:03:15.757370) dn = self._modify(modify_childs, ignore_license=ignore_license, response=response) (2018-04-05 21:03:15.757458) File "/usr/lib/pymodules/python2.7/univention/admin/handlers/__init__.py", line 1074, in _modify (2018-04-05 21:03:15.757785) self.lo.modify(self.dn, ml, ignore_license=ignore_license, serverctrls=serverctrls, response=response) (2018-04-05 21:03:15.757870) File "/usr/lib/pymodules/python2.7/univention/admin/uldap.py", line 505, in modify (2018-04-05 21:03:15.758073) raise univention.admin.uexceptions.ldapError(_err2str(msg), original_exception=msg) (2018-04-05 21:03:15.758379) univention.admin.uexceptions.ldapError: Object class violation: instantiation of abstract objectClass 'ucsschoolObject' not allowed
The installed version of ucs-school-import in Jenkins is 16.0.1-9, current is 16.0.1-17. Those results are not relevant for this bug anymore.
[4.3 036e3d5a] Bug #45379: ucsschoolObjectType is a multivalue property [4.3 42c9999f] Bug #45379: changelog [4.3 748f3c9b] Bug #45379: advisory update ucs-school-import (16.0.1-19)
Sorry the last change broke adding school users. Outer Exception catcher: TypeError("unhashable type: 'list'",) Traceback (most recent call last): File "/usr/lib/pymodules/python2.7/ucsschool/importer/frontend/cmdline.py", line 117, in main self.do_import() File "/usr/lib/pymodules/python2.7/ucsschool/importer/frontend/cmdline.py", line 95, in do_import importer.mass_import() File "/usr/lib/pymodules/python2.7/ucsschool/importer/mass_import/mass_import.py", line 70, in mass_import self.import_users() File "/usr/lib/pymodules/python2.7/ucsschool/importer/mass_import/mass_import.py", line 100, in import_users user_import.create_and_modify_users(imported_users) # 90% - 100% File "/usr/lib/pymodules/python2.7/ucsschool/importer/mass_import/user_import.py", line 144, in create_and_modify_users success = user.create(lo=self.connection) File "/usr/lib/pymodules/python2.7/ucsschool/importer/models/import_user.py", line 255, in create return super(ImportUser, self).create(lo, validate) File "/usr/lib/pymodules/python2.7/ucsschool/lib/models/base.py", line 427, in create success = self.create_without_hooks(lo, validate) File "/usr/lib/pymodules/python2.7/ucsschool/importer/models/import_user.py", line 259, in create_without_hooks return super(ImportUser, self).create_without_hooks(lo, validate) File "/usr/lib/pymodules/python2.7/ucsschool/lib/models/base.py", line 453, in create_without_hooks self.do_create(udm_obj, lo) File "/usr/lib/pymodules/python2.7/ucsschool/lib/models/user.py", line 253, in do_create success = super(User, self).do_create(udm_obj, lo) File "/usr/lib/pymodules/python2.7/ucsschool/lib/models/base.py", line 474, in do_create udm_obj.create() File "/usr/lib/pymodules/python2.7/univention/admin/handlers/__init__.py", line 483, in create dn = self._create(response=response, serverctrls=serverctrls) File "/usr/lib/pymodules/python2.7/univention/admin/handlers/__init__.py", line 1019, in _create al = self.call_udm_property_hook('hook_ldap_addlist', self, al) File "/usr/lib/pymodules/python2.7/univention/admin/handlers/__init__.py", line 864, in call_udm_property_hook changes = func(module, changes) File "/usr/lib/pymodules/python2.7/univention/admin/hooks.d/ucsschool_object_type.py", line 75, in hook_ldap_addlist for oc in object_type_to_object_classes[obj.info['ucsschoolObjectType']]: TypeError: unhashable type: 'list'
That change was a bit hasty... The following commits fix the error, and change the code to be more consistent by using lists everywhere. The test suite was adapted accordingly. Furthermore in a separate commit the object type for TeachersAndStaff user objects was changed from ['teacher_and_staff'] to ['staff', 'teacher']. This change has not been approved by everyone yet, so it may be reverted tomorrow. [4.3] 2ff670df Bug #45379: ucsschoolObjectType is a UDM multi-value property [4.3] 0d06f246 Bug #45379: ucsschoolObjectType is a UDM multi-value property [4.3] c46cb476 Bug #45379: ucsschoolObjectType is a UDM multi-value property [4.3] 83b9e2b2 Bug #45379: change object type for TeachersAndStaff from teacher_and_staff to ['staff', 'teacher'] [4.3] 0e4fb30c Bug #45379: changelog [4.3] 20661c21 Bug #45379: advisory update ucs-school-lib (11.0.1-8) ucs-school-import (16.0.1-20) ucs-test-ucsschool (5.0.2-39)
http://jenkins.knut.univention.de:8080/job/UCSschool-4.3/job/Install%20Singleserver/lastCompletedBuild/Config=s4,TestGroup=base1/testReport/90_ucsschool/101_exam_mode/test/ failed due to new S4 connector rejects which I found in connector-s4.log: 12.04.2018 02:10:52,626 LDAP (PROCESS): sync to ucs: Resync rejected dn: CN=sdv9othsfl,CN=Users,DC=autotest201,DC=local 12.04.2018 02:10:52,632 LDAP (PROCESS): sync to ucs: [ user] [ modify] uid=sdv9othsfl,cn=users,dc=autotest201,dc=local 12.04.2018 02:10:52,727 LDAP (ERROR ): Unknown Exception during sync_to_ucs 12.04.2018 02:10:52,727 LDAP (ERROR ): Traceback (most recent call last): File "/usr/lib/pymodules/python2.7/univention/s4connector/__init__.py", line 1588, in sync_to_ucs result = self.modify_in_ucs(property_type, object, module, position) File "/usr/lib/pymodules/python2.7/univention/s4connector/__init__.py", line 1365, in modify_in_ucs res = ucs_object.modify(serverctrls=serverctrls, response=response) File "/usr/lib/pymodules/python2.7/univention/admin/handlers/users/user.py", line 1672, in modify return super(object, self).modify(*args, **kwargs) File "/usr/lib/pymodules/python2.7/univention/admin/handlers/__init__.py", line 526, in modify dn = self._modify(modify_childs, ignore_license=ignore_license, response=response) File "/usr/lib/pymodules/python2.7/univention/admin/handlers/__init__.py", line 1069, in _modify ml = self.call_udm_property_hook('hook_ldap_modlist', self, ml) File "/usr/lib/pymodules/python2.7/univention/admin/handlers/__init__.py", line 864, in call_udm_property_hook changes = func(module, changes) File "/usr/lib/pymodules/python2.7/univention/admin/hooks.d/ucsschool_object_type.py", line 98, in hook_ldap_modlist if self.old_options != new_options: AttributeError: 'UcsschoolObjectType' object has no attribute 'old_options'
Preparing to unpack .../ucs-test-ucsschool_5.0.2-40A~4.3.0.201804111307_all.deb Entpacken von ucs-school-import (16.0.1-23A~4.3.0.201804111922) http://jenkins.knut.univention.de:8080/job/UCSschool-4.3/job/Install%20Singleserver/lastCompletedBuild/Config=s4,TestGroup=base1/testReport/90_ucsschool/79_ldap_acl_access_to_uidNumber0/test/ (2018-04-12 04:49:34.265630) Warning: Failed to remove 'users/user' object 'uid=schooladminA,cn=admins,cn=users,ou=schoolA,dc=autotest201,dc=local' (2018-04-12 04:49:34.292161) stdout='E: object not found\n' '' {} (2018-04-12 04:49:35.397423) Traceback (most recent call last): (2018-04-12 04:49:35.397452) File "79_ldap_acl_access_to_uidNumber0", line 165, in <module> (2018-04-12 04:49:35.397542) main() (2018-04-12 04:49:35.397568) File "79_ldap_acl_access_to_uidNumber0", line 109, in main (2018-04-12 04:49:35.397614) env = create_environment(schoolenv, udm, ucr) (2018-04-12 04:49:35.397637) File "79_ldap_acl_access_to_uidNumber0", line 75, in create_environment (2018-04-12 04:49:35.397660) school.name, school.dn = self.create_ou(ou_name='school%s' % (suffix,), name_edudc=hostname, name_admindc=admin_hostname) (2018-04-12 04:49:35.397685) File "/usr/lib/pymodules/python2.7/univention/testing/ucsschool.py", line 332, in create_ou (2018-04-12 04:49:35.397727) result = School(**kwargs).create(self.lo) (2018-04-12 04:49:35.397752) File "/usr/lib/pymodules/python2.7/ucsschool/lib/models/base.py", line 427, in create (2018-04-12 04:49:35.397852) success = self.create_without_hooks(lo, validate) (2018-04-12 04:49:35.397880) File "/usr/lib/pymodules/python2.7/ucsschool/lib/models/school.py", line 375, in create_without_hooks (2018-04-12 04:49:35.397928) self.add_host_to_dc_group(lo) (2018-04-12 04:49:35.397950) File "/usr/lib/pymodules/python2.7/ucsschool/lib/models/school.py", line 276, in add_host_to_dc_group (2018-04-12 04:49:35.397966) dc_udm_obj.modify() (2018-04-12 04:49:35.397984) File "/usr/lib/pymodules/python2.7/univention/admin/handlers/__init__.py", line 526, in modify (2018-04-12 04:49:35.398025) dn = self._modify(modify_childs, ignore_license=ignore_license, response=response) (2018-04-12 04:49:35.398046) File "/usr/lib/pymodules/python2.7/univention/admin/handlers/__init__.py", line 1076, in _modify (2018-04-12 04:49:35.398189) self._ldap_post_modify() (2018-04-12 04:49:35.398216) File "/usr/lib/pymodules/python2.7/univention/admin/handlers/computers/domaincontroller_slave.py", line 553, in _ldap_post_modify (2018-04-12 04:49:35.400526) univention.admin.handlers.simpleComputer.update_groups(self) (2018-04-12 04:49:35.400550) File "/usr/lib/pymodules/python2.7/univention/admin/handlers/__init__.py", line 2761, in update_groups (2018-04-12 04:49:35.400933) groupObject.modify(ignore_license=1) (2018-04-12 04:49:35.400956) File "/usr/lib/pymodules/python2.7/univention/admin/handlers/__init__.py", line 526, in modify (2018-04-12 04:49:35.401038) dn = self._modify(modify_childs, ignore_license=ignore_license, response=response) (2018-04-12 04:49:35.401061) File "/usr/lib/pymodules/python2.7/univention/admin/handlers/__init__.py", line 1074, in _modify (2018-04-12 04:49:35.401211) self.lo.modify(self.dn, ml, ignore_license=ignore_license, serverctrls=serverctrls, response=response) (2018-04-12 04:49:35.401234) File "/usr/lib/pymodules/python2.7/univention/admin/uldap.py", line 505, in modify (2018-04-12 04:49:35.401307) raise univention.admin.uexceptions.ldapError(_err2str(msg), original_exception=msg) (2018-04-12 04:49:35.401356) univention.admin.uexceptions.ldapError: Object class violation: instantiation of abstract objectClass 'ucsschoolObject' not allowed
[2018-04-11 21:54:51.102544] UCSTestSchool cleanup done (2018-04-11 21:54:51.102647) Traceback (most recent call last): (2018-04-11 21:54:51.102677) File "79_ldap_acl_access_to_uidNumber0", line 165, in <module> (2018-04-11 21:54:51.102721) main() (2018-04-11 21:54:51.102743) File "79_ldap_acl_access_to_uidNumber0", line 109, in main (2018-04-11 21:54:51.102807) env = create_environment(schoolenv, udm, ucr) (2018-04-11 21:54:51.102834) File "79_ldap_acl_access_to_uidNumber0", line 78, in create_environment (2018-04-11 21:54:51.102855) school.teacher_name, school.teacher_dn = self.create_user(school.name, username='teacher%s' % (suffix,), schools=schools, is_teacher=True, password=PASSWORD) (2018-04-11 21:54:51.102881) File "/usr/lib/pymodules/python2.7/univention/testing/ucsschool.py", line 541, in create_user (2018-04-11 21:54:51.102926) should_exist=True, (2018-04-11 21:54:51.102951) File "/usr/lib/pymodules/python2.7/univention/testing/utils.py", line 176, in verify_ldap_object (2018-04-11 21:54:51.102972) raise LDAPObjectValueMissing(msg) (2018-04-11 21:54:51.103015) univention.testing.utils.LDAPObjectValueMissing: DN: uid=teacherA,cn=lehrer,cn=users,ou=schoolA,dc=autotest208,dc=local (2018-04-11 21:54:51.103038) ucsschoolObjectType: ['teacher'], missing: 'a', c', e', h', r', t'
It was decided to postpone further development on this in favor of a different solution that includes enhanced ACLs support and support for multiple roles per object and different roles per OU. That solution is currently being investigated. For now we need a clean master branch. So the code in the state of comment 15 has been preserved in branch "dtroeder/45379_ucsschoolObjectType" and changes to the "4.3" branch have been reverted. [4.3] 8b66acbb Revert "Bug #45379: change object type for TeachersAndStaff from teacher_and_staff to ['staff', 'teacher']" [4.3] e61c76f0 Revert "Bug #45379: ucsschoolObjectType is a UDM multi-value property" [4.3] a17f0017 Revert "Bug #45379: ucsschoolObjectType is a UDM multi-value property" [4.3] 94013df4 Revert "Bug #45379: ucsschoolObjectType is a UDM multi-value property" [4.3] 893984c1 Revert "Bug #45379: ucsschoolObjectType is a multivalue property" [4.3] 6c4a4c78 Revert "Bug #45379: handle options change in hook, handle hook import error, remove listener" [4.3] 5b83cd55 Revert "Bug #45379: fix typo" [4.3] 552148da Revert "Bug #45379: fix typo" [4.3] 97d86ebd Revert "Bug #45379: verify objectClass" [4.3] 2573b6a5 Revert "Bug #45379: verify setting of ucsschoolObjectType attribute" [4.3] 2cd59b6a Revert "Bug #45379: mark administrator group, add object_type to OC mapping" [4.3] 62eff179 Revert "Bug #45379: mark ImportUser objects, update legacy import script, finish udm hook, add listener, add migration script" [4.3] c026433d Revert "Bug #45379: add objectClass only when creating an object" [4.3] d016e53a Revert "Bug #45379: add object type attribute to all UCS@school objects * create specific objectClasses * set attribute for some classes * create udm hook to add objectClasses" [4.3] 60023a74 Bug #45379: changelog (entry) and advisory (revert) [4.3] ade1340d Bug #45379: version bump [4.3] ac061b91 Bug #45379: advisory update ucs-school-lib (11.0.1-9) ucs-school-import (16.0.1-24) ucs-test-ucsschool (5.0.2-33) Set Bugzilla state to NEEDMOREINFO as further development is paused until a decision is made.
Accidentally deleted some changelog entries and ended up with a lower version than in repository. [4.3 1e6ab9cc] Bug #45379: readd deleted changelog entries and version bump ucs-test-ucsschool (5.0.2-43)
Make feature for now optional (UCR-switch).
https://hutten.knut.univention.de/mediawiki/index.php/UCS@school_LDAP-Rollen
Most UCS@school objects now have an ucsschool.lib property "ucsschool_roles" (LDAP attribute "ucsschoolRole"). It is added automatically when creating an object. There is a migration script that will add it to all existing objects: /usr/share/ucs-school-import/scripts/migrate_ucsschool_roles It supports choosing the object types to modify and a dry-run mode. It will not touch objects that already have a role set, except if used with "--force". ucs-tests have been adapted to check for roles on all objects. All code is in branch dtroeder/45379_ucsschoolRole. No packages have been built. de63c386c Bug #45379: add ucsschoolRole LDAP attribute 9caaa997e Bug #45379: add constants and mixin class to support ucsschool_role attribute 5931ca3a5 Bug #45379: add ucsschool_role attribute to ucsschool lib classes 0ed627db0 Bug #45379: add ucsschool_role attribute support to ucsschool import 07b50589f Bug #45379: ucs-tests check setting of roles ac134fab4 Bug #45379: add migration script When merging, the ucs-school-import packages dependency on python-ucs-school (from ucs-school-lib) must be updated to be the version created in this bug. More testing is required regarding moving objects from one OU into another. The behavior should be: * remove all roles from old OU * add minimal roles from new OU, only those clearly belonging to the object (e.g. "teacher", but not "school_admin") * leave roles from other OUs untouched
Reopen as discussed: Use default_roles instead of roles. Replace the assert with an exception in roles.py. And something I forgot: The new objectclasses: ucsschoolGroup, ucsschoolShare, ucsschoolServer all do the same? Would a more generic object class be enough here?
(In reply to Jürn Brodersen from comment #24) > Use default_roles instead of roles. > Replace the assert with an exception in roles.py. [dtroeder/45379_ucsschoolRole] 7f1325e51 Bug #45379: use exception instead of assertion > And something I forgot: > The new objectclasses: ucsschoolGroup, ucsschoolShare, ucsschoolServer all > do the same? Would a more generic object class be enough here? As discussed, we'll keep different objectclasses for future use. As discussed, the "roles" attribute was renamed to "default_roles". [dtroeder/45379_ucsschoolRole] 108c36573 Bug #45379: rename attribute with default roles In the meantime I found various problems: The create_ou script didn't mark the OU object. [dtroeder/45379_ucsschoolRole] 4c3cfd044 Bug #45379: fix create_ou not setting role on school (OU) object Moving a user didn't always change its ucsschoolRoles attributes in LDAP, because not always after a move() a modify() follows. A direct LDAP modification is now done after a move, because using a usschool.lib modify operation would result in a complex, recursive behavior, resulting in a remove(); create() of the object. I added a test that is similar to 80_move_users_into_another_ou, but simpler. [dtroeder/45379_ucsschoolRole] bdc04fd80 Bug #45379: add test that moves a user from school A to B to C using the import [dtroeder/45379_ucsschoolRole] cea1200e4 Bug #45379: modify roles in LDAP object directly after move(), because it's possible that no modify() will follow Diverse fixes: [dtroeder/45379_ucsschoolRole] 8f3eba3d9 Bug #45379: do not mask exceptions of later code [dtroeder/45379_ucsschoolRole] 0370515e9 Bug #45379: lower noise
I had two problems with the migration script: ----- ''' Object class violation: instantiation of abstract objectClass 'ucsschoolType' not allowed ''' For a user without any school object class ----- ''' Type or value exists: ucsschoolRole: value #19 provided more than once ''' For a user with both ucsschoolAdministrator and ucsschoolTeacher object classes ----- I think a better error message would be nice here And this doc string is weird: :param str oc: objectClass the is required for ucsschoolRole attribute the -> that?
Not a big deal but would it be possible to print the path of the log file at the end again? Otherwise it kinda gets lost in the other messages.
102_rename_class fails: Traceback (most recent call last): File "102_rename_class", line 178, in <module> main() File "102_rename_class", line 170, in main test_rename_class(schoolenv, school, old_name, new_name) File "102_rename_class", line 133, in test_rename_class check_ldap(school, old_name, new_name) File "102_rename_class", line 46, in check_ldap utils.verify_ldap_object(class_dn(new_name, school), expected_attr={'ucsschoolRole': create_ucsschool_role_string(role_school_class,school)}, should_exist=True) File "/usr/lib/pymodules/python2.7/univention/testing/utils.py", line 176, in verify_ldap_object raise LDAPObjectValueMissing(msg) univention.testing.utils.LDAPObjectValueMissing: DN: cn=vzd-vwiuwhw2lo,cn=klassen,cn=schueler,cn=groups,ou=vzd,dc=univention,dc=intranet ucsschoolRole: ['school_class:school:vzd'], missing: 'a', c', :', d', h', l', o', s', v', z', _' ucsschoolRole: ['school_class:school:vzd'], unexpected: 'school_class:school:vzd' ucsschoolRole needs to be an array I did not yet look at the tests
I did not yet look at the tests -> I did not yet look at the other tests
(In reply to Daniel Tröder from comment #21) > Make feature for now optional (UCR-switch). Sönke reminded me of this. I can't seem to find a way to do this nor is their any info on the trello card for this?
(In reply to Jürn Brodersen from comment #26) > I had two problems with the migration script: > ----- > ''' > Object class violation: instantiation of abstract objectClass > 'ucsschoolType' not allowed > ''' > For a user without any school object class > ----- The ucsschool.lib has backwards compatibility code that assumes a school user object, when it finds a user object in a school user container. Code was added to ignore such users. [dtroeder/45379_ucsschoolRole fff2d39f9] Bug #45379: ignore non-school user objects found in school user container > ''' > Type or value exists: ucsschoolRole: value #19 provided more than once > ''' > For a user with both ucsschoolAdministrator and ucsschoolTeacher object > classes > ----- [dtroeder/45379_ucsschoolRole 671245aa7] Bug #45379: fix school admin membership > I think a better error message would be nice here Not possible - this is formatted by the Python LDAP lib directly :/ > And this doc string is weird: > :param str oc: objectClass the is required for ucsschoolRole attribute > > the -> that? [dtroeder/45379_ucsschoolRole f20074354] Bug #45379: fix docstring (In reply to Jürn Brodersen from comment #27) > Not a big deal but would it be possible to print the path of the log file at > the end again? Otherwise it kinda gets lost in the other messages. Good idea. [dtroeder/45379_ucsschoolRole dd3821749] Bug #45379: print logfile path at end (In reply to Jürn Brodersen from comment #28) > 102_rename_class fails: [..] > ucsschoolRole needs to be an array Found that bug in various places... [dtroeder/45379_ucsschoolRole 23c7c9b36] Bug #45379: ucsschoolRole is a list (In reply to Jürn Brodersen from comment #30) > (In reply to Daniel Tröder from comment #21) > > Make feature for now optional (UCR-switch). > > Sönke reminded me of this. I can't seem to find a way to do this nor is > their any info on the trello card for this? I implemented this now rather crudely, expecting it to vanish without trace in the not to far future. All role saving and checking code has been wrapped in a UCRV-switch. Enable "ucsschool/feature/roles" to enable the roles code. [dtroeder/45379_ucsschoolRole 332097de3] Bug #45379: make roles handling a disabled feature (UCRV ucsschool/feature/roles)
The git branch dtroeder/45379_ucsschoolRole was merged to 4.3 and the packages built. [4.3] 30a687eca Bug #45379: Merge branch 'dtroeder/45379_ucsschoolRole' into 4.3 [4.3] e3a8b908f Bug #45379: changelog [4.3] 52917fc8e Bug #45379: advisory ucs-school-lib (11.0.1-14) ucs-school-import (16.0.2-20) ucs-test-ucsschool (5.0.2-68)
A similar traceback also happens with other test scripts: File "231_import-users_checks_in_dry-run", line 207, in <module> Test().run() File "/usr/share/ucs-test/90_ucsschool/essential/importusers_cli_v2.py", line 288, in run self.create_ous(schoolenv) File "/usr/share/ucs-test/90_ucsschool/essential/importusers_cli_v2.py", line 267, in create_ous res = schoolenv.create_multiple_ous(len(ous), name_edudc=self.ucr.get('hostname'), use_cache=self.use_ou_cache) File "/usr/lib/pymodules/python2.7/univention/testing/ucsschool.py", line 381, in create_multiple_ous ou_name, ou_dn = self.create_ou(None, name_edudc, name_admindc, displayName, name_share_file_server, use_cli, wait_for_replication, False) File "/usr/lib/pymodules/python2.7/univention/testing/ucsschool.py", line 334, in create_ou result = School(**kwargs).create(self.lo) File "/usr/lib/pymodules/python2.7/ucsschool/lib/models/base.py", line 435, in create success = self.create_without_hooks(lo, validate) File "/usr/lib/pymodules/python2.7/ucsschool/lib/models/school.py", line 374, in create_without_hooks success = super(School, self).create_without_hooks(lo, validate) File "/usr/lib/pymodules/python2.7/ucsschool/lib/models/base.py", line 463, in create_without_hooks self.do_create(udm_obj, lo) File "/usr/lib/pymodules/python2.7/ucsschool/lib/models/base.py", line 482, in do_create self._alter_udm_obj(udm_obj) File "/usr/lib/pymodules/python2.7/ucsschool/lib/models/school.py", line 420, in _alter_udm_obj return super(School, self)._alter_udm_obj(udm_obj) File "/usr/lib/pymodules/python2.7/ucsschool/lib/models/base.py", line 424, in _alter_udm_obj udm_obj[attr.udm_name] = value File "/usr/lib/pymodules/python2.7/univention/admin/handlers/__init__.py", line 376, in __setitem__ self.descriptions[key] KeyError: 'ucsschoolRole'
(In reply to Sönke Schwardt-Krummrich from comment #33) > A similar traceback also happens with other test scripts: > > File "231_import-users_checks_in_dry-run", line 207, in <module> > Test().run() > File "/usr/share/ucs-test/90_ucsschool/essential/importusers_cli_v2.py", > line 288, in run > self.create_ous(schoolenv) > File "/usr/share/ucs-test/90_ucsschool/essential/importusers_cli_v2.py", > line 267, in create_ous > res = schoolenv.create_multiple_ous(len(ous), > name_edudc=self.ucr.get('hostname'), use_cache=self.use_ou_cache) > File "/usr/lib/pymodules/python2.7/univention/testing/ucsschool.py", line > 381, in create_multiple_ous > ou_name, ou_dn = self.create_ou(None, name_edudc, name_admindc, > displayName, name_share_file_server, use_cli, wait_for_replication, False) > File "/usr/lib/pymodules/python2.7/univention/testing/ucsschool.py", line > 334, in create_ou > result = School(**kwargs).create(self.lo) > File "/usr/lib/pymodules/python2.7/ucsschool/lib/models/base.py", line > 435, in create > success = self.create_without_hooks(lo, validate) > File "/usr/lib/pymodules/python2.7/ucsschool/lib/models/school.py", line > 374, in create_without_hooks > success = super(School, self).create_without_hooks(lo, validate) > File "/usr/lib/pymodules/python2.7/ucsschool/lib/models/base.py", line > 463, in create_without_hooks > self.do_create(udm_obj, lo) > File "/usr/lib/pymodules/python2.7/ucsschool/lib/models/base.py", line > 482, in do_create > self._alter_udm_obj(udm_obj) > File "/usr/lib/pymodules/python2.7/ucsschool/lib/models/school.py", line > 420, in _alter_udm_obj > return super(School, self)._alter_udm_obj(udm_obj) > File "/usr/lib/pymodules/python2.7/ucsschool/lib/models/base.py", line > 424, in _alter_udm_obj > udm_obj[attr.udm_name] = value > File "/usr/lib/pymodules/python2.7/univention/admin/handlers/__init__.py", > line 376, in __setitem__ > self.descriptions[key] > KeyError: 'ucsschoolRole' This might be because an old version of this was installed in the past and the extended attributes were not updated.
Ok join script was already executed: univention-run-join-scripts --force --run-scripts 35ucs-school-import
Changing udm user options doesn't change the role. Previously there was a udm hook. See commit 745876f0a Also after changing a student to teacher through udm user options and calling the migration script again with the force option the old role is not removed.
For now, this feature will be released without support for modifying objectClasses / UDM options.
I think a lot of tests would fail because of an empty ucsschool_roles array from the umc server which is not expected. It looks like you started work for this in a73c485d1 but only for users. I added some code to only send the ucsschool_role attribute if the ucr feature variable is set, to make the tests happy. Feel free to change or revert this. [4.3 ee96fd970] Bug #45379: Make ucsschool role attribute internal if not activated [4.3 ad5a894ab] Revert "Bug #46740: add ucsschool_roles support to User test class" Reopen because of commit ee96fd970.
(In reply to Jürn Brodersen from comment #38) > I added some code to only send the ucsschool_role attribute if the ucr > feature variable is set, to make the tests happy. Feel free to change or > revert this. > [4.3 ee96fd970] Bug #45379: Make ucsschool role attribute internal if not > activated > [4.3 ad5a894ab] Revert "Bug #46740: add ucsschool_roles support to User test > class" Excellent. I wasn't aware of Attribute.internal - very useful. All Jenkins tests failed tonight, because the join script 35ucs-school-import.inst of 4.3 didn't run. The reason is, that its version is the same as in 4.2. I raised its version and rebuilt. I also raised the version of 40ucs-school-import-http-api.inst, because the init script / systemd unit for the Gunicorn process was renamed from "gunicorn" to "ucs-school-import-http-api", and the systemd unit enabling code is in that join script. (Not really part of this bug, but had to be done anyway.) [4.3] 8a1988f43 Bug #45379: bump join script version [4.3] 3b78ad63a Bug #45379: advisory update [4.3] a2cf2601d Bug #45379: advisory update
[4.3] c22bdb120 Bug #45379: add share classes to module import [4.3] dd965e6b4 Bug #45379: advisory update ucs-school-lib (11.0.1-16)
ok
UCS@school 4.3 v4 has been released. https://docs.software-univention.de/changelog-ucsschool-4.3v4-de.html If this error occurs again, please clone this bug.