Bug 58318 - Activate univentionObjectIdentifier for all LDAP objects with objectClass=univentionObject
Summary: Activate univentionObjectIdentifier for all LDAP objects with objectClass=uni...
Status: VERIFIED FIXED
Alias: None
Product: UCS
Classification: Unclassified
Component: LDAP
Version: UCS 5.2
Hardware: amd64 Linux
: P5 normal
Target Milestone: UCS 5.2-2
Assignee: Arvid Requate
QA Contact: Julia Bremer
URL: https://git.knut.univention.de/univen...
Keywords:
Depends on: 58252
Blocks: 58384
  Show dependency treegraph
 
Reported: 2025-05-23 16:08 CEST by Arvid Requate
Modified: 2025-06-11 15:29 CEST (History)
3 users (show)

See Also:
What kind of report is it?: Feature Request
What type of bug is this?: ---
Who will be affected by this bug?: ---
How will those affected feel about the bug?: ---
User Pain:
Enterprise Customer affected?: Yes
School Customer affected?:
ISV affected?: Yes
Waiting Support:
Flags outvoted (downgraded) after PO Review:
Ticket number:
Bug group (optional):
Customer ID:
Max CVSS v3 score:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Arvid Requate univentionstaff 2025-05-23 16:08:00 CEST
We want the univentionObjectIdentifier set for all objects, so
* Bug #58310 takes care of base.ldif
* This bug migrates other objects by running a script during package
  update on the primary to initialize univentionObjectIdentifier with entryUUID
* This bug changes the UCR default for
  directory/manager/object-identifier/autogeneration to true
* Bug #57393 adjusts all Directory Nodes to eq-index the attribute
Comment 1 Arvid Requate univentionstaff 2025-05-23 16:50:54 CEST
I think we also need to activate the new default for UDM via UCR policy for the entire domain. See MR in URL for a proposal.
Comment 2 Florian Best univentionstaff 2025-06-04 09:21:26 CEST
Another broken case:

436 times in /var/log/univention/connector-s4.log:
https://jenkins2022.knut.univention.de/job/UCS-5.2/job/UCS-5.2-2/job/AutotestJoin/SambaVersion=s4,Systemrolle=master-part-II/ws/test/connector-s4.log
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/univention/s4connector/__init__.py", line 1475, in sync_to_ucs
    result = self.property[property_type].ucs_sync_function(self, property_type, object)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/univention/s4connector/s4/dns.py", line 1606, in con2ucs
    ucs_host_record_create(s4connector, object)
  File "/usr/lib/python3/dist-packages/univention/s4connector/s4/dns.py", line 926, in ucs_host_record_create
    newRecord.create()
  File "/usr/lib/python3/dist-packages/univention/admin/handlers/__init__.py", line 606, in create
    dn = self._create(response=response, serverctrls=serverctrls, ignore_license=ignore_license)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/univention/admin/handlers/__init__.py", line 1335, in _create
    al = self._set_univentionObjectIdentifier(al)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/univention/admin/handlers/__init__.py", line 1309, in _set_univentionObjectIdentifier
    univention.admin.allocators.acquireUnique(self.lo, self.position, 'univentionObjectIdentifier', obj_identifier, 'univentionObjectIdentifier', scope='scope')
  File "/usr/lib/python3/dist-packages/univention/admin/allocators.py", line 246, in acquireUnique
    univention.admin.locking.lock(lo, position, type, value.encode('utf-8'), scope=scope)
  File "/usr/lib/python3/dist-packages/univention/admin/locking.py", line 96, in lock
    lo.add(dn, al)
  File "/usr/lib/python3/dist-packages/univention/admin/uldap.py", line 773, in add
    raise univention.admin.uexceptions.ldapError(_err2str(msg), original_exception=msg)
univention.admin.uexceptions.ldapError: LDAP Error: No such object.
Comment 3 Florian Best univentionstaff 2025-06-04 09:35:08 CEST
Many issues occurred when activating this globally, as revealed by our Jenkins tests:

1. subtree move of e.g. container/ou copied the identifier, but it's a read-only property for modifications
Test case: 59_udm.63_test_udm_containers.Test_ContainerOU.test_container_ou_rename

2. when initializing a module with a user template (like the self service does) the univentionObjectIdentifier property was copied from the template into the default value for the property in the target module.

3. Installing Apps on a memberserver was not possible anymore as memberservers weren't allowed to write the lock-attribute, which ensures uniqueness of the value.

4. blocklist entries couldn't be created anymore as they are created in a different LDAP database and the locking mechanism tried to create the temporary locks in that database.

5. DNS problems in S4 Connector, see comment 2.

TBC
Comment 4 Julia Bremer univentionstaff 2025-06-04 11:07:44 CEST
Problems 1-5 have been fixed by the following commits:

3c91c710e1 Bug #58318: For creating / searching locks, use the default LDAP base
5edf692076 fixup! Bug #58318: Only create univentionObjectIdentifier in the main LDAP database
2de9ca9f12 Bug #58318: Add note about container-udm-rest
b9db3f612f Bug #58318: Only create univentionObjectIdentifier in the main LDAP database
b43e28ba5b Bug #58318: Allow memberserver to write temporary lock objects for UniventionObjectIdentifier
2df5f3af40 Bug #58318: Fix UDM for activated univentionObjectIdentifier


1. subtree move of e.g. container/ou:
2df5f3af40
-> We now ignore the univentionObjectIdentifier during the creation of the intermediate move container

2. templates containing univentionObjectIdentifier tried to give their univentionObjectIdentifier to new objects
2df5f3af40
-> We now ignore univentionObjectIdentifier while applying the values from the template

3.  Installing Apps on a memberserver was not possible anymore
b43e28ba5b
-> We added ACLs to allow a memberserver to write temporary lock objects for univentionObjectIdentifier. We refrained from allowing them to write all temporary lock objects, because this would have allowed them to spam the lock objects for e.g. uid, uidNumber, gidNumber which would have been a DOS risk.
For univentionObjectIdentifier this risk is not applicable, since it contains random numbers. "Spamming" them would not create a DOS risk. 

4. blocklist entries couldn't be created anymore
b9db3f612f
-> We disabled the creation of a univentionObjectIdentifier for blocklist/list and blocklist/entry objects. Since they are created in a different LDAP base, the univentionObjectIdentifier check would need to iterate over all LDAP databases to check for uniqueness. These are internal objects and there is currently no use case for univentionObjectIdentifier on such objects. 
If we create more LDAP databases for different UDM objects, we may want to discuss how the approach should work in these cases.

5. S4 connector could not create msdcs objects anymore
3c91c710e1
During the sync of msdcs objects, the S4connector connects to base zoneName=AutoTest091.test,cn=dns,dc=AutoTest091,dc=test. During the creation of the lock object, UDM tried to create a lock object under cn=univentionObjectIdentifier,cn=temporary,cn=univention,zoneName=AutoTest091.test,cn=dns,dc=AutoTest091,dc=test.
We changed the function that creates univentionObjectIdentifier to always search for duplicates under the default ldap/base and to create lock objects under that base as well.

There may be more issues, but that seems to be all of them. Waiting for test results.
Comment 5 Arvid Requate univentionstaff 2025-06-04 18:59:57 CEST
Some more:

f9ac152fc88 | Make slapd ensure uniqueness of univentionObjectIdentifier
af7f65e1f83 | Make UDM lock for univentionObjectIdentifier optional
dd8fd3ecd9d | Relax syntax UUID4 to UUID1_or_UUID4

Package: univention-ldap
Version: 18.2.2
Branch: ucs_5.2-0-ucs5.2-2

Package: univention-directory-manager-modules
Version: 17.2.4
Branch: ucs_5.2-0-ucs5.2-2
Comment 6 Arvid Requate univentionstaff 2025-06-05 13:58:52 CEST
497ff2de77d | Activate slapo-unique on the primary (only)

Package: univention-ldap
Version: 18.2.3
Branch: ucs_5.2-0-ucs5.2-2
Comment 7 Florian Best univentionstaff 2025-06-05 14:53:19 CEST
6. UCS@school (and PS projects might) have LDAP ACL's which needs to be adapted as well.
Therefor uniqueness check is made optional in UDM and enforced via LDAP uniqueness constraint (only on the DC primary, which is the only write-able LDAP server).

7. The syntax check ensured that the values is a UUID4, while entryUUID is a UUID1.
As the value source could also be e.g. a objectGUID (which is a UUID4), both variants are now allowed.

8. Should the AD-Connector / S4-Connector use the objects objectGUID as value by default when syncing to UCS?
→ Create a new bug for this

9. The UCR variable 'directory/manager/object-identifier/autogeneration' still would allow to disable that feature.
As Ingo decided, the configurabillity should be removed.
Comment 8 Arvid Requate univentionstaff 2025-06-05 18:29:52 CEST
Yet another change requested:
* Remove directory/manager/object-identifier/autogeneration entirely

6d7cd259458 | Remove UCRV directory/manager/object-identifier/autogeneration

Successful build
Package: univention-directory-manager-modules
Version: 17.2.5
Branch: 5.2-0
Scope: ucs5.2-2

Package: ucs-test                                                                                                                                                                            
Version: 12.2.1                                                                                                                                                           
Branch: 5.2-0                                                                                                                                                                               
Scope: ucs5.2-2
Comment 9 Arvid Requate univentionstaff 2025-06-06 13:38:42 CEST
As discussed:

36a797b29d5 | Re-add UCRV directory/manager/object-identifier/autogeneration

Package: univention-directory-manager-modules                              
Version: 17.2.6                   
Branch: 5.2-0                                        
Scope: ucs5.2-2

Package: ucs-test                                                                                                                                                           
Version: 12.2.1                                                                                                                                                                              
Branch: 5.2-0                                                                                                                       
Scope: ucs5.2-2
Comment 10 Julia Bremer univentionstaff 2025-06-11 15:01:37 CEST
* OK: default of directory/manager/object-identifier/autogeneration is set to true
* OK:  directory/manager/object-identifier/autogeneration description has been adjusted to explicitely state that this feature shall not be deactivated
* ~OK: diagnostic check checks all univentionObject* objects for univentionObjectIdentifier (this does not include objects which are not univentionObjects)
* ~OK: Currently univentionObjectIdentifier is not added to dnsZone* objects -> we need a new bug for that
* OK: Blocklist objects don't get a univentionObjectIdentifier
* OK: Subtree move doesn't copy the containers univentionObjectIdentifier 
* OK: The lock object for univentionObjectIdentifier is always created under the default LDAP base
* OK: A LDAP uniqueness check has been activated (on primary only)
* OK: Creating a lock object is optional, since enforcing this would create lots of permission errors on different server roles as well as school replicas
* OK: Update to 5.2-2 is blocked if ucs-school-umc-exam-master with v 12.04 or below is installed.
* OK: Jenkins tests 5.2-2 
* OK: Manual ucs@school Jenkins test
* OK: Manual ox and o365 jenkins test run


Verified
Comment 11 Florian Best univentionstaff 2025-06-11 15:29:06 CEST
(In reply to Julia Bremer from comment #10)
> * ~OK: Currently univentionObjectIdentifier is not added to dnsZone* objects
> -> we need a new bug for thatBug #58384