Bug 47944 - Add settings/data udm module
Add settings/data udm module
Status: CLOSED FIXED
Product: UCS
Classification: Unclassified
Component: UDM (Generic)
UNSTABLE
Other Linux
: P5 normal (vote)
: UCS 4.3-2-errata
Assigned To: Daniel Tröder
Ole Schwiegert
:
Depends on:
Blocks: 40609 47942 48801
  Show dependency treegraph
 
Reported: 2018-10-10 13:42 CEST by Ole Schwiegert
Modified: 2019-03-16 18:48 CET (History)
5 users (show)

See Also:
What kind of report is it?: Feature Request
What type of bug is this?: ---
Who will be affected by this bug?: ---
How will those affected feel about the bug?: ---
User Pain:
Enterprise Customer affected?:
School Customer affected?:
ISV affected?:
Waiting Support:
Flags outvoted (downgraded) after PO Review:
Ticket number:
Bug group (optional):
Max CVSS v3 score:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Ole Schwiegert univentionstaff 2018-10-10 13:42:56 CEST

    
Comment 1 Daniel Tröder univentionstaff 2018-10-10 13:58:48 CEST
The UDM module is intended to hold binary or text blobs, that can be used to store source/binary code in LDAP.

It will be used to distribute univention-join hooks in the domain.

Code is in oschwieg/4.3-2/47944
Comment 2 Sönke Schwardt-Krummrich univentionstaff 2018-10-11 15:42:35 CEST
The module/LDAP object should also contain a "package version" (like e.g. registered LDAP schema). Then we can use ucs_registerLDAPextension to install only newer versions of the code blob. If a package with an older package version tries to change/register the blob, ucs_registerLDAPExtension will not change the LDAP.
Comment 3 Sönke Schwardt-Krummrich univentionstaff 2018-10-11 15:52:22 CEST
Please create cn=code,cn=univention,$LDAP_BASE during update/installation.
Comment 4 Daniel Tröder univentionstaff 2018-10-12 14:06:46 CEST
(In reply to Sönke Schwardt-Krummrich from comment #2)
> The module/LDAP object should also contain a "package version" (like e.g.
> registered LDAP schema). Then we can use ucs_registerLDAPextension to
> install only newer versions of the code blob. If a package with an older
> package version tries to change/register the blob, ucs_registerLDAPExtension
> will not change the LDAP.
2c83d0b2c2 Bug #47944: add "package" and "packageversion" properties

(In reply to Sönke Schwardt-Krummrich from comment #3)
> Please create cn=code,cn=univention,$LDAP_BASE during update/installation.
0249f8adc0 Bug #47944: create container for settings/code objects

Built univention-ldap and univention-directory-manager-modules in:
Branch: ucs_4.3-0
Scope: ucs-school-setup
Comment 5 Daniel Tröder univentionstaff 2018-10-12 16:10:50 CEST
settings/code was renamed to settings/data. The LDAP attributes and UDM properties have been adapted. A property "filename" was added.

56dfbfcfb4 Bug #47944: rename settings/code to settings/data
Comment 6 Daniel Tröder univentionstaff 2018-10-16 11:01:27 CEST
Added a test to verify ucs_registerLDAPExtension operations.

c926f4a5fc Bug #47944: add test for settings/data using ucs_registerLDAPExtension

→ ucs_registerLDAPExtension doesn't support "description" property
→ ucs_registerLDAPExtension doesn't support multiple "--data_meta" arguments


TODO: adapt version of univention-ldap-config dependency in debian/control after merging to 4.3-2.
Comment 7 Daniel Tröder univentionstaff 2018-10-16 12:39:46 CEST
Sönke will add the code for "description" and "--data_meta".
Comment 8 Daniel Tröder univentionstaff 2018-10-26 13:33:24 CEST
Please also make it possible to set the name.
Currently it is automatically set to the filename, and that means that the same file cannot be registered more than once.
Comment 9 Sönke Schwardt-Krummrich univentionstaff 2018-10-26 17:58:36 CEST
(In reply to Daniel Tröder from comment #8)
> Please also make it possible to set the name.
> Currently it is automatically set to the filename, and that means that the
> same file cannot be registered more than once.

I understand where you're going with this, but I haven't yet formed a final opinion about the following:

All other LDAP objects created with ucs_registerLDAPExtension get their name from the filename used as data source. We would now implement a different/unanticipated behavior here. Do we want that?

Until now, a file name for a UDM module, for example, could only be imported once. If you want to bring the module for 2 different UCS version ranges (4.1<=x<=4.2 and 4.3<=x<=4.4), you also have to use 2 different file names. This can have advantages, because e.g. in the file system it is immediately clear which version is still lying around, but also disadvantages, if it must be a certain file name.
Comment 10 Daniel Tröder univentionstaff 2018-10-27 11:35:01 CEST
I have now used it like this:

1) cp $FILENAME /tmp/$FILENAME.$(hostname -f)
2) register_ldap... --data "/tmp/$FILENAME.$(hostname -f)" ...

That's finde for me, as I don't intend to find the correct object by name anyway.
I filter using attributes "packagename", "data_type" and "data_meta".
Comment 11 Daniel Tröder univentionstaff 2018-10-29 15:14:23 CET
Bug #40609 depends on this, as settings/data objects are used there to distribute public RSA keys for UCS mail servers in the domain.
Comment 12 Sönke Schwardt-Krummrich univentionstaff 2018-10-30 14:21:51 CET
(In reply to Daniel Tröder from comment #10)
> I have now used it like this:
> 
> 1) cp $FILENAME /tmp/$FILENAME.$(hostname -f)
> 2) register_ldap... --data "/tmp/$FILENAME.$(hostname -f)" ...

I think this is currently the best solution. The object name is currently derived from the filename. If we separate filename and object name, we will have problems with the update mechanism.

> That's finde for me, as I don't intend to find the correct object by name
> anyway.
> I filter using attributes "packagename", "data_type" and "data_meta".

I don't think you should filter for the package name. Because what you want to find should be uniquely identified by the DataType and possibly by the meta data. If the package name is included in the search, it will become much more difficult later to register data of the same type in other packages or even manually.

packagename and packageversion are currently only required to determine if an update of the LDAP object is required.

If the search is done manually, the objectclass should be part of the filter.
At the moment it is not quite clear whether we can automatically extend the LDAP indexes in an erratum. If this is not the case and we can only extend the index with UCS 4.3-3, the LDAP filter should be selected so that as few objects as possible have to be found and opened to determine the result list.
Comment 13 Daniel Tröder univentionstaff 2018-10-30 15:08:37 CET
(In reply to Sönke Schwardt-Krummrich from comment #12)
> > I filter using attributes "packagename", "data_type" and "data_meta".
> 
> I don't think you should filter for the package name. Because what you want
> to find should be uniquely identified by the DataType and possibly by the
> meta data. If the package name is included in the search, it will become
> much more difficult later to register data of the same type in other
> packages or even manually.
Actually that's the reason I included "packagename" in the filter.
I understand "data_type" as a category and in my case it is "RSA_public_key".
I expect other packages to use the same category, but I don't want to use their keys. So I filter with "packagename=univention-mail-server" to get only "my" keys:

ldap_filter = (
	'(&'
	'(objectClass=univentionData)'
	'(univentionDataType=RSA_public_key)'
	'(univentionOwnedByPackage=univention-mail-server)'
	')'
)
Comment 14 Sönke Schwardt-Krummrich univentionstaff 2018-11-01 10:43:32 CET
(In reply to Daniel Tröder from comment #13)
> (In reply to Sönke Schwardt-Krummrich from comment #12)
> > > I filter using attributes "packagename", "data_type" and "data_meta".
> > 
> > I don't think you should filter for the package name. Because what you want
> > to find should be uniquely identified by the DataType and possibly by the
> > meta data. If the package name is included in the search, it will become
> > much more difficult later to register data of the same type in other
> > packages or even manually.
> Actually that's the reason I included "packagename" in the filter.
> I understand "data_type" as a category and in my case it is "RSA_public_key".
> I expect other packages to use the same category, but I don't want to use
> their keys. So I filter with "packagename=univention-mail-server" to get
> only "my" keys:
> 
> ldap_filter = (
> 	'(&'
> 	'(objectClass=univentionData)'
> 	'(univentionDataType=RSA_public_key)'
> 	'(univentionOwnedByPackage=univention-mail-server)'
> 	')'
> )

I think by using "(univentionDataType=RSA_public_key)" the search/the object is insufficiently specified. I would suggest something like 
univentionDataType=mail/signing/RSA_public_key

This way, the customer and other packages (e.g. 3rd party packages) are able to register additional keys (for a good reason). And each package (or the customer) is able to update the data objects in a sane way.
Comment 15 Sönke Schwardt-Krummrich univentionstaff 2018-11-02 14:58:08 CET
(In reply to Daniel Tröder from comment #7)
> Sönke will add the code for "description" and "--data_meta".

To keep the API identical to all other objects that may be registered via ucs_registerLDAPExtension, "description" may not be set via ucs_registerLDAPExtension.

But I implemented --data_meta which may be specified multiple times.

The ucs-test has been adapted accordingly.

The branches school/samba_vereinfachung and oschwieg/4.3-2/47944 are now in sync regarding ldap_extensions.py.
Comment 16 Sönke Schwardt-Krummrich univentionstaff 2018-11-02 15:07:15 CET
We suggest to use data_type like this: 
$context/$thing

for example "mail/sendercheck/rsa_key" or 
"join/pre-joinscript" 
"join/post-joinscript"
"join/pre-join"


To be honest, the data_type is not checked by the UDM module, but using a naming scheme similar to UCR variables helps to subdivide and categorize large amounts of settings/data objects.
Comment 17 Daniel Tröder univentionstaff 2018-11-02 15:38:10 CET
PS: both univentionDataType and univentionDataMeta have:
	EQUALITY caseIgnoreMatch
	SUBSTR caseIgnoreSubstringsMatch
Comment 18 Ole Schwiegert univentionstaff 2018-11-19 14:34:44 CET
Looks good so far to me. Please build the packages so I can do the final QA.
Comment 19 Daniel Tröder univentionstaff 2018-11-19 17:41:39 CET
[4.3-2] 13ca7e0c19 Bug #47944: Merge branch 'oschwieg/4.3-2/47944' into 4.3-2
[4.3-2] 4e0da28cd1 Bug #47944: update dependencies
[4.3-2] 523820228d Bug #47944: update dependencies
[4.3-2] 58351e4ad4 Bug #47944: advisories

univention-lib 7.0.0-14A~4.3.0.201811191459
univention-ldap 14.0.2-29A~4.3.0.201811191500
univention-directory-manager-modules 13.0.25-18A~4.3.0.201811191502

ucs-test can currently not be build, because of Bug #48182.
Comment 20 Arvid Requate univentionstaff 2018-11-19 19:46:28 CET
According to Bug 48182 this commit causes a regression:

[4.3-2] 4e0da28cd1 Bug #47944: update dependencies
Comment 21 Daniel Tröder univentionstaff 2018-11-20 07:36:38 CET
(In reply to Arvid Requate from comment #20)
> According to Bug 48182 this commit causes a regression:
> 
> [4.3-2] 4e0da28cd1 Bug #47944: update dependencies
The problem has been identified and split into a separate bug (Bug #48182) which blocks this one. The fix will have to be released together with this bug.
Comment 22 Ole Schwiegert univentionstaff 2018-11-20 10:39:30 CET
Advisory & Changelog: OK
settings/data object with expected values available: OK
register ldap extension accepts multiple meta fields: OK
tests run: OK
Comment 23 Stefan Gohmann univentionstaff 2018-11-20 19:11:10 CET
(In reply to Daniel Tröder from comment #21)
> (In reply to Arvid Requate from comment #20)
> > According to Bug 48182 this commit causes a regression:
> > 
> > [4.3-2] 4e0da28cd1 Bug #47944: update dependencies
> The problem has been identified and split into a separate bug (Bug #48182)
> which blocks this one. The fix will have to be released together with this
> bug.

If I see this correctly, the YAML files do not prevent this. Blocking bugs in Bugzilla is not enough.

That's why I open this bug again so that an erratum is not accidentally published.
Comment 24 Stefan Gohmann univentionstaff 2018-11-20 19:41:33 CET
If I read Bug #48182#c5 correctly, you have removed the dependency and this erratum can be released. So, I'm setting this bug back to verified and set the flags of Bug #48182.
Comment 25 Stefan Gohmann univentionstaff 2018-11-20 19:41:46 CET
(In reply to Stefan Gohmann from comment #24)
> If I read Bug #48182#c5 correctly, you have removed the dependency and this
> erratum can be released. So, I'm setting this bug back to verified and set
> the flags of Bug #48182.