Univention Bugzilla – Bug 43129
Enhance UDM Syntax for Generalized Time Syntax
Last modified: 2022-07-14 11:42:39 CEST
The "Generalized Time syntax" (OID value: 1.3.6.1.4.1.1466.115.121.1.24) implements a useful syntax to represent timestamps including the time-zone. By default it is already used in UCS (hdb.schema, dhcp.schema, krb5-kdc.schema) There are obviously use-cases where this syntax was chosen to represent timestamps in OpenLDAP. Example (found on https://www.sudo.ws/man/1.8.17/sudoers.ldap.man.html): attributetype ( 1.3.6.1.4.1.15953.9.1.8 NAME 'sudoNotBefore' DESC 'Start of time interval for which the entry is valid' EQUALITY generalizedTimeMatch ORDERING generalizedTimeOrderingMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.24 ) attributetype ( 1.3.6.1.4.1.15953.9.1.9 NAME 'sudoNotAfter' DESC 'End of time interval for which the entry is valid' EQUALITY generalizedTimeMatch ORDERING generalizedTimeOrderingMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.24 ) While it is not a big deal to store such informations in UCS once a schema extension is applied it is a challenge to find a matching syntax rule for UDM (see http://docs.software-univention.de/developer-reference-4.1.html#udm:syntax) The developer reference says that it is in general possible to write custom syntax extensions. It might be a considerable request to implement this syntax directly in the product.
Findings based on work done during the preparation of a PoC for a prospect who considers to migrate from OpenLDAP/Samba to UCS. see also #34070, we dont know if we run against this when implementing a custom syntax
Bug #34070 will not affect this. Did you implement a base for this already?
(In reply to Florian Best from comment #2) > Bug #34070 will not affect this. Did you implement a base for this already? not yet, as mentioned in #c1 this is a pre-sales enhancement request. I will discuss with the responsible sales person if/how/when this is a feature which is worth to implement in the product.
See Feedback Ticket#2016120821000528
Created attachment 9243 [details] Allow toLDAP() and fromLDAP() methods in syntax classes
Sorry, didn't add a comment to the patch. I want to propose a patch to admin/handlers/__init__.py that allows a syntax class to self-implement an LDAP mapping. The rationale behind this: * a syntax class inheriting from 'date' or 'date2' can benefit from inheritance (for instance, the usage of the DatePicker) * an implementor of the syntax class does not have to ask 'please add an LDAP mapping into the user class' everytime he comes across such situation * the data flow can be used for other types of LDAP maping too: for instance an UDM 'boolean' (showing up as a checkbox) that has to be mapped (and back!) to strings like 'ENABLED' and 'DISABLED' I have tried to implement it such that it doesn't affect any syntax classes that don't implement toLDAP() and fromLDAP() methods. To implement a mapping this way in a syntax class would look like: def fromLDAP(value): """ callback for a single value to convert LDAP -> UDM representation """ return do_some_conversion(value) def toLDAP(value): """ callback for a single value to convert UDM -> LDAP representation """ return do_some_other_conversion(value)
Created attachment 9244 [details] patch I enhanced the patch a little bit regarding style/error handling/etc. But I am unsure about whether we should apply it. I think we should definitely improve the syntax handling via Bug #38762 but I would not touch _post_map and _post_unmap for this. What we should do is to separate syntax rules from the used javascript widgets. Maybe we should also allow extended attributes to define a mapping / unmap function.
I second the OPs request. I just had to implement a UDM hook to handle conversion for a time field stored with exactly the syntax in comment#1 to/from the UDM syntax "iso8601Date", so I'd get a datepicker in UMC. It was not difficult to do it in a UDM hook, but far from elegant. Now if it25 would want to use that code for their extended attribute, they'd have to make a complete copy of the udm hook and change the attribute names in it: code duplication (and still not the right place for a ldap/udm mapping).
> Now if it25 would want to use that code for their extended attribute, > they'd have to make a complete copy of the udm hook and change the > attribute names in it: Exactly. If the proposed API is implemented, I can stop scratching my head how to implement things without touching the stock UDM classes. I know if I create my own UDM class then I can do arbitrary mapping there, but I can't do that if I add an Extended Attribute to the 'users/user' class. @florian: Thanks for pythonifying my patch, I know my style is miles away from pythonic. I don't insist on these hooks being done in _post_map() and _post_unmap(). Perhaps you know a better place. I've found only a few occurences where some class overrides these methods, and I guess it would be an easy task to call super() in overridden implementations.
Created attachment 9245 [details] patch for configurable mapping/unmapping I attached a patch which makes the map and unmap method configurable for extended attributes. A directory univention/admin/mapping.d/ is added which can be used for further (un)mapping functions. With this everything you want to do should be possible and no syntax classes or UDM core code gets violated. For example: cat >> /usr/lib/pymodules/python2.7/univention/admin/mapping.d/mydate.py def mapMyDate(value): return str(value / 60) def unmapMyDate(values): return int(values[0]) * 60 udm settings/extended_attribute modify --dn "$DN" --set mapMethod=mapMyDate --set unmapMethod=unmapMyDate. What do you think?
(In reply to Florian Best from comment #10) > udm settings/extended_attribute modify --dn "$DN" --set mapMethod=mapMyDate > --set unmapMethod=unmapMyDate. > > What do you think? This only works flawlessly if mapping.d/* files are also registered in LDAP.
Created attachment 9246 [details] patch for configurable mapping/unmapping (In reply to Sönke Schwardt-Krummrich from comment #11) > (In reply to Florian Best from comment #10) > > udm settings/extended_attribute modify --dn "$DN" --set mapMethod=mapMyDate > > --set unmapMethod=unmapMyDate. > > > > What do you think? > > This only works flawlessly if mapping.d/* files are also registered in LDAP. Okay, I added the LDAP registration part to the patch.
(In reply to Florian Best from comment #12) > Created attachment 9246 [details] > patch for configurable mapping/unmapping > > (In reply to Sönke Schwardt-Krummrich from comment #11) > > (In reply to Florian Best from comment #10) > > > udm settings/extended_attribute modify --dn "$DN" --set mapMethod=mapMyDate > > > --set unmapMethod=unmapMyDate. > > > > > > What do you think? > > > > This only works flawlessly if mapping.d/* files are also registered in LDAP. > Okay, I added the LDAP registration part to the patch. An alternative would be to use the existing hook files so that one would need to add functions to the hook files and specify a hook for the extended attribute. This would be less LDAP schema changes. But imho more intransparent and complexer.
Patched a current system inplace, generally it works: * UDM recognizes settings/udm_mapping object * ucs_registerLDAPExtension recognizes and registers --udm_mapping object * UMC allows editing all properties of mapping object * Extended attribute recognizes --mapMethod and --unmapMethod arguments * the registered map/unmap methods will be called. Generally I'm confused about the calling conventions of these methods. I got a mixup of different arguments (None, empty strings, Python Bool, strings in LDAP notation, strings in UDM notation, arrays, lists), and I can't recognize a rule when the mapping (LDAP -> UDM) should return an array, when to return a string or whatever. (for instance, if the function returns an UDM-typical date YYYY-MM-DD string, the date picker does NOT pick it up if i wrap it into an array, other UI elements require the returned string to be wrapped into an array, or a multivalue field would split a single string into all of its chars...) It would be nice to get some general rules about how to use the map/unmap methods.
Meanwhile, the mentioned customer project is in progress, so I had to rely on the possibility to add mapping methods to extended attributes. Not only for the initial purpose of mapping ISO8601 dates back and forth, but also for other mappings, for instance mapping of boolean states to arbitrary string values in LDAP. Currently both my development system and the customer's staging system have been patched manually, but it cannot remain that way as it prevents proper updates. Please consider implementing Florian's patch as proposed in #c12, or propose a different path to a solution. Thanks.
The "Generalized Time syntax" (OID value: 1.3.6.1.4.1.1466.115.121.1.24) is used in the UCS@school import code: * LDAP attribute "ucsschoolPurgeTimestamp": https://github.com/univention/ucs-school/blob/4.3/ucs-school-import/schema/ucs-school-import.schema#L46 * join script: - registration UDM hook: https://git.knut.univention.de/univention/ucsschool/blob/4.3/ucs-school-import/35ucs-school-import.inst#L118 - extended attribute: https://git.knut.univention.de/univention/ucsschool/blob/4.3/ucs-school-import/35ucs-school-import.inst#L366 * UDM hook: https://git.knut.univention.de/univention/ucsschool/blob/4.3/ucs-school-import/udm_hook/ucsschool_purge_timestamp.py The trick here is, that the extended attribute was registered with a known syntax class "iso8601Date", for which a mapping and a UMC widget already exist and are automatically used. What was left to do, was to map and unmap the UDM values (YYYY-MM-DD) to and from the expected LDAP format (Generalized Time syntax): * ldap2udm: '20090101000000Z' to '2009-01-01' * udm2ldap: '2009-01-01' to '20090101000000Z' That is done in the UDM hook in hook_open(), hook_ldap_addlist() and hook_ldap_modlist().
Created attachment 9722 [details] AttributeHook base class that also works While hooks arguably may not be the perfect place to put map/unmap functions, this works now and without any modifications to UCS (including UCS 4.2): Add a file /usr/lib/pymodules/python2.7/univention/admin/hooks.d/mymapping.py Include a new hook base class as uploaded and then derive from it: import univention.admin.uexceptions class MyMappingHook(AttributeHook): # non-sense boolean attribute mapping attribute_name = 'myAttribute' def map_attribute_value_to_ldap(self, value): if value == 'something invalid': # this is not great, but works reasonably well raise univention.admin.uexceptions.valueError('%s: Value may not be %r' % (self.attribute_name, value)) return 'FALSE' def map_attribute_value_to_udm(self, value): return 'TRUE' If this fits yours needs, we could add this base class in a new errata update.
This workaround works fairly well. I will close this bug. An errata update is difficult, because as soon as someone uses this new hook base class (i.e., derives from it), this errata update has to be installed on each and every UCS system. Otherwise loading the hook will fail. As of now, I would not release anything here. Feel free to REOPEN, if you think I am wrong. Or if the solution provided here does not work at all.
After some re-consideration, we chose to add AttributeHook in univention-directory-manager-modules 13.0.25-21A~4.3.0.201811271536 If you want to use it, make sure all your systems have installed this version or higher. This may be hindering you from adapting it today. But at some point in time, this new class will be installed on every maintained version of UCS.
- OK: code change - OK: functional test (via ucs-test script 40_extended_attribute_attributehook_value_mapping) - TODO: jenkins test Please note that this solution is not optimal, but currently the best solution that is backwards and future compatible. I attached an example hook, that can be used as base for a custom hook.
Created attachment 9766 [details] example-hook.py
I have added some debug output. I also experimented with obj.oldattr and obj.oldinfo to lower the requirements for the "map_attribute_value_to_udm" implementation but with no success. The Hook itself works fine, but the two implementations of the methods "map_attribute_value_to_udm" and "map_attribute_value_to_ldap" should really be double checked to verify that it works properly. We found that the hook was called with None during the object initialization which interfered with the actual setting of the attribute. This happened with udm-cli. Using the API did not show this error prone behavior.
Jenkins test was ok (that covered Dirk's changes)
<http://errata.software-univention.de/ucs/4.3/355.html>
(In reply to Florian Best from comment #12) > Created attachment 9246 [details] > patch for configurable mapping/unmapping > > (In reply to Sönke Schwardt-Krummrich from comment #11) > > (In reply to Florian Best from comment #10) > > > udm settings/extended_attribute modify --dn "$DN" --set mapMethod=mapMyDate > > > --set unmapMethod=unmapMyDate. > > > > > > What do you think? > > > > This only works flawlessly if mapping.d/* files are also registered in LDAP. > Okay, I added the LDAP registration part to the patch. We could need this patch for the userCertificate cool solution problem in UCS 5.0 in https://git.knut.univention.de/univention/prof-services/cool-solutions/-/issues/2.