Bug 53832 - Expose sizelimit parameter to the simple UDM search method
Expose sizelimit parameter to the simple UDM search method
Status: CLOSED FIXED
Product: UCS
Classification: Unclassified
Component: UDM (Generic)
UCS 4.4
Other Mac OS X 10.1
: P5 normal (vote)
: UCS 4.4-8-errata
Assigned To: Ole Schwiegert
Johannes Keiser
:
Depends on:
Blocks: 53833
  Show dependency treegraph
 
Reported: 2021-09-24 12:50 CEST by Ole Schwiegert
Modified: 2021-09-30 13:07 CEST (History)
1 user (show)

See Also:
What kind of report is it?: Feature Request
What type of bug is this?: ---
Who will be affected by this bug?: ---
How will those affected feel about the bug?: ---
User Pain:
Enterprise Customer affected?:
School Customer affected?:
ISV affected?:
Waiting Support:
Flags outvoted (downgraded) after PO Review:
Ticket number:
Bug group (optional):
Max CVSS v3 score:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Ole Schwiegert univentionstaff 2021-09-24 12:50:02 CEST
We need to be able to utilize the sizelimit parameter for LDAP queries in the "simple UDM API". For that the GenericModule.search() method needs to expose this parameter.
Comment 1 Ole Schwiegert univentionstaff 2021-09-24 14:21:09 CEST
Implemented on oschwieg/4.4-8/53832

Please QA and let me know when I can merge and build
Comment 2 Johannes Keiser univentionstaff 2021-09-24 15:33:50 CEST
OK: specifying sizelimit works for both searchDn() and lookup(); exception is thrown

~OK: exception message.
maybe
'[...] than the           sizelimit of {} allowed.'
'[...] than the specified sizelimit of {} allowed.'

Reads a little bit better that the sizelimit is an argument.

Would it be too confusing to use the same exception name as UDM

SearchLimitReached?


Otherwise can be merged
Comment 3 Ole Schwiegert univentionstaff 2021-09-27 07:49:30 CEST
Package: univention-directory-manager-modules
Version: 14.0.20-20A~4.4.0.202109270743
Branch: ucs_4.4-0
Scope: errata4.4-8

Package: ucs-test
Version: 9.0.7-69A~4.4.0.202109270747
Branch: ucs_4.4-0
Scope: errata4.4-8


The sizelimit option is now exposed in GenericModule.search and raises a SearchLimitReached Exception if the sizelimit is exceeded.

Both QA proposals were incorporated.
Comment 4 Johannes Keiser univentionstaff 2021-09-29 09:15:31 CEST
OK: tests look ok
OK: yaml
-> verified
Comment 5 Florian Best univentionstaff 2021-09-29 13:39:48 CEST
REOPEN:
You are introducing new python classes which are not available in UCS 5.0. During the upgrade to UCS 5.0 they are then missing but code is still running with your new version and dynamically re-imports other modules.
This causes errors like:

attachment 10836 [details] - connector-s4.log

208 rejects in /var/log/univention/connector-s4.log:

Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/univention/s4connector/__init__.py", line 1562, in sync_to_ucs
    result = self.modify_in_ucs(property_type, object, module, position)
  File "/usr/lib/python2.7/dist-packages/univention/s4connector/__init__.py", line 1303, in modify_in_ucs
    res = ucs_object.modify(serverctrls=serverctrls, response=response)
  File "/usr/lib/python2.7/dist-packages/univention/admin/handlers/__init__.py", line 650, in modify
    dn = self._modify(modify_childs, ignore_license=ignore_license, response=response)
  File "/usr/lib/python2.7/dist-packages/univention/admin/handlers/__init__.py", line 1325, in _modify
    def _modify(self, modify_childs=1, ignore_license=0, response=None, serverctrls=None):
  File "/usr/lib/python2.7/dist-packages/univention/admin/handlers/__init__.py", line 1056, in call_udm_property_hook
    """
  File "/usr/lib/python2.7/dist-packages/univention/admin/hooks.d/ucsschool_role_computers.py", line 120, in hook_ldap_modlist
    return self.add_ocs_and_ucschool_roles(obj, ml, "mod")
  File "/usr/lib/python2.7/dist-packages/univention/admin/hooks.d/ucsschool_role_computers.py", line 139, in add_ocs_and_ucschool_roles
    all_schools = dict((school.name, school.dn) for school in School.get_all(obj.lo))
  File "/usr/lib/python2.7/dist-packages/ucsschool/lib/models/school.py", line 733, in get_all
    lo, school=None, filter_str=filter_str, easy_filter=easy_filter
  File "/usr/lib/python2.7/dist-packages/ucsschool/lib/models/base.py", line 992, in get_all
    ret.append(cls.from_udm_obj(udm_obj, school, lo))
  File "/usr/lib/python2.7/dist-packages/ucsschool/lib/models/school.py", line 726, in from_udm_obj
    obj.educational_servers = obj.get_educational_server_names(lo)
  File "/usr/lib/python2.7/dist-packages/ucsschool/lib/models/school.py", line 389, in get_educational_server_names
    mod = UDM(lo).version(0).get("groups/group")
  File "/usr/lib/python2.7/dist-packages/univention/udm/udm.py", line 208, in get
    for module in plugins:
  File "/usr/lib/python2.7/dist-packages/univention/udm/plugins.py", line 76, in __iter__
    self.load()
  File "/usr/lib/python2.7/dist-packages/univention/udm/plugins.py", line 90, in load
    importlib.import_module('{}.{}'.format(self.python_path, pymodule_name))
  File "/usr/lib/python2.7/importlib/__init__.py", line 37, in import_module
    __import__(name)
  File "/usr/lib/python2.7/dist-packages/univention/udm/modules/policies_umc.py", line 35, in <module>
    from .generic import GenericModule, GenericObject, GenericObjectProperties
  File "/usr/lib/python2.7/dist-packages/univention/udm/modules/generic.py", line 50, in <module>
    from ..exceptions import (
ImportError: cannot import name SearchLimitReached

Traceback (most recent call last):
  File "/usr/bin/univention-portal-server", line 185, in refresh
    fd = self._refresh()
  File "/usr/bin/univention-portal-server", line 206, in _refresh
    localhost = self.udm.obj_by_dn(ldap_hostdn)
  File "/usr/lib/python2.7/dist-packages/univention/udm/udm.py", line 245, in obj_by_dn
    udm_module = self.get(uot)
  File "/usr/lib/python2.7/dist-packages/univention/udm/udm.py", line 208, in get
    for module in plugins:
  File "/usr/lib/python2.7/dist-packages/univention/udm/plugins.py", line 76, in __iter__
    self.load()
  File "/usr/lib/python2.7/dist-packages/univention/udm/plugins.py", line 90, in load
    importlib.import_module('{}.{}'.format(self.python_path, pymodule_name))
  File "/usr/lib/python2.7/importlib/__init__.py", line 37, in import_module
    __import__(name)
  File "/usr/lib/python2.7/dist-packages/univention/udm/modules/policies_umc.py", line 35, in <module>
    from .generic import GenericModule, GenericObject, GenericObjectProperties
  File "/usr/lib/python2.7/dist-packages/univention/udm/modules/generic.py", line 50, in <module>
    from ..exceptions import (
ImportError: cannot import name SearchLimitReached

You have to catch those import errors.
Comment 6 Ole Schwiegert univentionstaff 2021-09-29 13:49:04 CEST
What do you mean by catching those Exceptions? How did you solve this for other code? -- I need those imported classes after all, so what am I to do, if the classes are not available due to the import?
Comment 7 Ole Schwiegert univentionstaff 2021-09-29 13:49:29 CEST
* due to the update
Comment 8 Ole Schwiegert univentionstaff 2021-09-29 14:22:07 CEST
I wrapped the import in a try-except as discussed in chat. I also added a comment explaining the why.

Package: univention-directory-manager-modules
Version: 14.0.20-23A~4.4.0.202109291420
Branch: ucs_4.4-0
Scope: errata4.4-8
Comment 9 Johannes Keiser univentionstaff 2021-09-30 12:04:43 CEST
OK: import error is catched
OK: ImportError: cannot import name SearchLimitReached no longer occurs in jenkins tests

https://jenkins.knut.univention.de:8181/job/UCS-5.0/job/UCS-5.0-0/view/Default/job/AutotestUpgrade/263/SambaVersion=s4,Systemrolle=backup/testReport/99_end/01_var_log_tracebacks/test_fetch_logfiles_on_dc_master/
https://jenkins.knut.univention.de:8181/job/UCS-5.0/job/UCS-5.0-0/view/Default/job/AutotestUpgrade/SambaVersion=s4,Systemrolle=backup/264/testReport/99_end/01_var_log_tracebacks/



https://jenkins.knut.univention.de:8181/job/UCS-5.0/job/UCS-5.0-0/view/Default/job/AutotestUpgrade/263/SambaVersion=no-samba,Systemrolle=backup/testReport/99_end/01_var_log_tracebacks/test_fetch_logfiles_on_dc_master/
https://jenkins.knut.univention.de:8181/job/UCS-5.0/job/UCS-5.0-0/view/Default/job/AutotestUpgrade/SambaVersion=no-samba,Systemrolle=backup/264/testReport/99_end/01_var_log_tracebacks/

https://jenkins.knut.univention.de:8181/job/UCS-5.0/job/UCS-5.0-0/view/Default/job/AutotestUpgrade/263/SambaVersion=s4,Systemrolle=member/testReport/99_end/01_var_log_tracebacks/test_fetch_logfiles_on_dc_master/
https://jenkins.knut.univention.de:8181/job/UCS-5.0/job/UCS-5.0-0/view/Default/job/AutotestUpgrade/SambaVersion=s4,Systemrolle=member/264/testReport/99_end/01_var_log_tracebacks/test_fetch_logfiles_on_dc_master/

https://jenkins.knut.univention.de:8181/job/UCS-5.0/job/UCS-5.0-0/view/Default/job/AutotestUpgrade/263/SambaVersion=s4,Systemrolle=slave/testReport/99_end/01_var_log_tracebacks/test_fetch_logfiles_on_dc_master/
https://jenkins.knut.univention.de:8181/job/UCS-5.0/job/UCS-5.0-0/view/Default/job/AutotestUpgrade/SambaVersion=s4,Systemrolle=slave/264/testReport/99_end/01_var_log_tracebacks/test_fetch_logfiles_on_dc_master/

https://jenkins.knut.univention.de:8181/job/UCS-5.0/job/UCS-5.0-0/view/Default/job/AutotestUpgrade/263/SambaVersion=no-samba,Systemrolle=member/testReport/99_end/01_var_log_tracebacks/test_fetch_logfiles_on_dc_master/
https://jenkins.knut.univention.de:8181/job/UCS-5.0/job/UCS-5.0-0/view/Default/job/AutotestUpgrade/SambaVersion=no-samba,Systemrolle=member/264/testReport/99_end/01_var_log_tracebacks/test_fetch_logfiles_on_dc_master/

OK: yaml
-> verified