Bug 53833 - Expose sizelimit parameter to the simple UDM search method
Expose sizelimit parameter to the simple UDM search method
Status: CLOSED FIXED
Product: UCS
Classification: Unclassified
Component: UDM (Generic)
UCS 5.0
Other Mac OS X 10.1
: P5 normal (vote)
: UCS 5.0-0-errata
Assigned To: Ole Schwiegert
Johannes Keiser
:
Depends on: 53832
Blocks:
  Show dependency treegraph
 
Reported: 2021-09-24 12:50 CEST by Ole Schwiegert
Modified: 2021-09-30 13:40 CEST (History)
1 user (show)

See Also:
What kind of report is it?: Feature Request
What type of bug is this?: ---
Who will be affected by this bug?: ---
How will those affected feel about the bug?: ---
User Pain:
Enterprise Customer affected?:
School Customer affected?:
ISV affected?:
Waiting Support:
Flags outvoted (downgraded) after PO Review:
Ticket number:
Bug group (optional):
Max CVSS v3 score:


Attachments
portal.log (43.38 KB, text/x-log)
2021-09-29 09:53 CEST, Florian Best
Details
connector-s4.log (2.06 MB, text/x-log)
2021-09-29 10:01 CEST, Florian Best
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Ole Schwiegert univentionstaff 2021-09-24 12:50:45 CEST
+++ This bug was initially created as a clone of Bug #53832 +++

We need to be able to utilize the sizelimit parameter for LDAP queries in the "simple UDM API". For that the GenericModule.search() method needs to expose this parameter.
Comment 1 Ole Schwiegert univentionstaff 2021-09-24 14:26:32 CEST
Implemented on oschwieg/5.0-0/53833

Please QA and let me know when I can merge and build
Comment 2 Johannes Keiser univentionstaff 2021-09-27 11:26:12 CEST
OK: specifying sizelimit works for both searchDn() and lookup(); exception is thrown
OK: python2, python3
OK: exception message
OK: feedback from Bug #53832 is incorperated
OK: test case

can be merged
Comment 3 Ole Schwiegert univentionstaff 2021-09-27 11:49:49 CEST
Package: univention-directory-manager-modules
Version: 15.0.11-22A~5.0.0.202109271147
Branch: ucs_5.0-0
Scope: errata5.0-0

Package: ucs-test
Version: 10.0.6-52A~5.0.0.202109271144
Branch: ucs_5.0-0
Scope: errata5.0-0

The sizelimit option is now exposed in GenericModule.search and raises a SearchLimitReached Exception if the sizelimit is exceeded.
Comment 4 Johannes Keiser univentionstaff 2021-09-29 09:18:56 CEST
OK: tests look ok
OK: yaml
-> verified
Comment 5 Florian Best univentionstaff 2021-09-29 09:28:58 CEST
Traceback (most recent call last):
  File "/usr/bin/univention-portal-server", line 185, in refresh
    fd = self._refresh()
  File "/usr/bin/univention-portal-server", line 206, in _refresh
    localhost = self.udm.obj_by_dn(ldap_hostdn)
  File "/usr/lib/python2.7/dist-packages/univention/udm/udm.py", line 245, in obj_by_dn
    udm_module = self.get(uot)
  File "/usr/lib/python2.7/dist-packages/univention/udm/udm.py", line 208, in get
    for module in plugins:
  File "/usr/lib/python2.7/dist-packages/univention/udm/plugins.py", line 76, in __iter__
    self.load()
  File "/usr/lib/python2.7/dist-packages/univention/udm/plugins.py", line 90, in load
    importlib.import_module('{}.{}'.format(self.python_path, pymodule_name))
  File "/usr/lib/python2.7/importlib/__init__.py", line 37, in import_module
    __import__(name)
  File "/usr/lib/python2.7/dist-packages/univention/udm/modules/policies_umc.py", line 35, in <module>
    from .generic import GenericModule, GenericObject, GenericObjectProperties
  File "/usr/lib/python2.7/dist-packages/univention/udm/modules/generic.py", line 50, in <module>
    from ..exceptions import (
ImportError: cannot import name SearchLimitReached
Comment 6 Johannes Keiser univentionstaff 2021-09-29 09:40:22 CEST
(In reply to Florian Best from comment #5)
> Traceback (most recent call last):
>   File "/usr/bin/univention-portal-server", line 185, in refresh
>     fd = self._refresh()
>   File "/usr/bin/univention-portal-server", line 206, in _refresh
>     localhost = self.udm.obj_by_dn(ldap_hostdn)
>   File "/usr/lib/python2.7/dist-packages/univention/udm/udm.py", line 245,
> in obj_by_dn
>     udm_module = self.get(uot)
>   File "/usr/lib/python2.7/dist-packages/univention/udm/udm.py", line 208,
> in get
>     for module in plugins:
>   File "/usr/lib/python2.7/dist-packages/univention/udm/plugins.py", line
> 76, in __iter__
>     self.load()
>   File "/usr/lib/python2.7/dist-packages/univention/udm/plugins.py", line
> 90, in load
>     importlib.import_module('{}.{}'.format(self.python_path, pymodule_name))
>   File "/usr/lib/python2.7/importlib/__init__.py", line 37, in import_module
>     __import__(name)
>   File
> "/usr/lib/python2.7/dist-packages/univention/udm/modules/policies_umc.py",
> line 35, in <module>
>     from .generic import GenericModule, GenericObject,
> GenericObjectProperties
>   File "/usr/lib/python2.7/dist-packages/univention/udm/modules/generic.py",
> line 50, in <module>
>     from ..exceptions import (
> ImportError: cannot import name SearchLimitReached

Where is this happening. The SearchLimitReached class exists
Comment 7 Florian Best univentionstaff 2021-09-29 09:53:21 CEST
Created attachment 10835 [details]
portal.log

Somewhere during package upgrade?!
Comment 8 Florian Best univentionstaff 2021-09-29 10:01:37 CEST
Created attachment 10836 [details]
connector-s4.log

208 rejects in /var/log/univention/connector-s4.log:

Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/univention/s4connector/__init__.py", line 1562, in sync_to_ucs
    result = self.modify_in_ucs(property_type, object, module, position)
  File "/usr/lib/python2.7/dist-packages/univention/s4connector/__init__.py", line 1303, in modify_in_ucs
    res = ucs_object.modify(serverctrls=serverctrls, response=response)
  File "/usr/lib/python2.7/dist-packages/univention/admin/handlers/__init__.py", line 650, in modify
    dn = self._modify(modify_childs, ignore_license=ignore_license, response=response)
  File "/usr/lib/python2.7/dist-packages/univention/admin/handlers/__init__.py", line 1325, in _modify
    def _modify(self, modify_childs=1, ignore_license=0, response=None, serverctrls=None):
  File "/usr/lib/python2.7/dist-packages/univention/admin/handlers/__init__.py", line 1056, in call_udm_property_hook
    """
  File "/usr/lib/python2.7/dist-packages/univention/admin/hooks.d/ucsschool_role_computers.py", line 120, in hook_ldap_modlist
    return self.add_ocs_and_ucschool_roles(obj, ml, "mod")
  File "/usr/lib/python2.7/dist-packages/univention/admin/hooks.d/ucsschool_role_computers.py", line 139, in add_ocs_and_ucschool_roles
    all_schools = dict((school.name, school.dn) for school in School.get_all(obj.lo))
  File "/usr/lib/python2.7/dist-packages/ucsschool/lib/models/school.py", line 733, in get_all
    lo, school=None, filter_str=filter_str, easy_filter=easy_filter
  File "/usr/lib/python2.7/dist-packages/ucsschool/lib/models/base.py", line 992, in get_all
    ret.append(cls.from_udm_obj(udm_obj, school, lo))
  File "/usr/lib/python2.7/dist-packages/ucsschool/lib/models/school.py", line 726, in from_udm_obj
    obj.educational_servers = obj.get_educational_server_names(lo)
  File "/usr/lib/python2.7/dist-packages/ucsschool/lib/models/school.py", line 389, in get_educational_server_names
    mod = UDM(lo).version(0).get("groups/group")
  File "/usr/lib/python2.7/dist-packages/univention/udm/udm.py", line 208, in get
    for module in plugins:
  File "/usr/lib/python2.7/dist-packages/univention/udm/plugins.py", line 76, in __iter__
    self.load()
  File "/usr/lib/python2.7/dist-packages/univention/udm/plugins.py", line 90, in load
    importlib.import_module('{}.{}'.format(self.python_path, pymodule_name))
  File "/usr/lib/python2.7/importlib/__init__.py", line 37, in import_module
    __import__(name)
  File "/usr/lib/python2.7/dist-packages/univention/udm/modules/policies_umc.py", line 35, in <module>
    from .generic import GenericModule, GenericObject, GenericObjectProperties
  File "/usr/lib/python2.7/dist-packages/univention/udm/modules/generic.py", line 50, in <module>
    from ..exceptions import (
ImportError: cannot import name SearchLimitReached
Comment 9 Johannes Keiser univentionstaff 2021-09-29 10:08:11 CEST
Tracebacks are shown in all loglevels i assume?
Comment 10 Florian Best univentionstaff 2021-09-29 10:09:55 CEST
Loglevel = ERROR, so mostly yes.
Comment 11 Johannes Keiser univentionstaff 2021-09-29 10:27:02 CEST
I don't know: Upgrade works for me with no tracebacks.

Maybe it was due to this?
0f499f4c1c Bug #53833: Bump version due to build problems
Comment 12 Florian Best univentionstaff 2021-09-29 10:34:48 CEST
(In reply to Johannes Keiser from comment #11)
> I don't know: Upgrade works for me with no tracebacks.
> 
> Maybe it was due to this?
> 0f499f4c1c Bug #53833: Bump version due to build problems

No, this was skipped. The upgrade is from this night.

I think it would be better to just catch the ImportError and make a comment to remove it in UCS 5.1 or UCS 4.4-10.
Comment 13 Johannes Keiser univentionstaff 2021-09-29 10:45:49 CEST
Do you have a VM with a snapshot before the upgrade, where this occurs. I don't have a Traceback in my logs
Comment 14 Florian Best univentionstaff 2021-09-29 10:52:42 CEST
No that are the regular Jenkins runs.
For the S4-Connector thing you can e.g. install UCS@school before that and then join some Windows clients.
Comment 15 Florian Best univentionstaff 2021-09-29 13:36:13 CEST
I got the wrong bug. This is the 5.0 bug, but I actually need to REOPEN the 4.4 bug.
Comment 16 Ole Schwiegert univentionstaff 2021-09-29 14:27:35 CEST
The problem was fixed in the 4.4 bug.
Comment 17 Johannes Keiser univentionstaff 2021-09-30 12:22:46 CEST
OK: no changes needed for 5.0