Univention Bugzilla – Bug 47418
remove UMC code from schoolldap module
Last modified: 2020-02-06 17:23:34 CET
Every use of the ucsschool.lib eventually imports the module ucsschool.lib.schoolldap. That module contains code that relies on UMC library code, which in turn leads to importing a very long list of UMC module. The UMC code is however only used by UMC modules and for example not by the ucsschool-import or other scripts. It does bloat the memory footprint and slows down the start time of almost all ucs@school code. Move SchoolBaseModule into a separate module and use uldap functions instead of umc.ldap in LDAP_Connection.
Implemented this in branch dtroeder/47418_remove.UMC.code.from.schoolldap: [dtroeder/47418_remove.UMC.code.from.schoolldap] 96115c17f Bug #47418: split UMC code into separate module [dtroeder/47418_remove.UMC.code.from.schoolldap] d7cbc1d2e Bug #47418: adapt imports Before: $ /usr/bin/time python -c 'from ucsschool.lib.models import User' User time (seconds): 1.06 System time (seconds): 0.11 Elapsed (wall clock) time (h:mm:ss or m:ss): 0:01.18 Maximum resident set size (kbytes): 67268 After: $ /usr/bin/time python -c 'from ucsschool.lib.models import User' User time (seconds): 0.60 System time (seconds): 0.06 Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.68 Maximum resident set size (kbytes): 55632
Before: root@m125:~# python -v -c 'from ucsschool.lib.models import User' 2>&1 | egrep '^import' | cut -f 1 -d '#' | sort | uniq | wc -l 608 root@m125:~# python -v -c 'from ucsschool.lib.models import User' 2>&1 | egrep '^import' | cut -f 1 -d '#' | sort | uniq | egrep 'univention|ucs' | wc -l 226 After: root@m125:~# python -v -c 'from ucsschool.lib.models import User' 2>&1 | egrep '^import' | cut -f 1 -d '#' | sort | uniq | wc -l 497 root@m125:~# python -v -c 'from ucsschool.lib.models import User' 2>&1 | egrep '^import' | cut -f 1 -d '#' | sort | uniq | egrep 'univention|ucs' | wc -l 200
The caching behavior of UCSSchoolHelperAbstractClass.get_machine_connection() was restored: [dtroeder/47418_remove.UMC.code.from.schoolldap] 4fa4718b9 Bug #47418: restore caching behavior The imports were further reduced by splitting the LDAP connection code from the rest of the UMC code: [dtroeder/47418_remove.UMC.code.from.schoolldap] 6e4952dc7 Bug #47418: split LDAP connection decorator from other UMC code [dtroeder/47418_remove.UMC.code.from.schoolldap] ce5d7deac Bug #47418: adapt imports [dtroeder/47418_remove.UMC.code.from.schoolldap] ca20932c5 Bug #47418: fix typo
Fixed an import error. [dtroeder/47418_remove.UMC.code.from.schoolldap_4.4v2 34555bf49] Bug #47418: importing ucr from ucsschool.lib.models.utils leads to recursive import loop
from univention.lib.school_umc_ldap_connection import LDAP_Connection, set_bind_function This import is wrong and has to be fixed. The correct module is ucsschool.lib.[...] The faulty import is located in the school_umc_base.py
I fixed another broken import and started the branch test again. I checked some of the failing tests on a random basis on a local setup with the new packages and they all passed. I would propose merging the branch into 4.4 today we can perform the final QA on Monday.
Merged and built. [4.4] 482f8ce5a Bug #47418: split UMC code into separate module [4.4] 551d66f8b Bug #47418: restore caching behavior [4.4] 85ffc67b6 Bug #47418: split LDAP connection decorator from other UMC code [4.4] e5221fa05 Bug #47418: importing ucr from ucsschool.lib.models.utils leads to recursive import loop [4.4] 4c13c94fc Bug #47418: fix filename [4.4] b4920b11d Bug #47418: Merge branch 'dtroeder/47418_remove.UMC.code.from.schoolldap_4.4v2' into 4.4 [4.4] 65e914fe9 Bug #47418: version bump ucs-school-l10n-fr (4.0.0-2) ucs-school-lib (12.1.8) ucs-school-roleshares (7.0.0-2) ucs-school-s4-branch-site (7.0.0-1) ucs-school-umc-computerroom (11.0.0-9) ucs-school-umc-distribution (17.0.1-6) ucs-school-umc-exam (9.0.1-17) ucs-school-umc-groups (9.0.0-2) ucs-school-umc-helpdesk (15.0.0-1) ucs-school-umc-import (2.0.0-4) ucs-school-umc-internetrules (15.0.0-1) ucs-school-umc-lessontimes (7.0.0-1) ucs-school-umc-lists (2.0.0-4) ucs-school-umc-printermoderation (16.0.0-1) ucs-school-umc-rooms (16.1.0-2) ucs-school-umc-users (15.0.0-4) ucs-school-umc-wizards (11.0.0-7) univention-management-console-module-selective-udm (8.0.0-1)
[4.4] 2416defe2 Bug #47418: fix typo in import ucs-school-lib (12.1.9)
After the merge many tests failed. Also as far as I can see one of my fixes did not make it into your merge: From 4b27149f2dc3d58190eac9e97d0076e0e1a9e1fa Mon Sep 17 00:00:00 2001 From: Ole Schwiegert <schwiegert@univention.de> Date: Fri, 1 Nov 2019 09:43:12 +0100 Subject: [PATCH] Bug #47418: Add missing imports --- ucs-school-umc-rooms/umc/python/schoolrooms/__init__.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/ucs-school-umc-rooms/umc/python/schoolrooms/__init__.py b/ucs-school-umc-rooms/umc/python/schoolrooms/__init__.py index f72a641b1..a1b61b762 100644 --- a/ucs-school-umc-rooms/umc/python/schoolrooms/__init__.py +++ b/ucs-school-umc-rooms/umc/python/schoolrooms/__init__.py @@ -42,7 +42,7 @@ from univention.management.console.modules.decorators import sanitize import univention.admin.uexceptions as udm_exceptions from ucsschool.lib.school_umc_base import SchoolBaseModule, LDAP_Filter, SchoolSanitizer -from ucsschool.lib.school_umc_ldap_connection import LDAP_Connection +from ucsschool.lib.school_umc_ldap_connection import LDAP_Connection, USER_READ, USER_WRITE from ucsschool.lib.models import ComputerRoom, SchoolComputer from ucsschool.lib.models.utils import add_module_logger_to_schoollib -- 2.17.1 I am not sure if I forgot to push it to the remote repo, if so I am sorry.
I don't know what happened to 4b27149f2dc3d58190eac9e97d0076e0e1a9e1fa, it is'nt in dtroeder/47418_remove.UMC.code.from.schoolldap_4.4v2 anymore. Maybe I accidentally force pushed before pulling? The change is now in: [4.4 0b24d3c65] Bug #47418: fix import ucs-school-umc-distribution (17.0.1-6) # not changed, but had not been built ucs-school-roleshares (7.0.0-3) ucs-school-umc-rooms (16.1.0-3) ucs-test-ucsschool (6.0.74) I have started Jenkins jobs: 1. Publish U@S 4.4 scope to testing #1305 2.Install U@S 4.4 Singleserver #280
The Jenkins jobs U@S 4.4 Singleserver #280 didn't have any errors that were related to this bug. I updated the package version dependency in all packages and added advisories: [4.4] 9ce705483 Bug #47418: fix dependency [4.4] 79acb1c05 Bug #47418: advisories
Packages install: OK Changelog&Advisory: OK Tests: OK
UCS@school 4.4 v4 has been released. https://docs.software-univention.de/changelog-ucsschool-4.4v4-de.html If this error occurs again, please clone this bug.