Univention Bugzilla – Bug 31560
Performance problem with many servers
Last modified: 2013-07-26 14:28:49 CEST
The bug below improved performance for several syntax classes. These optimizations should also be applied to UCS_Server. +++ This bug was initially created as a clone of Bug #30991 +++ Opening the UDM users or computers module takes "very" long. groups/group.py#open() does not seem to be the culprit, since a "udm users/user list --filter uid=XXXXXX" from the command line is instant (<0.5s). "udm groups/group list" on the other hand takes 11 seconds, which matches roughly the time UMC UDM takes. 1213 groups. No groups in groups. No mixed hosts / users groups. The largest group has 3572 users. The user is member of 26 groups with 16,17²,21²,22,25,39,67,78,88,112,119³,121²,137,144,176,251,1945,2814,2872,3296,3572 members.
HOWTO: 1. Add "use_objects = False" to UCS_Server syntax class. 2. Change label = "%(fqdn)s" to '%(name)s.%(domain)s' (and probably add a comment there and in the handler where fqdn is set that these two statements have to be in sync). 3. Add a "def lookup_filter(filter_s=None):" to handlers/$udm_module for each $udm_module in UCS_Server.udm_modules. Look at groups/group for a very simple example (note how it is used in "def lookup") 4. But lookup_filter() above needs also the lo object to handle dnsAlias. This means that *every* lookup_filter function defined has to be extended and lo needs to be passable (but will be unused there). grep them, extend them. 5. umc-module-udm: pass the lo object whereever lookup_filter is used.
(In reply to comment #0) > The bug below improved performance for several syntax classes. These > optimizations should also be applied to UCS_Server. And also to DomainController. Workaround, to get input fields instead of drop downs: PREFIX="directory/manager/web/modules/policies" ucr set $PREFIX/ldapserver/properties/ldapServer/syntax=string ucr set $PREFIX/printserver/properties/printServer/syntax=string ucr set $PREFIX/repositoryserver/properties/repositoryServer/syntax=string
Done as stated in Comment#1 3.1-1: univention-management-console-module-udm 3.0.78-3.320.201307011450 univention-directory-manager-modules 8.0.143-18.1022.201307011502 3.2-0: univention-management-console-module-udm 4.0.2-1.321.201307011508 univention-directory-manager-modules 9.0.4-1.1023.201307011510 Changelog updated, YAML updated and created. 2013-06-25-univention-directory-manager-modules.yaml 2013-07-01-univention-management-console-module-udm.yaml QA: Note that all handlers that saw optimizations before have been slightely adjusted. Especially using dnsAlias should be tested carefully.
After setting up a larger environment (see Bug 29418), I spotted some more syntaxes which should be optimized IMHO: PrinterProtocol PrinterProducerList UserName GroupName ServicePrint_FQDN nfsShare Here the performance with a JavaScript test code... ==================== var Deferred = require('dojo/Deferred'); var tools = require('umc/tools'); var deferred = new Deferred(); deferred.resolve(); var syntaxes = [ 'UCS_Server', 'DomainController', 'GroupDN', 'WritableShare', 'DNS_ReverseZone', 'DNS_ForwardZone', 'UserID', 'GroupID', 'PrinterProtocol', 'PrinterProducerList', 'UserName', 'GroupName', 'ServicePrint_FQDN', 'nfsShare' ]; syntaxes.forEach(function(syntax) { deferred = deferred.then(function() { console.log('### BEGIN', syntax); var d = new Date(); return tools.umcpCommand('udm/syntax/choices', { syntax: syntax }).then(function(response) { console.log('### END', syntax, 'time:', (new Date() - d) / 1000.0, 'elements:', response.result.length ); }); }); }); ==================== And here the output: ==================== ### BEGIN UCS_Server ### END UCS_Server time: 3.552 elements: 610 ### BEGIN DomainController ### END DomainController time: 2.404 elements: 406 ### BEGIN GroupDN ### END GroupDN time: 0.275 elements: 1034 ### BEGIN WritableShare ### END WritableShare time: 0.259 elements: 1 ### BEGIN DNS_ReverseZone ### END DNS_ReverseZone time: 0.287 elements: 3 ### BEGIN DNS_ForwardZone ### END DNS_ForwardZone time: 0.26 elements: 2 ### BEGIN UserID ### END UserID time: 1.131 elements: 5062 ### BEGIN GroupID ### END GroupID time: 0.307 elements: 1035 ### BEGIN PrinterProtocol ### END PrinterProtocol time: 0.282 elements: 8 ### BEGIN PrinterProducerList ### END PrinterProducerList time: 2.023 elements: 63 ### BEGIN UserName ### END UserName time: 40.61 elements: 5062 ### BEGIN GroupName ### END GroupName time: 7.655 elements: 1034 ### BEGIN ServicePrint_FQDN ### END ServicePrint_FQDN time: 0.297 elements: 1 ### BEGIN nfsShare ### END nfsShare time: 5.38 elements: 2400 ==================== Especially UserName, GroupName (→ adding printer/printer share), nfsShare (→ NFS mount policy) should be optimized.
(In reply to Alexander Kläser from comment #4) > After setting up a larger environment (see Bug 29418), I spotted some more > syntaxes which should be optimized IMHO: > > PrinterProtocol > PrinterProducerList > ... > ### BEGIN PrinterProtocol > ### END PrinterProtocol time: 0.282 elements: 8 > ### BEGIN PrinterProducerList > ### END PrinterProducerList time: 2.023 elements: 63 Those two can be ignored, PrinterProducerList will be handled via Bug 31376. PrinterProtocol contains only 8 elements.
(In reply to Alexander Kläser from comment #4) > Especially UserName, GroupName (→ adding printer/printer share), nfsShare (→ > NFS mount policy) should be optimized. Okay, added in univention-directory-manager-modules 8.0.143-21.1027.201307041354 and univention-directory-manager-modules 9.0.7-1.1028.201307041356
Performance test with applied erratum changes: ==================== ### BEGIN UCS_Server ### END UCS_Server time: 0.398 elements: 610 ### BEGIN DomainController ### END DomainController time: 0.29 elements: 406 ### BEGIN GroupDN ### END GroupDN time: 0.297 elements: 1034 ### BEGIN WritableShare ### END WritableShare time: 0.27 elements: 1 ### BEGIN DNS_ReverseZone ### END DNS_ReverseZone time: 0.278 elements: 3 ### BEGIN DNS_ForwardZone ### END DNS_ForwardZone time: 0.276 elements: 2 ### BEGIN UserID ### END UserID time: 1.012 elements: 5062 ### BEGIN GroupID ### END GroupID time: 0.297 elements: 1035 ### BEGIN PrinterProtocol ### END PrinterProtocol time: 0.282 elements: 8 ### BEGIN PrinterProducerList ### END PrinterProducerList time: 1.94 elements: 63 ### BEGIN UserName ### END UserName time: 0.917 elements: 5062 ### BEGIN GroupName ### END GroupName time: 0.29 elements: 1034 ### BEGIN ServicePrint_FQDN ### END ServicePrint_FQDN time: 0.284 elements: 1 ### BEGIN nfsShare ### END nfsShare time: 0.76 elements: 2400 ==================== Performance looks much better now. Same number of elements in lists are returned. Before I found the following UDM objects to be problematic: computer user share printer policy: LDAP server policy: NFS mount policy: Print quota policy: Print server policy: Repository server In general, opening the detail page for one of these objects now takes 2-4 sec (only printer takes 7 sec as the syntax PrinterProducerList has not been optimized). When opening the policy print quota, it now only takes about 3 sec where sizelimit is set to 10000 and thus the combobox contains about 5000 users. That is indeed nice :) ! Performance test → OK YAML files → OK, adapted 2013-07-01-univention-management-console-module-udm.yaml slightly (change "Two syntaxes" to "Five syntaxes") 3.2 changelog → OK, adapted formatting sligthly 3.2 changes → OK, changes have been applied to 3.2
http://errata.univention.de/ucs/3.1/142.html
http://errata.univention.de/ucs/3.1/147.html