Univention Bugzilla – Bug 29509
Standardsuchen nicht komplett im Index
Last modified: 2013-03-25 19:57:22 CET
Die Standardsuchen in UMC sind nicht komplett im LDAP Index. Benutzer: Nov 30 18:24:48 backup442 slapd[9724]: <= bdb_substring_candidates: (employeeNumber) not indexed Rechner: Nov 30 18:25:43 backup442 slapd[9724]: <= bdb_substring_candidates: (univentionInventoryNumber) not indexed Nov 30 18:25:43 backup442 slapd[9724]: <= bdb_substring_candidates: (aRecord) not indexed Nov 30 18:25:43 backup442 slapd[9724]: <= bdb_substring_candidates: (associatedDomain) not indexed Nov 30 18:25:43 backup442 slapd[9724]: <= bdb_substring_candidates: (macAddress) not indexed Das ist vor allem in großen Umgebungen problematisch.
Folgendes Kommando findet noch mehr Eigenschaften, die nicht im Index stehen: ==================== > ucr=$(ucr search ldap/index) > for i in /usr/share/pyshared/univention/admin/handlers/*/*.py; do > j=${i#*admin\/handlers/} > sed -n '/univention.admin.property/{h;d}; G; s/.*default_search.*\n.*\W\(\w*\)\W\s*:\s*univention.admin.property.*/\1/p' $i | while read k; do > [ -n "$k" -a -z "$(echo "$ucr" | grep "\\<$k\\>")" ] && echo "$k - $j" > done > done | sort ==================== Ausgabe: ==================== address - dns/ptr_record.py base - settings/license.py cname - dns/alias.py ddnsHostname - policies/dhcp_dnsupdate.py defaultClientGroup - settings/default.py defaultComputerGroup - settings/default.py defaultDomainControllerGroup - settings/default.py defaultDomainControllerMBGroup - settings/default.py defaultGroup - settings/default.py defaultMemberServerGroup - settings/default.py domain - computers/domaincontroller_backup.py domain - computers/domaincontroller_master.py domain - computers/domaincontroller_slave.py domain - computers/ipmanagedclient.py domain - computers/linux.py domain - computers/macos.py domain - computers/memberserver.py domain - computers/ubuntu.py domain - computers/windows_domaincontroller.py domain - computers/windows.py domain_name - policies/dhcp_dns.py employeeNumber - users/user.py firstname - users/user.py flavor - settings/umc_operationset.py host - dhcp/host.py host - shares/share.py inventoryNumber - computers/computer.py inventoryNumber - computers/ipmanagedclient.py inventoryNumber - computers/linux.py inventoryNumber - computers/macos.py inventoryNumber - computers/memberserver.py inventoryNumber - computers/ubuntu.py inventoryNumber - computers/windows_domaincontroller.py inventoryNumber - computers/windows.py ip - computers/computer.py ip - computers/domaincontroller_backup.py ip - computers/domaincontroller_master.py ip - computers/domaincontroller_slave.py ip - computers/ipmanagedclient.py ip - computers/linux.py ip - computers/macos.py ip - computers/memberserver.py ip - computers/ubuntu.py ip - computers/windows_domaincontroller.py ip - computers/windows.py lastname - users/user.py longDescription - settings/extended_attribute.py longDescription - settings/extended_options.py mac - computers/computer.py mac - computers/domaincontroller_backup.py mac - computers/domaincontroller_master.py mac - computers/domaincontroller_slave.py mac - computers/ipmanagedclient.py mac - computers/linux.py mac - computers/macos.py mac - computers/memberserver.py mac - computers/ubuntu.py mac - computers/windows_domaincontroller.py mac - computers/windows.py mailAddress - groups/group.py mailAddress - mail/lists.py mailDomain - mail/folder.py model - shares/printer.py netmask - networks/network.py network - networks/network.py operatingSystem - computers/linux.py operatingSystem - computers/ubuntu.py operatingSystem - computers/windows_domaincontroller.py operatingSystem - computers/windows.py primaryGroup - computers/ubuntu.py primaryGroup - computers/windows_domaincontroller.py primaryGroup - computers/windows.py printablename - shares/share.py printeruri - settings/printeruri.py printmodel - settings/printermodel.py printServer - policies/printserver.py profile - policies/desktop.py ptr_record - dns/ptr_record.py repositoryServer - policies/repositoryserver.py sambaName - shares/print.py server - dhcp/server.py serverRole - computers/domaincontroller_backup.py serverRole - computers/domaincontroller_master.py serverRole - computers/domaincontroller_slave.py serverRole - computers/memberserver.py serverRole - computers/windows_domaincontroller.py service - dhcp/service.py shortDescription - settings/extended_attribute.py shortDescription - settings/extended_options.py subnet - dhcp/sharedsubnet.py subnet - dhcp/subnet.py subnet - dns/reverse_zone.py translationLongDescription - settings/extended_attribute.py translationLongDescription - settings/extended_options.py translationShortDescription - settings/extended_attribute.py translationShortDescription - settings/extended_options.py uri - shares/printer.py usernames - settings/prohibited_username.py username - users/passwd.py username - users/user.py zone - dns/forward_zone.py ====================
@Comment 1: This is probably not very useful as UDM propterties are mapped to OpenLDAP attributes: mapping.register('inventoryNumber', 'univentionInventoryNumber') It's probably more useful to take this approach: ucr set ldap/debug/level=stats; invoke-rc.d slapd restart ## perform all kinds of operations grep 'not indexed' /var/log/syslog
Update, die Suche war nach UDM- und nicht nach LDAP-Attributen. ==================== > ucr=$(ucr search ldap/index) > for i in /usr/share/pyshared/univention/admin/handlers/*/*.py; do > # get module name > mod=${i#*admin\/handlers/} > # cache all UDM->LDAP mappings > map=$(grep mapping.register $i) > # find all default_search attributes > sed -n '/univention.admin.property/{h;d}; G; s/.*default_search.*\n.*\W\(\w*\)\W\s*:\s*univention.admin.property.*/\1/p' $i | while read k; do > [ -z "$k" ] && continue > # convert UDM attribute to LDAP attribute > l=$(echo "$map" | sed -n 's/.*register(\W*\<'"$k"'\>\W*\<\(\w*\)\>.*/\1/p') > if [ -z "$l" ]; then > l=$k > fi > # see whether the attribute is in the index > [ -z "$(echo "$ucr" | grep "\\<$l\\>")" ] && echo "$k [=$l] - $mod" > done > done | sort ==================== Ausgabe: ==================== > base [=univentionLicenseBaseDN] - settings/license.py > ddnsHostname [=univentionDhcpDdnsHostname] - policies/dhcp_dnsupdate.py > defaultClientGroup [=univentionDefaultClientGroup] - settings/default.py > defaultComputerGroup [=univentionDefaultComputerGroup] - settings/default.py > defaultDomainControllerGroup [=univentionDefaultDomainControllerGroup] - settings/default.py > defaultDomainControllerMBGroup [=univentionDefaultDomainControllerMasterGroup] - settings/default.py > defaultGroup [=univentionDefaultGroup] - settings/default.py > defaultMemberServerGroup [=univentionDefaultMemberserverGroup] - settings/default.py > domain [=associatedDomain] - computers/domaincontroller_backup.py > domain [=associatedDomain] - computers/domaincontroller_master.py > domain [=associatedDomain] - computers/domaincontroller_slave.py > domain [=associatedDomain] - computers/ipmanagedclient.py > domain [=associatedDomain] - computers/linux.py > domain [=associatedDomain] - computers/macos.py > domain [=associatedDomain] - computers/memberserver.py > domain [=associatedDomain] - computers/ubuntu.py > domain [=associatedDomain] - computers/windows_domaincontroller.py > domain [=associatedDomain] - computers/windows.py > domain_name [=univentionDhcpDomainName] - policies/dhcp_dns.py > employeeNumber [=employeeNumber] - users/user.py > firstname [=firstname] - users/user.py > flavor [=umcOperationSetFlavor] - settings/umc_operationset.py > host [=univentionShareHost] - shares/share.py > inventoryNumber [=univentionInventoryNumber] - computers/computer.py > inventoryNumber [=univentionInventoryNumber] - computers/ipmanagedclient.py > inventoryNumber [=univentionInventoryNumber] - computers/linux.py > inventoryNumber [=univentionInventoryNumber] - computers/macos.py > inventoryNumber [=univentionInventoryNumber] - computers/memberserver.py > inventoryNumber [=univentionInventoryNumber] - computers/ubuntu.py > inventoryNumber [=univentionInventoryNumber] - computers/windows_domaincontroller.py > inventoryNumber [=univentionInventoryNumber] - computers/windows.py > ip [=ip] - computers/computer.py > ip [=ip] - computers/domaincontroller_backup.py > ip [=ip] - computers/domaincontroller_master.py > ip [=ip] - computers/domaincontroller_slave.py > ip [=ip] - computers/ipmanagedclient.py > ip [=ip] - computers/linux.py > ip [=ip] - computers/macos.py > ip [=ip] - computers/memberserver.py > ip [=ip] - computers/ubuntu.py > ip [=ip] - computers/windows_domaincontroller.py > ip [=ip] - computers/windows.py > lastname [=lastname] - users/user.py > longDescription [=univentionUDMOptionLongDescription] - settings/extended_options.py > mailDomain [=mailDomain] - mail/folder.py > model [=univentionPrinterModel] - shares/printer.py > netmask [=univentionNetmask] - networks/network.py > network [=univentionNetwork] - networks/network.py > operatingSystem [=univentionOperatingSystem] - computers/linux.py > operatingSystem [=univentionOperatingSystem] - computers/ubuntu.py > operatingSystem [=univentionOperatingSystem] - computers/windows_domaincontroller.py > operatingSystem [=univentionOperatingSystem] - computers/windows.py > primaryGroup [=primaryGroup] - computers/ubuntu.py > primaryGroup [=primaryGroup] - computers/windows_domaincontroller.py > primaryGroup [=primaryGroup] - computers/windows.py > printablename [=printablename] - shares/share.py > printeruri [=printerURI] - settings/printeruri.py > printmodel [=printerModel] - settings/printermodel.py > printServer [=univentionPrintServer] - policies/printserver.py > profile [=univentionDesktopProfile] - policies/desktop.py > repositoryServer [=univentionRepositoryServer] - policies/repositoryserver.py > sambaName [=sambaName] - shares/print.py > shortDescription [=univentionUDMOptionShortDescription] - settings/extended_options.py > translationLongDescription [=translationLongDescription] - settings/extended_attribute.py > translationLongDescription [=translationLongDescription] - settings/extended_options.py > translationShortDescription [=translationShortDescription] - settings/extended_attribute.py > translationShortDescription [=translationShortDescription] - settings/extended_options.py > uri [=univentionPrinterURI] - shares/printer.py > usernames [=prohibitedUsername] - settings/prohibited_username.py > username [=username] - users/passwd.py > username [=username] - users/user.py ====================
This would boil down to: associatedDomain employeeNumber primaryGroup printerModel printerURI prohibitedUsername umcOperationSetFlavor univentionDefaultClientGroup univentionDefaultComputerGroup univentionDefaultDomainControllerGroup univentionDefaultDomainControllerMasterGroup univentionDefaultGroup univentionDefaultMemberserverGroup univentionDesktopProfile univentionDhcpDdnsHostname univentionDhcpDomainName univentionInventoryNumber univentionLicenseBaseDN univentionNetmask univentionNetwork univentionOperatingSystem univentionPrinterModel univentionPrinterURI univentionPrintServer univentionRepositoryServer univentionShareHost univentionUDMOptionLongDescription univentionUDMOptionShortDescription additionally, grepping the UDM modules for '(ad|ml).append' we would have aAAARecord dhcpPermitList dhcpRange gecos krb5KDCFlags krb5KeyVersionNumber krb5MaxLife krb5MaxRenew krb5PasswordEnd krb5ValidEnd pwhistory sambaBadPasswordCount sambaKickoffTime sambaMungedDial sambaPasswordHistory sambaPwdLastSet shadowExpire shadowLastChange shadowMax sOARecord univentionAllowedEmailGroups univentionAllowedEmailUsers univentionCron univentionIpRange univentionMailACL univentionMailSharedFolderDeliveryAddress univentionNagiosNotificationOptions univentionNagiosTimeperiod univentionNextIp univentionRepositoryCron univentionSyntaxLDAPAttribute univentionSyntaxLDAPValue + userPassword sambaNTpassowrd sambaLMPassword k5key The question is, which of these attributes are search for by value equality, which by substring and which by peresence? And which are just retrived from the object itself. AFAIK the indices do not improve the return of values, e.g. ldapsearch -x foo=this bar would benefit from foo beeing indexed, but not from bar beeing indexed.
(In reply to comment #4) > The question is, which of these attributes are search for by value equality, > which by substring and which by peresence? And which are just retrived from the > object itself. Those in default_search (the ones grepped for by Alexander) are all searched by substring as UMC-UDM wraps asterisks around every search since 3.1-0. All module's attributes in default_search are concatenated by OR.
(In reply to comment #5) > Those in default_search (the ones grepped for by Alexander) are all searched by > substring as UMC-UDM wraps asterisks around every search since 3.1-0. All > module's attributes in default_search are concatenated by OR. See also Bug#30533
I used the umc-command udm/nav/object/query (flavour navigation) to identify non-indexed attributes in umc's default search. ---------- ucr set ldap/debug/level=stats invoke-rc.d slapd restart eval "$(ucr shell)" for i in $(udm modules -h | grep "^ "); do if [ "$i" = "users/passwd" -o "$i" = "users/self" ]; then continue fi params="'container':'cn=shares,$ldap_base'" params="$params,'objectType':'$i'" params="$params,'objectProperty':'None'" params="$params,'objectPropertyValue':'l*'" umc-command -U Administrator -P univention \ udm/nav/object/query -e -o "{$params}" \ -f 'navigation' -r done grep 'not indexed' /var/log/syslog | \ awk -F '(' '{print $2}' | awk -F ')' '{print $1}' | \ sort -u ---------- /var/log/syslog then complains about those attributes: aRecord associatedDomain employeeNumber macAddress name pTRRecord relativeDomainName univentionInventoryNumber univentionOperatingSystem univentionSyntaxDescription univentionUDMPropertyLongDescription univentionUDMPropertyShortDescription which i added to recommended_ldap_index_sub in univention-ldap-server.postinst. I think that's enough for now. ( still got Mar 7 15:39:04 master slapd[30671]: <= bdb_substring_candidates: (pTRRecord) not indexed in syslog because of Bug #20199 )
Running the script tells my that "ou" is triggered by container/ou. Maybe it should also go into ldap/index/sub.
(In reply to comment #8) > Running the script tells my that "ou" is triggered by container/ou. Maybe it > should also go into ldap/index/sub. ok
Changelog adjusted and verified.
UCS 3.1-1 has been released: http://download.univention.de/doc/release-notes-3.1-1_en.pdf http://download.univention.de/doc/release-notes-3.1-1.pdf If this error occurs again, please use "Clone This Bug".