Attachment 10350 Details for Bug 51236

[patch] suggested patch

skip-loop.patch (text/plain), 3.24 KB, created by Ralf Haferkamp on 2020-05-06 20:53 CEST

(hide)

Description:

Filename:

MIME Type:

Creator: Ralf Haferkamp

Created: 2020-05-06 20:53 CEST

Size: 3.24 KB

patch

obsolete

>commit c1a2a4bccbaf53b61b0510a64503b802ccfd7388
>Author: Ralf Haferkamp <rhafer@suse.com>
>Date:   Wed May 6 20:19:06 2020 +0200
>
>    replication: Avoid expensive nested loop when loglevel below INFO
>    
>    handler() goes into a nested for loop to figure out whether the
>    entry read from LDAP differs from the one it got from the cache.
>    The purpose of that seems to be to log a message (at "INFO" level)
>    when they differ.
>    
>    That nested for loop can be every expensive. E.g. for very large
>    groups. (We ran into this with a group with roughly 100000 values
>    in the memberUid and uniqueMember attributes).
>    
>    This patch skips the loop when the currently set loglevel is
>    below "INFO". It might make sense only go even further and to the
>    logs only and "DEBUG" level. (Or skip the checks if the number of
>    attribute values is beyond a certain size).
>
>diff --git a/management/univention-directory-replication/replication.py b/management/univention-directory-replication/replication.py
>index 97d69e7a80..0e380c3b41 100644
>--- a/management/univention-directory-replication/replication.py
>+++ b/management/univention-directory-replication/replication.py
>@@ -829,30 +829,31 @@ def handler(dn, new, listener_old, operation):
> 		if not isinstance(l, LDIFObject):
> 			old = getOldValues(l, dn)
> 
>-			# Check if both entries really match
>-			match = 1
>-			if len(old) != len(listener_old):
>-				ud.debug(ud.LISTENER, ud.INFO, 'replication: LDAP keys=%s; listener keys=%s' % (list(old.keys()), list(listener_old.keys())))
>-				match = 0
>-			else:
>-				for k in old:
>-					if k in EXCLUDE_ATTRIBUTES:
>-						continue
>-					if k not in listener_old:
>-						ud.debug(ud.LISTENER, ud.INFO, 'replication: listener does not have key %s' % (k,))
>-						match = 0
>-						break
>-					if len(old[k]) != len(listener_old[k]):
>-						ud.debug(ud.LISTENER, ud.INFO, 'replication: LDAP and listener values diff for %s' % (k,))
>-						match = 0
>-						break
>-					for v in old[k]:
>-						if v not in listener_old[k]:
>-							ud.debug(ud.LISTENER, ud.INFO, 'replication: listener does not have value for key %s' % (k,))
>+			if ud.get_level(ud.LISTENER) >= ud.INFO:
>+				# Check if both entries really match
>+				match = 1
>+				if len(old) != len(listener_old):
>+					ud.debug(ud.LISTENER, ud.INFO, 'replication: LDAP keys=%s; listener keys=%s' % (list(old.keys()), list(listener_old.keys())))
>+					match = 0
>+				else:
>+					for k in old:
>+						if k in EXCLUDE_ATTRIBUTES:
>+							continue
>+						if k not in listener_old:
>+							ud.debug(ud.LISTENER, ud.INFO, 'replication: listener does not have key %s' % (k,))
>+							match = 0
>+							break
>+						if len(old[k]) != len(listener_old[k]):
>+							ud.debug(ud.LISTENER, ud.INFO, 'replication: LDAP and listener values diff for %s' % (k,))
> 							match = 0
> 							break
>-			if not match:
>-				ud.debug(ud.LISTENER, ud.INFO, 'replication: old entries from LDAP server and Listener do not match')
>+						for v in old[k]:
>+							if v not in listener_old[k]:
>+								ud.debug(ud.LISTENER, ud.INFO, 'replication: listener does not have value for key %s' % (k,))
>+								match = 0
>+								break
>+				if not match:
>+					ud.debug(ud.LISTENER, ud.INFO, 'replication: old entries from LDAP server and Listener do not match')
> 		else:
> 			old = listener_old
>

Actions: View | Diff

Attachments on bug 51236: 10350