Bug 41999 - Quadratic scaling in s4cache.py (speed up _update_entry)
Quadratic scaling in s4cache.py (speed up _update_entry)
Status: CLOSED FIXED
Product: UCS
Classification: Unclassified
Component: S4 Connector
UCS 4.1
Other Linux
: P5 enhancement (vote)
: UCS 4.1-3-errata
Assigned To: Janek Walkenhorst
Stefan Gohmann
:
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2016-08-15 18:02 CEST by Janek Walkenhorst
Modified: 2016-09-29 17:31 CEST (History)
4 users (show)

See Also:
What kind of report is it?: Bug Report
What type of bug is this?: 2: Improvement: Would be a product improvement
Who will be affected by this bug?: 2: Will only affect a few installed domains
How will those affected feel about the bug?: 2: A Pain – users won’t like this once they notice it
User Pain: 0.046
Enterprise Customer affected?:
School Customer affected?: Yes
ISV affected?:
Waiting Support:
Flags outvoted (downgraded) after PO Review:
Ticket number:
Bug group (optional): Large environments, UCS Performance
Max CVSS v3 score:


Attachments
Patch against r71582 UCS 4.1-2, but will probably work with many other versions (1.45 KB, patch)
2016-08-15 18:02 CEST, Janek Walkenhorst
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Janek Walkenhorst univentionstaff 2016-08-15 18:02:30 CEST
Created attachment 7871 [details]
Patch against r71582 UCS 4.1-2, but will probably work with many other versions

The attached patch adds indexes to enable efficient lookup
  by data.guid_id (and data.attribute_id) (as used
    by _update_entry()
    by get_entry()
    by remove_entry()
  )
  by attributes.attribute (as used
    by _get_attr_id()
    by _update_entry()
  )
  by guids.guid (as used
    by _get_guid_id()
  )
to the SQLite database of the S4Cache, if they do not already exist, during __init__().

Otherwise more objects lead to more work per object, i.e. quadratic complexity.

Customer reported speedup from 10s of seconds per object to 10s of objects per second.


Additionally the patch adds indexes to enable efficient lookup
  by s4_lock.guid (as used
    by is_s4_locked()
    by unlock_s4()
  )
  by ucs_lock.uuid (as used
    by is_ucs_locked()
    by unlock_ucs()
  )
to the SQLite database of the LockingDB, if they do not already exist, during __init__().

Otherwise more locks lead to more work per lock, i.e. quadratic complexity.


Finally I note that
  self.s4cache in LockingDB.__init__() seems like dead code
and
  self.s4cache in S4Cache.__init__() and .add_entry() seems like unused code or a waste of memory
.
Comment 1 Janek Walkenhorst univentionstaff 2016-08-24 19:24:40 CEST
Patch applied.
Advisory: univention-s4-connector.yaml
Tests: OK
Comment 2 Stefan Gohmann univentionstaff 2016-08-30 06:37:26 CEST
YAML: OK

Code review 4.1-3 (r71908): OK

Code review 4.2 (r71909): OK

Tests: OK

ucs-test: OK
Comment 3 Janek Walkenhorst univentionstaff 2016-09-07 18:41:47 CEST
<http://errata.software-univention.de/ucs/4.1/255.html>