Bug 51813 - improve user resource search speed
Summary: improve user resource search speed
Status: CLOSED FIXED
Alias: None
Product: UCS@school
Classification: Unclassified
Component: HTTP-API (Kelvin)
Version: UCS@school 4.4
Hardware: Other Linux
: P5 normal
Target Milestone: ---
Assignee: Daniel Tröder
QA Contact: Tobias Wenzel
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2020-08-12 16:17 CEST by Daniel Tröder
Modified: 2023-05-26 15:39 CEST (History)
2 users (show)

See Also:
What kind of report is it?: Feature Request
What type of bug is this?: ---
Who will be affected by this bug?: ---
How will those affected feel about the bug?: ---
User Pain:
Enterprise Customer affected?:
School Customer affected?:
ISV affected?:
Waiting Support:
Flags outvoted (downgraded) after PO Review:
Ticket number:
Bug group (optional):
Customer ID:
Max CVSS v3 score:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Daniel Tröder univentionstaff 2020-08-12 16:17:26 CEST
The search operation of the user resource of the Kelvin API service is inefficient and slow, when the search is not narrowed down to one OU.
It then iterates over all OUs, searching for the queries user objects.
This requires one UDM REST API query per OU.
When there are lots of OUs this takes a lot of time.
Make the user resource search operation more efficient.
Comment 1 Daniel Tröder univentionstaff 2020-08-12 16:31:37 CEST
The user search is now done using only one UDM REST API query. Even in small installations it is now at least 4 times faster.

The logging code was improved to additionally. It now detects when the FastAPI server ist started from the command line (when developing) and adds a console logger.

I accidentally commited directly to feature/kelvin, instead of a feature branch.
If this impedes a release, it must be reverted. Sorry.

[feature/kelvin] 2f081a11d Bug #51813: also log to stderr when started from a terminal
[feature/kelvin] 38b25be70 Bug #51813: find all matching users with one lookup
Comment 2 Daniel Tröder univentionstaff 2020-10-19 10:42:00 CEST
The code was moved to git branch dtroeder/51828_reuse_exam_users_kelvin as it was in the way of a release.
Comment 3 Daniel Tröder univentionstaff 2020-10-20 11:20:37 CEST
Looks like we didn't make a release in the meantime and the code stayed in features/kelvin.

I added the changelog entry that was forgotten for the 1.1.2 release:

[feature/kelvin 943fe7e41] Bug #51813: add changelog entry for previous release 1.1.2
Comment 4 Tobias Wenzel univentionstaff 2020-10-20 13:21:50 CEST
QA -> All OK -> build & Upload to testapp center

Code
- the amount of calls to the udm rest api is significantly reduced
- code looks good
- loglevel is set to debug, if started from terminal

The tests are green (or rather flaky as before). To test the increase of speed, I created four more schools and compared the runtime of test_route_user before and after the patch. After the patch the code executed about a minute faster.

Changelog -> OK
Comment 5 Daniel Tröder univentionstaff 2020-10-20 13:51:21 CEST
The Docker image could not be built, because the linters found problems.

[feature/kelvin 3ff9e8577] Bug #51813: fix build (make linters happy)
[feature/kelvin e05cf254f] Bug #51813: bump version to 1.2.0

The version was raised to "1.2.0" and the image built.
REPOSITORY                                                       TAG                 IMAGE ID            CREATED             SIZE
docker-upload.software-univention.de/ucsschool-kelvin-rest-api   1.2.0               10b9749ab7ba        35 seconds ago      390MB
Comment 6 Tobias Wenzel univentionstaff 2020-10-20 14:18:12 CEST
QA -> All OK -> VERIFY

Tests are passing -> OK
Code (code formatting) -> OK
New version 1.2.0 in testapp center -> OK
installed from testapp center -> OK
Comment 7 Daniel Tröder univentionstaff 2021-02-23 09:29:52 CET
A Kelvin API app with the changes in this bug has been published.