Bug 56809 - Deletion of big OUs via UDM takes hours to complete.
Deletion of big OUs via UDM takes hours to complete.
Status: NEEDMOREINFO
Product: UCS
Classification: Unclassified
Component: UDM (Generic)
UCS 5.0
Other Linux
: P5 normal (vote)
: ---
Assigned To: UMC maintainers
UMC maintainers
:
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2023-11-08 15:26 CET by Oliver Friedrich
Modified: 2023-11-10 09:45 CET (History)
3 users (show)

See Also:
What kind of report is it?: Bug Report
What type of bug is this?: 2: Improvement: Would be a product improvement
Who will be affected by this bug?: 1: Will affect a very few installed domains
How will those affected feel about the bug?: ---
User Pain:
Enterprise Customer affected?:
School Customer affected?:
ISV affected?:
Waiting Support:
Flags outvoted (downgraded) after PO Review:
Ticket number:
Bug group (optional): Large environments
Max CVSS v3 score:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Oliver Friedrich univentionstaff 2023-11-08 15:26:32 CET
When I delete a big OU via UDM, there is no progress for a long time, but after HOURS it is deleted:

time udm container/ou remove --dn ou=delmasse31,<ldap base>
Object removed: ou=delmasse31,<ldap base>
real    621m23,798s
user    0m0,067s
sys     0m0,006s

The OU delmasse contained about 30k objects, mainly school users.
(I can provide the LDIF file before deletion on request)

univention-app info:

UCS: 5.0-5 errata857
Installed: admindiary-backend=1.0 admindiary-frontend=1.0 itslearning=5.0-ucs1 ox-connector=2.2.7 privacyidea-saml=2.1.2 self-service=5.0 self-service-backend=5.0 ucsschool=5.0 v4 4.4/dovecot-connector=1.0.2 4.4/ucsschool-id-connector=2.2.5 4.4/ucsschool-kelvin-rest-api=1.8.8 4.4/ucsschool-veyon-proxy=4.7.4.14-0
Upgradable: privacyidea-saml dovecot-connector ucsschool-id-connector ucsschool-kelvin-rest-api

udm settings/udm_hook list | sed -ne 's/DN: cn=//p'
schoolOU
schoolAdminGroup
ucsschool_user_options
ucsschool_purge_timestamp
ucsschool_role_computers
monitoring
oxAccess
oxContextRW
oxGroupHook
oxUserDefaults
Comment 2 Daniel Tröder univentionstaff 2023-11-08 15:41:32 CET
Type of bug is "improvement".
What are your expectations?
Please provide KPIs.
Comment 3 Oliver Friedrich univentionstaff 2023-11-10 09:01:01 CET
I repeated this on a non-UCS-School system (no apps, 5.0-5 errata872, same HW specs)

## create users

A small script for creation....

#!/bin/bash

OU=delmasse01

START_TIME=$(date +%s)

for i in {1..30000}; do
        udm users/user create \
                --ignore_exists \
                --position "cn=users,ou=$OU,$(ucr get ldap/base)" \
                --set lastname=Doe \
                --set username="user_$i" \
                --set unixhome="/home/user_$i" \
                --set password="univention_$i"
        echo "Created user_$i"
done

duration="$(($(date +%s)-START_TIME))"
echo "Created 30k users in $duration seconds"

Created 30k users in 18636 seconds

## delete

time udm container/ou remove --dn ou=delmasse01,$(ucr get ldap/base)
Object removed: ou=delmasse01,<ldap base>
real    1155m50,602s
user    0m0,113s
sys     0m0,034s

- slapd was at 100% CPU, 2 cores
Comment 4 Daniel Tröder univentionstaff 2023-11-10 09:43:08 CET
1. Have you waited after the object-creation until the replication (Listener) was done, before starting the deletion?

2. School OUs should not contain 30k users. That's not a realistic scenario.

The performance will be very bad, because of the massive size of the "Domain Users $OU" group!
Comment 6 Daniel Tröder univentionstaff 2023-11-10 09:45:42 CET
Not only the "Domain Users $OU" group but also the "$ROLE $OU" groups will be massive.
OpenLDAP does not work well with huge multi-value attributes.