Univention Bugzilla – Bug 54797
Notifier indexing slows rapidly down the last pending 999 transactions to index
Last modified: 2023-03-08 16:36:28 CET
If /usr/share/univention-directory-notifier/univention-translog stat shows an AssertionError this indicates that the transaction.index is corrupt or uses wrong data. AssertionError: (25775045, 11269137) So removing the /var/lib/univention-ldap/notify/transaction.index and restarting the univention-directory-notifier it starts reindexing. In larger environments it starts really fast 2022-05-25 13:17:40,881:WARNING:Index=1962147 < translog=11269137 entries 2022-05-25 13:18:20,087:WARNING:Index=11268138 < translog=11269137 entries → 11269137−11268138 = 999 2022-05-25 13:19:02,318:WARNING:Index=11268138 < translog=11269137 entries 2022-05-25 13:20:05,220:WARNING:Index=11268140 < translog=11269137 entries This seems always to slow down the last 999 transactions to index. Approximately 45 sec for one index entry. -------------------------------------------- In the other environment it is the same: → 7120940−7119941 = 999 2022-05-24 12:37:22,619:WARNING:Index=2859957 < translog=7120940 entries 2022-05-24 12:37:46,503:WARNING:Index=7119941 < translog=7120940 entries 2022-05-24 12:37:57,075:WARNING:Index=7119941 < translog=7120940 entries So indexing finished next day. During this time replication is not working at all.
<https://forge.univention.org/bugzilla/show_bug.cgi?id=41687#c9> 26. from 3 years ago includes the analysis and the patch
Without Philipp's explanations of what the Notifier was doing, I would have been helpless. The notifier takes a long time to rebuild the index. A log entry that it is doing this now would also be very helpful. For me it looked like the notifier has no error, but also does not work. Enterprise customer affected. Added ticket number.
Decided to not fix in favor of Bug #54837 *** This bug has been marked as a duplicate of bug 54837 ***
I don't understand why it is a duplicate. Will this issue be automatically fixed when the old versions of the protocol have been removed?
(In reply to Stefan Gohmann from comment #5) > I don't understand why it is a duplicate. Will this issue be automatically > fixed when the old versions of the protocol have been removed? Yes: only UDN/UDL protocol version 2 requires the text-file `/var/lib/univention-ldap/notify/transaction`, where UDN itself provides the DN itself and does not delegate this to `ldapsearch -b cn=translog`. The index-file `…/transaction.index` for it is not required any more, but UDN still spends time on maintaining it.
[5.0-3] 8d838f97f2 feat(udn): univention-translog reindex doc/errata/staging/univention-directory-notifier.yaml | 15 +++++++++++++++ management/univention-directory-notifier/debian/changelog | 6 ++++++ management/univention-directory-notifier/univention-translog | 32 +++++++++++++++++++++++++++++++- 3 files changed, 52 insertions(+), 1 deletion(-) Package: univention-directory-notifier Version: 14.0.6-3A~5.0.0.202303031515 Branch: ucs_5.0-0 Scope: errata5.0-3 [5.0-3] 29399309e0 Bug #54797: univention-directory-notifier 14.0.6-3A~5.0.0.202303031515 doc/errata/staging/univention-directory-notifier.yaml | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-) QA: cd /var/lib/univention-ldap/notify truncate -s 1000 ./transaction.index /usr/share/univention-directory-notifier/univention-translog -v reindex ls -l transaction.* /usr/share/univention-directory-notifier/univention-translog stat systemctl status univention-directory-notifier.service
[5.0-3] 6cfd2a78d0 feat(udn): univention-translog reindex doc/errata/staging/univention-directory-notifier.yaml | 2 +- management/univention-directory-notifier/debian/changelog | 6 ++++++ management/univention-directory-notifier/univention-translog | 2 ++ 3 files changed, 9 insertions(+), 1 deletion(-) Package: univention-directory-notifier Version: 14.0.6-4A~5.0.0.202303031532 Branch: ucs_5.0-0 Scope: errata5.0-3 [5.0-3] 8aee1f58f2 Bug #54797: univention-directory-notifier 14.0.6-4A~5.0.0.202303031532 doc/errata/staging/univention-directory-notifier.yaml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
Created attachment 11048 [details] univention-translog backport to Python 2.7 / UCS 4.4-x
OK: Test as described in comment 7 OK: /usr/share/univention-directory-notifier/univention-translog reindex OK: yaml Verified
Cloned that bug for UCS 4.4 https://forge.univention.org/bugzilla/show_bug.cgi?id=55863
<https://errata.software-univention.de/#/?erratum=5.0x604>