Bug 54795 - Broken univention-translog import --min
Broken univention-translog import --min
Status: CLOSED FIXED
Product: UCS
Classification: Unclassified
Component: Notifier (univention-directory-notifier)
UCS 4.4
Other Linux
: P5 normal (vote)
: UCS 4.4-9-errata
Assigned To: Philipp Hahn
Julia Bremer
:
Depends on: 54794
Blocks:
  Show dependency treegraph
 
Reported: 2022-05-25 07:36 CEST by Philipp Hahn
Modified: 2022-06-01 16:42 CEST (History)
2 users (show)

See Also:
What kind of report is it?: Bug Report
What type of bug is this?: 4: Minor Usability: Impairs usability in secondary scenarios
Who will be affected by this bug?: 2: Will only affect a few installed domains
How will those affected feel about the bug?: 5: Blocking further progress on the daily work
User Pain: 0.229
Enterprise Customer affected?:
School Customer affected?:
ISV affected?:
Waiting Support:
Flags outvoted (downgraded) after PO Review:
Ticket number:
Bug group (optional): bitesize
Max CVSS v3 score:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Philipp Hahn univentionstaff 2022-05-25 07:36:30 CEST
+++ This bug was initially created as a clone of Bug #54794 +++

`univention-translog import --min XXX` does not work.

There's a bug in `univention-translog import --min`, which ist *not* used.
> 492 start = max(1, min(c_tid, p_tid, s_tid), translog.first)
which should be
> 492 start = max(1, min(c_tid, p_tid, s_tid), translog.first, opt.min)

root@primary:~# /usr/share/univention-directory-notifier/univention-translog -v  import -i -m 12 -M 12
2022-05-24 12:04:48,267:INFO:Reading transactions from file '/var/lib/univention-ldap/notify/transaction'...
2022-05-24 12:04:48,267:INFO:Index of size 9935 contains 1103 entries
2022-05-24 12:04:48,275:INFO:Processing transaction from offset 0 .. 12
2022-05-24 12:04:48,276:CRITICAL:ldap_add(Transaction(tid=1, dn='zoneName=schein.ig,cn=dns,dc=schein,dc=ig', command='a')): Already exists

-i` == `--index` → "Do not use index to find transactions"
`-m` == `--min` → "First transaction ID to process"
`-M` == `--max` → "Last transaction ID to process"
`--min`/`-m` and `--max`/`-M` are *hard* limits, while `--count`/`-c` and `--percent`/`-p` and `--size`/`-s` are *soft* limits: The later are used to limit the number of TIDs to import; otherwise in a busy domain any Backup/Replica would need to import all transactions starting at TID 1; technical any DC only needs the TID starting with the lowest ListenerID in the domain: any previous TID already has been processed by *all* listeners; only the following TIDs are required for that (and maybe other) Listeners. As we do not have any feedback from Listeners back to Primary/Backups, we do not know the lowest TID in the domain. As an approximation we only import the last 100.000 TIDs or last 10 MiB from the transaction file, whichever is earlier.
Your given command will export 10⁵ TID or 10 MiB from the end of the file, but only if they are in range [12…12]. So only TID 12.
Your index (which is *not* used despite the message "Index of size …" as you have given `-i`; otherwise the message would have been "Processing transactions %d .. %d") indicates it contains only the last 1103 TIDs, the other 9935-1103=8802 TID are not in the index.
As `univention-translog --import -i` is not allowed to use the index, it must walk *all* TID in `translog` *from the beginning*, starting with TID 1.
Comment 1 Philipp Hahn univentionstaff 2022-05-25 07:40:35 CEST
[4.4-9] ef83b37454 refactor[UDN]: Improve univention-translog --help
 management/univention-directory-notifier/univention-translog | 32 ++++++++++++++++----------------
 1 file changed, 16 insertions(+), 16 deletions(-)

[4.4-9] 751d0262e0 refactor[UDN]: Remove default action="store"
 management/univention-directory-notifier/univention-translog | 14 +++++++-------
 1 file changed, 7 insertions(+), 7 deletions(-)

[4.4-9] 0166561a66 fix[UDN]: Heed univention-translog import --min TID
 doc/errata/staging/univention-directory-notifier.yaml        | 10 ++++++++++
 management/univention-directory-notifier/debian/changelog    |  6 ++++++
 management/univention-directory-notifier/univention-translog |  2 +-
 3 files changed, 17 insertions(+), 1 deletion(-)

Package: univention-directory-notifier
Version: 13.0.3-9A~4.4.0.202205250738
Branch: ucs_4.4-0
Scope: errata4.4-9

[4.4-9] b850686abf Bug #54795: univention-directory-notifier 13.0.3-9A~4.4.0.202205250738
 doc/errata/staging/univention-directory-notifier.yaml | 10 +++++-----
 1 file changed, 5 insertions(+), 5 deletions(-)
Comment 2 Julia Bremer univentionstaff 2022-05-27 14:59:40 CEST
OK: --min has effect
OK: help text adjustments