Univention Bugzilla – Bug 49201
Extend univention-translog by various consistency checks
Last modified: 2019-04-04 14:15:19 CEST
univention-translog should be extended by various consistency checks: - Check for duplicates in the transaction file including a fix option - Check for missing transactions (transaction holes) including a fix option - Check for listener/listener enumeration (it must continue the transaction ID from the transaction file) - A diff between cn=translg and the entries in the transaction files In each case it should be printed which consequences the adjustments have, e.g. a rejoin of the backups.
It should be released after Bug #49198 has been released.
[4.4-0] 1f433145ef Bug #49201 udn: Implement 'univention-translog check --fix' [4.4-0] e2b94ce26b Bug #49201 udn: Implement 'univention-translog check --fix' .../univention-directory-notifier/debian/changelog | 6 + .../univention-translog | 409 ++++++++++++++++++++- 2 files changed, 412 insertions(+), 3 deletions(-) [4.4-0] 822e0b649d Bug #49201 udn: Rename command line argument names .../univention-translog | 26 +++++++++++----------- 1 file changed, 13 insertions(+), 13 deletions(-) [4.4-0] c889b44a90 Bug #49201 udn: More mypy annotations management/univention-directory-notifier/univention-translog | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-) [4.4-0] c51fc04b67 Bug #49201 udn: Add index dump command .../univention-translog | 23 ++++++++++++++++++++++ 1 file changed, 23 insertions(+) [4.4-0] 86d773181b Bug #49201 udn: Fix debug message management/univention-directory-notifier/univention-translog | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) Package: univention-directory-notifier Version: 13.0.1-12A~4.4.0.201904032046 Branch: ucs_4.4-0 Scope: errata4.4-0 [4.4-0] 80094d4d3c Bug #49201: univention-directory-notifier 13.0.1-12A~4.4.0.201904032046 doc/errata/staging/univention-directory-notifier.yaml | 15 +++++++++++++++ 1 file changed, 15 insertions(+) TODO: Also check cn=translog TODO: More tests TODO: Maybe also fix syntax errors - see TODO in code TODO: Backport after QA to UCS-4.3-3 (Bug #49202) [4.4-0] 80094d4d3c Bug #49201: univention-directory-notifier 13.0.1-12A~4.4.0.201904032046 doc/errata/staging/univention-directory-notifier.yaml | 15 +++++++++++++++ 1 file changed, 15 insertions(+)
Some issues: 1. Holes ------------------------------------------------------------------------------- I've stopped the notifier, created two containers and removed two lines from the transaction file. Afterwards I executed: /usr/share/univention-directory-notifier/univention-translog check -f The output: [...] 2019-03-12 08:13:36,094:ERROR:/var/lib/univention-ldap/notify/transaction:3360:'1694 uid=stefan,cn=users,dc=deadlock44,dc=intranet m\n': Hole after '1692 dc=deadlock44,dc=intranet m' 2019-03-12 08:13:36,094:ERROR:/var/lib/univention-ldap/notify/transaction:3361:'1693 dc=deadlock44,dc=intranet m\n': Repeated line after '1694 uid=stefan,cn=users,dc=deadlock44,dc=intranet m' 2019-03-12 08:13:36,095:ERROR:/var/lib/univention-ldap/notify/transaction:3362:'1695 uid=stefan,cn=users,dc=deadlock44,dc=intranet m\n': Hole after '1693 dc=deadlock44,dc=intranet m' 2019-03-12 08:13:36,095:ERROR:/var/lib/univention-ldap/notify/transaction:3363:'1694 dc=deadlock44,dc=intranet m\n': Repeated line after '1695 uid=stefan,cn=users,dc=deadlock44,dc=intranet m' 2019-03-12 08:13:36,095:ERROR:/var/lib/univention-ldap/notify/transaction:3364:'1696 cn=Domain Users,cn=groups,dc=deadlock44,dc=intranet m\n': Hole after '1694 dc=deadlock44,dc=intranet m' 2019-03-12 08:13:36,096:ERROR:/var/lib/univention-ldap/notify/transaction:3365:'1695 dc=deadlock44,dc=intranet m\n': Repeated line after '1696 cn=Domain Users,cn=groups,dc=deadlock44,dc=intranet m' 2019-03-12 08:13:36,096:ERROR:/var/lib/univention-ldap/notify/transaction:3366:'1697 cn=S-1-5-21-429048034-1787900510-999614333-1118,cn=sid,cn=temporary,cn=univention,dc=deadlock44,dc=intranet a\n': Hole after '1695 dc=deadlock44,dc=intranet m' 2019-03-12 08:13:36,097:ERROR:/var/lib/univention-ldap/notify/transaction:3367:'1696 dc=deadlock44,dc=intranet m\n': Repeated line after '1697 cn=S-1-5-21-429048034-1787900510-999614333-1118,cn=sid,cn=temporary,cn=univention,dc=deadlock44,dc=intranet a' 2019-03-12 08:13:36,097:ERROR:/var/lib/univention-ldap/notify/transaction:3368:'1698 uid=stefan,cn=users,dc=deadlock44,dc=intranet m\n': Hole after '1696 dc=deadlock44,dc=intranet m' Traceback (most recent call last): File "/usr/share/univention-directory-notifier/univention-translog", line 1141, in <module> exit(main()) File "/usr/share/univention-directory-notifier/univention-translog", line 399, in main return opt.func(opt) or 0 File "/usr/share/univention-directory-notifier/univention-translog", line 651, in check translog.run() File "/usr/share/univention-directory-notifier/univention-translog", line 753, in run self.fixit() File "/usr/share/univention-directory-notifier/univention-translog", line 823, in fixit self.fix_fill() File "/usr/share/univention-directory-notifier/univention-translog", line 926, in fix_fill assert not self.needs_fill AssertionError root@master441:~# diff transaction /var/lib/univention-ldap/notify/transaction.bak 25,26d24 < 25 cn=Univention,cn=packages,cn=univention,dc=deadlock44,dc=intranet a < 26 cn=Fernwartung,cn=packages,cn=univention,dc=deadlock44,dc=intranet a root@master441:~# cat /var/lib/univention-ldap/listener/listener 1699 cn=x1,dc=deadlock44,dc=intranet a 1700 cn=x2,dc=deadlock44,dc=intranet a root@master441:~# tail -n 3 /var/lib/univention-ldap/notify/transaction.bak 1696 cn=Domain Users,cn=groups,dc=deadlock44,dc=intranet m 1697 cn=S-1-5-21-429048034-1787900510-999614333-1118,cn=sid,cn=temporary,cn=univention,dc=deadlock44,dc=intranet a 1698 uid=stefan,cn=users,dc=deadlock44,dc=intranet m root@master441:~# tail -n 3 /var/lib/univention-ldap/notify/transaction 1697 cn=S-1-5-21-429048034-1787900510-999614333-1118,cn=sid,cn=temporary,cn=univention,dc=deadlock44,dc=intranet a 1696 dc=deadlock44,dc=intranet m 1698 uid=stefan,cn=users,dc=deadlock44,dc=intranet m root@master441:~# ------------------------------------------------------------------------------- 2. Messages ------------------------------------------------------------------------------- Since we have a fix command, we should show it instead of or at least in addition to the SDB link. root@master441:~# /usr/share/univention-directory-notifier/univention-translog check 2019-03-12 08:20:10,078:ERROR:/var/lib/univention-ldap/notify/transaction:25:'27 cn=Tools,cn=packages,cn=univention,dc=deadlock44,dc=intranet a\n': Hole after '24 zoneName=deadlock44.intranet,cn=dns,dc=deadlock44,dc=intranet m' /var/lib/univention-ldap/notify/transaction needs fixing: - missing transactions in sequence See <https://help.univention.com/t/problem-umc-diagnostic-module-complains-about-problems-with-udn-replication/11707/1> for details. root@master441:~# So, maybe something like: You can rerun the tool with the option -f in order to try to fix this issue. See <https://help.univention.com/t/problem-umc-diagnostic-module-complains-about-problems-with-udn-replication/11707/1> for more details. ------------------------------------------------------------------------------- 3. Different duplicates ------------------------------------------------------------------------------- I've duplicated the id 1699 and changed the DN from cn=x1 to cn=x2: root@master441:~# tail -n 5 /var/lib/univention-ldap/notify/transaction.bak 1697 cn=S-1-5-21-429048034-1787900510-999614333-1118,cn=sid,cn=temporary,cn=univention,dc=deadlock44,dc=intranet a 1698 uid=stefan,cn=users,dc=deadlock44,dc=intranet m 1699 cn=x2,dc=deadlock44,dc=intranet a 1699 cn=x1,dc=deadlock44,dc=intranet a 1700 cn=x2,dc=deadlock44,dc=intranet a root@master441:~# The check fixed it into: root@master441:~# tail -n 5 /var/lib/univention-ldap/notify/transaction 1696 cn=Domain Users,cn=groups,dc=deadlock44,dc=intranet m 1697 cn=S-1-5-21-429048034-1787900510-999614333-1118,cn=sid,cn=temporary,cn=univention,dc=deadlock44,dc=intranet a 1698 uid=stefan,cn=users,dc=deadlock44,dc=intranet m 1699 cn=x2,dc=deadlock44,dc=intranet a 1700 cn=x2,dc=deadlock44,dc=intranet a root@master441:~# I think in this case the reset is required. ------------------------------------------------------------------------------- 4. Backup ------------------------------------------------------------------------------- The check failed on a DC Backup: root@backup442:~# /usr/share/univention-directory-notifier/univention-translog check 2019-03-12 08:32:33,942:ERROR:/var/lib/univention-ldap/last_id: Invalid last id: [Errno 2] No such file or directory: '/var/lib/univention-ldap/last_id' /var/lib/univention-ldap/last_id needs manual fixing! See <https://help.univention.com/t/problem-umc-diagnostic-module-complains-about-problems-with-udn-replication/11707/1> for details. root@backup442:~# The DC Backup don't have a last_id. ------------------------------------------------------------------------------- The following checks work very well: - Duplicates for the same entry - wrong last_id I've created for the diff tool between cn=translog and the transaction file a new bug since it is currently not important: Bug #49225.
(In reply to Stefan Gohmann from comment #3) > 1. Holes [4.4-0] af1aa86643 Bug #49201 udn: Fix hole filling > 2. Messages [4.4-0] 9bdbc474ba Bug #49201 udn: Extend help messages > 3. Different duplicates [4.4-0] d99a7a93cd Bug #49201 udn: Fix sorting [4.4-0] 5d06565de1 Bug #49201 udn: Check sort success > 4. Backup [4.4-0] 91d9a41430 Bug #49201 udn: Only check UCS Master and Backups [4.4-0] b1665e332f Bug #49201 udn: Recommend re-join on non-Master [4.4-0] 8f1124c434 Bug #49201 udn: Fix last_id checking > The following checks work very well: > - Duplicates for the same entry > - wrong last_id > > I've created for the diff tool between cn=translog and the transaction file > a new bug since it is currently not important: Bug #49225. 5. Always just renumber listener/listener [4.4-0] 0f6ac209e4 Bug #49201 udn: Always renumber listener/listener 6. Cosmetic patches [4.4-0] 347fef774d Bug #49201 udn: Fix spelling error [4.4-0] 4844bde784 Bug #49201 udn: Add copyright 7. Add test suite - run tests/run manually - not yet from debian/rules [4.4-0] 0fd6d93ac0 Bug #49201 udn: Fix univention-tranlog check --fix [4.4-0] d4cbe5a4db Bug #49201 udn: Fix univention-tranlog check --fix [4.4-0] 61c85c6aba Bug #49201 udn: Add tests [4.4-0] f0a345c74c Bug #49201 udn: Add argument to skip stopping/starting services Package: univention-directory-notifier Version: 13.0.1-13A~4.4.0.201904041212 Version: 13.0.1-13A~4.4.0.201904041219 Branch: ucs_4.4-0 Scope: errata4.4-0 [4.4-0] 705176ed80 Bug #49201: univention-directory-notifier 13.0.1-13A~4.4.0.201904041219 doc/errata/staging/univention-directory-notifier.yaml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
(In reply to Philipp Hahn from comment #4) > (In reply to Stefan Gohmann from comment #3) > > 1. Holes > > [4.4-0] af1aa86643 Bug #49201 udn: Fix hole filling OK, it works now. > > 2. Messages > > [4.4-0] 9bdbc474ba Bug #49201 udn: Extend help messages It looks good. > > 3. Different duplicates > > [4.4-0] d99a7a93cd Bug #49201 udn: Fix sorting > [4.4-0] 5d06565de1 Bug #49201 udn: Check sort success It works now. > > 4. Backup > > [4.4-0] 91d9a41430 Bug #49201 udn: Only check UCS Master and Backups > [4.4-0] b1665e332f Bug #49201 udn: Recommend re-join on non-Master > [4.4-0] 8f1124c434 Bug #49201 udn: Fix last_id checking OK, it works now. > > > The following checks work very well: > > - Duplicates for the same entry > > - wrong last_id > > > > I've created for the diff tool between cn=translog and the transaction file > > a new bug since it is currently not important: Bug #49225. > > 5. Always just renumber listener/listener > > [4.4-0] 0f6ac209e4 Bug #49201 udn: Always renumber listener/listener OK > 6. Cosmetic patches > > [4.4-0] 347fef774d Bug #49201 udn: Fix spelling error > [4.4-0] 4844bde784 Bug #49201 udn: Add copyright > > 7. Add test suite - run tests/run manually - not yet from debian/rules > > [4.4-0] 0fd6d93ac0 Bug #49201 udn: Fix univention-tranlog check --fix > [4.4-0] d4cbe5a4db Bug #49201 udn: Fix univention-tranlog check --fix > [4.4-0] 61c85c6aba Bug #49201 udn: Add tests > [4.4-0] f0a345c74c Bug #49201 udn: Add argument to skip stopping/starting > services OK > Package: univention-directory-notifier > Version: 13.0.1-13A~4.4.0.201904041212 > Version: 13.0.1-13A~4.4.0.201904041219 > Branch: ucs_4.4-0 > Scope: errata4.4-0 > > [4.4-0] 705176ed80 Bug #49201: univention-directory-notifier > 13.0.1-13A~4.4.0.201904041219 > doc/errata/staging/univention-directory-notifier.yaml | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) OK
<http://errata.software-univention.de/ucs/4.4/33.html>