Univention Bugzilla – Bug 48729
Add tool to clean up old cn=translog entries
Last modified: 2019-08-14 14:54:52 CEST
We should have a tool to remove entries from cn=translog with an id that is lower than a specified value (or maybe something like "keep only the last X entries").
Currently happening on customer site. Database has been set recently to 8GB approx. two weeks ago and has reached again the size limit. Customer increased again (now to ~17G). The issue is not about empty pages, the translog already uses 48.6% of the 17GB: ======================================================================= root@ucs-master:~# used_pages=$(mdb_stat -e /var/lib/univention-ldap/translog | sed -n 's/^ *Number of pages used: //p') root@ucs-master:~# max_pages=$(mdb_stat -e /var/lib/univention-ldap/translog | sed -n 's/^ *Max pages: //p') root@ucs-master:~# python -c "print('%.1f used' % (float($used_pages) / $max_pages * 100))" 48.6 used root@ucs-master:~# ucr get ldap/database/mdb/maxsize 17179869184 ======================================================================= An unlimited growth of the database will cause severe issues at some stage (/-fs usage and so on). As no workaround available and customer disk is getting filled up tagging as "waiting support"
[4.4-0] 66b3ff5ac8 Bug #48729 udn: Implement prune command [4.4-0] a6dbca4664 Bug #48729 udn: Implement prune command .../univention-directory-notifier/debian/changelog | 6 ++ .../univention-translog | 119 ++++++++++++++++++++- 2 files changed, 124 insertions(+), 1 deletion(-) [4.4-0] 8d46c51abb Bug #48729 udn: Add documentation .../univention-translog | 123 +++++++++++++++++++-- 1 file changed, 114 insertions(+), 9 deletions(-) [4.4-0] 756502a095 Bug #48729 udn: Reuse index.remove() code .../univention-translog | 22 ++++++++++++++-------- 1 file changed, 14 insertions(+), 8 deletions(-) [4.4-0] 2411f0f436 Bug #48729 udn: Reuse restart() code management/univention-directory-notifier/univention-translog | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) [4.4-0] eaafcdbd35 Bug #48729 udn: Fix indention .../univention-translog | 172 ++++++++++----------- 1 file changed, 86 insertions(+), 86 deletions(-) Package: univention-directory-notifier Version: 13.0.1-15A~4.4.0.201905031308 Branch: ucs_4.4-0 Scope: errata4.4-0 [4.4-0] cfa10faa12 Bug #48729: univention-directory-notifier 13.0.1-15A~4.4.0.201905031308 doc/errata/staging/univention-directory-notifier.yaml | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+) Prune every transaction before 10: ./univention-translog -vvvv prune 10 Keep only the last 10 transactions: ./univention-translog -vvvv prune -10
Ok works, two minor points: * Is this a typo in the log message or what does that mean: log.info('Keeeling last %d transactions before %d', -opt.trans, translog.last) * I have to specify at least two -v to see any info messages. I would expect to see some output when I specify --verbose once: root@master10:~# /usr/share/univention-directory-notifier/univention-translog -v --dry-run prune 102 root@master10:~# /usr/share/univention-directory-notifier/univention-translog -vv --dry-run prune 102 2019-05-02 14:53:23,648:INFO:Index of size 13265 contains 1473 entries 2019-05-02 14:53:23,648:INFO:Purging/keeping 100..102..1472 2019-05-02 14:53:23,684:INFO:Deleted reqSession=100,cn=translog 2019-05-02 14:53:23,685:INFO:Deleted reqSession=101,cn=translog * Idea for improvement for later: Backup removed translog file entries in a compressed file. But I would not do this now. * Advisory: Ok Reopening for you comments or changes.
(In reply to Arvid Requate from comment #3) > * Is this a typo in the log message or what does that mean: > log.info('Keeeling last %d transactions before %d', -opt.trans, > translog.last) Fixed > * I have to specify at least two -v to see any info messages. I would expect > to see some output when I specify --verbose once: Increase default debug level. > * Idea for improvement for later: Backup removed translog file entries in a > compressed file. But I would not do this now. The old file is already moved to .bak [4.4-0] 0ce117fe13 Bug #48729 udl: Fix spelling errors and increase verbosity management/univention-directory-notifier/debian/changelog | 6 ++++++ management/univention-directory-notifier/univention-translog | 6 +++--- 2 files changed, 9 insertions(+), 3 deletions(-) Package: univention-directory-notifier Version: 13.0.1-16A~4.4.0.201905080824 Branch: ucs_4.4-0 Scope: errata4.4-0 [4.4-0] 1571569321 Bug #48729: univention-directory-notifier 13.0.1-16A~4.4.0.201905080824 doc/errata/staging/univention-directory-notifier.yaml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
Verified: * Prune transactions older than 10: /usr/share/univention-directory-notifier/univention-translog -v --dry-run prune 10 * Prune transactions, keeping the last 1000 : /usr/share/univention-directory-notifier/univention-translog -v --dry-run prune -1000 ## negative number Other small improvements for the log messages are postponed to Bug #49430
<http://errata.software-univention.de/ucs/4.4/81.html>
Attempt to use the new prune command on Master UCS to reduce the size of the translog database. The translog database has reached almost 87% capacity (of 16GB). Initially attempt to keep only the last 1,000,000 entries (of 15,538,649) using the following command: root@ucs-master:~# /usr/share/univention-directory-notifier/univention-translog -v prune -1000000 Right at Deleted reqSession=399999,cn=translog the following error was thrown: Traceback (most recent call last): File "/usr/share/univention-directory-notifier/univention-translog", line 1406, in <module> exit(main()) File "/usr/share/univention-directory-notifier/univention-translog", line 420, in main return opt.func(opt) or 0 File "/usr/share/univention-directory-notifier/univention-translog", line 1182, in prune prune_ldap(opt) File "/usr/share/univention-directory-notifier/univention-translog", line 1237, in prune_ldap rtype, rdata, rmsgid, serverctrls = ld.result3(response) File "/usr/lib/python2.7/dist-packages/ldap/ldapobject.py", line 514, in result3 resp_ctrl_classes=resp_ctrl_classes File "/usr/lib/python2.7/dist-packages/ldap/ldapobject.py", line 521, in result4 ldap_result = self._ldap_call(self._l.result4,msgid,all,timeout,add_ctrls,add_intermediates,add_extop) File "/usr/lib/python2.7/dist-packages/ldap/ldapobject.py", line 106, in _ldap_call result = func(*args,**kwargs) ldap.SIZELIMIT_EXCEEDED: {'desc': 'Size limit exceeded'} Re-running the exact same prune command: CRITICAL:Already purged. Restoring the snapshot. Attempt pruning a smaller amount of entries at one time (keeping 15,500,000 of 15,538,649). This attempt also resulted in the same timeout error as above.