Univention Bugzilla – Bug 40595
squidguard does not remove db backing files in /var/tmp/
Last modified: 2016-03-16 15:01:08 CET
+++ This bug was initially created as a clone of Bug #40541 +++
In /var/tmp were lot's of files (~15000; 77GB) from the last 3 years.
The files were owned by proxy:proxy and contained database information.
I digged into the code of squidguard and libdb and for me it looks like if
squidguard is using an unnamed in-memory btree via libdb. If the amount of
data still fits into the in memory cache, no file is created. If too much data
is stored within the btree, a temporary backing file is created in /var/tmp
(if nothing else is defined in DB_ENV).
Currently I have no idea who is responsible for the backing file. If I'm
right, squidguard does not get an file handle for the backing file and is
therefore unable to take care of it.
It is possible to define a different tmpdir for the backing files but e.g.
/tmp is no real solution since we have multiple squidguard instances running
which are quite often restarted. On each start of an instance, a new backing
file is created.
The problem got much worse since the increase of started squidguard instances (2→20).
ucs-school-webproxy now uses the "dbtemp" option of squidguard. dbtemp is set to /var/lib/ucs-school-webproxy/bdbtemp/, so all "BDB*" files are automatically placed there.
A cronjob checks regulary (every 30min) if there are BDB* files without open file handle (→ lsof). If this is the case, the unused/unhandled files are removed.
The undocumented UCR variable proxy/squidguard/cron/cleanup_bdb_tempfiles may be used to change the cron interval.
r67268 | Bug #40541: regulary purge outdated BDB* files / use new squidguard option "dbtemp"
Please note: changes in ucs-school-webproxy have been mistakenly committed with bug number 40541!
I built the package:
root@xen3:~# for ((i=0; i<=20; i++)); do touch "/var/lib/ucs-school-webproxy/bdbtemp/BDB$i"; done
>>> l = [open('/var/lib/ucs-school-webproxy/bdbtemp/BDB%s' % (i,)) for i in range(10)]
root@xen3:~# /usr/share/ucs-school-webproxy/purge_outdated_squidguard_bdb_files --dry-run
UNUSED BDB FILE: /var/lib/ucs-school-webproxy/bdbtemp/BDB14
UNUSED BDB FILE: /var/lib/ucs-school-webproxy/bdbtemp/BDB20
UNUSED BDB FILE: /var/lib/ucs-school-webproxy/bdbtemp/BDB16
UNUSED BDB FILE: /var/lib/ucs-school-webproxy/bdbtemp/BDB11
UNUSED BDB FILE: /var/lib/ucs-school-webproxy/bdbtemp/BDB12
UNUSED BDB FILE: /var/lib/ucs-school-webproxy/bdbtemp/BDB10
UNUSED BDB FILE: /var/lib/ucs-school-webproxy/bdbtemp/BDB18
UNUSED BDB FILE: /var/lib/ucs-school-webproxy/bdbtemp/BDB17
UNUSED BDB FILE: /var/lib/ucs-school-webproxy/bdbtemp/BDB19
UNUSED BDB FILE: /var/lib/ucs-school-webproxy/bdbtemp/BDB15
UNUSED BDB FILE: /var/lib/ucs-school-webproxy/bdbtemp/BDB13
root@xen3:~# ls /var/lib/ucs-school-webproxy/bdbtemp | wc -l
UCS@school 4.1 v5 has been released.
If this error occurs again, please clone this bug.