Univention Bugzilla – Bug 40595
squidguard does not remove db backing files in /var/tmp/
Last modified: 2016-03-16 15:01:08 CET
+++ This bug was initially created as a clone of Bug #40541 +++ Ticket#2016012821000323 In /var/tmp were lot's of files (~15000; 77GB) from the last 3 years. The files were owned by proxy:proxy and contained database information. I digged into the code of squidguard and libdb and for me it looks like if squidguard is using an unnamed in-memory btree via libdb. If the amount of data still fits into the in memory cache, no file is created. If too much data is stored within the btree, a temporary backing file is created in /var/tmp (if nothing else is defined in DB_ENV). Currently I have no idea who is responsible for the backing file. If I'm right, squidguard does not get an file handle for the backing file and is therefore unable to take care of it. It is possible to define a different tmpdir for the backing files but e.g. /tmp is no real solution since we have multiple squidguard instances running which are quite often restarted. On each start of an instance, a new backing file is created. The problem got much worse since the increase of started squidguard instances (2→20). See also: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=721314
ucs-school-webproxy now uses the "dbtemp" option of squidguard. dbtemp is set to /var/lib/ucs-school-webproxy/bdbtemp/, so all "BDB*" files are automatically placed there. A cronjob checks regulary (every 30min) if there are BDB* files without open file handle (→ lsof). If this is the case, the unused/unhandled files are removed. The undocumented UCR variable proxy/squidguard/cron/cleanup_bdb_tempfiles may be used to change the cron interval. ucs-school-webproxy (12.0.3-1): r67268 | Bug #40541: regulary purge outdated BDB* files / use new squidguard option "dbtemp" Please note: changes in ucs-school-webproxy have been mistakenly committed with bug number 40541!
I built the package: Package: ucs-school-webproxy Version: 12.0.3-1.116.201602091554 Branch: ucs_4.1-0 Scope: ucs-school-4.1
root@xen3:~# for ((i=0; i<=20; i++)); do touch "/var/lib/ucs-school-webproxy/bdbtemp/BDB$i"; done root@xen3:~# python >>> l = [open('/var/lib/ucs-school-webproxy/bdbtemp/BDB%s' % (i,)) for i in range(10)] ^Z root@xen3:~# /usr/share/ucs-school-webproxy/purge_outdated_squidguard_bdb_files --dry-run UNUSED BDB FILE: /var/lib/ucs-school-webproxy/bdbtemp/BDB14 UNUSED BDB FILE: /var/lib/ucs-school-webproxy/bdbtemp/BDB20 UNUSED BDB FILE: /var/lib/ucs-school-webproxy/bdbtemp/BDB16 UNUSED BDB FILE: /var/lib/ucs-school-webproxy/bdbtemp/BDB11 UNUSED BDB FILE: /var/lib/ucs-school-webproxy/bdbtemp/BDB12 UNUSED BDB FILE: /var/lib/ucs-school-webproxy/bdbtemp/BDB10 UNUSED BDB FILE: /var/lib/ucs-school-webproxy/bdbtemp/BDB18 UNUSED BDB FILE: /var/lib/ucs-school-webproxy/bdbtemp/BDB17 UNUSED BDB FILE: /var/lib/ucs-school-webproxy/bdbtemp/BDB19 UNUSED BDB FILE: /var/lib/ucs-school-webproxy/bdbtemp/BDB15 UNUSED BDB FILE: /var/lib/ucs-school-webproxy/bdbtemp/BDB13 root@xen3:~# fg ^C root@xen3:~# /usr/share/ucs-school-webproxy/purge_outdated_squidguard_bdb_files root@xen3:~# ls /var/lib/ucs-school-webproxy/bdbtemp | wc -l 0
UCS@school 4.1 v5 has been released. If this error occurs again, please clone this bug.