Bug 40592 - Add option dbtemp to squidguard
Add option dbtemp to squidguard
Product: UCS
Classification: Unclassified
Component: Squid
UCS 4.1
Other Linux
: P5 normal (vote)
: UCS 4.1-0-errata
Assigned To: Sönke Schwardt-Krummrich
Florian Best
Depends on: 40541 40595
  Show dependency treegraph
Reported: 2016-02-05 12:11 CET by Sönke Schwardt-Krummrich
Modified: 2016-02-11 13:58 CET (History)
2 users (show)

See Also:
What kind of report is it?: ---
What type of bug is this?: ---
Who will be affected by this bug?: ---
How will those affected feel about the bug?: ---
User Pain:
Enterprise Customer affected?:
School Customer affected?:
ISV affected?:
Waiting Support:
Ticket number:
Bug group (optional):
Max CVSS v3 score:


Note You need to log in before you can comment on or make changes to this bug.
Description Sönke Schwardt-Krummrich univentionstaff 2016-02-05 12:11:50 CET
squidguard is part of UCS, so the patched package should be delivered also via errata updates, so avoid update problems later on (see bug 40541).

+++ This bug was initially created as a clone of Bug #40541 +++


In /var/tmp were lot's of files (~15000; 77GB) from the last 3 years.
The files were owned by proxy:proxy and contained database information.

I digged into the code of squidguard and libdb and for me it looks like if 
squidguard is using an unnamed in-memory btree via libdb. If the amount of 
data still fits into the in memory cache, no file is created. If too much data 
is stored within the btree, a temporary backing file is created in /var/tmp 
(if nothing else is defined in DB_ENV).

Currently I have no idea who is responsible for the backing file. If I'm 
right, squidguard does not get an file handle for the backing file and is 
therefore unable to take care of it.

It is possible to define a different tmpdir for the backing files but e.g. 
/tmp is no real solution since we have multiple squidguard instances running 
which are quite often restarted. On each start of an instance, a new backing 
file is created.

The problem got much worse since the increase of started squidguard instances (2→20).

See also:
Comment 1 Sönke Schwardt-Krummrich univentionstaff 2016-02-09 12:23:46 CET
squidguard has been patched. A new config option "dbtemp" has been added. If defined, the bdb backing files will be placed within that directory. If unset, squidguard shows the old behaviour (using /var/tmp as temp dir for backing files).

r67234 | Bug #40592: updated squidguard.yaml
r67215 | Bug #40592: added squidguard.yaml

Please note: squidguard has been patched and built in ucs-school-4.1 AND errata4.1-0 to avoid problems with security/debian updates later on.
The build version of the errata scope is larger than the version of the ucs-school scope.

Comment 2 Florian Best univentionstaff 2016-02-09 17:46:55 CET
I can't reproduce the issue. Code change looks okay (.bison file was not adapted).

OK: squidguard 1.5-1.24.201602051323
OK: squidguard 1.5-1.26.201602051325
Comment 3 Sönke Schwardt-Krummrich univentionstaff 2016-02-09 20:41:00 CET
(In reply to Florian Best from comment #2)
> I can't reproduce the issue. Code change looks okay (.bison file was not
> adapted).

I rechecked the Makefile. The only part where bison files are touched are the following two targets:

y.tab.c y.tab.h: sg.y sg.h
        @if [ "$(YACC)" = "yacc" ]; then \
           if [ ! -x $(YACC) ]; then \
              echo " " ; \
              echo "No yacc/bison found. Copy prepared files for y.tab.h and y.tab.c over. " ; \
              echo " " ; \
              cp y.tab.h.bison y.tab.h ; \
              cp y.tab.c.bison y.tab.c ; \
           else \
              $(YACC) -d sg.y  ; \
           fi ; \
        else \
           $(YACC) -d sg.y  ; \
        fi ;

y.tab.c.bison y.tab.h.bison: sg.y sg.h
        $(YACC) -d sg.y
        mv -f y.tab.c y.tab.c.bison
        mv -f y.tab.h y.tab.h.bison

In the beginning of the Makefile, YACC is defined to "bison -y", therefore the code reduces to:

y.tab.c y.tab.h: sg.y sg.h
        $(YACC) -d sg.y

y.tab.c.bison y.tab.h.bison: sg.y sg.h
        $(YACC) -d sg.y
        mv -f y.tab.c y.tab.c.bison
        mv -f y.tab.h y.tab.h.bison

So the .bison files are recreated during build and do not have to be patched.
Comment 4 Janek Walkenhorst univentionstaff 2016-02-11 13:58:40 CET