Univention Bugzilla – Bug 30184
SquidGuard - integrate usage of external blacklists
Last modified: 2015-05-19 11:00:50 CEST
Asked via ticket #2013012521001021: It would be nice having the possiblity to integrate external blacklists for SquidGuard via UCR. See <http://wiki.ubuntuusers.de/squidGuard#Blacklist-installieren> for reference.
also asked for in 2013052721001519 often requested is the "shalla blacklist": http://www.shallalist.de/
*** Bug 31747 has been marked as a duplicate of this bug. ***
An implementation was created at Ticket #2013062621001224.
requested on Ticket#: 2013082021002249 again
This feature is requested again in association with UCS@school 3.2 R2: http://forum.univention.de/viewtopic.php?t=3364&p=11859#p11859
The squidguard configuration should be able to import/include external filter lists. By default in UCS@school the proxy checks are performed in the following order: 1) if defined, evaluate only the *whitelist* for the current computerroom (if the computer is member of that room) 2) if defined, evaluate the filter blacklist/whitelist with the highest priority that is assigned to one of the user's groups 3) allow the access, if 1) and 2) do not apply Usecase 1) is only a whitelist and everything else is blocked → it makes no sense to add the external filter list here. In case of 2) a filter is assigned to one of the user's groups. In case this filter is a white list, the external filter is also useless. In case of a blacklist, the external filter list should be optionally processed. It should be configurable via UCR if the external filter list has to be processed in this case (one global variable for all groups). If a external list is configured, it should be always processed in case 3). The external filter list is usually provided as raw text files. Split into 2 files - domains/ip addresses only and URLs only. squidguard should be able to include both. A UCR variable should define the text file names. A special update command should be implemented that converts the text file into a squidgard database file. Please test with the mentioned shalla list and check if there are performance issues with 4 mio entries.
External blacklists may now be placed below /var/lib/ucs-school-webproxy/ either directly or in a subdirectory. 2 new UCR variables have been introduced: 1) proxy/filter/global/blacklists/domains 2) proxy/filter/global/blacklists/urls These variables should contain the filenames of either domainname blacklists or url blacklists. The filenames have to be relative to /var/lib/ucs-school-webproxy and have to separated by spaces, e.g.: /var/lib/ucs-school-webproxy/extblacklist1/domains /var/lib/ucs-school-webproxy/extblacklist1/urls /var/lib/ucs-school-webproxy/bl2/list-domains /var/lib/ucs-school-webproxy/bl2/list-urls /var/lib/ucs-school-webproxy/bl3-dom /var/lib/ucs-school-webproxy/bl3-urls ucr set proxy/filter/global/blacklists/domains=\ "extblacklist1/domains bl2/list-domains bl3-dom" ucr set proxy/filter/global/blacklists/urls=\ "extblacklist1/urls bl2/list-urls bl3-urls" The UCR module ucs-school-webproxy.py concatenates all files of one type (domain/url) and call squidguard to convert it into a squidguard db file. Please note: this is only done, if the UCR variables are set/have changed! Depending on the size of the blacklist files, this may take several seconds! In the test environment, 1.6 mio entries took about 12 seconds. ucs-school-webproxy_11.0.3-2.81.201504241306_all.deb xml changelog entry has been added
This somehow does not work for me. I have no further internet rules set up. Tried with a teacher, student and Administrator. # cd /var/lib/ucs-school-webproxy # wget "$shallalist" # tar xvzf shallalist.tar.gz # ucr set proxy/filter/global/blacklists/domains="$(find BL -name domains | tr '\n' ' ')" \ proxy/filter/global/blacklists/urls="$(find BL -name urls | tr '\n' ' ')" # invoke-rc.d squid restart # curl --proxy http://$USER:univention@10.200.27.10:3128/ "http://$(shuf -n 1 global-blacklist-domains)/" -i # curl --proxy http://$USER:univention@10.200.27.10:3128/ "http://$(shuf -n 1 global-blacklist-urls)" -i The blacklist database is successfully created and seems to be loaded: 2015-04-27 16:52:13 [27861] init domainlist /var/lib/ucs-school-webproxy//global-blacklist-domains 2015-04-27 16:52:13 [27861] INFO: loading dbfile /var/lib/ucs-school-webproxy//global-blacklist-domains.db 2015-04-27 16:52:13 [27861] init urllist /var/lib/ucs-school-webproxy//global-blacklist-urls 2015-04-27 16:52:13 [27861] INFO: loading dbfile /var/lib/ucs-school-webproxy//global-blacklist-urls.db
(In reply to Florian Best from comment #8) > This somehow does not work for me. I have no further internet rules set up. > Tried with a teacher, student and Administrator. I fixed a logic bug that has slipped in in a last minute change. Package has been rebuilt.
(In reply to Sönke Schwardt-Krummrich from comment #9) > (In reply to Florian Best from comment #8) > > This somehow does not work for me. I have no further internet rules set up. > > Tried with a teacher, student and Administrator. > > I fixed a logic bug that has slipped in in a last minute change. > Package has been rebuilt. OK, now it is fine! I could not experience any performance problems. The global blacklist overwrites every whitelist. > The filenames have to be relative to /var/lib/ucs-school-webproxy Well, technically this is not enforced. Changelog: OK
UCS@school 4.0 R2 v1 has been released: http://docs.univention.de/release-notes-ucsschool-4.0R2v1-de.html If this error occurs again, please use "Clone This Bug".