Bug 37031 - Content scanner could block preup.sh
Content scanner could block preup.sh
Status: CLOSED FIXED
Product: UCS
Classification: Unclassified
Component: Update - univention-updater
UCS 3.2
Other Linux
: P5 normal (vote)
: UCS 3.2-4-errata
Assigned To: Philipp Hahn
Felix Botner
:
Depends on:
Blocks: 37345
  Show dependency treegraph
 
Reported: 2014-11-26 08:59 CET by Tim Petersen
Modified: 2014-12-17 12:53 CET (History)
2 users (show)

See Also:
What kind of report is it?: ---
What type of bug is this?: ---
Who will be affected by this bug?: ---
How will those affected feel about the bug?: ---
User Pain:
Enterprise Customer affected?:
School Customer affected?:
ISV affected?:
Waiting Support:
Ticket number:
Bug group (optional):
Max CVSS v3 score:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Tim Petersen univentionstaff 2014-11-26 08:59:09 CET
In a customer environment, dansguardian blocks *.sh *.sh.gpg (which is default) - so the updater fails while preup verification.

We should make this more transparent - the updater should recognize the situation and show hint show to disable content scanner/proxy instead.
Comment 1 Stefan Gohmann univentionstaff 2014-11-27 06:02:13 CET
(In reply to Tim Petersen from comment #0)
> We should make this more transparent - the updater should recognize the
> situation and show hint show to disable content scanner/proxy instead.

Or the updater could at least give a hint about the possibility.
Comment 2 Philipp Hahn univentionstaff 2014-12-01 15:09:47 CET
This is caused by a broken DansGuardian configuration, which return "200 OK" even for a filtered URL. AFAIK "403 FORBIDDEN" should be returned, as repeating the request won't fix the problem. See <http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html>

This mis-configuration will break any automatic tool which depends on proper use of the status code, because parsing a text response is too error prone.

The updater check the script file to start with "#!" and the signature file for "BEGIN PGP SIGNATURE" to detect any proxy transparently returning "text/html"instead of "text/x-sh" or "text/plain" (or whatever is currently configured on the repository server.)


r56354 | Bug #37031 Updater: Detect broken Dansguardian proxy
r56353 | Bug #37031 Updater: Detect broken Dansguardian proxy

Package: univention-updater
Version: 10.0.51-2.1339.201412011456
Branch: ucs_4.0-0
Scope: errata4.0-0

Package: univention-updater
Version: 9.0.44-5.1340.201412011502
Branch: ucs_3.2-0
Scope: errata3.2-4

r56355 | Bug #37031 Updater: Detect broken Dansguardian proxy YAML
Comment 3 Felix Botner univentionstaff 2014-12-10 17:20:32 CET
1.

dansguardian by default also blocks x-gzip files (Packages.gz). In this situation the updater can't create a proper repository list (sources.list is empty) but does not fail, sets version/version and claims that the update was successful (fastest update to 4.0-0 i have ever seen). This should be handled too.

2. (removed x-gzip from lists/defaultgroup-bannedmimetypelist on the proxy)

updater ->

Update to = 4.0-0
Traceback (most recent call last):
  File "/usr/share/univention-updater/univention-updater", line 597, in <module>
    msg = 'Update aborted due to configuration error: %s' % e
  File "/usr/lib/pymodules/python2.6/univention/updater/errors.py", line 90, in __str__
    return "Proxy configuration error: %s" % self.args[1]
IndexError: tuple index out of range

3. 

/usr/share/pyshared/univention/updater/tools.py +1731 and +1737

raise ProxyError("Failed to fetch '%s' - maybe blocked by a proxy?")

missing value for '%s'
Comment 4 Philipp Hahn univentionstaff 2014-12-15 11:56:02 CET
(In reply to Felix Botner from comment #3)
> 1.
> dansguardian by default also blocks x-gzip files (Packages.gz). In this
> situation the updater can't create a proper repository list (sources.list is
> empty) but does not fail, sets version/version and claims that the update
> was successful (fastest update to 4.0-0 i have ever seen). This should be
> handled too.

DansGuardian is lying to every http using application and transparently modifying the requested data. By not modifying the HTTP return code it is breaking any non-human-using application, as adding content inspection is insane. Status codes are for automatic procession and DansGuardian is breaking that.
The updater is only checking for the existence of the URL - it is not downloading it and thus can't check it's content! I now use size=0 as an indicator for the existence of a broken DansGuardian configuration
r56803 | Bug #37031 Up: Detect broken DansGuardian proxy

DansGuardian also breaks our App-Center: http://appcenter.software-univention.de/meta-inf/4.0/index.json.gz

See Bug #32387.


> 2. (removed x-gzip from lists/defaultgroup-bannedmimetypelist on the proxy)
...
>     return "Proxy configuration error: %s" % self.args[1]
...
> 3. 
...
> raise ProxyError("Failed to fetch '%s' - maybe blocked by a proxy?")

r56680 | Bug #37031 Updater: Detect broken Dansguardian proxy
 Pass two arguments to ProxyError(uri, reason)

Package: univention-updater
Version: 10.0.51-10.1355.201412151124
Branch: ucs_4.0-0
Scope: errata4.0-0

r56804 | Bug #37031 Up: Detect broken DansGuardian proxy YAML
 2014-12-01-univention-updater.yaml
Comment 5 Philipp Hahn univentionstaff 2014-12-15 13:42:55 CET
4.0-0 forward port cloned to Bug #37345

3.2-4:
r56829 | Bug #37031 Up: Detect broken DansGuardian proxy 3.2

r56830 | Bug #37031 Up: Detect broken DansGuardian proxy YAML
 2014-12-01-univention-updater.yaml
Comment 6 Felix Botner univentionstaff 2014-12-15 15:30:44 CET
OK - standard dansguardian

Error: Update aborted due to configuration error: Proxy configuration error: http://univention-repository.knut.univention.de/4.0/maintained/4.0-0/all/Packages.gz download blocked by proxy?
exitcode of univention-updater: 1


OK - dansguardian with application/x-gzip and *.gz

Error: Update aborted due to configuration error: Proxy configuration error: http://univention-repository.knut.univention.de/4.0/maintained/4.0-0/all/preup.sh download blocked by proxy?
exitcode of univention-updater: 1

OK - dansguardian +with *.sh

update to UCS 4.0 works

OK - errata4.0-0 + YAML
OK - errata3.2-4 + YAML
Comment 7 Moritz Muehlenhoff univentionstaff 2014-12-17 12:53:55 CET
http://errata.univention.de/ucs/3.2/253.html