Univention Bugzilla – Bug 44489
Error while installing spamassassin
Last modified: 2020-08-12 16:18:11 CEST
While installing univention-spamassassin (as part of the kopano-core installation) the following is logged in umc-module-appcenter.log As the error from the webpage is html formatted, it was displayed in huge letters in the progress bar. 27.04.17 14:37:28.149 MODULE ( PROCESS ) : univention-spamassassin (7.0.1-1.77.201511032351) wird eingerich tet ... 27.04.17 14:37:28.150 MODULE ( PROCESS ) : univention-spamassassin wird konfiguriert. 27.04.17 14:37:28.190 MODULE ( PROCESS ) : univention-spamassassin wird konfiguriert. 27.04.17 14:37:30.300 MODULE ( PROCESS ) : Create mail/antivir/spam 27.04.17 14:37:30.300 MODULE ( PROCESS ) : Create mail/antispam/requiredhits 27.04.17 14:37:30.300 MODULE ( PROCESS ) : Create mail/antispam/learndaily 27.04.17 14:37:30.300 MODULE ( PROCESS ) : Create mail/antispam/rules/autoupdate 27.04.17 14:37:30.300 MODULE ( PROCESS ) : Create mail/antispam/autostart 27.04.17 14:37:30.300 MODULE ( PROCESS ) : Multifile: /etc/postfix/main.cf 27.04.17 14:37:30.300 MODULE ( PROCESS ) : Module: kopano-cfg 27.04.17 14:37:30.300 MODULE ( PROCESS ) : Multifile: /etc/postfix/master.cf 27.04.17 14:37:32.575 MODULE ( PROCESS ) : http: GET http://sa-update.secnap.net/1786640.tar.gz request failed: 404 Not Found: <html> <head><title>404 Not Found</title></head> <body bgcolor="white"> <center><h1>404 Not Found</h1></center> <hr><center>nginx/1.6.2</center> </body> </html>
My guess is, this is the error output from su debian-spamd -c "sa-update --gpghomedir /var/lib/spamassassin/sa-update-keys" in univention-spamassassin/debian/univention-spamassassin.postinst The sa-update.secnap.net mirror seems broken. IMHO it is good to immediately update the SpamAssassin rules, but a network dependent and potentially long blocking operation like that should be forked out of the installation process. I suggest "at now". This would prevent the HTML being displayed.
For TT 2020-06-18/19 we're using AWS with UCS-4.4-3. The Slave itself has no public usable IP address. Because of Bug #50613 no proxy server is configured for https://. PS: `sa-update -v` shows, that http:// is used instead of https://, so even not even a proxy for http:// was configured. During the setup of the DC Slave `univention-spamassassin` is installed. Calling `sa-update` fails as https:// does not work because of the missing proxy. This leads to `spamassassin.service` not being started: # systemctl status spamassassin.service ● spamassassin.service - Perl-based spam filter using text analysis Loaded: loaded (/lib/systemd/system/spamassassin.service; enabled; vendor preset: enabled) Active: failed (Result: exit-code) since Fri 2020-06-19 15:56:18 CEST; 37min ago Process: 3163 ExecStart=/usr/sbin/spamd -d --pidfile=/var/run/spamd.pid $OPTIONS (code=exited, status=255) CPU: 301ms Jun 19 15:56:17 slave1 systemd[1]: spamassassin.service: Unit entered failed state. Jun 19 15:56:17 slave1 systemd[1]: spamassassin.service: Failed with result 'exit-code'. Jun 19 15:56:18 slave1 systemd[1]: spamassassin.service: Service hold-off time over, scheduling restart. Jun 19 15:56:18 slave1 systemd[1]: Stopped Perl-based spam filter using text analysis. Jun 19 15:56:18 slave1 systemd[1]: spamassassin.service: Start request repeated too quickly. Jun 19 15:56:18 slave1 systemd[1]: Failed to start Perl-based spam filter using text analysis. Jun 19 15:56:18 slave1 systemd[1]: spamassassin.service: Unit entered failed state. Jun 19 15:56:18 slave1 systemd[1]: spamassassin.service: Failed with result 'exit-code'. An update from UCS-4.4-3 to UCS-4.4-4 then aborts as it contains an upgrade of `spamassassin`; during the `postinst` the service is restarted, which again fails due the missing https:// proxy. This failure then aborts the whole upgrade. This happend in all 4 environments. Running `sa-update` manually after setting environment variable `https_proxy` through UCR variable `proxy/https` followed by `systemctl restart spamassassin.service` fixed the problem. # ls -al /var/lib/spamassassin/3.004002/updates_spamassassin_org/ insgesamt 8 drwxr-xr-x 2 debian-spamd debian-spamd 4096 Jun 18 11:33 . drwxr-xr-x 3 debian-spamd debian-spamd 4096 Jun 18 11:33 .. # grep -i -e spamassassin -e sa-update /var/log/apt/term.log Vormals nicht ausgewähltes Paket spamassassin wird gewählt. Vorbereitung zum Entpacken von .../85-spamassassin_3.4.2-1~deb9u1_all.deb ... Entpacken von spamassassin (3.4.2-1~deb9u1) ... Vormals nicht ausgewähltes Paket univention-spamassassin wird gewählt. Vorbereitung zum Entpacken von .../90-univention-spamassassin_9.0.0-6A~4.3.0.201803091404_all.deb ... Entpacken von univention-spamassassin (9.0.0-6A~4.3.0.201803091404) ... spamassassin (3.4.2-1~deb9u1) wird eingerichtet ... Erstelle Home-Verzeichnis »/var/lib/spamassassin« nicht. univention-spamassassin (9.0.0-6A~4.3.0.201803091404) wird eingerichtet ... File: /etc/spamassassin/90_univention.cf File: /etc/cron.daily/univention-spamassassin File: /etc/default/spamassassin File: /etc/logrotate.d/univention-spamassassin Updating spamassassin rules... Cannot open file /var/lib/spamassassin/3.004002/updates_spamassassin_org/MIRRORED.BY: No such file or directory at /usr/bin/sa-update line 1600. Jun 18 11:34:24.271 [10704] dbg: generic: SpamAssassin version 3.4.2 Jun 18 11:34:24.271 [10704] dbg: generic: Perl 5.024001, PREFIX=/usr, DEF_RULES_DIR=/usr/share/spamassassin, LOCAL_RULES_DIR=/etc/spamassassin, LOCAL_STATE_DIR=/var/lib/spamassassin Jun 18 11:34:24.378 [10704] dbg: config: using "/etc/spamassassin" for site rules pre files Jun 18 11:34:24.378 [10704] dbg: config: read file /etc/spamassassin/init.pre Jun 18 11:34:24.378 [10704] dbg: config: read file /etc/spamassassin/sa-compile.pre Jun 18 11:34:24.378 [10704] dbg: config: read file /etc/spamassassin/v310.pre Jun 18 11:34:24.378 [10704] dbg: config: read file /etc/spamassassin/v312.pre Jun 18 11:34:24.378 [10704] dbg: config: read file /etc/spamassassin/v320.pre Jun 18 11:34:24.378 [10704] dbg: config: read file /etc/spamassassin/v330.pre Jun 18 11:34:24.379 [10704] dbg: config: read file /etc/spamassassin/v340.pre Jun 18 11:34:24.379 [10704] dbg: config: read file /etc/spamassassin/v341.pre Jun 18 11:34:24.379 [10704] dbg: config: read file /etc/spamassassin/v342.pre Jun 18 11:34:24.379 [10704] dbg: config: using "/var/lib/spamassassin/3.004002" for sys rules pre files Jun 18 11:34:24.379 [10704] dbg: config: using "/var/lib/spamassassin/3.004002" for default rules dir config: no rules were found! Do you need to run 'sa-update'? Updating spamassassin rules... done Restarting spamassassin (via systemctl): spamassassin.serviceJob for spamassassin.service failed because the control process exited with error code. See "systemctl status spamassassin.service" and "journalctl -xe" for details. Vorbereitung zum Entpacken von .../27-spamassassin_3.4.2-1~deb9u3_all.deb ... Entpacken von spamassassin (3.4.2-1~deb9u3) über (3.4.2-1~deb9u1) ... spamassassin (3.4.2-1~deb9u3) wird eingerichtet ... Job for spamassassin.service failed because the control process exited with error code. See "systemctl status spamassassin.service" and "journalctl -xe" for details. invoke-rc.d: initscript spamassassin, action "restart" failed. ● spamassassin.service - Perl-based spam filter using text analysis Loaded: loaded (/lib/systemd/system/spamassassin.service; enabled; vendor preset: enabled) Jun 19 13:27:04 slave1 systemd[1]: spamassassin.service: Unit entered faile…ate. Jun 19 13:27:04 slave1 systemd[1]: spamassassin.service: Failed with result…de'. dpkg: Fehler beim Bearbeiten des Paketes spamassassin (--configure): sa-compile hängt ab von spamassassin; aber: Paket spamassassin ist noch nicht konfiguriert. spamassassin # grep -n . /var/lib/dpkg/info/univention-spamassassin.postinst … 68: if [ -f /etc/cron.daily/spamassassin ] ; then 69: echo "Updating spamassassin rules..." 70: python -c 'import pty; pty.spawn(["/etc/cron.daily/spamassassin"])' 71: echo "Updating spamassassin rules... done" 72: else 73: echo "ERROR: the update of the spamassassin rules failed. Please run 'sa-update' manually." 74: fi Wrong! Wrong! Wrong!
Following Bug #50613 comment 3: The HTTP proxy configuration is not written to a file sourced by systemd: Every service get's a clean environment with only minimal environment variables set. /etc/cron.daily/spamassassin uses `env http_proxy=$http_proxy`, but as it is started from cron, which is started by systemd, the environment variable is missing: # xargs -0n1 -a /proc/`pidof cron`/environ echo LANG=de_DE.UTF-8 PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin INVOCATION_ID=5d9671ba207d4d97917e57dd81aef767 JOURNAL_STREAM=8:14087 READ_ENV=yes # cat /tmp/sa-update.log /usr/bin/sa-update --gpghomedir /var/lib/spamassassin/sa-update-keys HOME=/var/lib/spamassassin http_proxy= PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin LANG= PWD=/ # ucr search --brief ^proxy/ proxy/http: http://localhost:3128 proxy/https: http://localhost:3128 proxy/no_proxy: <empty>
Again: TT 2020-07-01/02
We have two problems here (1) cron does not respect the env variable http_proxy and (2), described by Phillip, univention-system-setup also does not respect http_proxy in usr/lib/univention-system-setup/scripts/30_net/18proxy we correctly set the UCR proxy/http according to the profile, but calling setup-join.sh never gets the env variable so the during the setup (installation and subsequent update) http_proxy is not set apt-get works though, because it uses its own proxy configuration This scenario file can be used to reproduce the problem [Global] # These settings can be overwritten by the instance logfile: autotest-spamassassin.log # ucs-kt-get kvm settings kvm_server: [ENV:KVM_BUILD_SERVER] kvm_user: [ENV:KVM_USER] kvm_dhcp: 1 kvm_interface: eth0 kvm_extra_label: spamassassin-[ENV:UCS_VERSION] kvm_template: [ENV:KVM_TEMPLATE] kvm_ucsversion: [ENV:KVM_UCSVERSION] kvm_architecture: amd64 kvm_memory: 2G recover: 2 environment: UCS_VERSION=[ENV:UCS_VERSION] UCS_TEST_RUN=[ENV:UCS_TEST_RUN] RELEASE_UPDATE=[ENV:RELEASE_UPDATE] ERRATA_UPDATE=[ENV:ERRATA_UPDATE] TARGET_VERSION=[ENV:TARGET_VERSION] JENKINS_WS=[ENV:JOB_URL] [proxy] kvm_template: ucs-master kvm_operating_system: Others command1: . utils.sh && basic_setup . utils.sh && basic_setup_ucs_role Administrator univention [proxy_IP] . utils.sh && import_license univention-install -y univention-squid command2: files: ~/ec2/license/license.secret /etc/ [master] profile: hostname="master" domainname="autotest.local" windows/domain="AUTOTEST" ldap/base="dc=autotest,dc=local" root_password="univention" locale/default="de_DE.UTF-8:UTF-8" packages_install="univention-mail-server" packages_remove="" server/role="domaincontroller_master" nameserver1="[proxy_IP]" update/system/after/setup="True" proxy/http="http://[proxy_IP]:3128" command1: # set ucr set squid/allowfrom='10.207.0.0/16' on the squid server ! . utils.sh; basic_setup . utils.sh; add_tech_key_authorized_keys . utils.sh; rotate_logfiles ucr set nameserver/external=true echo -e '#!/bin/sh\nset -x\nroute del default' > /usr/lib/univention-system-setup/scripts/30_net/99route && chmod 755 /usr/lib/univention-system-setup/scripts/30_net/99route echo -e '#!/bin/sh\nset -x\nroute add default gw 10.207.0.1' > /usr/lib/univention-system-setup/scripts/90_postjoin/99route && chmod 755 /usr/lib/univention-system-setup/scripts/90_postjoin/99route # simple workaround #sed -i 's|\(30_net\)|\1\n. /etc/profile|' /usr/lib/univention-system-setup/scripts/setup-join.sh . utils.sh; run_setup_join systemctl status spamassassin.service command2: files: ~/ec2/scripts/activate-errata-test-scope.sh /root/ ~/ec2/license/license.secret /etc/ after the installation spamassasin is in a "failed" state systemctl status spamassassin.service stdout: exit-code) since Thu 2020-08-06 16:46:11 CEST; 8min ago CPU: 195ms Aug 06 16:46:11 master systemd[1]: spamassassin.service: Control process exited, code=exited status=255 Aug 06 16:46:11 master systemd[1]: Failed to start Perl-based spam filter using text analysis. and subsequent update may fail. If you enable the "workaround" (. /etc/profile to get new proxy environment variables, i'm not sure is this is a valid fix, just prove the point) http_poxy is exported in setup-join.sh and the spamassassin can be successfully installed.
https://forge.univention.org/bugzilla/show_bug.cgi?id=50613#c3 describes some details
7bae3d8f6a Bug #44489: yaml 5b17d1961b Bug #44489: changelog 009661d3d4 Bug #44489: Make http(s)_proxy env variables usable in spamassassin cronjob Successful build Package: univention-spamassassin Version: 10.0.0-4A~4.4.0.202008062105 Branch: ucs_4.4-0 Scope: errata4.4-5 User: jbremer -------------------- We added the UCR Variables http(s)_proxy to /etc/default/spamassassin which is executed by the cron job. When using Felix cfg and adding the line . utils.sh; jenkins_updates so that it uses the new univention-spamassassin package, the test succeeds and the spamassassin service is running. Even though this is not a general solution, it fixes this specific problem.
OK works, but could you remove the "export" and add a comment before the proxy settings
b6c790b5cc Make env vars http(s)_proxy usable by spamassassin cron job ce9aafdbcd yaml Package: univention-spamassassin Version: 10.0.0-5A~4.4.0.202008101131
732e643f3f register all ucr variables da3787befd yaml univention-spamassassin 10.0.0-6A~4.4.0.202008101143
OK - works -> unset https_proxy -> unset http_proxy -> bash -x /etc/cron.daily/spamassassin + CRON=0 + test -f /etc/default/spamassassin + . /etc/default/spamassassin ++ ENABLED=1 ++ OPTIONS='--create-prefs --max-children 10 --helper-home-dir' ++ PIDFILE=/var/run/spamd.pid ++ CRON=1 ++ http_proxy=http://10.200.7.160:3128 ++ https_proxy=http://10.200.7.160:3128 + test -x /usr/bin/sa-update + test -x /etc/init.d/spamassassin + command -v gpg + '[' 1 = 0 ']' + '[' '!' -t 0 ']' + umask 022 + env -i LANG=de_DE.UTF-8 PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/bin/X11 http_proxy=http://10.200.7.160:3128 start-stop-daemon --chuid debian-spamd:debian-spamd --start --exec /usr/bin/sa-update -- --gpghomedir /var/lib/spamassassin/sa-update-keys + case $? in OK - yaml Ok - test with cfg
<https://errata.software-univention.de/#/?erratum=4.4x709>