Bug 45968 - update postfix in 4.2 to enable upgrade to 4.3
update postfix in 4.2 to enable upgrade to 4.3
Status: CLOSED FIXED
Product: UCS
Classification: Unclassified
Component: Mail
UCS 4.3
Other Linux
: P5 normal (vote)
: UCS 4.3
Assigned To: Daniel Tröder
Sönke Schwardt-Krummrich
: interim-2
Depends on: 45963 46460
Blocks: 47117
  Show dependency treegraph
 
Reported: 2018-01-02 14:28 CET by Daniel Tröder
Modified: 2018-06-04 08:29 CEST (History)
2 users (show)

See Also:
What kind of report is it?: Release Management
What type of bug is this?: ---
Who will be affected by this bug?: ---
How will those affected feel about the bug?: ---
User Pain:
Enterprise Customer affected?:
School Customer affected?:
ISV affected?:
Waiting Support:
Flags outvoted (downgraded) after PO Review:
Ticket number:
Bug group (optional):
Max CVSS v3 score:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Daniel Tröder univentionstaff 2018-01-02 14:28:12 CET
In a UCS 4.3 updated from 4.2 Postfix 3 doesn't start because the file /etc/postfix/dynamicmaps.cf contains errors. (In a new installation it is OK.)

Jan  2 11:51:35 m60 postfix/smtpd[17884]: error: unsupported dictionary type: ldap
Jan  2 11:51:35 m60 postfix/smtpd[17884]: warning: unsupported dictionary type: ldap (no/postfix-ldap.so: No such f


The problem is known (https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=859805) and the update procedure has been fixed in jessie in version 2.11.3-1+deb8u2.

Upgrade Postfix in UCS 4.2 to allow a successful upgrade to UCS 4.3.
Comment 1 Daniel Tröder univentionstaff 2018-01-02 15:22:50 CET
dtroeder@dimma:~$ repo_admin.py -U -r 4.2-0-0 -s errata4.2-3 -d jessie -p postfix

dtroeder@dimma:~$ repo_stat.py postfix
[..]
Version 2.11.3-1	Rev 97588	Date 2017-06-07 02:37:38
	Release 4.2-0-0
Version 2.11.3-1+deb8u2	Rev 122798	Date 2018-01-02 15:16:47
	Release 4.2-0-0	Scope errata4.2-3
Version 3.1.6-0+deb9u1	Rev 118271	Date 2017-11-04 13:41:58
	Release 4.3-0-0

dtroeder@dimma:~$ b42-scope errata4.2-3 postfix

Package: postfix
Version: 2.11.3-1+deb8u2A~4.2.0.201801021517
Branch: ucs_4.2-0
Scope: errata4.2-3

[..]

gcc -shared -Wl,-soname,libpostfix-global.so.1 -o libglobal.a abounce.o anvil_clnt.o been_here.o bounce.o bounce_log.o canon_addr.o cfg_parser.o cleanup_strerror.o cleanup_strflags.o clnt_stream.o conv_time.o db_common.o debug_peer.o debug_process.o defer.o deliver_completed.o deliver_flock.o deliver_pass.o deliver_request.o dict_proxy.o domain_list.o dot_lockfile.o dot_lockfile_as.o dsb_scan.o dsn.o dsn_buf.o dsn_mask.o dsn_print.o dsn_util.o ehlo_mask.o ../util/dict_cdb.c:398:2: warning: ignoring return value of 'ftruncate', declared with attribute warn_unused_result
 [-Wunused-result]
  ftruncate(fd, 0);
  ^
gcc -Wmissing-prototypes -Wformat -Wno-comment -I. -I../../include -DDEBIAN -DMAX_DYNAMIC_MAPS -DHAS_PCRE -DHAS_LDA
P -DUSE_LDAP_SASL -DHAS_SQLITE -DMYORIGIN_FROM_FILE -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -DHAS_CDB -DHAS_MYSQ
L -I/usr/include/mysql -DHAS_PGSQL -I/usr/include/postgresql -DHAS_SQLITE -I/usr/include -DHAS_SSL -I/usr/include/o
penssl -DUSE_SASL_AUTH -I/usr/include/sasl -DUSE_CYRUS_SASL -DUSE_TLS -fPIC  -O2 -I. -I../../include -DLINUX3 -c di
ct_pgsql.c
gcc -Wmissing-prototypes -Wformat -Wno-comment -I. -I../../include -DDEBIAN -DMAX_DYNAMIC_MAPS -DHAS_PCRE -DHAS_LDA
P -DUSE_LDAP_SASL -DHAS_SQLITE -DMYORIGIN_FROM_FILE -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -DHAS_CDB -DHAS_MYSQ
L -I/usr/include/mysql -DHAS_PGSQL -I/usr/include/postgresql -DHAS_SQLITE -I/usr/include -DHAS_SSL -I/usr/include/o
penssl -DUSE_SASL_AUTH -I/usr/include/sasl -DUSE_CYRUS_SASL -DUSE_TLS -fPIC  -O2 -I. -I../../include -DLINUX3 -c di
ct_sqlite.c
gcc -shared -Wl,-soname,libpostfix-global.so.1 -o libglobal.a abounce.o anvil_clnt.o been_here.o bounce.o bounce_lo
g.o canon_addr.o cfg_parser.o cleanup_strerror.o cleanup_strflags.o clnt_stream.o conv_time.o db_common.o debug_pee
r.o debug_process.o defer.o deliver_completed.o deliver_flock.o deliver_pass.o deliver_request.o dict_proxy.o domai
n_list.o dot_lockfile.o dot_lockfile_as.o dsb_scan.o dsn.o dsn_buf.o dsn_mask.o dsn_print.o dsn_util.o ehlo_mask.o 
ext_prop.o file_id.o flush_clnt.o header_opts.o header_token.o input_transp.o int_filt.o is_header.o log_adhoc.o mail_addr.o mail_addr_crunch.o mail_addr_find.o mail_addr_map.o mail_command_client.o mail_command_server.o mail_conf.o mail_conf_bool.o mail_conf_int.o mail_conf_long.o mail_conf_raw.o mail_conf_str.o mail_conf_time.o mail_connect.o mail_copy.o mail_date.o mail_dict.o mail_error.o mail_flush.o mail_open_ok.o mail_params.o mail_pathname.o mail_queue.o mail_run.o mail_scan_dir.o mail_stream.o mail_task.o mail_trigger.o maps.o mark_corrupt.o match_parent_style.o mbox_conf.o mbox_open.o mime_state.o mkmap_db.o mkmap_dbm.o mkmap_lmdb.o mkmap_open.o mkmap_sdbm.o msg_stats_print.o msg_stats_scan.o mynetworks.o mypwd.o namadr_list.o off_cvt.o opened.o own_inet_addr.o pipe_command.o post_mail.o quote_821_local.o quote_822_local.o rcpt_buf.o rcpt_print.o rec_attr_map.o rec_streamlf.o rec_type.o recipient_list.o record.o remove.o resolve_clnt.o resolve_local.o rewrite_clnt.o scache_clnt.o scache_multi.o scache_single.o sent.o smtp_stream.o split_addr.o string_list.o strip_addr.o sys_exits.o timed_ipc.o tok822_find.o tok822_node.o tok822_parse.o tok822_resolve.o tok822_rewrite.o tok822_tree.o trace.o user_acl.o valid_mailhost_addr.o verify.o verify_clnt.o verp_sender.o wildcard_inet_addr.o xtext.o delivered_hdr.o fold_addr.o header_body_checks.o mkmap_proxy.o data_redirect.o match_service.o mail_conf_nint.o addr_match_list.o mail_conf_nbool.o smtp_reply_footer.o safe_ultostr.o verify_sender_addr.o dict_memcache.o mail_version.o memcache_proto.o server_acl.o mkmap_fail.o haproxy_srvr.o ../../lib/libutil.a -lssl -lcrypto -lsasl2 -lpthread -L/var/build/temp/tmp.74h3vuLJVc/postfix-2.11.3/debian -ldb -lnsl -lresolv
gcc -shared -Wl,-soname,dict_cdb.so -o dict_cdb.so dict_cdb.o mkmap_cdb.o -lcdb -L. -lutil
gcc -shared -Wl,-soname,dict_mysql.so -o dict_mysql.so dict_mysql.o -lmysqlclient -L. -lutil -lglobal
./libglobal.a: file not recognized: File truncated
collect2: error: ld returned 1 exit status
Makefile:161: recipe for target 'dict_mysql.so' failed
make: *** [dict_mysql.so] Error 1
Comment 2 Sönke Schwardt-Krummrich univentionstaff 2018-01-02 21:33:06 CET
(In reply to Daniel Tröder from comment #0)
> The problem is known
> (https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=859805) and the update
> procedure has been fixed in jessie in version 2.11.3-1+deb8u2.
> 
> Upgrade Postfix in UCS 4.2 to allow a successful upgrade to UCS 4.3.

This way, we have to ensure that the user has installed a certain errata level with a newer postfix version under UCS 4.2.

Have you considered to install a new version of dynamicmaps.cf in the preup.sh of univention-installer? This way, we don't have urge the user to install a UCS 4.2 errata.
Comment 3 Daniel Tröder univentionstaff 2018-01-03 10:55:23 CET
(In reply to Sönke Schwardt-Krummrich from comment #2)
> (In reply to Daniel Tröder from comment #0)
> > The problem is known
> > (https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=859805) and the update
> > procedure has been fixed in jessie in version 2.11.3-1+deb8u2.
> > 
> > Upgrade Postfix in UCS 4.2 to allow a successful upgrade to UCS 4.3.
> 
> This way, we have to ensure that the user has installed a certain errata
> level with a newer postfix version under UCS 4.2.
I thought that a safe assumption...

> Have you considered to install a new version of dynamicmaps.cf in the
> preup.sh of univention-installer? This way, we don't have urge the user to
> install a UCS 4.2 errata.
As the content of dynamicmaps.cf depends on the installed (additional) Postfix-packages (postfix-ldap, postfix-mysql, ...) this means duplicating the maintainer code from the 2.11.3-1+deb8u2 package... which is actually not so much...

Turns out the the Postfix3 [extension] packages edit dynamicmaps.cf in their respective postinst→configure part. So removing the file prior to the dist-upgrade lets them recreate it when they are upgraded.

Successfully tested upgrade with postfix-ldap, postfix-cdb, postfix-mysql, postfix-pcre, postfix-pgsql packages installed. After the upgrade:

--------------------------------------------------------
root@m52:~# diff /etc/postfix/dynamicmaps.cf /etc/postfix/dynamicmaps.cf.ucs42
1,6c1,11
< pcre	postfix-pcre.so	dict_pcre_open	
< mysql	postfix-mysql.so	dict_mysql_open	
< ldap	postfix-ldap.so	dict_ldap_open	
< cdb	postfix-cdb.so	dict_cdb_open	mkmap_cdb_open
< pgsql	postfix-pgsql.so	dict_pgsql_open	
< sqlite	postfix-sqlite.so	dict_sqlite_open	
---
> # Postfix dynamic maps configuration file.
> #
> #type	location of .so file			open function	(mkmap func)
> #====	================================	=============	============
> tcp	/usr/lib/postfix/dict_tcp.so		dict_tcp_open	
> sqlite	/usr/lib/postfix/dict_sqlite.so		dict_sqlite_open	
> ldap	/usr/lib/postfix/dict_ldap.so		dict_ldap_open	
> cdb	/usr/lib/postfix/dict_cdb.so		dict_cdb_open	mkmap_cdb_open
> mysql	/usr/lib/postfix/dict_mysql.so		dict_mysql_open	
> pcre	/usr/lib/postfix/dict_pcre.so		dict_pcre_open	
> pgsql	/usr/lib/postfix/dict_pgsql.so		dict_pgsql_open	

(The "tcp" map is now internal and not dynamically linked anymore.)
--------------------------------------------------------

The actual preup script was not tested, because the new version hadn't been uploaded to the repo yet.

4f74c637: remove /etc/postfix/dynamicmaps.cf in preup.sh to let Postfix3 (extension) packages recreate it with new format

univention-updater 13.0.0-10A~4.3.0.201801030933
Comment 4 Felix Botner univentionstaff 2018-01-15 13:57:09 CET
please redirect the output to the log file ( >>"$UPDATER_LOG" 2>&1 )
Comment 5 Daniel Tröder univentionstaff 2018-01-15 16:41:02 CET
(In reply to Felix Botner from comment #4)
> please redirect the output to the log file ( >>"$UPDATER_LOG" 2>&1 )
[4.3-0 1427915337] Bug #45968: redirect the output to log file

univention-updater 13.0.1-4A~4.3.0.201801151632
Comment 6 Florian Best univentionstaff 2018-01-26 14:15:49 CET
It seems the mail tests are currently failing due to a old config file in postfix.
Comment 7 Florian Best univentionstaff 2018-01-26 14:22:45 CET
It seems the preup.sh was executed twice. Probably the code should be able to handle this?
Comment 8 Daniel Tröder univentionstaff 2018-01-29 10:58:50 CET
[4.3-0 2a23cb4a7ae] Bug #45968: remove dynamicmaps.cf only if it contains old paths

univention-updater  13.0.1-7A~4.3.0.201801291042
Comment 9 Sönke Schwardt-Krummrich univentionstaff 2018-02-07 17:15:47 CET
REOPEN: code change (see below)
OK: functional change
OK: tests
OK: changelog entry
OK: changelog xml
OK: package built

Only a small issue/change:
The preup.sh does not check if the backup file already exists and might overwrite the target file. Suggestion for a fix (please update changelog-4.3-0.xml accordingly):

--- a/base/univention-updater/script/preup.sh
+++ b/base/univention-updater/script/preup.sh
@@ -472,7 +472,8 @@ esac
 if grep -q 'usr/lib/postfix' /etc/postfix/dynamicmaps.cf; then
 	echo "Removing /etc/postfix/dynamicmaps.cf. Creating backup in"
 	echo "/etc/postfix/dynamicmaps.cf.postfix2."
-	mv -fv /etc/postfix/dynamicmaps.cf /etc/postfix/dynamicmaps.cf.postfix2 >>"$UPDATER_LOG" 2>&1
+	fn="$(mktemp /etc/postfix/dynamicmaps.cf.backup-postfix2.XXXXXXXX)"
+	mv -fv /etc/postfix/dynamicmaps.cf "$fn" >>"$UPDATER_LOG" 2>&1
 fi
 
 # Bug 45935: backup squid conf before update
Comment 10 Daniel Tröder univentionstaff 2018-02-13 09:55:20 CET
[4.3-0 be366cac64] Bug #45968: use random filename for backup
[4.3-0 540e43fb2f] Bug #45968: update changelog

univention-updater (13.0.1-13)
Comment 11 Felix Botner univentionstaff 2018-02-14 10:02:38 CET
-> univention-upgrade  --ignoressh --ignoreterm --disable-app-updates
...
Checking for space on /: OK
Checking for package status: OK
Checking LDAP schema: OK
Removing /etc/postfix/dynamicmaps.cf. Creating backup in
/etc/postfix/dynamicmaps.cf.backup-postfix2.iuEEvO26.
...


please redirect the "Removing ...postfix..." output to the updater.log
Comment 12 Daniel Tröder univentionstaff 2018-02-14 11:00:10 CET
[4.3-0 210991247e] Bug #45968: redirect output to logfile

univention-updater (13.0.1-16)
Comment 13 Sönke Schwardt-Krummrich univentionstaff 2018-02-25 16:40:20 CET
(In reply to Daniel Tröder from comment #10)
> [4.3-0 be366cac64] Bug #45968: use random filename for backup
> [4.3-0 540e43fb2f] Bug #45968: update changelog
> [4.3-0 210991247e] Bug #45968: redirect output to logfile
> 
> univention-updater (13.0.1-16)
OK: code review
Comment 14 Stefan Gohmann univentionstaff 2018-03-14 14:38:12 CET
UCS 4.3 has been released:
 https://docs.software-univention.de/release-notes-4.3-0-en.html
 https://docs.software-univention.de/release-notes-4.3-0-de.html

If this error occurs again, please use "Clone This Bug".