Bug 42249 - support virtual mail aliases
support virtual mail aliases
Status: CLOSED FIXED
Product: UCS
Classification: Unclassified
Component: Mail
UCS 4.2
Other Linux
: P5 enhancement (vote)
: UCS 4.2-0-errata
Assigned To: Daniel Tröder
Sönke Schwardt-Krummrich
:
Depends on:
Blocks: 38672 44705 44948 44949
  Show dependency treegraph
 
Reported: 2016-09-05 10:40 CEST by Daniel Tröder
Modified: 2017-08-07 10:13 CEST (History)
8 users (show)

See Also:
What kind of report is it?: Bug Report
What type of bug is this?: 1: Cosmetic issue or missing function but workaround exists
Who will be affected by this bug?: 2: Will only affect a few installed domains
How will those affected feel about the bug?: 2: A Pain – users won’t like this once they notice it
User Pain: 0.023
Enterprise Customer affected?: Yes
School Customer affected?:
ISV affected?: Yes
Waiting Support:
Flags outvoted (downgraded) after PO Review:
Ticket number: 2016062321000118
Bug group (optional): External feedback
Max CVSS v3 score:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Daniel Tröder univentionstaff 2016-09-05 10:40:32 CEST
UCS currently does not support creation of virtual mail aliases (only user@fqdn using mail/alias/.* for typical system addresses).

A simple UCR based method would be:
$ ucr set mail/virtual_alias/troeder@univention.de=daniel@troeder.de,daniel@gmail.com
which creates a line in /etc/postfix/virtual:
---
troeder@univention.de daniel@troeder.de,daniel@gmail.com
---
It'd have to run 
$ postmap /etc/postfix/virtual
afterwards, so a UCR module would be necessary.


The other option, which would be more dynamic, would also work in large-scale installations and would be more in line with the rest of the UCS mail stack, would be to extend the LDAP schema and add an entry to virtual_alias_maps. Management of such aliases would work through a UDM and a UMC module.


This feature has been requested multiple times, last time in Ticket #2016062321000118.
Comment 1 Philipp Hahn univentionstaff 2016-09-05 11:43:35 CEST
WORKS-FOR-ME: UCRV mail/postfix/virtual/enabled from Bug #22369
but it seems to be undocumented - ask Sönke
Comment 2 Daniel Tröder univentionstaff 2016-09-05 16:06:02 CEST
(In reply to Philipp Hahn from comment #1)
> WORKS-FOR-ME: UCRV mail/postfix/virtual/enabled from Bug #22369
> but it seems to be undocumented - ask Sönke
mail/postfix/virtual/enabled enables virtual user accounts: addresses with domains other than $(hostname -f), but only domains and mailboxes can be managed by UCS-means, not alias of the form "<user>@<some-domain> → <some>@<where-else>".
Comment 4 Sönke Schwardt-Krummrich univentionstaff 2016-09-13 14:00:46 CEST
(In reply to Daniel Tröder from comment #2)
> (In reply to Philipp Hahn from comment #1)

> > WORKS-FOR-ME: UCRV mail/postfix/virtual/enabled from Bug #22369
> > but it seems to be undocumented - ask Sönke
> mail/postfix/virtual/enabled enables virtual user accounts: addresses with
> domains other than $(hostname -f), but only domains and mailboxes can be
> managed by UCS-means, not alias of the form "<user>@<some-domain> →
> <some>@<where-else>".

The UCS mailsystem has to be responsible for <some-domain> (otherwise the mails would only be relayed). If this is the case, you can use mailinglist objects. This is currently the best and simpliest solution:
Mailinglist: 
foo@univention.de ==> troeder@univention.de, daniel@troeder.de, daniel@gmail.com

Only drawback: it's currently not obvious, which address is connected to which local user.

Btw: a combination of mailinglists and mailAlternativeAddress is currently NOT possible! Only one of those 2 is evaluated.
Comment 5 Daniel Tröder univentionstaff 2016-09-19 08:31:12 CEST
This does not solve the problem of the requests. They don't want a mailing list with a role account address. They want mails to an existing user accounts email address, forwarded to additional external addresses.
Comment 6 Stefan Gohmann univentionstaff 2016-12-13 08:10:39 CET
The Enterprise Customer affected flag is set but neither a Ticket number is referenced nor a Customer ID is set. Please set a Ticket number or a Customer ID. Otherwise the Enterprise Customer affected flag will be reset.
Comment 7 Ralf Becker 2017-02-09 12:38:57 CET
Mein Kunde, die Grüne Landtagsfraktion in Berlin, hat mich nochmal darauf angesprochen, ob es dafür inzwischen einen Termin gäbe.

Geht konkret darum beliebige Forwards über ein multivalued LDAP Attribute zu ermöglichen.

Ralf
Comment 8 Ralf Becker 2017-04-06 15:42:55 CEST
Any news on this? 

Is it already in 4.2?

Ralf
Comment 9 Daniel Tröder univentionstaff 2017-04-20 15:44:35 CEST
Implementation idea:

Extend LDAP schema:
* multi-value email attribute
* copy-to-self bool attribute

UDM hook
* when storing into LDAP:
** append mailPrimaryAddress if copy-to-self is True
** set copy-to-self to True if mailPrimaryAddress is in email list
* when loading from LDAP always remove mailPrimaryAddress from email list
Comment 10 Ralf Becker 2017-04-24 10:20:03 CEST
That approach certainly works for us and is sufficient.

An other approach we support is a distinct (multi-value) attribute for aliases, then you don't need all the logic of adding and removing self address. Thought you still need a copy-to-self or forwardOnly bool attribute, in case user/email-address should have no mailbox at all, but just forward (and you want to support forwards without mailbox).

Is there any ETA yet for that feature request?

Ralf
Comment 11 Daniel Tröder univentionstaff 2017-04-26 13:27:59 CEST
r78944 - univention-ldap: add support for virtual mail aliases
r78945 - univention-directory-manager-modules: add support for virtual mail aliases
r78946 - univention-mail-postfix: add support for virtual mail aliases, separate maps for virtual mailboxes and virtual aliases
r78947: advisories

Package: univention-ldap
Version: 13.0.7-6A~4.2.0.201704261322
Branch: ucs_4.2-0
Scope: errata4.2-0

Package: univention-directory-manager-modules
Version: 12.0.16-7A~4.2.0.201704261323
Branch: ucs_4.2-0
Scope: errata4.2-0

Package: univention-mail-postfix
Version: 11.0.1-1A~4.2.0.201704261324
Branch: ucs_4.2-0
Scope: errata4.2-0
Comment 12 Daniel Tröder univentionstaff 2017-04-28 14:54:32 CEST
r78993: Had to remove the dependency on the package that contains the LDAP schema to prevent a circular dependency

Package: univention-directory-manager-modules
Version: 12.0.17-3A~4.2.0.201704281453
Branch: ucs_4.2-0
Scope: errata4.2-0
Comment 13 Felix Botner univentionstaff 2017-05-04 13:00:50 CEST
Package: univention-mail-postfix
Depends:
...
+ univention-ldap-config (>= 13.0.7-6),


This leads to the installation of univention-ldap-config and slapd during the update of a memberserver.

Why a dependency to univention-ldap-config. The schema files in this package are evaluated only on the UCS master (backup?). All other servers get the ldap schema via the ldap replication. 

univention-ldap-config
Description: UCS - common LDAP configuration
 This package contains the general configuration for the
 OpenLDAP server such as the base LDIF file and the
 schema.
Pre-Depends: slapd
Comment 14 Daniel Tröder univentionstaff 2017-05-05 08:50:41 CEST
Yes - well... now UDM and Postfix will fail, if the master is not updated first...

r79102: remove dependency on univention-ldap-config
r79103: advisory update

univention-mail-postfix 11.0.1-2A~4.2.0.201705050845
Comment 15 Felix Botner univentionstaff 2017-05-05 09:18:27 CEST
(In reply to Daniel Tröder from comment #14)
> Yes - well... now UDM and Postfix will fail, if the master is not updated
> first...

What does this mean "will fail", postfix is longer usable or postfix can't handle the new feature if the master is not updated?
Comment 16 Daniel Tröder univentionstaff 2017-05-05 09:46:14 CEST
Just tested it: Postfix will work, but won't send mails to the aliases, as the result_attribute in /etc/postfix/ldap.external_aliases will be unknown -> no results (errors seem to be silently ignored) -> no aliases.

In my short test (on a dc master!) it seems, that UDM will to not crash and allow modifying existing users, but modifications to the alias field will result in:
---------------------------------------------------------------
The LDAP object could not be saved: LDAP Error Undefined attribute type: univentionExternalMailAlias: attribute type undefined
---------------------------------------------------------------

I guess this is acceptable. So marking as resolved.

My tests were made on a dc master! Please test this in a distributed environment.
Comment 17 Sönke Schwardt-Krummrich univentionstaff 2017-05-29 14:04:00 CEST
+attributetype ( 1.3.6.1.4.1.10176.1010.1.47 NAME 'univentionExternalMailAlias'
+       SUBSTR caseIgnoreSubstringsMatch
+       DESC 'External mail addresses to forward the users emails to'
+       EQUALITY caseIgnoreIA5Match
+       SYNTAX 1.3.6.1.4.1.1466.115.121.1.26 )

IA5 string is not sufficient for mail adresses. This should be UTF-8:
→ 1.3.6.1.4.1.1466.115.121.1.15

> In addition to the above ASCII characters, international characters above 
> U+007F, encoded as UTF-8, are permitted by RFC 6531, though mail systems may 
> restrict which characters to use when assigning local-parts.
Source: https://en.wikipedia.org/wiki/Email_address

→ REOPEN

The UCS manual has to be updated, too.

→ REOPEN
Comment 18 Daniel Tröder univentionstaff 2017-05-29 16:04:34 CEST
(In reply to Sönke Schwardt-Krummrich from comment #17)
> +attributetype ( 1.3.6.1.4.1.10176.1010.1.47 NAME
> 'univentionExternalMailAlias'
> +       SUBSTR caseIgnoreSubstringsMatch
> +       DESC 'External mail addresses to forward the users emails to'
> +       EQUALITY caseIgnoreIA5Match
> +       SYNTAX 1.3.6.1.4.1.1466.115.121.1.26 )
> 
> IA5 string is not sufficient for mail adresses. This should be UTF-8:
> → 1.3.6.1.4.1.1466.115.121.1.15
> 
> > In addition to the above ASCII characters, international characters above 
> > U+007F, encoded as UTF-8, are permitted by RFC 6531, though mail systems may 
> > restrict which characters to use when assigning local-parts.
> Source: https://en.wikipedia.org/wiki/Email_address
> 
> → REOPEN
r79726: support UTF-8 in email aliases
r79728: advisory update

univention-ldap 13.0.7-8A~4.2.0.201705291603

> The UCS manual has to be updated, too.
> 
> → REOPEN
Split into bug: Bug #44705
Comment 19 Sönke Schwardt-Krummrich univentionstaff 2017-06-01 18:12:52 CEST
* /etc/postfix/ldap.external_aliases has been added
  query_filter = (&(objectClass=univentionMail)(|(mailAlternativeAddress=%s)(mailPrimaryAddress=%s))(!(univentionCanonicalRecipientRewriteEnabled=1)))
  result_attribute = univentionExternalMailAlias
  result_format = %s

* virtual_mailbox_maps and virtual_alias_maps have been tidied up.
  virtual_alias_maps now only contains lookup tables for address rewriting.
  virtual_mailbox_maps now only contains lookup tables for valid addresses that match $virtual_mailbox_domains.
  → ldap.groups has been removed from virtual_mailbox_maps
  → ldap.distlist has been removed from virtual_mailbox_maps
  → ldap.virtual has been removed from virtual_mailbox_maps, instead 
    ldap.virtual_mailbox has been added (see below)
  → ldap.external_aliases has been added to virtual_alias_maps

** /etc/postfix/ldap.virtual modified
  -query_filter = (&(objectClass=univentionMail)(|(mailAlternativeAddress=%s)(mailPrimaryAddress=%s))(!(univentionCanonicalRecipientRewriteEnabled=1)))
  +query_filter = (&(objectClass=univentionMail)(mailAlternativeAddress=%s)(!(univentionCanonicalRecipientRewriteEnabled=1)))
   result_attribute = mailPrimaryAddress
   result_format = %s

** /etc/postfix/ldap.virtual_mailbox added
  query_filter = (&(objectClass=univentionMail)(mailPrimaryAddress=%s)(!(univentionCanonicalRecipientRewriteEnabled=1)))
  result_attribute = mailPrimaryAddress
  result_format = %s
  → added due to changes of ldap.virtual

REOPEN
  → ldap.sharedfolderremote has been removed from virtual_alias_maps
  → this will break mailAlternativeAddress of remote shared folders, because the 
    mailAlternativeAddress is no longer rewritten to mailPrimaryAddress
  → should be moved to virtual_access_maps

REOPEN
  → ldap.sharedfolderlocal has been removed from virtual_alias_maps
  → this will break mailAlternativeAddress of local shared folders, because the 
    mailAlternativeAddress is no longer rewritten to mailPrimaryAddress
  → should be moved to virtual_access_maps

* master.cf: "-o receive_override_options=no_address_mappings" is set for 
  smtpd:10025
  → TODO: check if canonical maps are still working
Comment 20 Daniel Tröder univentionstaff 2017-06-02 12:49:56 CEST
r80015:
* split ldap.sharedfolderlocal for mailAlternativeAddress
* move ldap.sharedfolderremote to virtual_alias_maps
* add UCRV update code

univention-mail-postfix 11.0.1-5A~4.2.0.201706021248
Comment 21 Daniel Tröder univentionstaff 2017-06-02 15:59:54 CEST
* Property names have been changed to be more consistent with existing properties.
* A UDM syntax lets the user choose the desired action in a drop-down.
* The OX integration was adapted to use the new alias feature. The same cleanup was done as for the UCS Postfix integration.

r80016: change wording, UDM/UMC grouping and syntax
r80017: rename properties
r80018: refactoring for consistent naming
r80020: rename mailForwardAddress property
r80022 by QA: added ucs-test scripts for mailForwardAddress tests
r80027: add support for external aliases, split tables for aliases and mailboxes

Branch: ucs_4.2-0 Scope: errata4.2-0

Package: univention-mail-postfix
Version: 11.0.1-6A~4.2.0.201706021429

Package: univention-directory-manager-modules
Version: 12.0.17-9.1A~4.2.0.201706021427

Package: univention-ldap
Version: 13.0.7-10A~4.2.0.201706021426

----

Package: univention-mail-postfix-ox
Version: 8.0.0-3A~4.2.0.201706021551
Branch: ucs_4.2-0
Scope: oxse4ucs
Comment 22 Daniel Tröder univentionstaff 2017-06-06 10:13:22 CEST
r80037: add translations
r80038: update advisory

Package: univention-directory-manager-modules
Version: 12.0.17-10A~4.2.0.201706061011
Comment 23 Sönke Schwardt-Krummrich univentionstaff 2017-06-12 14:06:35 CEST
(In reply to Daniel Tröder from comment #22)
> r80037: add translations
> r80038: update advisory
> 
> Package: univention-directory-manager-modules
> Version: 12.0.17-10A~4.2.0.201706061011

In univention-directory-manager-modules/modules/univention/admin/handlers/users/user.py the long descriptions of mailForwardAddress and mailForwardCopyToSelf are not encapsulated in _(…) and therefore not translated.
→ REOPEN
Comment 24 Daniel Tröder univentionstaff 2017-06-12 14:49:46 CEST
r80120: add translations
r80122: update advisory

Package: univention-directory-manager-modules
Version: 12.0.17-11A~4.2.0.201706121448
Comment 25 Daniel Tröder univentionstaff 2017-06-13 12:57:06 CEST
r80139: depend on updated univention-mail-postfix

Package: univention-mail-postfix-ox    Branch: ucs_4.2-0
Version: 8.0.0-4A~4.2.0.201706131256   Scope: oxse4ucs
Comment 26 Sönke Schwardt-Krummrich univentionstaff 2017-06-13 14:08:06 CEST
* OK: DE translations are missing in UMC
* OK: manual ucs-test script run was successful
* OK: changed dependencies
* advisories
** OK: univention-mail-postfix
** OK: univention-directory-manager-modules
** OK: univention-ldap
* automatic migration/modifcation of mail/postfix/virtual/{alias,mailbox}/maps
** OK: initial installation
** FAIL: OX is installed on system with latest errata
    → will be handled in next OX update
** OK: installation of latest errata on OX system
** OK: Kopano systems → no special case since Kopano uses default UCS mail stack
Comment 28 Sönke Schwardt-Krummrich univentionstaff 2017-08-07 10:13:53 CEST
Already published as OX 7.8.4-ucs2