Bug 45115 - Make apps available as subdomain at $appid.$domain
Make apps available as subdomain at $appid.$domain
Status: CLOSED FIXED
Product: UCS
Classification: Unclassified
Component: App Center
UCS 4.4
Other Linux
: P5 enhancement (vote)
: UCS 4.4-2-errata
Assigned To: Dirk Wiesenthal
Johannes Keiser
:
Depends on: 50436 50437
Blocks: 50445
  Show dependency treegraph
 
Reported: 2017-08-01 14:32 CEST by Felix Botner
Modified: 2019-11-27 14:20 CET (History)
4 users (show)

See Also:
What kind of report is it?: Feature Request
What type of bug is this?: ---
Who will be affected by this bug?: ---
How will those affected feel about the bug?: ---
User Pain:
Enterprise Customer affected?:
School Customer affected?:
ISV affected?:
Waiting Support:
Flags outvoted after Product Owner Review:
Ticket number:
Bug group (optional): External feedback
Max CVSS v3 score:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Felix Botner univentionstaff 2017-08-01 14:32:00 CEST
We should think about making the app available via appid.domain (not only fqdn/app)
Comment 1 Nico Gulden univentionstaff 2019-01-18 08:39:06 CET
This feature was requested by a user in the forum: https://help.univention.com/t/custom-url-for-webmail-portal/10889/
Comment 2 Felix Bartels (Kopano) 2019-01-18 10:44:47 CET
While a general support for this in the platform would be nice, app maintainers can already implement such a functionality on configure_host (done so for mattermost and minio, see https://stash.z-hub.io/projects/K4U/repos/mattermost/browse/configure_host).
Comment 3 Felix Botner univentionstaff 2019-10-29 17:51:13 CET
merged the changes

I don't like the "create certficates for all hosts" stuff. So i changed the listener: If a service 'wildcard-certficate' is set on the host object, the listener should create such a certificate.

This is just a POC, please check/decide if this is a way to go.

univention-apache
 + template fixes
 + removed (comment) IncludeOptional, does not work, i get 
  Okt 29 17:34:09 member apachectl[5733]: apache2: Syntax error on line 
  225 of /etc/apache2/apache2.conf: Syntax error on line 15 of 
  /etc/apache2/sites-enabled/include-vhosts.conf: 
  Could not open configuration file

univention-appcenter
 + set 'wildcard-certficate' service on host
 + call univention-fetch-certificate

univention-ssl
 + listener changes for 'wildcard-certficate'
 + mask * for "mawk"

univention-join
 + download univention-fetch-certificate exactly one certificate
   (given as parameter)
Comment 4 Florian Best univentionstaff 2019-11-06 07:37:21 CET
FYI: join.log contains the following error message:

Configure 08univention-apache.inst Tue Nov  5 23:35:58 CET 2019
2019-11-05 23:35:58.371264004+01:00 (in joinscript_init)
Module ssl disabled.
To activate the new configuration, you need to run:
  systemctl restart apache2
Considering dependency setenvif for ssl:
Module setenvif already enabled
Considering dependency mime for ssl:
Module mime already enabled
Considering dependency socache_shmcb for ssl:
Module socache_shmcb already enabled
Enabling module ssl.
See /usr/share/doc/apache2/README.Debian.gz on how to configure SSL and create self-signed certificates.
To activate the new configuration, you need to run:
  systemctl restart apache2
Site default-ssl disabled.
To activate the new configuration, you need to run:
  systemctl reload apache2
Enabling site default-ssl.
To activate the new configuration, you need to run:
  systemctl reload apache2
ERROR: Site include-vhosts does not exist!
ERROR: Site include-vhosts does not exist!
Reloading apache2 configuration (via systemctl): apache2.service.
2019-11-05 23:35:58.903248264+01:00 (in joinscript_save_current_version)

https://jenkins.knut.univention.de:8181/job/UCS-4.4/job/UCS-4.4-2/job/AutotestJoin/SambaVersion=s4,Systemrolle=master/ws/test/join.log
Comment 5 Erik Damrose univentionstaff 2019-11-13 12:16:16 CET
changes to the appcenter package have been reverted in 9c28994f + 0ff91b41
Comment 6 Dirk Wiesenthal univentionstaff 2019-11-14 11:11:56 CET
Fixed in
  univention-apache 11.0.1-11A~4.4.0.201911141049
(univention-add-vhost + UCR template)
  univention-ssl fix: 13.0.0-5A~4.4.0.201910311222
(certificate listener + univention-fetch-certificate)
  univention-join fix: 11.0.1-26A~4.4.0.201910311226
(uses univention-fetch-certificate)

I also added 23_apache/30_vhosts in ucs-test.

This is a quick overview (see also Bug#50445):
  Any App that needs to add a VHost by creating a dedicated file in /etc/apache2/sites-available can now do the following in the join script:
  univention-add-vhost "$FQDN" "$PORT" [--ssl] --conffile /var/lib/univention-appcenter/apps/$myapp/data/apache.conf

A new virtual host is created inside the newly added /etc/apache2/sites-available/univention-vhosts.conf.

If ssl is on (or PORT == 443), a new wildcard certificate is created by the DC Master and downloaded by localhost. This certificate can be used for "*.$hostname.$domainname". The certificate is added to the vhost entry. No further configuration is done, except for a "ServerName" directive.

(This certificate routine is only used if "$FQDN" is below "$hostname.$domainname". But every value for "$FQDN" is acceptable)

Actual configuration is done in /var/lib/univention-appcenter/apps/$myapp/data/apache.conf instead - ReverseProxy, WebSockets, etc. This file could be changed in the App's configure script, for example.

In the unjoin script, do this:
  univention-add-vhost "$FQDN" "$PORT" --remove

Administrators may even add their own custom files if needed:
  ucr set ucr set apache2/vhosts/$FQDN/$PORT/files="$OLD_VALUE,/my/apache.conf"
Comment 7 Johannes Keiser univentionstaff 2019-11-14 18:17:39 CET
Misc:
  OK automatic download of certificate on all roles
  FAIL no settings/service object for "Wildcard Certificate" exists
      add this as test to 30_vhosts?

univention-add-vhost command:
  OK --binddn
  OK --bindpwdfile
  OK --ssl
  OK --aliases --conffile (help could mention that it is comma seperated)
      OK --aliases a1,a2
      OK --aliases a1 --aliases a2
  OK --cert
  OK --private-key
  OK --ca
  FAIL? --remove
      dns/aliases objects from --aliases are not removed

ucr variables:
  OK Variables: apache2/vhosts/.*/enabled
  OK Variables: apache2/vhosts/.*/files
  FAIL? Variables: apache2/vhosts/.*/aliases
      no new dns/alias objects are created  (also no unused dns/alias objects are deleted)
  OK Variables: apache2/vhosts/.*/ssl/certificate
  OK Variables: apache2/vhosts/.*/ssl/key
  OK Variables: apache2/vhosts/.*/ssl/ca
  FAIL? Variables: apache2/vhosts/.*/ssl/certificatechain
      unused?

30_vhosts test:
  FAIL the "host $HOST" call is flaky right after the reloads
      i added a sleep 5 before it, then it was better
  FAIL? after some univention-add-vhost calls only apache2 is reloaded (effectively ignoring univention-add-vhost output)
  OK master
  FAIL backup
      Host myapp.backup71.mydomain.intranet not found: 3(NXDOMAIN)
  FAIL slave
      Host myapp.slave72.mydomain.intranet not found: 3(NXDOMAIN)
      missing --binddn --bindpwdfile
  ~OK member
      missing --binddn --bindpwdfile
Comment 8 Dirk Wiesenthal univentionstaff 2019-11-15 11:53:12 CET
(In reply to Johannes Keiser from comment #7)

>   FAIL no settings/service object for "Wildcard Certificate" exists
>       add this as test to 30_vhosts?

Yes, fixed.

>   FAIL? --remove
>       dns/aliases objects from --aliases are not removed

Yes, fixed.

>   FAIL? Variables: apache2/vhosts/.*/aliases
>       no new dns/alias objects are created  (also no unused dns/alias
>   FAIL? Variables: apache2/vhosts/.*/ssl/certificatechain
>       unused?

If univention-add-vhost is not used, but instead only "ucr set", we should not trigger any LDAP changes (let alone that on a memberserver, this would be impossible). certificatechain is not used by the script as we do not see the benefit. But maybe someone relies on it? We should let it just in the template there for now.

> 
> 30_vhosts test:
>   FAIL the "host $HOST" call is flaky right after the reloads
>       i added a sleep 5 before it, then it was better

Yes, I added a sleep 5. We will have to keep in mind that bind sometimes needs some time. This is fine if you just use "univention-add-vhost" in the last line, but may be problematic if your script wants to work immediately with the new FQDN.

>   FAIL? after some univention-add-vhost calls only apache2 is reloaded
> (effectively ignoring univention-add-vhost output)

That is okay. We do not add a new DNS entry, so we only need to reload apache2.

>   FAIL backup
>       Host myapp.backup71.mydomain.intranet not found: 3(NXDOMAIN)

Seems to be a problem with your local listener, not a general problem.

>   FAIL slave
>       Host myapp.slave72.mydomain.intranet not found: 3(NXDOMAIN)

Seems to be a problem with your local listener, not a general problem.

>       missing --binddn --bindpwdfile

Yes, fixed.

>   ~OK member
>       missing --binddn --bindpwdfile

Yes, fixed.
Comment 9 Johannes Keiser univentionstaff 2019-11-15 14:06:53 CET
(In reply to Dirk Wiesenthal from comment #8)
> (In reply to Johannes Keiser from comment #7)
> 
> >   FAIL no settings/service object for "Wildcard Certificate" exists
> >       add this as test to 30_vhosts?
> 
> Yes, fixed.

OK

> >   FAIL? --remove
> >       dns/aliases objects from --aliases are not removed
> 
> Yes, fixed.

OK

> >   FAIL? Variables: apache2/vhosts/.*/aliases
> >       no new dns/alias objects are created  (also no unused dns/alias
> >   FAIL? Variables: apache2/vhosts/.*/ssl/certificatechain
> >       unused?
> 
> If univention-add-vhost is not used, but instead only "ucr set", we should
> not trigger any LDAP changes (let alone that on a memberserver, this would
> be impossible). certificatechain is not used by the script as we do not see
> the benefit. But maybe someone relies on it? We should let it just in the
> template there for now.

OK
  
> > 30_vhosts test:
> >   FAIL the "host $HOST" call is flaky right after the reloads
> >       i added a sleep 5 before it, then it was better
> 
> Yes, I added a sleep 5. We will have to keep in mind that bind sometimes
> needs some time. This is fine if you just use "univention-add-vhost" in the
> last line, but may be problematic if your script wants to work immediately
> with the new FQDN.

OK
 
> >   FAIL? after some univention-add-vhost calls only apache2 is reloaded
> > (effectively ignoring univention-add-vhost output)
> 
> That is okay. We do not add a new DNS entry, so we only need to reload
> apache2.

OK
 
> >   FAIL backup
> >       Host myapp.backup71.mydomain.intranet not found: 3(NXDOMAIN)
> 
> Seems to be a problem with your local listener, not a general problem.

OK works now
 
> >   FAIL slave
> >       Host myapp.slave72.mydomain.intranet not found: 3(NXDOMAIN)
> 
> Seems to be a problem with your local listener, not a general problem.

OK works now
 
> >       missing --binddn --bindpwdfile
> 
> Yes, fixed.

OK
 
> >   ~OK member
> >       missing --binddn --bindpwdfile
> 
> Yes, fixed.

OK

OK univention-ssl yaml
OK univention-join yaml
OK univention-apache yaml (fe65d8d0cb Bug #45115: yaml)

-> verified
Comment 11 Felix Botner univentionstaff 2019-11-21 09:34:58 CET
23_apache.30_vhost still fails on https://jenkins.knut.univention.de:8181/job/UCS-4.4/job/UCS-4.4-2/job/AutotestJoin/lastCompletedBuild/SambaVersion=no-samba,Systemrolle=member/testReport/

And in the AD Member Modes tests. This is a bit of a problem. The whole concept does not work on AD Member Mode, because the Windows Server is the main DNS server in this scenario. What now?
Comment 12 Dirk Wiesenthal univentionstaff 2019-11-25 17:30:15 CET
Fixed in
  univention-apache 11.0.1-13A~4.4.0.201911251550

Now it is possible to do
  univention-add-vhost --dont-reload-services

If not given (default), apache2 and bind9 are reloaded if necessary. Additionally, a DNS lookup is performed after a bind9 reload. This is repeated four times over the course of 20 seconds until it succeeds.

This should not only solve the problems with the test, it also makes a lot of "sleep 3" statements in the test unnecessary; i dropped those lines.
Comment 13 Dirk Wiesenthal univentionstaff 2019-11-25 17:33:22 CET
(In reply to Felix Botner from comment #11)
> 23_apache.30_vhost still fails on
> https://jenkins.knut.univention.de:8181/job/UCS-4.4/job/UCS-4.4-2/job/
> AutotestJoin/lastCompletedBuild/SambaVersion=no-samba,Systemrolle=member/
> testReport/
> 
> And in the AD Member Modes tests. This is a bit of a problem. The whole
> concept does not work on AD Member Mode, because the Windows Server is the
> main DNS server in this scenario. What now?

AD Member Mode is a bit trickier as we use AD's DNS server, not ours. So I skip the test for now in AD mode. The fix described earlier should solve some problems on Non-Master systems in a S4 scenario where DNS reacts a bit slower as we cannot force to reload any relevant service locally.

I will add a note about that in the manual.
Comment 14 Johannes Keiser univentionstaff 2019-11-27 10:53:45 CET
OK code changes
OK tests / jenkins
OK services are reloaded by default / can be disabled
OK yaml
-> verified