Bug 47320 - Domain Join of Ubuntu18 in a domain ending with ".local" does not work
Domain Join of Ubuntu18 in a domain ending with ".local" does not work
Status: CLOSED FIXED
Product: UCS
Classification: Unclassified
Component: Univention Domain Join (Ubuntu)
UCS 4.3
Other Linux
: P5 normal (vote)
: ---
Assigned To: Richard Ulmer
UCS maintainers
:
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2018-07-09 13:49 CEST by Christian Völker
Modified: 2018-09-26 16:09 CEST (History)
5 users (show)

See Also:
What kind of report is it?: Bug Report
What type of bug is this?: 6: Setup Problem: Issue for the setup process
Who will be affected by this bug?: 2: Will only affect a few installed domains
How will those affected feel about the bug?: 5: Blocking further progress on the daily work
User Pain: 0.343
Enterprise Customer affected?: Yes
School Customer affected?:
ISV affected?:
Waiting Support: Yes
Flags outvoted (downgraded) after PO Review:
Ticket number: 2018062721000796
Bug group (optional):
Max CVSS v3 score:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Christian Völker univentionstaff 2018-07-09 13:49:26 CEST
Try to join domain on a Ubuntu 18.04 with the client fails with error like "domain controller not found".

"ping dc" works fine
"ping dc.domain.local" fails with "hostname not found".

dig and host work fine, too.


Client tries to resolve the host $dchostname.$domain which fails.

On Linux Distros the domain name ".local" is reserved for multicast DNS (mDNS) and thus the avahi daemon tries to resolve this fqdn. Which fails as there is no mDNS in place.

So a hostname ending with ".local" never gets resolved through bind.

To fix:
Edit /etc/nsswitch.conf  and replace

#hosts:          files mdns4_minimal [NOTFOUND=return] dns mdns4

by

hosts:          files mdns4_minimal dns mdns4

After a service restart avahi forwards failed .local requests to the proper DNS server.

Ubuntu resolves then properly and joins the domain.
Comment 1 Arvid Requate univentionstaff 2018-07-09 15:34:20 CEST
We already have something like that in the code. Maybe we need to put this adjustment before the ping (and roll it back in case something fails).
Comment 2 Christian Völker univentionstaff 2018-07-10 14:19:50 CEST
Tested behaviour on a .local domain with Ubuntu 18.04:

When /etc/nsswitch.conf is set to:

hosts:          files mdns4_minimal dns mdns4

it takes up to five seconds to resolve target.domain.local.

Only removing the mdns4_minimal appears to work properly without a delay:

hosts:          files dns mdns4


Changing the default domain name from "local" to "alocal" in /etc/avahi/avahi-daemon.conf does not change behaviour at all. Only editing nsswitch.conf works (see above)
Comment 4 Christian Völker univentionstaff 2018-07-20 09:59:54 CEST
Even with the change in nsswitch.conf the join does not work!

Before the join the DNS is properly configured:
==================================================
root@ubuntu18:~# systemd-resolve --status
Global
          DNSSEC NTA: 10.in-addr.arpa
                      16.172.in-addr.arpa
[...]
                      local
                      private
                      test

Link 2 (ens160)
      Current Scopes: DNS
       LLMNR setting: yes
MulticastDNS setting: no
      DNSSEC setting: no
    DNSSEC supported: no
         DNS Servers: 10.250.220.10
          DNS Domain: ucs.local
==================================================

And it resolves properly internal and external names:
==================================================
root@ubuntu18:~# dig _domaincontroller_master._tcp.ucs.local. -t SRV

; <<>> DiG 9.11.3-1ubuntu1.1-Ubuntu <<>> _domaincontroller_master._tcp.ucs.local. -t SRV
;; global options: +cmd
;; Got answer:
;; WARNING: .local is reserved for Multicast DNS
;; You are currently testing what happens when an mDNS query is leaked to DNS
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 56234
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 65494
;; QUESTION SECTION:
;_domaincontroller_master._tcp.ucs.local. IN SRV

;; ANSWER SECTION:
_domaincontroller_master._tcp.ucs.local. 900 IN	SRV 0 0 0 master.ucs.local.

;; Query time: 2 msec
;; SERVER: 127.0.0.53#53(127.0.0.53)
;; WHEN: Fri Jul 20 09:54:39 CEST 2018
;; MSG SIZE  rcvd: 95

root@ubuntu18:~# ping www.univention.de -c 1
PING www.univention.de (78.47.199.152) 56(84) bytes of data.
64 bytes from www.univention.de (78.47.199.152): icmp_seq=1 ttl=52 time=34.6 ms

--- www.univention.de ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 34.662/34.662/34.662/0.000 ms
==================================================

But running the join script cli fails with error:
==================================================
root@ubuntu18:~# univention-domain-join-cli 
Automatically detected the domain 'ucs.local'.
Please enter the user name of a domain administrator: administrator
Please enter the password for administrator: 
Warning: /etc/ldap/ldap.conf already exists.
Warning: /etc/krb5.conf already exists.
Created a backup of all configuration files, that will be modified at '/var/univention-backup/20180720073442_domain-join'.
Configuring ipv4 DNS servers for ens160.
Applying new settings to ens160.
Verbindung »Kabelgebundene Verbindung 1« wurde erfolgreich deaktiviert (aktiver D-Bus-Pfad: /org/freedesktop/NetworkManager/ActiveConnection/1)
Verbindung wurde erfolgreich aktiviert (aktiver D-Bus-Pfad: /org/freedesktop/NetworkManager/ActiveConnection/2)
An error occurred. Please check /var/log/univention/domain-join-cli.log for more information.
==================================================

Checking /varlog/univention/domaibn-join-cli.log shows:
==================================================
root@ubuntu18:~# cat /var/log/univention/domain-join-cli.log 
2018-07-20 09:34:32,757 userinfo INFO Automatically detected the domain 'ucs.local'.
2018-07-20 09:34:42,138 userinfo WARNING Warning: /etc/ldap/ldap.conf already exists.
2018-07-20 09:34:42,274 userinfo WARNING Warning: /etc/krb5.conf already exists.
2018-07-20 09:34:42,275 userinfo INFO Created a backup of all configuration files, that will be modified at '/var/univention-backup/20180720073442_domain-join'.
2018-07-20 09:34:42,327 userinfo INFO Configuring ipv4 DNS servers for ens160.
2018-07-20 09:34:42,370 userinfo INFO Applying new settings to ens160.
2018-07-20 09:34:42,691 userinfo CRITICAL An error occurred. Please check /var/log/univention/domain-join-cli.log for more information.
2018-07-20 09:34:42,692 debugging CRITICAL All nameservers failed to answer the query _domaincontroller_master._tcp.ucs.local. IN SRV: Server 127.0.0.53 UDP port 53 answered SERVFAIL
Traceback (most recent call last):
  File "/usr/sbin/univention-domain-join-cli", line 195, in <module>
    distribution_joiner.join_domain()
  File "/usr/lib/python2.7/dist-packages/univention_domain_join/distributions/ubuntu.py", line 95, in join_domain
    DnsConfigurator(self.nameservers, self.domain).configure_dns()
  File "/usr/lib/python2.7/dist-packages/univention_domain_join/utils/general.py", line 40, in root_wrapper
    return_value = func(*args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/univention_domain_join/join_steps/dns_configurator.py", line 78, in configure_dns
    self.check_if_dns_works()
  File "/usr/lib/python2.7/dist-packages/univention_domain_join/join_steps/dns_configurator.py", line 83, in check_if_dns_works
    resolver.query('_domaincontroller_master._tcp.%s.' % (self.domain,), 'SRV')
  File "/usr/lib/python2.7/dist-packages/dns/resolver.py", line 947, in query
    raise NoNameservers(request=request, errors=errors)
NoNameservers: All nameservers failed to answer the query _domaincontroller_master._tcp.ucs.local. IN SRV: Server 127.0.0.53 UDP port 53 answered SERVFAIL
==================================================

Checking DNS now the "DNS domain" (aka "search") is missing in setting (see last line):
==================================================
root@ubuntu18:~# systemd-resolve --status
Global
          DNSSEC NTA: 10.in-addr.arpa
[...]
                      local
                      private
                      test

Link 2 (ens160)
      Current Scopes: DNS
       LLMNR setting: yes
MulticastDNS setting: no
      DNSSEC setting: no
    DNSSEC supported: no
         DNS Servers: 10.250.220.10
==================================================

It now does not resolve the DC local any more:
==================================================
root@ubuntu18:~# dig _domaincontroller_master._tcp.ucs.local. -t SRV

; <<>> DiG 9.11.3-1ubuntu1.1-Ubuntu <<>> _domaincontroller_master._tcp.ucs.local. -t SRV
;; global options: +cmd
;; Got answer:
;; WARNING: .local is reserved for Multicast DNS
;; You are currently testing what happens when an mDNS query is leaked to DNS
;; ->>HEADER<<- opcode: QUERY, status: SERVFAIL, id: 43996
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 65494
;; QUESTION SECTION:
;_domaincontroller_master._tcp.ucs.local. IN SRV

;; Query time: 0 msec
;; SERVER: 127.0.0.53#53(127.0.0.53)
;; WHEN: Fri Jul 20 09:50:47 CEST 2018
;; MSG SIZE  rcvd: 68
==================================================

But external domains still get resolved as before:
==================================================
root@ubuntu18:~# ping www.univention.de -c 1
PING www.univention.de (78.47.199.152) 56(84) bytes of data.
64 bytes from www.univention.de (78.47.199.152): icmp_seq=1 ttl=52 time=34.6 ms

--- www.univention.de ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 34.662/34.662/34.662/0.000 ms
==================================================



So the script removes the "DNS Domain" from DNS-configuriation which somehow is related to .local-Domain.
Comment 5 Christian Völker univentionstaff 2018-07-20 10:16:46 CEST
In a domain not ending with ".local" join of Ubuntu 18 runs smoothly and finally joins.
Comment 6 Arvid Requate univentionstaff 2018-08-07 12:47:50 CEST
Surprise: Actually it's a behavior of systemd-resolved

https://github.com/systemd/systemd/issues/8852
Comment 7 Richard Ulmer univentionstaff 2018-08-08 13:49:34 CEST
I have now committed a fix for Ubuntu 18.04. It adds the domain name as a search domain, so that systemd-resolve starts resolving .local domain names.

While debugging I also found, that the join was slower when using .local domain names. I guess this is due to /etc/nsswitch.conf telling the system to first try using mDNS to resolve .local domains, which takes some time before it fails and the system continues to try regular DNS. I have changed the order, so that regular DNS is now tried before mDNS. I've applied this change to all (supported) versions of the tool.

univention-domain-join (1.0-17ubuntu1 - Ubuntu 18.04)
138946270ee0 | Bug #47320: Add changelog entry
24012f85d679 | Bug #47320: Adapt nsswitch.conf to avoid slow DNS
0920d2f959b9 | Bug #47320: Add support for .local domains

univention-domain-join (1.0-16ubuntu3 - Ubuntu 16.04)
607a52042008 | Bug #47320: Add changelog entry
0f40fa192486 | Bug #47320: Adapt nsswitch.conf to avoid slow DNS

univention-domain-join (1.0-16ubuntu4 - Ubuntu 14.04)
2fa60ae5bb3c | Bug #47320: Add changelog entry
267f8a975c36 | Bug #47320: Add changelog entry
c282638872c3 | Bug #47320: Adapt nsswitch.conf to avoid slow DNS
Comment 8 Arvid Requate univentionstaff 2018-08-28 18:23:48 CEST
> 24012f85d679 | Bug #47320: Adapt nsswitch.conf to avoid slow DNS

IIRC when debugging this together, we had to do a
  systemctl restart systemd-resolved

after setting ipv4.dns-search via nmcli. I'm surprised that it is missing in all of the 18.04 commits. Did you check that it works without?
Comment 9 Richard Ulmer univentionstaff 2018-08-29 12:33:51 CEST
I have tried the domain join on Ubuntu 18.04 with the proposed changes and a .local domain. It seems like DNS works right after running univention-domain-join, even without restarting systemd-resolved.
Comment 10 Arvid Requate univentionstaff 2018-08-30 14:04:37 CEST
Ok, works, so let's build & sign packages for PPA upload.
Comment 11 Felix Botner univentionstaff 2018-09-13 13:31:56 CEST
upload
Comment 12 Richard Ulmer univentionstaff 2018-09-18 08:46:01 CEST
Packages are uploaded an built.
Comment 13 Felix Botner univentionstaff 2018-09-18 10:09:02 CEST
OK - installation from ppa:univention-dev/ppa (18.04 )
Comment 14 Stefan Gohmann univentionstaff 2018-09-26 16:09:08 CEST
(In reply to Felix Botner from comment #13)
> OK - installation from ppa:univention-dev/ppa (18.04 )

It is already released.