Univention Bugzilla – Bug 38285
adconnector/check_domain() GSSAPI failed
Last modified: 2017-09-15 11:31:27 CEST
We received the following traceback, 4.0-1 errata152 (Walle). Execution of command 'adconnector/check_domain' has failed: Traceback (most recent call last): File "%PY2.7%/univention/management/console/modules/__init__.py", line 176, in _decorated return function(self, request, *args, **kwargs) File "%PY2.7%/univention/management/console/modules/decorators.py", line 188, in _response return function(self, request) File "%PY2.7%/univention/management/console/modules/decorators.py", line 316, in _response result = _multi_response(self, request) File "%PY2.7%/univention/management/console/modules/decorators.py", line 460, in _response return list(function(self, iterator, *nones)) File "%PY2.7%/univention/management/console/modules/decorators.py", line 282, in _fake_func yield function(self, *args) File "%PY2.7%/univention/management/console/modules/adconnector/__init__.py", line 377, in check_domain admember.check_ad_account(ad_domain_info, username, password) File "%PY2.7%/univention/lib/admember.py", line 235, in check_ad_account lo_ad.lo.sasl_interactive_bind_s("", auth) File "/usr/lib/python2.7/dist-packages/ldap/ldapobject.py", line 892, in sasl_interactive_bind_s res = self._apply_method_s(SimpleLDAPObject.sasl_interactive_bind_s,*args,**kwargs) File "/usr/lib/python2.7/dist-packages/ldap/ldapobject.py", line 860, in _apply_method_s return func(self,*args,**kwargs) File "/usr/lib/python2.7/dist-packages/ldap/ldapobject.py", line 236, in sasl_interactive_bind_s return self._ldap_call(self._l.sasl_interactive_bind_s,who,auth,RequestControlTuples(serverctrls),RequestControlTuples(clientctrls),sasl_flags) File "/usr/lib/python2.7/dist-packages/ldap/ldapobject.py", line 106, in _ldap_call result = func(*args,**kwargs) LOCAL_ERROR: {'info': 'SASL(-1): generic failure: GSSAPI Error: Miscellaneous failure (see text) (Matching credential (ldap/xbmc.desk76.local@DESK76.LOCAL) not found)', 'desc': 'Local error'}
Reported again, 4.0-2 errata279 (Walle)
Reported again, 4.0-3 errata288 (Walle)
Possibly the Kerberos realm has not been correct in these cases and we can improve the check to avoid the traceback. More info on this is welcome in case this happens again.
(In reply to Arvid Requate from comment #3) > Possibly the Kerberos realm has not been correct in these cases and we can > improve the check to avoid the traceback. More info on this is welcome in > case this happens again. I can't give more information. I only get the traceback and an optional remark.
Reported again, 4.0-3 errata320 (Walle) LOCAL_ERROR: {'info': 'SASL(-1): generic failure: GSSAPI Error: Miscellaneous failure (see text) (Matching credential (ldap/dhcp.fritz.box@FRITZ.BOX) not found)', 'desc': 'Local error'}
The Kerberos Principals are the only indication of what might be causing this. Are people attempting to enter the addresses of their xmbc or fritz boxes?
(ldap/fd00::d505:9c6e:256d:cec8@SSP.LOCAL) (ldap/snIactIve00.active.sni@ACTiVE.SNi)
And another time, again a fritz.box (ldap/win-6nbvsod4jra.fritz.box@FRITZ.BOX). Are people thinking that they need to enter the credentials of their router? Or is the server automatically detected by some DNS lookup? Remark: Neu Installation in eine Citrix Xen VM. Version: 4.1-0 errata1 (Vahr)
In check_ad_account we connect to ad_server_name = ad_domain_info["DC DNS Name"] ## this is the PDC DNS Name and in univention-ad-connector/umc/python/adconnector/__init__.py we set: ad_domain_info = admember.lookup_adds_dc(ad_server_address) ## This does CLDAP so, either of those delivers something unexpected. Anyway, we should try: except: around it and log the info returned by lookup_adds_dc, so we have the chance to ask for this info in case somebody contacts support about a failing adtakeover.
Reported again, 4.1-0 errata29 (Vahr) (ldap/terminalserver.fritz.box@FRITZ.BOX
Reported again, 4.1-0 errata44 (Vahr) LOCAL_ERROR: {'info': 'SASL(-1): generic failure: GSSAPI Error: Miscellaneous failure (see text) (Matching credential (ldap/forestdnszones.perry.home@PERRY.HOME) not found)', 'desc': 'Local error'}
Reported again, 4.1-1 errata160 (Vahr) LOCAL_ERROR: {'info': 'SASL(-1): generic failure: GSSAPI Error: Miscellaneous failure (see text) (Matching credential (ldap/rks.intern.***.de@INTERN.***.DE) not found)', 'desc': 'Local error'}
Version: 4.1-1 errata174 (Vahr) Remark: can't join a R2012R2 active directory. This is the error message, when trying to connect. ****** is a domain admin, and the password provided is correct. GSSAPI Error: Miscellaneous failure (see text) (Matching credential (ldap/****.dip0.t-ipconnect.de@DIP0.T-IPCONNECT.DE) not found)
Reported again, 4.1-0 errata75 (Vahr) ldap/w2008ad1.****.nl@****.NL
Currently my best guess is that the SPN ldap/<FQDN-of-AD-DC>@<AD-Realm> is missing in the situations generating this traceback. It should be easy to test this to verify this hypothesis. OTOH it would surprise me if the AD-Domain is still fully functional without that SPN.
I got this problem in one of my test environments (UCS 4.1-2 e220) feel free to appraise the behavior between .42.90 (master) and .42.94 (w2k12)
Created attachment 7852 [details] management-console-module-adconnector.log This logfile shows the situation of Comment 17. Apparently the sasl_interactive_bind_s attempts to get a service ticket for an invalid service principal name, where the domainname and the REALM are those from the UCS domain, not for the AD domain: =============================================================================== 28.07.16 17:12:22.013 MODULE ( PROCESS ) : AD Info: {'Domain': 'ad.Univention.local', 'LDAP Base': 'DC=ad,DC=Univention,DC=local', 'Forest': 'ad.Univention.local', 'Cli ent Site': 'Default-First-Site-Name', 'DC Netbios Name': 'W2K12X64-42-94', 'DC DNS Name': 'W2k12x64-42-94.ad.Univention.local', 'Netbios Domain': 'AD', 'DC IP': '10.200.42.94 ', 'Server Site': 'Default-First-Site-Name'} 28.07.16 17:12:22.348 MODULE ( WARN ) : Time sync failed, trying to authenticate anyway. Original exception: Remote clock is behind local clock by more than 360 seco nds, refusing to turn back time. 28.07.16 17:12:22.726 MODULE ( PROCESS ) : Prepare Kerberos UCR settings 28.07.16 17:12:22.731 MODULE ( PROCESS ) : Setting UCR variables: [u'kerberos/defaults/dns_lookup_kdc=true', u'kerberos/realm=AD.UNIVENTION.LOCAL'] 28.07.16 17:12:23.769 MODULE ( PROCESS ) : Unsetting UCR variables: [u'kerberos/kdc', u'kerberos/kpasswdserver', u'kerberos/adminserver'] 28.07.16 17:12:25.874 MODULE ( PROCESS ) : Die Ausführung des Kommandos adconnector/check_domain ist fehlgeschlagen: Traceback (most recent call last): File "/usr/lib/pymodules/python2.7/univention/management/console/base.py", line 283, in execute function(self, request) File "/usr/lib/pymodules/python2.7/univention/management/console/modules/decorators.py", line 190, in _response return function(self, request) File "/usr/lib/pymodules/python2.7/univention/management/console/modules/decorators.py", line 318, in _response result = _multi_response(self, request) File "/usr/lib/pymodules/python2.7/univention/management/console/modules/decorators.py", line 462, in _response return list(function(self, iterator, *nones)) File "/usr/lib/pymodules/python2.7/univention/management/console/modules/decorators.py", line 284, in _fake_func yield function(self, *args) File "/usr/lib/pymodules/python2.7/univention/management/console/modules/adconnector/__init__.py", line 356, in check_domain admember.check_ad_account(ad_domain_info, username, password) File "/usr/lib/pymodules/python2.7/univention/lib/admember.py", line 236, in check_ad_account lo_ad.lo.sasl_interactive_bind_s("", auth) File "/usr/lib/python2.7/dist-packages/ldap/ldapobject.py", line 892, in sasl_interactive_bind_s res = self._apply_method_s(SimpleLDAPObject.sasl_interactive_bind_s,*args,**kwargs) File "/usr/lib/python2.7/dist-packages/ldap/ldapobject.py", line 860, in _apply_method_s return func(self,*args,**kwargs) File "/usr/lib/python2.7/dist-packages/ldap/ldapobject.py", line 236, in sasl_interactive_bind_s return self._ldap_call(self._l.sasl_interactive_bind_s,who,auth,RequestControlTuples(serverctrls),RequestControlTuples(clientctrls),sasl_flags) File "/usr/lib/python2.7/dist-packages/ldap/ldapobject.py", line 106, in _ldap_call result = func(*args,**kwargs) *LOCAL_ERROR: {'info': 'SASL(-1): generic failure: GSSAPI Error: Miscellaneous failure (see text) (Matching credential (ldap/w2k12x64-42-94.univention.test@UNIVENTION.TEST) n*ot found)', 'desc': 'Local error'} =============================================================================== The Kerberos Ticket cache I found (Aug 4 15:57 /tmp/krb5cc_0) contained a TGT from the AD KDC: =============================================================================== root@master-42-90:~# klist -c /tmp/krb5cc_0 Credentials cache: FILE:/tmp/krb5cc_0 Principal: Administrator@AD.UNIVENTION.LOCAL Issued Expires Principal Aug 4 15:56:12 2016 Aug 5 01:56:12 2016 krbtgt/AD.UNIVENTION.LOCAL@AD.UNIVENTION.LOCAL =============================================================================== So the kinit worked.
Created attachment 7853 [details] config-registry.replog The corresponding UCR history. There have been several attempts. Note that hosts/static/10.200.42.94 hadn't been set What makes things even more weird: When I tried to replay the issue directly in an interactive python session on the host, it worked fine. And after that, the UMC worked as well... This is from the history: ============================================================================ root@master-42-90:~# mv /tmp/krb5cc_0 /tmp/krb5cc_01 root@master-42-90:~# python >>> from univention.lib import admember >>> ad_domain_info={'Domain': 'ad.Univention.local', 'LDAP Base': 'DC=ad,DC=Univention,DC=local', 'Forest': 'ad.Univention.local', 'Cli ent Site': 'Default-First-Site-Name', 'DC Netbios Name': 'W2K12X64-42-94', 'DC DNS Name': 'W2k12x64-42-94.ad.Univention.local', 'Netbios Domain': 'AD', 'DC IP': '10.200.42.94 ', 'Server Site': 'Default-First-Site-Name'} >>> admember.check_ad_account(ad_domain_info, 'Administrator', 'univention.99') Setting nameserver1 File: /etc/resolv.conf Create kerberos/defaults/dns_lookup_kdc Setting kerberos/realm File: /var/lib/samba/private/krb5.conf File: /etc/samba/base.conf File: /etc/krb5.conf Multifile: /etc/samba/smb.conf File: /etc/heimdal-kdc/kdc.conf Unsetting kerberos/kdc Unsetting kerberos/kpasswdserver Unsetting kerberos/adminserver File: /etc/krb5.conf Create hosts/static/10.200.42.94 Multifile: /etc/hosts Setting nameserver1 File: /etc/resolv.conf Setting kerberos/realm Create kerberos/kdc Create kerberos/kpasswdserver Create kerberos/adminserver File: /var/lib/samba/private/krb5.conf File: /etc/samba/base.conf File: /etc/krb5.conf Multifile: /etc/samba/smb.conf File: /etc/heimdal-kdc/kdc.conf Unsetting kerberos/defaults/dns_lookup_kdc File: /etc/krb5.conf True ============================================================================ So everything successfull. After that I had a new ticket cache, correctly containing a TGT plus a TGS for the correct ldap/w2k12x64-42-94.ad.univention.local@AD.UNIVENTION.LOCAL: 3,1K Aug 4 16:43 /tmp/krb5cc_0 The difference might be that, according to the UCR history my interactive call has set a new UCR variable: ============================================================================ 2016-08-04 16:43:18: set hosts/static/10.200.42.94=W2k12x64-42-94.ad.Univention.local old:[Previously undefined] ============================================================================ So I'm led to believe that in this case the IP 10.200.42.94 has been resolvable by the UMC all the time via a PTR record that resolved to an FQDN defined in the UCS domain: W2K12X64-42-94.univention.test. The last four changes in the /var/lib/univention-ldap/notify/transaction file confirm this assumption: ============================================================================ 1575 cn=adconnector,cn=apps,cn=univention,dc=univention,dc=test a 1576 univentionAppID=adconnector_10.0,cn=adconnector,cn=apps,cn=univention,dc=univention,dc=test a 1577 univentionAppID=adconnector_10.0,cn=adconnector,cn=apps,cn=univention,dc=univention,dc=test m 1578 uid=Administrator,cn=users,dc=univention,dc=test m 1579 cn=master-42-90,cn=dc,cn=computers,dc=univention,dc=test m 1580 cn=UNIVENTION_ADCONNECTOR,cn=nagios,dc=univention,dc=test m 1581 univentionAppID=adconnector_10.0,cn=adconnector,cn=apps,cn=univention,dc=univention,dc=test m 1582 cn=2015,cn=uidNumber,cn=temporary,cn=univention,dc=univention,dc=test a 1583 cn=2015,cn=gidNumber,cn=temporary,cn=univention,dc=univention,dc=test a 1584 cn=2015,cn=gidNumber,cn=temporary,cn=univention,dc=univention,dc=test d 1585 cn=uidNumber,cn=temporary,cn=univention,dc=univention,dc=test m 1586 cn=2015,cn=uidNumber,cn=temporary,cn=univention,dc=univention,dc=test d 1587 cn=W2K12X64-42-94$,cn=uid,cn=temporary,cn=univention,dc=univention,dc=test a 1588 cn=W2K12X64-42-94,cn=computers,dc=univention,dc=test a 1589 cn=W2K12X64-42-94$,cn=uid,cn=temporary,cn=univention,dc=univention,dc=test d 1590 cn=W2K12X64-42-94,cn=computers,dc=univention,dc=test m 1591 cn=W2K12X64-42-94,cn=computers,dc=univention,dc=test m 1592 cn=Windows Hosts,cn=groups,dc=univention,dc=test m 1593 cn=W2K12X64-42-94,cn=computers,dc=univention,dc=test m 1594 cn=w2k12x64-42-94,cn=computers,dc=univention,dc=test m 1595 relativeDomainName=W2k12x64-42-94,zoneName=univention.test,cn=dns,dc=univention,dc=test a 1596 relativeDomainName=94,zoneName=42.200.10.in-addr.arpa,cn=dns,dc=univention,dc=test a 1597 zoneName=univention.test,cn=dns,dc=univention,dc=test m 1598 zoneName=42.200.10.in-addr.arpa,cn=dns,dc=univention,dc=test m 1599 cn=W2K12X64-42-94,cn=computers,dc=univention,dc=test d 1600 cn=Windows Hosts,cn=groups,dc=univention,dc=test m 1601 cn=Windows Hosts,cn=groups,dc=univention,dc=test m 1602 relativeDomainName=W2K12X64-42-94,zoneName=univention.test,cn=dns,dc=univention,dc=test d 1603 zoneName=univention.test,cn=dns,dc=univention,dc=test m 1604 relativeDomainName=94,zoneName=42.200.10.in-addr.arpa,cn=dns,dc=univention,dc=test d 1605 zoneName=42.200.10.in-addr.arpa,cn=dns,dc=univention,dc=test m ============================================================================ The last zone object shows the time of that operation: modifyTimestamp: 20160804134534Z Since dns/backend is set to samba4 this can be confirmed by looking at the corresponding SOA in Samba: whenChanged: 20160804134539.0Z That's UTC, currently corresponding to 15:45: root@master-42-90:~# date -R -d'Thu, 04 Aug 2016 13:45:34 +0' Thu, 04 Aug 2016 15:45:39 +0200 The GSSAPI tracebacks show that they are around that time: ============================================================================ 04.08.16 13:13:39.597 MODULE ( PROCESS ) : Die Ausführung des Kommandos adconnector/check_domain ist fehlgeschlagen: 04.08.16 15:40:22.677 MODULE ( PROCESS ) : Die Ausführung des Kommandos adconnector/check_domain ist fehlgeschlagen: 04.08.16 15:45:55.567 MODULE ( PROCESS ) : Die Ausführung des Kommandos adconnector/check_domain ist fehlgeschlagen: 04.08.16 15:47:03.331 MODULE ( PROCESS ) : Die Ausführung des Kommandos adconnector/check_domain ist fehlgeschlagen: 04.08.16 15:57:23.843 MODULE ( PROCESS ) : Die Ausführung des Kommandos adconnector/check_domain ist fehlgeschlagen: ============================================================================ The last one is bit odd, it's more than 10 minutes after the the DNS record has been deleted. We can even see the timestamp of the deleted object in Samba/AD: ============================================================================ root@master-42-90:~# univention-s4search "DC=94*" --cross-ncs --show-deleted whenChanged isDeleted # record 1 dn: dc=94\0ADEL:e8e71b16-6043-4ee0-a67f-c264cbe4aadf,CN=Deleted Objects,DC=DomainDnsZones,DC=univention,DC=test isDeleted: TRUE whenChanged: 20160804134539.0Z ============================================================================ So, the morale of the whole story is, that something (bind9? nscd?) seems to cache this for an overly long time (the last failed attempt was 10 minutes after the PTR record has been deleted!). While I doubt that every traceback report at this bug is based on a similar situation, I would recommend to *always* set a hosts/static entry in UCR. The code is already present, I'll attach a patch.
Created attachment 7854 [details] flush_nscd_and_set_hosts_static.patch This patch causes the ncsd hosts cache to be flushed and an hosts/static entry to be set unconditionally.
There is another lesson in this, which Felix just pointed out: From all those tracebacks above I always assumed that all cases have been attempts to configure a UCS Master for AD "member mode". But it's just the admember.check_ad_account method getting called here also for the "normal" AD Connector setup. That method temporarily modifies critical UCR variables kerberos/realm, kerberos/kdc and such to configure Kerberos (client) to use the AD DC. That's perfectly ok for the AD "member mode", because we need it there later anyway, but it's completely unnecessary for the "normal" AD-Connector setup. It's even more hazardous, if the UCS domain is already configured as a Samba/AD domain. Maybe we should change this and have another method univention.connector.ad.check_ad_account which doesn't require Kerberos.
Reported again, 4.1-2 errata174 (Vahr)
We now get also traceback reports during the initial UMC system setup. This traceback has been reported: Version: 4.1-3 errata234 (Vahr) Traceback(dc6b67b806d1aab65bc82615750da21a): Execution of command 'setup/check/credentials wizard' has failed: Traceback (most recent call last): File "%PY2.7%/univention/management/console/base.py", line 283, in execute function(self, request) File "%PY2.7%/univention/management/console/modules/decorators.py", line 318, in _response result = _multi_response(self, request) File "%PY2.7%/univention/management/console/modules/decorators.py", line 462, in _response return list(function(self, iterator, *nones)) File "%PY2.7%/univention/management/console/modules/decorators.py", line 284, in _fake_func yield function(self, *args) File "%PY2.7%/univention/management/console/modules/setup/__init__.py", line 780, in check_credentials return util.check_credentials_ad(nameserver, address, username, password) File "%PY2.7%/univention/management/console/modules/setup/util.py", line 1051, in check_credentials_ad check_ad_account(ad_domain_info, username, password) File "%PY2.7%/univention/lib/admember.py", line 236, in check_ad_account lo_ad.lo.sasl_interactive_bind_s("", auth) File "/usr/lib/python2.7/dist-packages/ldap/ldapobject.py", line 892, in sasl_interactive_bind_s res = self._apply_method_s(SimpleLDAPObject.sasl_interactive_bind_s,*args,**kwargs) File "/usr/lib/python2.7/dist-packages/ldap/ldapobject.py", line 860, in _apply_method_s return func(self,*args,**kwargs) File "/usr/lib/python2.7/dist-packages/ldap/ldapobject.py", line 236, in sasl_interactive_bind_s return self._ldap_call(self._l.sasl_interactive_bind_s,who,auth,RequestControlTuples(serverctrls),RequestControlTuples(clientctrls),sasl_flags) File "/usr/lib/python2.7/dist-packages/ldap/ldapobject.py", line 106, in _ldap_call result = func(*args,**kwargs) LOCAL_ERROR: {'info': 'SASL(-1): generic failure: GSSAPI Error: Miscellaneous failure (see text) (Matching credential (ldap/failsafe.****@****.COM) not found)', 'desc': 'Local error'}
*** Bug 38667 has been marked as a duplicate of this bug. ***
Reported again, 4.1-0 errata0 (Vahr) Remark: I don't know what to do at this point... It gives me this error after attempting Active Directory Connection, it has a different computer name but the same domain name as the primary domain controller.
See Comment 16: > Apparently the sasl_interactive_bind_s attempts to get a service ticket for an invalid service principal name, where the domainname and the REALM are those from the UCS domain, not for the AD domain Maybe in the UMC System Setup case kerberos isn't set up properly yet? Anyway implementing Comment 20 and Comment 21 is recommended.
*** Bug 36802 has been marked as a duplicate of this bug. ***
Reported again, 4.1-4 errata324 (Vahr)
> Reported again, 4.1-4 errata324 (Vahr) I don't think these +1 reports add any information.
Reported again, 4.1-4 errata324 (Vahr), during system setup.
Move setup issues to 4.2-0-errata.
Reported again during system setup. Version: 4.1-4 errata324 (Vahr)
Version: 4.1-4 errata324 (Vahr) Remark: this error show up after try to join existing domain
Reported again, 4.2-0 errata2 (Lesum)
(In reply to Arvid Requate from comment #29) > > Reported again, 4.1-4 errata324 (Vahr) > > I don't think these +1 reports add any information. I add them nevertheless to demonstrate that this problem often occurs and is one of the most reported problem we have. This has impact on priorization of this bug.
Package rebuilt with patch from Comment 20. Advisory: univention-lib.yaml
Reported again, 4.2-0 errata0 (Lesum)
Code review: OK Tests: OK YAML: OK
<http://errata.software-univention.de/ucs/4.2/5.html>
Reported again, 4.1-4 errata420 (Vahr) Remark: Tried to Join to AD DS.
Reported again, 4.1-4 errata420 (Vahr)
Reported again, 4.2-0 errata0 (Lesum) Remark: Trying to join server 2012 ad domain..
Reported again at Ticket#2017080221000358. The user referred to https://help.univention.com/t/kein-ad-join-des-ucs-masters-moglich/3534/5 for a solution. Using `ucr set hosts/static/192.168.0.100=w2k8-32.ad.example.com` helped him.