Univention Bugzilla – Bug 50492
Windows login fails in UCS Samba/AD domain after changing password in MS AD domain
Last modified: 2020-03-18 12:27:41 CET
Setup: UCS Samba/AD domain with AD-Connector configured to sync with MS AD domain. Trigger: User changes his AD password, e.g. while logged in at a Windows client machine that is joined into the MS AD domain (or admin resets AD password via MS ADUC gui). Problem: After that the user login against UCS Samba/AD fails with new password, e.g. when logging in with a second Windows client that is joined into the UCS domain. Instead Windows login is possible with the old password.
Other symptoms visible for Support: * kinit in UCS only possible with *old* PW * simple bind against OpenLDAP only possible with *old* PW (because userPassword = {K5KEY}) * smbclient (w/o kerberos) possible with *new* PW Analysis: The AD-Connector only synchronizes the NT-Hash from/to MS AD. It does not synchronize Kerberos hashes. That's fine for OpenLDAP, but causes the S4-Connector to not update the Kerberos-Keys stored in the Samba/AD attribute "supplementalCredentials". It cannot, because OpenLDAP doesn't have the required AES and/or DES keys.
Idea for a workaround: Make the KDC prefer the arcfour hashes: ucr set \ kerberos/defaults/enctypes/permitted="arcfour-hmac-md5 aes256-cts-hmac-sha1-96 aes128-cts-hmac-sha1-96 des-cbc-crc des-cbc-md5 des-cbc-md4 des3-hmac-sha1 des3-cbc-sha1" ucr unset kerberos/defaults/enctypes/tgs kerberos/defaults/enctypes/tkt /etc/init.d/samba restart Caveat: Worked in my artifical test-VM with Samba domain function level 2008-R2. Not tested in production yet. Idea for a solution: 1. Make the AD-Connector read the supplementalCredentials from MS AD and write them into OpenLDAP krb5key attributes. They are salted with the UPN from the MS AD domain. 2. Check that the S4-Connector synchronizes these "alien" keys properly to Samba/AD (worked in my test setup). 3. Adjust our OpenLDAP patch 12_k5pwd.patch to use the salt stored in the krb5key(s) when checking the user password during simple bind. This seems to work with Heimdal with and without Samba. Heimdal seems to use the salt that is stored in the krb5Key/supplementalCredentials rather than the actual principal name of the user. There is one caveat though: If the MS AD domain has a different set of Kerberos keys than the UCS Samba/AD domain, this could result in logon errors. E.g. I had issues when implating DES-only hashes from a Samba/AD domain with function level 2003 into a 2008-R2 level Samba/AD domain account. In this case I was able to authenticate but kerberized logon e.g. to a Samba/AD share failed with NT_STATUS_LOGON_FAILURE. But that's a condition we could communicate or even check in the AD-Connector.
Increasing as customer can not update their passwords properly. Have not found a working workaround. Getting urgent
Successful build Package: univention-ad-connector Version: 13.0.0-30A~4.4.0.202003041103 Branch: ucs_4.4-0 Scope: errata4.4-3 User: jbremer Successful build Package: openldap Version: 2.4.45+dfsg-1~bpo9+1A~4.4.0.202003031624 Branch: ucs_4.4-0 Scope: errata4.4-3 User: jbremer The Ad-Connector has been adjusted to sync the Kerberos Hashes from supplementalCredentials to the ldap attribute krb5Key. The k5pwd openldap overlay module has been adjusted to use the correct salt, found in the keys itself instead of the default salt. These adjustments are in svn in patch: 14_univention-k5pwd-use-correct-salt.quilt Test 55_adconnector/050sync_password_sync has been adapted to check kinit with every etype after changing the password in AD. The UCR Variable con.*/ad/mapping/user/password/kerberos/enabled has been added to enable/disable the sync of kerberos hashes. If unset, the new kerberos hashes won't be synced, which is the default right now. A WIP of sdb article can be found here: https://help.univention.com/t/activating-the-synchronization-of-kerberos-hashes-with-ad-connector/14454
e21d950f97 Bug #50429: yaml 286de9e7f1 Bug #50429: Add ucr variable to disable kerberos hash sync d3594dfdab Bug #50429: version bump cfa65eb5c2 Bug #50429: yaml 7339eac89e Bug #50491: Get krb5 hashes from ad d84d4a862c Bug #50492: openldap yaml 1659845ca2 Bug #50492: Test Kerberos hash sync with kinit 28011cd5fd Bug #50492: version bump cbf3f17dd9 Bug #50492: Merge branch 'jbremer/bug50492' into 4.4-3 4894950664 Bug #50492: Version bump 8830bcc099 Bug #50492: yaml Commits were done with different bugnumbers, i hope i found everything.
We discussed some implementation details yesterday More notes: The ucs-test 55_adconnector/050sync_password_sync now always sets the kerberos pw sync to yes, the behavior without syncing kerberos hashses is not tested anymore. We should have a separate section in the test for that. Also, the AD connector is not restarted at the end after resetting the UCRv. In the tests we also found that the s4 connector can not sync the KRB hashes synced from an AD into S4, because we have to sync exactly 2 respective 4 keys. Traceback: in password_sync_ucs_to_s4 06.03.2020 14:59:00.112 LDAP (WARNING): sync failed, saved as rejected /var/lib/univention-connector/s4/1583501026.171020 06.03.2020 14:59:00.113 LDAP (WARNING): Traceback (most recent call last): File "/usr/lib/python2.7/dist-packages/univention/s4connector/__init__.py", line 891, in __sync_file_from_ucs if ((old_dn and not self.sync_from_ucs(key, mapped_object, pre_mapped_ucs_dn, unicode(old_dn, 'utf8'), old, new)) or (not old_dn and not self.sync_from_ucs(key, mapped_object, pre_mapped_ucs_dn, old_dn, old, new))): File "/usr/lib/python2.7/dist-packages/univention/s4connector/s4/__init__.py", line 2610, in sync_from_ucs f(self, property_type, object) File "/usr/lib/python2.7/dist-packages/univention/s4connector/s4/password.py", line 639, in password_sync_ucs_to_s4 s4connector.lo_s4.lo.modify_ext_s(compatible_modstring(object['dn']), modlist, serverctrls=[ctrl_bypass_password_hash]) File "/usr/lib/python2.7/dist-packages/ldap/ldapobject.py", line 374, in modify_ext_s resp_type, resp_data, resp_msgid, resp_ctrls = self.result3(msgid,all=1,timeout=self.timeout) File "/usr/lib/python2.7/dist-packages/ldap/ldapobject.py", line 514, in result3 resp_ctrl_classes=resp_ctrl_classes File "/usr/lib/python2.7/dist-packages/ldap/ldapobject.py", line 521, in result4 ldap_result = self._ldap_call(self._l.result4,msgid,all,timeout,add_ctrls,add_intermediates,add_extop) File "/usr/lib/python2.7/dist-packages/ldap/ldapobject.py", line 106, in _ldap_call result = func(*args,**kwargs) CONSTRAINT_VIOLATION: {'info': '0000202F: PrimaryKerberos num_keys != 2 at ../../source4/dsdb/samdb/ldb_modules/password_hash.c:414', 'desc': 'Constraint violation'}
Created attachment 10304 [details] adapted patch 92_allow_missing_des-cbc to allow every combination of keys For some reason a local security policy called Network security: Configure encryption types allowed for Kerberos was set on my machine, to allow all enctypes, which is why it "worked on my machine"(tm). By using this policy, we can get a variety of different amounts of hashes, while Samba wants exactly 4 hashes of the kind: aes256-cts-hmac-sha1-96, aes128-cts-hmac-sha1-96, des-cbc-crc, des-cbc-md5. We can "fill" the key lists with dummy hashes (hashes without any key value) in the S4-Connector, to make Samba happy, OR we could loosen the Restrictions in Samba, which seem outdated, and do not seem to have any purpose but to make it harder to write "bad" data into the password attributes. In my first tests with this patch applied, it did not make a difference if these constraints are there or not.
454f7ad8bf Bug #50492: yaml 83695bd9f4 Bug #50492: yaml 53db07bb4d Bug #50492: version bump b34deeaed5 Bug #50492: version bump ef4a9325e5 Bug #50492: Changelog 5f58121f85 Bug #50492: Adjust testcase for password sync 78de0ee599 Bug #50492: Pass "dummy" hashes to samba if etypes are not supported Successful build Package: univention-ad-connector Version: 13.0.0-32A~4.4.0.202003081641 Branch: ucs_4.4-0 Scope: errata4.4-3 User: jbremer Successful build Package: univention-s4-connector Version: 13.0.2-65A~4.4.0.202003081635 Branch: ucs_4.4-0 Scope: errata4.4-3 User: jbremer In default configuration, Windows Server 2008r2 and up do not send any des-cbc-crc by default. Samba needs an exact number of keys though, which is why a "dummy" hash is now created in the S4-Connector if the DES-CBC-CRC is missing. Customers can set the allowed encryption types for kerberos, which mean they can disable the other des-key too. I tried passing a dummy hash in this case too, but if the whole Primary:Kerberos package (des-cbc-crc and des-cb-md5) is replaced with dummy-hashes, it raises a "Subcontext" (?) Error on ndr_pack . I documented in the sdb article, that at least the enctypes DES_CBC_MD5, RC4_HMAC_MD5, AES128_HMAC_SHA1and AES256_HMAC_SHA1 need to be enabled on the AD DC for the kerberos hash sync. https://help.univention.com/t/activating-the-synchronization-of-kerberos-hashes-with-ad-connector/14454 I adjusted the test case, so that the password sync without the hash sync enabled is still tested. The new Jenkins Job, which tests s4connector AND adconnector failed once at setup, and the second time seemed to hang after 15+ hours.
For QA: Setup a UCS Master with Active Directory compatible Domain Controller and Ad Connection installed, Create (bidirectional) Ad-Connection in syncmode After installing the new packages : ucr set 'connector/ad/mapping/user/password/kerberos/enabled=yes' /etc/init.d/univention-ad-connector restart create a user in UCS, reset the password in RSAT "Users and Computers" by right-clicking on the user object. If the sync was succesfull, there are 3 krb5key entries on the user-object. Test kinit with the encryption types. kinit -e <enctype> <username> It needs to work at least with (des-cbc-crc is disabled by default): aes256-cts-hmac-sha1-96 aes128-cts-hmac-sha1-96 arcfour-hmac-md5 des-cbc-md5 Check all the boxes in GPO_name**\Computer Configuration\Windows Settings\Security Settings\Local Policies\Security Options\“Network security: Configure encryption types allowed for Kerberos” reset password again and check if authentication also works with des-cbc-crc. Test if ldapsearch with simplebind works with the new password, since openldap was not able to handle foreign hashes with foreign salts before.
FYI: https://lists.samba.org/archive/cifs-protocol/2008-September/000469.html and https://docs.microsoft.com/en-us/openspecs/windows_protocols/ms-samr/fa61e5fc-f8fb-4d5b-9695-c724af0c3829 explains the origin of the keytype -140 dummy hash and states that the key value in that case is the NT-Hash. I'm unsure if that matters though. If Samba is fine with a "None" value then I don't object.
Created attachment 10306 [details] Alternative 14_univention-k5pwd-use-correct-salt.quilt QA for the adjusted OpenLDAP patch: Ok, works. Maybe we could have kept the old krb5_get_pw_salt default Salt strategy in case the ekey.salt is empty. This is what we had in very early stages of UCS: krb5Keys that where salted with the defult salt but the didn't explicitely contain this salt. Heimdal falls back to the default salt in that case. Long story short: The slapd works and doesn't crash even if the krb5Key doesn't explicitly mention the salt. In that case the simplebind just simply fails -- and the S4-Connector rejects those theoretical types of krb5Key anyway with a traceback. No regression. The attached quilt patch would be an alternative, just in case.
QA: * Code review: Ok * Functional test: Ok * SDB-article: Ok, I've done some wording changes in cooperation with Julia. Basically it's better to always explicitly write "Microsoft AD", that makes it easier to understand which side we are talking about.
Found some tracebacks in the ucs-test.log, which seem to happen because of this change in the s4connector 14 times: Traceback (most recent call last): File "/usr/lib/python2.7/dist-packages/univention/s4connector/__init__.py", line 891, in __sync_file_from_ucs if ((old_dn and not self.sync_from_ucs(key, mapped_object, pre_mapped_ucs_dn, unicode(old_dn, 'utf8'), old, new)) or (not old_dn and not self.sync_from_ucs(key, mapped_object, pre_mapped_ucs_dn, old_dn, old, new))): File "/usr/lib/python2.7/dist-packages/univention/s4connector/s4/__init__.py", line 2610, in sync_from_ucs f(self, property_type, object) File "/usr/lib/python2.7/dist-packages/univention/s4connector/s4/password.py", line 609, in password_sync_ucs_to_s4 supplementalCredentials_new = calculate_supplementalCredentials(krb5Key, supplementalCredentials) File "/usr/lib/python2.7/dist-packages/univention/s4connector/s4/password.py", line 399, in calculate_supplementalCredentials krb3_blob = ndr_pack(krb) File "/usr/lib/python2.7/dist-packages/samba/ndr.py", line 33, in ndr_pack return ndr_pack() RuntimeError: (7, 'Subcontext Error') ===================================================================== The change that the s4-connector now always creates the Kerberos:Primary package seems to be the issue here, which I reverted. I also set the nt_hash as the value of the dummy key to match the samba documentation. Package: univention-s4-connector Version: 13.0.2-66A~4.4.0.202003122223 Branch: ucs_4.4-0 Scope: errata4.4-3 = /var/univention/buildsystem2/logs/ucs_4.4-0-0-errata4.4-3/univention-s4-connector_13.0.2-66A~4.4.0.202003122223_all.deb'. 9fda6b32a0 Bug #50492: Write nt-hash into new dummy hash
Ok, confirmed.
<http://errata.software-univention.de/ucs/4.4/493.html> <http://errata.software-univention.de/ucs/4.4/494.html> <http://errata.software-univention.de/ucs/4.4/495.html>