Bug 40662 - Test the support for trusted domains as AD DC with Samba 4.7.11
Test the support for trusted domains as AD DC with Samba 4.7.11
Status: RESOLVED WONTFIX
Product: UCS
Classification: Unclassified
Component: Samba4
UCS 4.2
Other Linux
: P5 enhancement (vote)
: ---
Assigned To: Arvid Requate
Samba maintainers
https://hutten.knut.univention.de/med...
:
Depends on: 41644 42624 44963
Blocks: 40665 45326
  Show dependency treegraph
 
Reported: 2016-02-15 16:55 CET by Stefan Gohmann
Modified: 2019-02-25 16:17 CET (History)
12 users (show)

See Also:
What kind of report is it?: Feature Request
What type of bug is this?: ---
Who will be affected by this bug?: ---
How will those affected feel about the bug?: ---
User Pain:
Enterprise Customer affected?:
School Customer affected?:
ISV affected?:
Waiting Support:
Flags outvoted (downgraded) after PO Review:
Ticket number: 2017030221000401, 2017112821000241
Bug group (optional):
Max CVSS v3 score:


Attachments
setup_forest_trust.sh (11.81 KB, text/x-sh)
2016-12-21 16:44 CET, Arvid Requate
Details
GPO evaluation failing with forest trust (1.72 MB, application/gzip)
2018-02-21 17:05 CET, Eduard Mai
Details
GPO evaluation successful without trust (2.07 MB, application/gzip)
2018-02-21 17:06 CET, Eduard Mai
Details
GPO evaluation succeedes. No trust configured. (2.07 MB, application/gzip)
2018-03-01 12:40 CET, Eduard Mai
Details
GPO evaluation succeedes. No trust configured. (9.31 MB, application/gzip)
2018-03-01 12:40 CET, Eduard Mai
Details
GPO evaluation fails with configured trust. (2.84 MB, application/gzip)
2018-03-01 12:41 CET, Eduard Mai
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Stefan Gohmann univentionstaff 2016-02-15 16:55:16 CET
Please test the support for trusted domains as AD DC with Samba 4.4.
Comment 1 Stephan Hendl 2016-04-19 15:08:02 CEST
Are there any new results available? Is there a timeline for testing?
Comment 2 Arvid Requate univentionstaff 2016-04-19 19:28:55 CEST
We tested again with Samba 4.3.7.

In my setup the "AD trusts UCS" configuration worked in the sense of allowing AD logon for UCS users. The other direction "UCS trusts AD" worked in the more restricted sense of a) allowing AD users to get a Kerberos ticket from the UCS domain and b) enabling authentication via winbind. But there is additional work required depending on the use case.

The steps below are given as an example to show the current state but are in no way meant as recommendations. Security aspects should be considered carefully in each case.

DNS Preparation:
================
In the first step, you need to have both sides forward DNS lookups for the other domain to the corresponding authoritative name server (usually the other DC).

## Assume the UCS DC master.ucsdom.local has IP 10.200.8.10
## Assume the AD DC dc1.addom.local has IP 10.200.8.20

UCS Side:
=========
root@master:~# cat >> /etc/bind/local.conf.samba4 <<%EOR

zone "addom.local" {
      type forward;
      forwarders { 10.200.8.20; };
};
%EOR
root@master:~# /etc/init.d/bind9 restart
root@master:~# host dc1.addom.local
dc1.addom.local has address 10.200.8.20

AD Side:
========
On the Windows AD DC "Conditional Forwarding" needs to be configured for the UCS DNS domain:

* Open the DNS server console
* Expand the DNS server console tree
* Right click on Conditional Forwarders
* From the context menu select New Conditional Forwarder
* In the field "DNS domain" enter the FQDN of the UCS domain
* Click the "IP addresses of the master servers" list and enter the IP address of the UCS DC


Setting up the trust:

UCS Side:
=========
Log into the UCS System as root.

Check the Active Directory Domain Function Level: Raise the Domain and Forest level on the UCS Server to 2008 R2 if necessary (that's the default in UCS 4.1).

Use the following command to create a bidirectional trust:

root@master:~# samba-tool domain trust create addom.local \
                    -k no -UADDOM\\Administrator%ADAdminPassword \
                    --type=external --create-location=local \
                    --skip-validation
New Incoming Trust Password:      ## Password.1
Retype Incoming Trust Password:   ## Password.1
New Outgoing Trust Password:      ## Password.2
Retype Outgoing Trust Password:   ## Password.2
## More output

Note: In case the goal is to set up a unidirectional trust relationship, e.g. AD trusting UCS, the option --direction=incoming may be specified.

## Check:
root@master10:~# samba-tool domain trust list 
Type[External] Transitive[No]  Direction[BOTH]     Name[addom.local]

## After these steps Samba/AD should contain this account:
root@master:~# univention-s4search 'sAMAccountName=ADDOM$' accountExpires


AD Side:
=======
Terminology: "outgoing" trust from AD corresponds to "incoming" trust under UCS (and vice versa). Bidirectional trust can be set up in one go in the AD GUI or in separate steps. I usually prefer to set it up in separate steps, so I can check what's going on. So I describe setting up "outgoing" here first:


Start the "Active Directory Domains and Trusts" as Administrator on the AD DC.
Select your Domain, right click and select Properties

## Outgoing trust

* Select the "Trusts" tab and click on "New Trust..."
* The New Trust Wizard opens, continue and enter the "Trust Name". This can be the workgroup/netbios domain name (UCR: windows/domain) or the DNS domainname.
* Trust Type: External trust
* Direction of Trust: One-Way: outgoing
* Sides of Trust: This domain only
* Outgoing Trust Authentication Level: Domain-wide authentication
* Enter the password given to samba-tool for "Incoming Trust" ("Password.1")
* Continue>Continue>Confirm Outgoing Trust: Yes>Continue.

## The validation of the external outgoing trust to the UCS domain should validate ok at this point.

## After these steps the native AD should contain a corresponding account for the Samba/AD-Domain:
root@master:~# ldbsearch -H ldap://<Name or IP of AD Server> \
                         -UADDOM\\Administrator%ADAdminPassword \
                         flatName='UCSDOM'


Afterwards you should be able to log into a Windows Client attached to the Windows Domain with a UCS User.


## Incoming trust
* Select the "Trusts" tab and click on "New Trust..."
* The New Trust Wizard opens, continue and enter the "Trust Name". This can be * the workgroup/netbios domain name (UCR: windows/domain) or the DNS domainname.
* The wizard should ask if the existing unidirectional trust should be converted to a bidirectional trust, select yes.
* Sides of Trust: This domain only
* Outgoing Trust Authentication Level: Domain-wide authentication
* Enter the password given to samba-tool for "Outgoing Trust" ("Password.2")
* Continue>Continue>Confirm Incoming Trust: No>Continue.

After this step it should be possible for a root user logged into the UCS DC to obtain a Kerberos ticket for AD users, if the password is known. Note that the Kerberos Realm is case sensitive:

root@master10:~# kinit administrator@ADDOM.LOCAL
administrator@ADDOM.LOCAL's Password: 
## should work
root@master10:~# wbinfo -a ADDOM+Administrator
## should work

Please note at this point that there are additional steps required on the UCS side to make this trust direction (UCS trusts AD) useful for specific use cases. For example there is no ID mapping configured yet to map the AD user SIDs to Posix UIDs in UCS domain. This will require additional research and modifications to configuration and possibly to software code to make this mapping work in an automatic fashion like it did with Samba/NT (aka "Samba 3"). There will also be some obvious steps required like:

root@master10:~# univention-install libpam-winbind libnss-winbind
root@master10:~# ucr set auth/methods="krb5 ldap unix winbind"

but that's not sufficient yet. E.g. I get this in auth.log when trying to log in via SSH as ADDOM+Administrator:

=======================================================================
Apr 19 13:02:09 unassigned-hostname sshd[16102]: pam_winbind(sshd:auth): request wbcLogonUser failed: WBC_ERR_AUTH_ERROR, PAM error: PAM_AUTH_ERR (7), NTSTATUS: NT_STATUS_WRONG_PASSWORD, Error message was: Wrong Password
Apr 19 13:02:09 unassigned-hostname sshd[16102]: pam_winbind(sshd:auth): user 'ADDOM+Administrator' denied access (incorrect password or invalid membership)
=======================================================================
Comment 3 Arvid Requate univentionstaff 2016-06-08 15:36:59 CEST
Maybe relevant for UCS/Samba trusts AD:
 * https://bugzilla.samba.org/show_bug.cgi?id=11830
Comment 4 Arvid Requate univentionstaff 2016-06-23 16:17:09 CEST
Works 100%, at least with Windows 2008 R2, for a PoC script see URL field above.

Tested: "external" & "forest" trust, both transitive and --not-transitive.


But the limitations mentioned in the Samba 4.3.0 release notes apply:

  - Both sides of the trust need to fully trust each other!
  - No SID filtering rules are applied at all!
  - This means DCs of domain A can grant domain admin rights
    in domain B.
  - It's not possible to add users/groups of a trusted domain
    into domain groups.
Comment 5 Arvid Requate univentionstaff 2016-08-02 11:06:09 CEST
After a while the trust breaks, might be a lifetime issue of a kerberos-ticket or something, I'm still trying to debug.

In case of failure the samba-tool domain trust validate returns:

OK: LocalValidation: DC[\\WIN-SJ085F6TDA3.w2k8r2d2.ar] CONNECTION[WERR_OK] TRUST[WERR_OK] VERIFY_STATUS_RETURNED
ERROR: LOCAL_DC[MASTER45]: NETLOGON_CONTROL_REDISCOVER failed - ERROR(0x0000054F) - WERR_INTERNAL_ERROR

That's during the first failing call. On every next call the error message looks like this:

ERROR: LocalValidation: DC[\\WIN-SJ085F6TDA3.w2k8r2d2.ar] CONNECTION[WERR_NO_LOGON_SERVERS] TRUST[WERR_NO_LOGON_SERVERS] VERIFY_STATUS_RETURNED

It's a bit hard to debug because the problem only occurs after some time and the last time I got gdb rigged to the winbindd PID that is dedicated to the AD domain the problem disappeared after the samba-tool call ran into a timeout.
Comment 6 Stefan Gohmann univentionstaff 2016-10-17 06:48:31 CEST
It should be re-checked after fixing Bug #42624.
Comment 7 Arvid Requate univentionstaff 2016-10-25 17:28:32 CEST
Same behavior with Samba 4.5.1. I've opened a Ticket.
Comment 8 Stefan Gohmann univentionstaff 2016-11-03 09:56:45 CET
Not a blocker for the release.
Comment 9 Arvid Requate univentionstaff 2016-11-07 11:44:39 CET
Ok, with Samba 4.5.1 tests showed that the issue of Comment 5 (WERR_INTERNAL_ERROR / WERR_NO_LOGON_SERVERS) looks like resolved.

Next we are looking into https://bugzilla.samba.org/show_bug.cgi?id=11517 .
Comment 10 Arvid Requate univentionstaff 2016-11-07 11:45:41 CET
I assume the WERR_INTERNAL_ERROR / WERR_NO_LOGON_SERVERS issue has been fixed with upstream Samba Bug 12369.
Comment 11 Arvid Requate univentionstaff 2016-11-09 18:25:24 CET
For https://wrms.catalyst.net.nz/wr.php?request_id=265170 I created detailed test logs. The machine part of a GPO is still applied after configuring the bidirectional (transitive) forest trust, but the user part is not. When running gpupdate /force a corresponding error message is shown and the event viewer logs an error:

Event-ID: 1110

"The processing of Group Policy failed. Windows could not determine if the user and computer accounts are in the same forest. Ensure the user domain name matches the name of a trusted domain that resides in the same forest as the computer account."
Comment 12 Arvid Requate univentionstaff 2016-12-21 16:44:04 CET
Created attachment 8320 [details]
setup_forest_trust.sh

I used the attached script to setup the "forest" type trust. It seems to be stable with the winbind adjustments of samba 4.5.1.

A modified version of that script can be used to create an "external" type trust instead. Trust types explained:

https://technet.microsoft.com/en-us/library/dd560679(WS.10).aspx
https://technet.microsoft.com/en-us/library/cc730798(v=ws.11).aspx

Both types of trust currently cause GPO evaluation to fail on a Windows 7 client.
With "forest" trust application of the "machine" part of the GPO failed.
With "external" trust, application of the "user" part of the GPO failed.
But this difference may have been purely by chance.

Comparing Samba/AD side Wireshark logs with and without trust didn't show any totally obvious differences, so further debugging is required here.
Comment 13 herpolaert 2017-05-03 10:56:01 CEST
Hello. I try to create trust relationship between two Samba4. Sometimes, it works and sometimes, I have two errors :

smb_signing_good: BAD SIG: seq 1
ERROR: REMOTE_DC[pdc.sc.local]: failed to connect lsa server - ERROR(0xC0000022) - Access denied

smb_signing_good: BAD SIG: seq 1
ERROR: REMOTE_DC[pdc.sc.local]: failed to connect netlogon server - ERROR(0xC0000022) - Access denied

Any idea ?
Comment 14 Arvid Requate univentionstaff 2017-06-20 14:10:08 CEST
Ticket #2017030221000401 provided feedback about another issue:

The customer tried to add a user from the trusted domain into a security group of the trusting domain, using the ADUC tool. This works, when the trusting domain is a Windows domain, but it fails when the trusting domain is a UCS domain. The error message indicates that the Windows user is not found.
Comment 15 acr 2017-07-07 10:07:33 CEST
samba-4.6.5

after set transitive forest trust with win2008R2
problems with processing GPO
some logs in samba show problems with kerberos constrained delegation

Kerberos: TGS-REQ W7-WRK1$@SFB.TEST.SIP.OFFICE.GL from ipv4:192.168.100.22:50515 for w7-wrk1$\@SFB.TEST.SIP.OFFICE.GL@SFB.TEST.SIP.OFFICE.GL [canonicalize, request-anonymous, renewable, forwardable]
Kerberos: Bad request for constrained delegation
Kerberos: constrained delegation from W7-WRK1$@SFB.TEST.SIP.OFFICE.GL (w7-wrk1$\@SFB.TEST.SIP.OFFICE.GL@SFB.TEST.SIP.OFFICE.GL) as W7-WRK1$@SFB.TEST.SIP.OFFICE.GL to w7-wrk1$\@SFB.TEST.SIP.OFFICE.GL@SFB.TEST.SIP.OFFICE.GL not allowed
Kerberos: Failed building TGS-REP to ipv4:192.168.100.22:50515


problem with check constrained delegation
in /kdc/krb5tgs.c
function check_constrained_delegation

if(!krb5_realm_compare(context, client->entry.principal, server->entry.principal)) {
	ret = KRB5KDC_ERR_BADOPTION;
	kdc_log(context, config, 0,
	    "Bad request for constrained delegation");
	return ret;
}

this function just strcmp of realms char*

no idea why it failed
Comment 16 Arvid Requate univentionstaff 2017-07-10 18:53:18 CEST
Cool, thanks a lot for reporting! Some time ago I've created a patch for that particular corner of the code  in the standalone Heimdal KDC, see Bug 37687#c5 - but in the end we didn't apply that patch, because it was not necessary to fix the issue central to that other bug.

I just checked: The patch for that bug actually applies to the Samba/source4 builtin Heimdal code too, so we should check if it fixes the GPO issue.

I've created Bug 44963 to evaluate that patch for an 4.2 Errata update.
Comment 17 Arvid Requate univentionstaff 2017-12-13 21:58:20 CET
Eduard tested the patch from Bug 44963 and it didn't fix the GPO issue in the AD trust scenario. In fact, the "Kerberos: constrained delegation" error message is  also logged in the case where the GPO evalation works (and the code doesn't even come that far in the case where GPO evaluation aborts).

We now have full network traces from boot to error with "gpupdate /force /target:user" -- from both, the non-working (with domain trust) and the working (w/o domain trust) case.

First analysis shows that the client attempts (yet another) DCERPC bind against NETLOGON in the last step before error, signalling to GENSEC that the "schannel" mechanism shall be used. Samba then looks up a session for the client machine name in its session cache, fails and reports a bind_nak. And that's it. This is about package 74 of a short trace just capturing the gpupdate call. Comparing that to the successfull "gpupdate /force /target:user" that happens after delting the AD domain trust, we see in the successfull case, that the windows client doesn't suggest the use of "schannel" in that particular DCERPC bind against NETLOGON. It's unclear why the client decides differently in both cases. So, currently we are in the unfortunate situation that we have to make sense of differing behaviour of the (close source) Windows client. The full network traces (plus level 10 logs) contain the answer, but it's probably no easy task to spot the source of this issue.
Comment 18 Eduard Mai univentionstaff 2018-02-21 17:05:30 CET
Created attachment 9410 [details]
GPO evaluation failing with forest trust
Comment 19 Eduard Mai univentionstaff 2018-02-21 17:06:16 CET
Created attachment 9411 [details]
GPO evaluation successful without trust
Comment 20 Eduard Mai univentionstaff 2018-02-21 17:21:58 CET
Domain trust was reevaluated with Samba 4.7.5. The test setup consisted of a UCS 4.3 master configured to provide an AD DC via Samba. A Windows 7 client was joined into that domain. To test Policy evaluation a login as the UCS domain administrator was performed on the Windows client. Then the gpupdate tool was used to force the client to refetch User and Group Policies from the UCS DC.
This is successful when no trust to other domains is configured, logs and network packet traces are in attachment 9411 [details].

After that a forest trust was configured with samba-tool. The trusted domain was provided with a Windows Server 2008 instance. The same action with gpupdate was performed again. In this case the client failed to evaluate the group policy. See attchment 9410 for logs and network packet traces.
Comment 21 Eduard Mai univentionstaff 2018-03-01 12:40:05 CET
Created attachment 9426 [details]
GPO evaluation succeedes. No trust configured.
Comment 22 Eduard Mai univentionstaff 2018-03-01 12:40:44 CET
Created attachment 9427 [details]
GPO evaluation succeedes. No trust configured.
Comment 23 Eduard Mai univentionstaff 2018-03-01 12:41:23 CET
Created attachment 9428 [details]
GPO evaluation fails with configured trust.
Comment 24 Eduard Mai univentionstaff 2018-03-01 14:26:17 CET
An adjusted test run was performed.

A UCS DC with Samba 4.7.5 was set up. A Windows 7 Client then joined the UCS domain. It was then reconfigured to evaluate Group Policies synchronously during boot and login. Further more additional logging was activated for gpsvc (Group Policy Client Service) and the netlogon service.
The logs and network traces in attachment 9427 [details] cover a whole boot process of the Windows Client.

Again a trust between the UCS DC and an AD DC was established. The client was then rebooted. Attachment 9428 [details] is the result of this boot process.

Please note those two lines from netlogon log when a trust is configured.
02/28 16:48:50 [CRITICAL] DsrEnumerateDomainTrusts: Can't get Domain List from cache: 0x54B
02/28 16:48:50 [MISC] DsrEnumerateDomainTrusts: returns: 1311

The return value of 1311 (0x51F) is documented here:
https://msdn.microsoft.com/en-us/library/cc237254.aspx
Comment 26 Arvid Requate univentionstaff 2018-03-13 23:32:40 CET
Samba 4.8.0 has been released. According to the release notes some improvements have been made in the area of inter domain trusts, so we should check again and generate fresh traces, sorry.
Comment 27 Arvid Requate univentionstaff 2018-06-05 18:36:00 CEST
I've tested with Samba 4.8.2 and collected detailed logs and network traces for a/b comparison and posted them to this upstream bug:

* https://bugzilla.samba.org/show_bug.cgi?id=11517#c6
Comment 28 Arvid Requate univentionstaff 2018-10-23 14:11:38 CEST
Samba 4.7.11, released today, claims to fix the upstream issue:

 * BUG 11517: Fix Samba GPO issue when Trust is enabled.
 * BUG 13539: krb5-samba: Interdomain trust uses different salt principal.

Also included in Samba 4.9.
Comment 29 Arvid Requate univentionstaff 2019-02-25 16:17:52 CET
Works with Samba 4.10