Bug 34222 - Samba4 @school test: DNS SRV record replication
Summary: Samba4 @school test: DNS SRV record replication
Status: RESOLVED FIXED
Alias: None
Product: UCS Test
Classification: Unclassified
Component: S4 Connector
Version: unspecified
Hardware: Other Linux
: P5 normal
Target Milestone: ---
Assignee: Dmitry Galkin
QA Contact:
URL:
Keywords:
Depends on: 38064
Blocks:
  Show dependency treegraph
 
Reported: 2014-03-03 13:16 CET by Arvid Requate
Modified: 2017-05-22 14:42 CEST (History)
2 users (show)

See Also:
What kind of report is it?: ---
What type of bug is this?: ---
Who will be affected by this bug?: ---
How will those affected feel about the bug?: ---
User Pain:
Enterprise Customer affected?:
School Customer affected?:
ISV affected?:
Waiting Support:
Flags outvoted (downgraded) after PO Review:
Ticket number:
Bug group (optional):
Customer ID:
Max CVSS v3 score:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Arvid Requate univentionstaff 2014-03-03 13:16:46 CET
In UCS@School the S4-Connector filters Samba4- and Kerberos-specific DNS records. A couple of replication cases need to be checked:

* Modification in UDM -> Samba/ADDS (only local system should be written into Samba/ADDS)

* Modification in Samba/ADDS -> UDM (no replication)

This needs to be checked in a UCS@school Master and also on a Slave in a multi-server domain. As a third dimension, the UCS@school singlemaster should be checked as well.

Looking at ucs-school-metapackage/62ucs-school-* the following records are relevant at least:

_ldap._tcp._msdcs
_ldap._tcp.pdc._msdcs
_ldap._tcp.dc._msdcs
_ldap._tcp.gc._msdcs
_gc._tcp
_kerberos._tcp
_kerberos._udp
_kerberos._tcp.dc._msdcs
_kpasswd._tcp
_kpasswd._udp
_kerberos._tcp.default-first-site-name._sites.gc._msdcs
_kerberos._tcp.default-first-site-name._sites.dc._msdcs
_kerberos._tcp.default-first-site-name._sites
_ldap._tcp.default-first-site-name._sites.gc._msdcs
_ldap._tcp.default-first-site-name._sites.dc._msdcs
_ldap._tcp.default-first-site-name._sites
_gc._tcp.default-first-site-name._sites

( See/check also http://technet.microsoft.com/en-us/library/cc961719.aspx ).
Comment 1 Dmitry Galkin univentionstaff 2014-10-27 15:53:09 CET
r54957:
  * Bug #34222:
    90_ucsschool/97_samba4_dns_srv_replication: test the DNS SRV
    records replication.

The following behavior was observed and it's assumed as a correct one by the test, please confirm:

1. DC-Master in multi-school setup: replication does not work neither from openLDAP -> Samba, nor the other way from Samba -> openLDAP. This is determined by the UCR vars 'connector/s4/mapping/dns/srv_record/*'"ignore" that make connector ignore changes to a number of records.

2. DC-Slave in a multi-school setup: does not work neither from openLDAP -> Samba, nor the other way around. This is determined by the UCR vars 'connector/s4/mapping/dns/srv_record/*' with hardcoded values.

3. DC-Master as a single-master setup: replication works both ways: openLDAP <---> Samba.

The list of DNS SRV records that are checked is a bit different from the one above, please confirm as well if that is OK.
The differences are:
'_ldap._tcp._msdcs' - does not exist;
'_kerberos-adm._tcp' - this record was not in the original list, but present and seems to be relevant as it is used in installation scripts for both Master (62ucs-school-master.inst) and Slave (62ucs-school-slave.inst);
'_kerberos._tcp.Default-First-Site-Name._sites.gc._msdcs' - does not exist;
'_ldap._tcp.Default-First-Site-Name._sites.gc._msdcs' - is not ignored by the S4-Connector (perhaps, it should be?).
Comment 2 Ammar Najjar univentionstaff 2014-12-16 10:42:30 CET
The test script produced a traceback in the last Jenkins run:
http://jenkins.knut.univention.de:8080/job/UCSschool%204.0/job/UCSschool%204.0%20Singleserver/11/SambaVersion=s4/testReport/90_ucsschool/97_samba4_dns_srv_replication/test/


Checking that DNS SRV '_ldap._tcp.pdc._msdcs' record has the same attribute values in LDAP and Samba.
### FAIL ###
An error occured while trying to get the '_ldap._tcp.pdc._msdcs' DNS SRV record via 'univention-s4search': 'univention-s4search dc=_ldap._tcp.pdc._msdcs', STDERR:
 WARNING: No path in service IPC$ - making it unavailable!
NOTE: Service IPC$ is flagged unavailable.

###      ###
.
.
.
Modifying the DNS SRV '_ldap._tcp.pdc._msdcs' record in Samba using 'ldbmodify', delete(-ing) a test value:
### FAIL ###
An error occured while trying to modify the '_ldap._tcp.pdc._msdcs' SRV record via 'ldbmodify': 'ldbmodify -H /var/lib/samba/private/sam.ldb --user=Administrator%univention -k no'. STDERR:
 'ERR: (No such attribute) "attribute 'dnsRecord': no matching attribute value while deleting attribute on 'DC=_ldap._tcp.pdc._msdcs,DC=autotest201.local,CN=MicrosoftDNS,CN=System,DC=autotest201,DC=local'" on DN DC=_ldap._tcp.pdc._msdcs,DC=autotest201.local,CN=MicrosoftDNS,CN=System,dc=autotest201,dc=local at block before line 3
'
###      ###
Comment 3 Dmitry Galkin univentionstaff 2014-12-16 11:46:46 CET
(In reply to Ammar Najjar from comment #2)
> The test script produced a traceback in the last Jenkins run:
> http://jenkins.knut.univention.de:8080/job/UCSschool%204.0/job/UCSschool%204.
> 0%20Singleserver/11/SambaVersion=s4/testReport/90_ucsschool/
> 97_samba4_dns_srv_replication/test/

With an update to UCS 4, the following message commonly appears in stderr:

>  WARNING: No path in service IPC$ - making it unavailable!
> NOTE: Service IPC$ is flagged unavailable.

which makes problems for a bunch of tests in ucs@school, as some are checking stderr for any messages, not for concrete onces. Would be nice to fix the cause of the messages or suppress them somehow if possible, otherwise at least the following tests should be modified:

90_ucsschool.90_samba4_sysvol_replication.test
90_ucsschool.91_samba4_gpc_two_way_replication.test
90_ucsschool.92_samba4_check_denied_user_creation.test
90_ucsschool.95_samba4_client_join_on_slave.test
90_ucsschool.97_samba4_dns_srv_replication.test
Comment 4 Arvid Requate univentionstaff 2014-12-16 13:39:45 CET
> otherwise at least the following tests should be modified:

Please create a new bug for that.
Comment 5 Dmitry Galkin univentionstaff 2014-12-16 15:17:42 CET
(In reply to Arvid Requate from comment #4)
> Please create a new bug for that.

Opened: Bug #37362
Comment 6 Dmitry Galkin univentionstaff 2015-01-20 16:02:33 CET
As following fixed:
https://forge.univention.org/bugzilla/show_bug.cgi?id=37362
I mark this one fixed.
Comment 7 Arvid Requate univentionstaff 2015-02-05 20:21:46 CET
Something still fails in the test:

========================================================================
Modifying the DNS SRV '_ldap._tcp.pdc._msdcs' record in Samba using 'ldbmodify', delete(-ing) a test value:
### FAIL ###
An error occured while trying to modify the '_ldap._tcp.pdc._msdcs' SRV record via 'ldbmodify': 'ldbmodify -H /var/lib/samba/private/sam.ldb --user=Administrator%univention -k no'. STDERR:
 'ERR: (No such attribute) "attribute 'dnsRecord': no matching attribute value while deleting attribute on 'DC=_ldap._tcp.pdc._msdcs,DC=autotest201.local,CN=MicrosoftDNS,CN=System,DC=autotest201,DC=local'" on DN DC=_ldap._tcp.pdc._msdcs,DC=autotest201.local,CN=MicrosoftDNS,CN=System,dc=autotest201,dc=local at block before line 3
'
###      ###
========================================================================

http://jenkins.knut.univention.de:8080/job/UCSschool%204.0/job/UCSschool%204.0%20Singleserver/16/SambaVersion=s4/testReport/90_ucsschool/97_samba4_dns_srv_replication/test/
Comment 8 Dmitry Galkin univentionstaff 2015-02-06 11:04:34 CET
(In reply to Arvid Requate from comment #7)
> Something still fails in the test:
> 

> http://jenkins.knut.univention.de:8080/job/UCSschool%204.0/job/UCSschool%204.
> 0%20Singleserver/16/SambaVersion=s4/testReport/90_ucsschool/
> 97_samba4_dns_srv_replication/test/


In this specific case, there was a well-known message in the output:
An error occured while trying to get the '_ldap._tcp.pdc._msdcs' DNS SRV record via 'univention-s4search': 'univention-s4search dc=_ldap._tcp.pdc._msdcs', STDERR:
 WARNING: No path in service IPC$ - making it unavailable!
NOTE: Service IPC$ is flagged unavailable.

--> I guess the changes from Bug #37362 were not synchronized to Appcenter back than.


But looking into newer test outputs: http://jenkins.knut.univention.de:8080/job/UCSschool%204.0/job/UCSschool%204.0%20Multiserver/16/SambaVersion=s4-only-master/testReport/90_ucsschool/97_samba4_dns_srv_replication/test/

Checking the DNS SRV '_ldap._tcp.pdc._msdcs' record replication from openLDAP to Samba:
### FAIL ###
The replication from openLDAP to Samba did not work in a case it should have worked. Record '_ldap._tcp.pdc._msdcs'. Test record values: '['53', '777', '63256', 'ucs_test.hostname.local']', state in Samba '(['0'], ['100'], ['389'], ['master203.autotest203.local'])'. The following values were replicated: '[]'
###      ###

Seems like replication does not work. Perhaps you can have a look at Comment 1 of this bug.


An in the following case: http://jenkins.knut.univention.de:8080/job/UCSschool%204.0/job/UCSschool%204.0%20Multiserver/16/SambaVersion=s4-school-only/testReport/90_ucsschool/97_samba4_dns_srv_replication/test/

We have lots of tracebacks from 'univention-dnsedit'
Comment 9 Dmitry Galkin univentionstaff 2015-02-20 10:12:11 CET
As of the last test run at: http://jenkins.knut.univention.de:8080/job/UCSschool%204.0/job/UCSschool%204.0%20Singleserver/22/SambaVersion=s4-all-components/testReport/junit/90_ucsschool/97_samba4_dns_srv_replication/test/

Checking the DNS SRV '_ldap._tcp.pdc._msdcs' record replication from openLDAP to Samba:
### FAIL ###
The replication from openLDAP to Samba did not work in a case it should have worked. Record '_ldap._tcp.pdc._msdcs'. Test record values: '['53', '777', '63256', 'ucs_test.hostname.local']', state in Samba '(['0'], ['100'], ['389'], ['master201.autotest201.local'])'. The following values were replicated: '[]'
###      ###


The replication did not work. Several test runs in a row on the same configuration the replication did work. Does not seem to be a timing issue as there is a function that waits for replication.

On another configuration, the same behavior can be observed. Last run failed, several times in a row before that passed: http://jenkins.knut.univention.de:8080/job/UCSschool%204.0/job/UCSschool%204.0%20Singleserver/SambaVersion=s4/22/testReport/junit/90_ucsschool/97_samba4_dns_srv_replication/test/

I mark Bug fixed. Please pay attention to Comment 1 when reviewing.
Comment 10 Dmitry Galkin univentionstaff 2015-02-27 14:55:14 CET
The test is disabled for now:
r58521:
  * 90_ucsschool/97_samba4_dns_srv_replication: disabled the test as it fails
    in some configurations (Bug #34222).

Should be reviewed and re-enabled after the problem is fixed.


Currently test fails in:

1. Multiserver "s4-school-only" with trace from univention-dnsedit: http://jenkins.knut.univention.de:8080/job/UCSschool%204.0/job/UCSschool%204.0%20Multiserver/21/SambaVersion=s4-school-only/testReport/junit/90_ucsschool/97_samba4_dns_srv_replication/test/

2. Multiserver "s4-only-master" where replication does not work: http://jenkins.knut.univention.de:8080/job/UCSschool%204.0/job/UCSschool%204.0%20Multiserver/21/SambaVersion=s4-only-master/testReport/90_ucsschool/97_samba4_dns_srv_replication/test/

3. Also Singleserver "s4-all-components", no replication: http://jenkins.knut.univention.de:8080/job/UCSschool%204.0/job/UCSschool%204.0%20Singleserver/22/SambaVersion=s4-all-components/testReport/90_ucsschool/97_samba4_dns_srv_replication/test/

4. Singleserver "s4", no replication:
http://jenkins.knut.univention.de:8080/job/UCSschool%204.0/job/UCSschool%204.0%20Singleserver/22/SambaVersion=s4/testReport/90_ucsschool/97_samba4_dns_srv_replication/test/


But works in other configurations.
Comment 11 Ammar Najjar univentionstaff 2015-03-10 10:25:35 CET
Script is also skipped on 4.0-1 and 3.2-5. Please enable when this bug is fixed.
Comment 12 Arvid Requate univentionstaff 2015-03-10 14:33:31 CET
Reply to Comment 1:

> please confirm:
> 
> 1. DC-Master in multi-school setup: replication does not work neither from
> openLDAP -> Samba, nor the other way from Samba -> openLDAP. This is
> determined by the UCR vars 'connector/s4/mapping/dns/srv_record/*'"ignore"
> that make connector ignore changes to a number of records.

Yes, exactly: Samba on all Samba4 DCs in the central school department is left on it's own with regards to the care about these DNS-Records. This is achieved by setting the corresponding UCR-Variables to "ignore". My idea about the test was to check that the end result of all replication (Listener, S4-Connectors and DRS) is the intended one.


> 2. DC-Slave in a multi-school setup: does not work neither from openLDAP ->
> Samba, nor the other way around. This is determined by the UCR vars
> 'connector/s4/mapping/dns/srv_record/*' with hardcoded values.

Exactly, the Samba-Slave-PDCs run their own S4-Connector and these are instructed to write the values specified in the UCR-Variables. The end result should be checked.


> 3. DC-Master as a single-master setup: replication works both ways:
> openLDAP <---> Samba.

Yes, that's probably ok since there are no Slave-PDCs which would need to get masked out.

> The list of DNS SRV records that are checked is a bit different from the
> one above, please confirm as well if that is OK.
> The differences are:
> '_ldap._tcp._msdcs' - does not exist;

Typo, this should have been just "_ldap._tcp"

> '_kerberos-adm._tcp' - this record was not in the original list, but present
> and seems to be relevant as it is used in installation scripts for both
> Master (62ucs-school-master.inst) and Slave (62ucs-school-slave.inst);

Yes, you may test that as well. Currently it's probably not particularly  relevant as Samba doesn't offer a kadmind (port 749) interface yet AFAIK.

>  '_kerberos._tcp.Default-First-Site-Name._sites.gc._msdcs' - does not exist;

Ok, copy and paste error, just skip it.

> '_ldap._tcp.Default-First-Site-Name._sites.gc._msdcs' - is not ignored
> by the S4-Connector (perhaps, it should be?).

Yes, thanks, good point, I created Bug 37994 for this.
Comment 13 Dmitry Galkin univentionstaff 2015-03-17 15:20:22 CET
Modified the list of records according to Comment 12:

r59120:
  * 97_samba4_dns_srv_replication: changed list of test records (Bug #34222).


And opened a Bug #38064.


Are there any ideas regarding the other environments where the test failed (Comment 10)? Should it run in those configurations? are there any restrictions apply or that indicates a bug?
Comment 14 Dmitry Galkin univentionstaff 2015-04-20 16:55:32 CEST
Ok, the retab done with r58113 broke the "sed" statement used to determine the domain configuration -> this led to system recognized as single server DC-Master in case it was actually multiserver with DC-Slaves in the domain by the test.

 And thus wrong test scenario was selected...

r59984:
  * 90_ucsschool/97_samba4_dns_srv_replication: fix regular expression
    after retab (Bug #34222).
Comment 15 Dmitry Galkin univentionstaff 2015-04-21 11:23:53 CEST
I've checked the test once more:

1. The test determines possible scenarios.
2. If replication should not work - > it will check that no values were replicated.
3. If the replication should work -> it will check that every single value was replicated.
4. Added Workaround for Bug #38064.


Test works in different scenarios:

On DC-Master (Multi): http://hutten.knut.univention.de/pastebin/m526f28a9
On DC-Slave: http://hutten.knut.univention.de/pastebin/m52dc9392

So I've enabled it for 4.0-1:
r60007:
  * 90_ucsschool/97_samba4_dns_srv_replication: some 'style' changes;
    enabled the test (Bug #34222)