Bug 34216 - Samba4 @school test: GPC replication Master S4 to Slave S4
Summary: Samba4 @school test: GPC replication Master S4 to Slave S4
Status: RESOLVED WONTFIX
Alias: None
Product: UCS@school
Classification: Unclassified
Component: ucs-test
Version: UCS@school 4.1 R2
Hardware: Other Linux
: P5 normal
Target Milestone: ---
Assignee: Lukas Oyen
QA Contact:
URL:
Keywords:
Depends on:
Blocks: 34214
  Show dependency treegraph
 
Reported: 2014-03-03 12:03 CET by Arvid Requate
Modified: 2019-02-05 21:43 CET (History)
3 users (show)

See Also:
What kind of report is it?: ---
What type of bug is this?: ---
Who will be affected by this bug?: ---
How will those affected feel about the bug?: ---
User Pain:
Enterprise Customer affected?:
School Customer affected?:
ISV affected?:
Waiting Support:
Flags outvoted (downgraded) after PO Review:
Ticket number:
Bug group (optional):
Customer ID:
Max CVSS v3 score:
requate: Patch_Available+


Attachments
Reduce flakiness in 91_samba4_gpc_two_way_replication (2.73 KB, patch)
2017-01-02 16:51 CET, Lukas Oyen
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Arvid Requate univentionstaff 2014-03-03 12:03:01 CET
In UCS@School the Group Policy Container (GPC) Objects are replicated via OpenLDAP. Some test cases should be implemented to check this:

* Verify that a GPO created in the Samba directory on the UCS@school Samba4 DC Master is replicated to the Samba directory of a school server (UCS@school Samba4 DC Slave).

* Link the GPO to the School OU in the Samba directory of the Master and verify that this link is replicated to the Samba directory of the school slave as well.
Comment 1 Dmitry Galkin univentionstaff 2014-08-26 15:23:32 CEST
r53037:
  * Bug #34214 and #34216:
    90_ucsschool/91_samba4_gpc_two_way_replication: test the GPC objects and 
    links replication from DC-Master to DC-Slave or from DC-Slave to DC-Master;
    90_ucsschool/essential/test_samba4.py: a helper class for Samba4 related
    tests.

Test covers cases in this bug and bug #34214.
Comment 2 Arvid Requate univentionstaff 2014-09-11 18:02:34 CEST
Running the test in my environment on a DC Slave resulted in a traceback. I guess you have to add "-k no" to the samba-tool command.

=============================================================================
running command: samba-tool gpo create ucs_test_school_gpo_cr2e033o -H ldap://master90.ar321s1.school --username Administrator --password univention

An error message while creating a GPO using 'samba-tool' on the remote host 'ldap://master90.ar321s1.school'. STDERR:
SPNEGO(gssapi_krb5) creating NEG_TOKEN_INIT failed: NT_STATUS_INVALID_PARAMETER
GSS client Update(krb5)(1) Update failed:  Miscellaneous failure (see text): Matching credential (cifs/master90.ar321s1.school@AR321S1.SCHOOL) not found
ERROR(runtime): Error connecting to 'master90.ar321s1.school' using SMB - (-1073741715, 'Logon failure')
=============================================================================

The "running command" debug message was introduced by me... But it would actually be a good idea to output the command in case of an error. Maybe that's possible?
Comment 3 Dmitry Galkin univentionstaff 2014-09-12 12:05:11 CEST
(In reply to Arvid Requate from comment #2)
> Running the test in my environment on a DC Slave resulted in a traceback. I
> guess you have to add "-k no" to the samba-tool command.

Strange..have not previously happened in my env.

> The "running command" debug message was introduced by me... But it would
> actually be a good idea to output the command in case of an error. Maybe
> that's possible?

r53616:
  * 90_ucsschool/91_samba4_gpc_two_way_replication: added more debug output,
    made sure kerberos authentication won't be used (Bug #34216);
    90_ucsschool/essential/test_samba4.py: added more debug output.
Comment 4 Dmitry Galkin univentionstaff 2014-09-12 12:07:29 CEST
P.S. The test does not wait for SYSVOL replication to happen, as there is a Bug #35619
Comment 5 Dmitry Galkin univentionstaff 2014-10-28 11:43:50 CET
I've noticed the test failed:
http://jenkins.knut.univention.de:8080/job/UCSschool%203.2%20Multiserver/107/SambaVersion=s4-school-only-with-slave/testReport/junit/90_ucsschool/91_samba4_gpc_two_way_replication/test/

as in that Jenkins template (204) we have no "univention-samba4" on DC-Master. Will update the test to take it into account.
Comment 6 Dmitry Galkin univentionstaff 2014-10-28 12:16:27 CET
r55012:
  * Bug #34216:
    90_ucsschool/91_samba4_gpc_two_way_replication: skip the test when there is
    no Samba4 on the DC-Master.
Comment 7 Arvid Requate univentionstaff 2015-02-05 20:44:33 CET
That patch is only in branches/ucs-3.2/ucs-school-3.2r2, please merge it also to branches/ucs-4.0/ucs-school-4.0. Maybe you could put that code into the test_samba4 library module so that it can be used in Bug 34224 Comment 5 as well.
Comment 8 Dmitry Galkin univentionstaff 2015-02-06 11:39:21 CET
(In reply to Arvid Requate from comment #7)
> That patch is only in branches/ucs-3.2/ucs-school-3.2r2, please merge it
> also to branches/ucs-4.0/ucs-school-4.0. Maybe you could put that code into
> the test_samba4 library module so that it can be used in Bug 34224 Comment 5
> as well.

Yep, thanks!

r57823:
  * 90_ucsschool/91_samba4_gpc_two_way_replication and
    essential/test_samba4.py: applied missing changes
    from ucs school 3.2r2 (Bug #34216).
Comment 9 Arvid Requate univentionstaff 2015-02-16 19:55:54 CET
I think the "samba-tool gpo create" command requires the -H option here, too, to select the specific server where the GPO should be created.
Comment 10 Dmitry Galkin univentionstaff 2015-02-17 15:05:52 CET
(In reply to Arvid Requate from comment #9)
> I think the "samba-tool gpo create" command requires the -H option here,
> too, to select the specific server where the GPO should be created.

The host is always specified for the test, please check the source:

               cmd = ("samba-tool", "gpo", "create", display_name,
                       self.host_or_ip, self.remote_host, "-k", "no",
                       "--username", self.admin_username,
                       "--password", self.admin_password)

there is either a hostname or an IP is used.


Test run:
http://jenkins.knut.univention.de:8080/job/UCSschool%204.0/job/UCSschool%204.0%20Singleserver/21/SambaVersion=s4-with-slave/testReport/90_ucsschool/91_samba4_gpc_two_way_replication/test/
Comment 11 Dmitry Galkin univentionstaff 2015-02-17 15:11:03 CET
(In reply to Dmitry Galkin from comment #10)
> Test run:
> http://jenkins.knut.univention.de:8080/job/UCSschool%204.0/job/UCSschool%204.
> 0%20Singleserver/21/SambaVersion=s4-with-slave/testReport/90_ucsschool/
> 91_samba4_gpc_two_way_replication/test/


Those test results are actually for the other case: replication from Slave to Master, i.e. Bug #34214
Comment 12 Dmitry Galkin univentionstaff 2015-02-20 12:55:56 CET
(In reply to Dmitry Galkin from comment #11)
> (In reply to Dmitry Galkin from comment #10)
> > Test run:
> > http://jenkins.knut.univention.de:8080/job/UCSschool%204.0/job/UCSschool%204.
> > 0%20Singleserver/21/SambaVersion=s4-with-slave/testReport/90_ucsschool/
> > 91_samba4_gpc_two_way_replication/test/
> 
> 
> Those test results are actually for the other case: replication from Slave
> to Master, i.e. Bug #34214

Also see Bug #34214 Comment 9.
Comment 13 Stefan Gohmann univentionstaff 2016-12-07 13:28:02 CET
The test case currently fails:

[2016-12-06 22:44:10.107789] 
[2016-12-06 22:44:10.107882] Obtaining Administrator username and password for the test from the UCR
[2016-12-06 22:44:10.119038] 
[2016-12-06 22:44:10.119087] Current server role is DC-Master, trying to find a DC-Slave in the domain for the test
[2016-12-06 22:44:10.119142] 
[2016-12-06 22:44:10.119185]  create_and_run_process(('udm', 'computers/domaincontroller_slave', 'list', '--filter', 'service=Samba 4'), shell=False)
[2016-12-06 22:44:10.294787] 
[2016-12-06 22:44:10.294818]  create_and_run_process(('sed', '-n', 's/^  ip: //p'), shell=False)
[2016-12-06 22:44:10.297335] 
[2016-12-06 22:44:10.297357] The DC-Slave(s) with the following IP address(-es) were found in the domain: '['10.210.4.169', '10.210.231.220', '10.210.202.118', '10.210.88.114']'
[2016-12-06 22:44:10.297374] The following DC-Slave '10.210.4.169' will be selected as the remote host for the test
[2016-12-06 22:44:10.297386] Creating a Group Policy Object (GPO) on the host '10.210.4.169' with a display name 'ucs_test_school_gpo_k06gibfq' using 'samba-tool'
[2016-12-06 22:44:10.297403]  create_and_run_process(('samba-tool', 'gpo', 'create', 'ucs_test_school_gpo_k06gibfq', '--ipaddress', '10.210.4.169', '-k', 'no', '--username', 'Administrator', '--password', 'univention'), shell=False)
[2016-12-06 22:44:12.589710] 
[2016-12-06 22:44:12.589748] Samba-tool produced the following output: GPO 'ucs_test_school_gpo_k06gibfq' created as {33815E7B-1F8D-4442-9CC5-A02ACE6618E4}
[2016-12-06 22:44:12.589763] Checking if GPO '{33815E7B-1F8D-4442-9CC5-A02ACE6618E4}' was replicated to the current DC
[2016-12-06 22:44:42.620933] 
[2016-12-06 22:44:42.620973]  create_and_run_process(('samba-tool', 'gpo', 'show', '{33815E7B-1F8D-4442-9CC5-A02ACE6618E4}', '--ipaddress=10.210.84.36', '--username', 'Administrator', '--password', 'univention'), shell=False)
[2016-12-06 22:44:43.560646] 
[2016-12-06 22:44:43.560678] Selecting the School OU for the test
[2016-12-06 22:44:43.560692]  create_and_run_process(('udm', 'computers/domaincontroller_slave', 'list', '--filter', 'service=Samba 4'), shell=False)
[2016-12-06 22:44:44.013200] 
[2016-12-06 22:44:44.013232]  create_and_run_process(('sed', '-n', 's/^DN: //p'), shell=False)
[2016-12-06 22:44:44.031207] 
[2016-12-06 22:44:44.031230] select_school_ou: SchoolSearchBase found these OUs: ['ou=School1,dc=autotest300,dc=local', 'ou=School3,dc=autotest300,dc=local', 'ou=School2,dc=autotest300,dc=local']
[2016-12-06 22:44:44.031246] Linking 'ou=School1,dc=autotest300,dc=local' container and '{33815E7B-1F8D-4442-9CC5-A02ACE6618E4}' GPO on the remote host '10.210.4.169' using 'samba-tool'
[2016-12-06 22:44:44.031262]  create_and_run_process(('samba-tool', 'gpo', 'setlink', 'ou=School1,dc=autotest300,dc=local', '{33815E7B-1F8D-4442-9CC5-A02ACE6618E4}', '--ipaddress', '10.210.4.169', '-k', 'no', '--username', 'Administrator', '--password', 'univention'), shell=False)
[2016-12-06 22:44:45.179999] 
[2016-12-06 22:44:45.180034] Executing cmd: ('samba-tool', 'gpo', 'setlink', 'ou=School1,dc=autotest300,dc=local', '{33815E7B-1F8D-4442-9CC5-A02ACE6618E4}', '--ipaddress', '10.210.4.169', '-k', 'no', '--username', 'Administrator', '--password', 'univention')
[2016-12-06 22:44:45.180047] An error message while creating a GPO link using 'samba-tool' on the remote host '10.210.4.169'. STDERR:
[2016-12-06 22:44:45.180058] ERROR(<type 'exceptions.KeyError'>): uncaught exception - 'No such element'
[2016-12-06 22:44:45.180068]   File "/usr/lib/python2.7/dist-packages/samba/netcmd/__init__.py", line 176, in _run
[2016-12-06 22:44:45.180078]     return self.run(*args, **kwargs)
[2016-12-06 22:44:45.180089]   File "/usr/lib/python2.7/dist-packages/samba/netcmd/gpo.py", line 624, in run
[2016-12-06 22:44:45.180098]     cmd_getlink().run(container_dn, H, sambaopts, credopts, versionopts)
[2016-12-06 22:44:45.180107]   File "/usr/lib/python2.7/dist-packages/samba/netcmd/gpo.py", line 524, in run
[2016-12-06 22:44:45.180117]     if msg['gPLink']:
[2016-12-06 22:44:45.180124] ### FAIL ###
[2016-12-06 22:44:45.231723] The linked School OU (Container) was not referenced in the 'samba-tool' output
[2016-12-06 22:44:45.231736] ###      ###
[2016-12-06 22:45:15.204754] 
[2016-12-06 22:45:15.204791] Removing previously created Group Policy Object (GPO) with a reference: {33815E7B-1F8D-4442-9CC5-A02ACE6618E4}
[2016-12-06 22:45:15.204807]  create_and_run_process(('samba-tool', 'gpo', 'del', '{33815E7B-1F8D-4442-9CC5-A02ACE6618E4}', '--username=Administrator', '--password=univention'), shell=False)
[2016-12-06 22:45:17.600685] 
[2016-12-06 22:45:17.600719] Executing cmd: ('samba-tool', 'gpo', 'del', '{33815E7B-1F8D-4442-9CC5-A02ACE6618E4}', '--username=Administrator', '--password=univention')
[2016-12-06 22:45:17.600731] An error message while removing the GPO using 'samba-tool':
[2016-12-06 22:45:17.600747] ERROR(<type 'exceptions.SystemError'>): uncaught exception - error return without exception set
[2016-12-06 22:45:17.600761]   File "/usr/lib/python2.7/dist-packages/samba/netcmd/__init__.py", line 176, in _run
[2016-12-06 22:45:17.600770]     return self.run(*args, **kwargs)
[2016-12-06 22:45:17.600780]   File "/usr/lib/python2.7/dist-packages/samba/netcmd/gpo.py", line 1088, in run
[2016-12-06 22:45:17.600788]     conn.deltree(sharepath)
[2016-12-06 22:45:17.600799] Samba-tool produced the following output:
[2016-12-06 22:45:17.600808] GPO {33815E7B-1F8D-4442-9CC5-A02ACE6618E4} is linked to containers
[2016-12-06 22:45:17.600817]     Removed link from OU=School1,DC=autotest300,DC=local.

http://jenkins.knut.univention.de:8080/job/UCSschool%204.1/job/UCSschool%204.1%20(R2)%20Large%20Environment/42/testReport/master300.90_ucsschool/91_samba4_gpc_two_way_replication/master300_test/

I've disabled the test case: r75060
Comment 14 Lukas Oyen univentionstaff 2017-01-02 16:51:57 CET
Created attachment 8327 [details]
Reduce flakiness in 91_samba4_gpc_two_way_replication

I see two possible reasons for the flakiness of the given test case:

1) The replication timeout is set to low. This is a simple `time.sleep()`.
2) The wrong (non-school) slave is selected.

Details for 2): The testcase detects, that it is running on a S4-Master and searches for S4-Slaves. If multiple are found, the first is selected. Whichever is the first may vary from run to run. In every failed test-run in the `UCSschool 4.1 (R2) Large Environment` I examined, the selected slave was a school slave (one of slave300-s1, slave300-s2,slave300-s3) and not the slave3002.

The attached patch limits the slave-search to those, that are configured as UCS@School slaves. This limits the test-case to what is described in comment #1. This may reduce the flakiness, but not necessarily fixes the error.
Comment 15 Lukas Oyen univentionstaff 2017-01-24 13:29:35 CET
Uploaded the attempt to reduce flakiness in r76050. Improvements to school-OU selection were uploaded in r76051. Test is reenabled.
Comment 16 Florian Best univentionstaff 2017-01-24 14:28:57 CET
(In reply to Lukas Oyen from comment #15)
> Uploaded the attempt to reduce flakiness in r76050. Improvements to
> school-OU selection were uploaded in r76051. Test is reenabled.
REOPEN: don't use univention.uldap! use univention.admin.uldap instead.
Comment 17 Florian Best univentionstaff 2017-01-24 14:29:49 CET
(In reply to Florian Best from comment #16)
> (In reply to Lukas Oyen from comment #15)
> > Uploaded the attempt to reduce flakiness in r76050. Improvements to
> > school-OU selection were uploaded in r76051. Test is reenabled.
> REOPEN: don't use univention.uldap! use univention.admin.uldap instead.
→ See Bug #41368
Comment 18 Lukas Oyen univentionstaff 2017-01-24 16:32:00 CET
(In reply to Florian Best from comment #17)
> (In reply to Florian Best from comment #16)
> > (In reply to Lukas Oyen from comment #15)
> > > Uploaded the attempt to reduce flakiness in r76050. Improvements to
> > > school-OU selection were uploaded in r76051. Test is reenabled.
> > REOPEN: don't use univention.uldap! use univention.admin.uldap instead.
> → See Bug #41368

Updated in r76064 to use `univention.admin.uldap`.
Comment 19 Sönke Schwardt-Krummrich univentionstaff 2019-02-05 21:43:21 CET
This issue has been filled against UCS@school 4.1 (R2). The maintenance with
bug and security fixes for UCS@school 4.1 (R2) has ended on 5th of April 2018.

Customers still on UCS 4.1 are encouraged to update to UCS 4.3 (or later). 
Please contact your partner or Univention for any questions.

If this issue still occurs in newer UCS versions, please use "Clone this bug"
or simply reopen the issue. In this case please provide detailed information on
how this issue is affecting you.