Comment 1 Daniel Tröder 2020-01-12 12:49:18 CET

All tests except for one succeed now.

The test "01_school_resource → test_05_create_ou" cannot work atm, because OU creation is not yet implemented.

[feature/kelvin dc71a6197] Bug #50678: make BB-API ucs-tests for Kelvin-API more reliable

Comment 2 Jürn Brodersen 2020-01-15 09:24:53 CET

Is the app not updated?

Test 03_user_resource:


======================================================================
ERROR: test_06_create_user_without_name (__main__.Test)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "03_user_resource", line 291, in test_06_create_user_without_name
    self.assertEqual(result["name"], expected_username)
KeyError: u'name'

======================================================================
FAIL: test_02_create_user_parallel_from_external_different_classes (__main__.Test)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "03_user_resource", line 69, in test_02_create_user_parallel_from_external_different_classes
    self.compare_import_user_and_resource(user, result)
  File "/usr/lib/pymodules/python2.7/univention/testing/ucsschool/kelvin_api.py", line 269, in compare_import_user_and_resource
    self.assertSetEqual(set(import_user.udm_properties.keys()), set(v.keys()))
AssertionError: Items in the first set but not the second:
u'employeeType'
u'description'
u'title'
u'organisation'
u'uidNumber'
u'phone'
u'gidNumber'

======================================================================
FAIL: test_03_create_user_parallel_from_external_same_classes (__main__.Test)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "03_user_resource", line 136, in test_03_create_user_parallel_from_external_same_classes
    self.compare_import_user_and_resource(user, result)
  File "/usr/lib/pymodules/python2.7/univention/testing/ucsschool/kelvin_api.py", line 269, in compare_import_user_and_resource
    self.assertSetEqual(set(import_user.udm_properties.keys()), set(v.keys()))
AssertionError: Items in the first set but not the second:
u'employeeType'
u'description'
u'title'
u'organisation'
u'uidNumber'
u'phone'
u'gidNumber'

======================================================================
FAIL: test_04_partial_update_user_parallel_from_external_different_classes (__main__.Test)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "03_user_resource", line 211, in test_04_partial_update_user_parallel_from_external_different_classes
    self.compare_import_user_and_resource(user, result)
  File "/usr/lib/pymodules/python2.7/univention/testing/ucsschool/kelvin_api.py", line 269, in compare_import_user_and_resource
    self.assertSetEqual(set(import_user.udm_properties.keys()), set(v.keys()))
AssertionError: Items in the first set but not the second:
u'employeeType'
u'description'
u'title'
u'organisation'
u'uidNumber'
u'phone'
u'gidNumber'

======================================================================
FAIL: test_05_rename_single_user (__main__.Test)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "03_user_resource", line 278, in test_05_rename_single_user
    self.compare_import_user_and_resource(user, resource_new)
  File "/usr/lib/pymodules/python2.7/univention/testing/ucsschool/kelvin_api.py", line 269, in compare_import_user_and_resource
    self.assertSetEqual(set(import_user.udm_properties.keys()), set(v.keys()))
AssertionError: Items in the first set but not the second:
u'employeeType'
u'description'
u'title'
u'organisation'
u'uidNumber'
u'phone'
u'gidNumber'

======================================================================
FAIL: test_07_move_teacher_one_school_only (__main__.Test)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "03_user_resource", line 346, in test_07_move_teacher_one_school_only
    self.compare_import_user_and_resource(user, resource_new)
  File "/usr/lib/pymodules/python2.7/univention/testing/ucsschool/kelvin_api.py", line 269, in compare_import_user_and_resource
    self.assertSetEqual(set(import_user.udm_properties.keys()), set(v.keys()))
AssertionError: Items in the first set but not the second:
u'employeeType'
u'description'
u'title'
u'organisation'
u'uidNumber'
u'phone'
u'gidNumber'

======================================================================
FAIL: test_08_move_teacher_remove_primary (__main__.Test)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "03_user_resource", line 426, in test_08_move_teacher_remove_primary
    self.compare_import_user_and_resource(user, resource_new)
  File "/usr/lib/pymodules/python2.7/univention/testing/ucsschool/kelvin_api.py", line 269, in compare_import_user_and_resource
    self.assertSetEqual(set(import_user.udm_properties.keys()), set(v.keys()))
AssertionError: Items in the first set but not the second:
u'employeeType'
u'description'
u'title'
u'organisation'
u'uidNumber'
u'phone'
u'gidNumber'

======================================================================
FAIL: test_09_move_teacher_remove_primary_with_classes (__main__.Test)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "03_user_resource", line 507, in test_09_move_teacher_remove_primary_with_classes
    self.compare_import_user_and_resource(user, resource_new)
  File "/usr/lib/pymodules/python2.7/univention/testing/ucsschool/kelvin_api.py", line 269, in compare_import_user_and_resource
    self.assertSetEqual(set(import_user.udm_properties.keys()), set(v.keys()))
AssertionError: Items in the first set but not the second:
u'employeeType'
u'description'
u'title'
u'organisation'
u'uidNumber'
u'phone'
u'gidNumber'

======================================================================
FAIL: test_10_move_teacher_remove_primary_no_classes_in_new_school (__main__.Test)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "03_user_resource", line 589, in test_10_move_teacher_remove_primary_no_classes_in_new_school
    self.compare_import_user_and_resource(user, resource_new)
  File "/usr/lib/pymodules/python2.7/univention/testing/ucsschool/kelvin_api.py", line 269, in compare_import_user_and_resource
    self.assertSetEqual(set(import_user.udm_properties.keys()), set(v.keys()))
AssertionError: Items in the first set but not the second:
u'employeeType'
u'description'
u'title'
u'organisation'
u'uidNumber'
u'phone'
u'gidNumber'

======================================================================
FAIL: test_11_move_teacher_remove_primary_with_classes_and_rename (__main__.Test)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "03_user_resource", line 672, in test_11_move_teacher_remove_primary_with_classes_and_rename
    self.compare_import_user_and_resource(user, resource_new)
  File "/usr/lib/pymodules/python2.7/univention/testing/ucsschool/kelvin_api.py", line 269, in compare_import_user_and_resource
    self.assertSetEqual(set(import_user.udm_properties.keys()), set(v.keys()))
AssertionError: Items in the first set but not the second:
u'employeeType'
u'description'
u'title'
u'organisation'
u'uidNumber'
u'phone'
u'gidNumber'

======================================================================
FAIL: test_12_modify_teacher_remove_all_classes (__main__.Test)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "03_user_resource", line 726, in test_12_modify_teacher_remove_all_classes
    self.assertEqual(resource_new['school_classes'], {})
AssertionError: {u'bm3ts7': [u'lvdd', u'pvid']} != {}
- {u'bm3ts7': [u'lvdd', u'pvid']}
+ {}

----------------------------------------------------------------------
Ran 14 tests in 534.230s

FAILED (failures=10, errors=1)

Comment 3 Daniel Tröder 2020-01-22 11:07:42 CET

Thanks for adding wait_for_*() to reduce the flakiness.
For me all tests are green now.

Comment 4 Jürn Brodersen 2020-01-22 11:11:55 CET

Tests are looking good now :)

-> Verified

Comment 5 Daniel Tröder 2021-02-23 09:29:54 CET

A Kelvin API app with the changes in this bug has been published.