Bug 52607 - Migrate ucs-test-ucsschool to Python 3
Migrate ucs-test-ucsschool to Python 3
Status: CLOSED FIXED
Product: UCS@school
Classification: Unclassified
Component: General
UCS@school 5.0
Other Linux
: P5 normal (vote)
: UCS@school 5.0 v1
Assigned To: Florian Best
Felix Botner
: interim-1
Depends on:
Blocks: 52578
  Show dependency treegraph
 
Reported: 2021-01-12 15:13 CET by Florian Best
Modified: 2021-11-29 17:20 CET (History)
1 user (show)

See Also:
What kind of report is it?: Development Internal
What type of bug is this?: ---
Who will be affected by this bug?: ---
How will those affected feel about the bug?: ---
User Pain:
Enterprise Customer affected?:
School Customer affected?:
ISV affected?:
Waiting Support:
Flags outvoted (downgraded) after PO Review:
Ticket number:
Bug group (optional):
Max CVSS v3 score:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Florian Best univentionstaff 2021-01-12 15:13:55 CET
The package ucs-test-ucsschool needs to be migrated to Python 3.

See: https://hutten.knut.univention.de/mediawiki/index.php/Python_3_Migration

As the package is using python-samba, which is not available for Python 2 anymore, the package/scripts needs to be migrated to UCS@school 5.0.
Comment 2 Florian Best univentionstaff 2021-06-14 11:09:50 CEST
Traceback (most recent call last):
  File "30_import-create_ou_alter_dhcpd_base.py", line 14, in <module>
    eio.import_ou_alter_dhcpd_base_flag(use_cli_api=True, use_python_api=False)
  File "/usr/lib/python3/dist-packages/univention/testing/ucsschool/importou.py", line 1065, in import_ou_alter_dhcpd_base_flag
    remove_ou(ou_name)
  File "/usr/lib/python3/dist-packages/univention/testing/ucsschool/importou.py", line 97, in remove_ou
    schoolenv.cleanup_ou(ou_name)
  File "/usr/lib/python3/dist-packages/univention/testing/ucsschool/ucs_test_school.py", line 392, in cleanup_ou
    self.remove_dcs_from_global_groups(ou_name)
  File "/usr/lib/python3/dist-packages/univention/testing/ucsschool/ucs_test_school.py", line 428, in remove_dcs_from_global_groups
    for dn in unique_member_attr
  File "/usr/lib/python3/dist-packages/univention/testing/ucsschool/ucs_test_school.py", line 429, in <listcomp>
    if dn.endswith(",ou={},{}".format(ou_name, self.ldap_base))
TypeError: endswith first arg must be bytes or a tuple of bytes, not str
Comment 3 Florian Best univentionstaff 2021-06-14 11:13:13 CEST
Traceback (most recent call last):
  File "00_installation_infos.py", line 14, in <module>
    from univention.testing.ucsschool.importusers_cli_v2 import ImportTestbase
  File "/usr/lib/python3/dist-packages/univention/testing/ucsschool/importusers_cli_v2.py", line 35, in <module>
    from univention.testing.ucsschool.importusers import Person
  File "/usr/lib/python3/dist-packages/univention/testing/ucsschool/importusers.py", line 11, in <module>
    import smbpasswd
ModuleNotFoundError: No module named 'smbpasswd'
Comment 4 Florian Best univentionstaff 2021-06-14 11:21:08 CEST
(In reply to Florian Best from comment #3)
> Traceback (most recent call last):
>   File "00_installation_infos.py", line 14, in <module>
>     from univention.testing.ucsschool.importusers_cli_v2 import
> ImportTestbase
>   File
> "/usr/lib/python3/dist-packages/univention/testing/ucsschool/
> importusers_cli_v2.py", line 35, in <module>
>     from univention.testing.ucsschool.importusers import Person
>   File
> "/usr/lib/python3/dist-packages/univention/testing/ucsschool/importusers.py",
> line 11, in <module>
>     import smbpasswd
> ModuleNotFoundError: No module named 'smbpasswd'

Fixed in:
ucs-test-ucsschool (7.0.2)
83ea0bc7a180 | Bug #52607: migrate from smbpasswd to python-passlib
Comment 5 Florian Best univentionstaff 2021-06-14 16:35:31 CEST
Traceback (most recent call last):
  File "96_samba4_sid_allocation.py", line 302, in <module>
    exit(TestSambaSIDAllocation.main())
  File "96_samba4_sid_allocation.py", line 206, in main
    initial_next_rid = self.s4_search("(objectClass=rIDSet)", "rIDNextRID")
  File "96_samba4_sid_allocation.py", line 81, in s4_search
    stdout, stderr = self.create_and_run_process(cmd)
  File "/usr/lib/python3/dist-packages/univention/testing/ucsschool/test_samba4.py", line 61, in create_and_run_process
    stdout = self.remove_samba_warnings(stdout)
  File "/usr/lib/python3/dist-packages/univention/testing/ucsschool/test_samba4.py", line 44, in remove_samba_warnings
    input_str = input_str.replace("WARNING: No path in service IPC$ - making it unavailable!", "") 
TypeError: a bytes-like object is required, not 'str'
Comment 6 Florian Best univentionstaff 2021-06-14 19:39:49 CEST
Traceback (most recent call last):
  File "242_import-users_test_password_hashes.py", line 77, in <module>
    Test().run()
  File "/usr/lib/python3/dist-packages/univention/testing/ucsschool/importusers_cli_v2.py", line 324, in run 
    self.test()
  File "242_import-users_test_password_hashes.py", line 51, in test
    fn_csv = self.create_csv_file(person_list=person_list, mapping=config["csv"]["mapping"])
  File "/usr/lib/python3/dist-packages/univention/testing/ucsschool/importusers_cli_v2.py", line 481, in create_csv_file
    properties2headers = {v: k for k, v in header2properties.iteritems()}
AttributeError: 'dict' object has no attribute 'iteritems'
Comment 7 Florian Best univentionstaff 2021-06-14 19:44:05 CEST
Traceback (most recent call last):
  File "248_migrate_ucsschool_import_user.py", line 107, in main
    writer.writerow(["Firstname", "Lastname", "Some Stuff", "record_uid"])
  File "/usr/lib/python3.7/tempfile.py", line 620, in func_wrapper
    return func(*args, **kwargs)   
TypeError: a bytes-like object is required, not 'str'
Comment 8 Florian Best univentionstaff 2021-06-16 00:18:20 CEST
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/univention/testing/ucsschool/importusers_cli_v2.py", line 324, in run
    self.test()
  File "244_import_user_pre_read_pyhook.py", line 59, in test
    fn_csv = self.create_csv_file(person_list=person_list, mapping=config["csv"]["mapping"])
  File "/usr/lib/python3/dist-packages/univention/testing/ucsschool/importusers_cli_v2.py", line 484, in create_csv_file
    random.shuffle(header_row)
  File "/usr/lib/python3.7/random.py", line 278, in shuffle
    x[i], x[j] = x[j], x[i]
TypeError: 'dict_keys' object is not subscriptable
Comment 9 Florian Best univentionstaff 2021-09-02 16:46:48 CEST
All tests have been migrated to Python 3. Most test cases have been migrated to pytest.
The changes have also been backported to UCS@school 4.4.

ucs-test-ucsschool (7.0.2)
df376c56312d | Bug #52607: add more debug output into hook failure
5324292b0185 | Bug #52607: remove python-pytest-catchlog dependency
7904e8ff2108 | Bug #52607: [ucs-test-ucsschool] Change hashbangs to Python 3

ucs-test-ucsschool (6.0.245)
6d2f81c0367a | Bug #52607: debian/changelog
443ce837c3f7 | Bug #52607: reindent with black
d211d584d20d | Bug #52607: migrate ipaddr to ipaddress
633c88d97758 | Bug #52607: [ucs-test-ucsschool]: migrate tests to pytest and make them Python 3 compatible
c3950e50b31b | Bug #52607: migrate from smbpasswd to python-passlib
cc049d2b2827 | Bug #52607: ignore __pycache__ directory to be present in hooks directory
b92466dc717e | Bug #52607: test if test fails with different mail domain
517962ddd0b1 | Bug #52607: s/coding=utf-8/-*- coding: utf-8 -*-/g
e3a3f6b696ce | Bug #52607: fix format string during logging
4f09aabf179e | Bug #52607: fix format string during logging
f5c9f7753389 | Bug #52607: set missing executable flags
73630b57635e | Bug #52607: migrate 51_ucs-school-configurable-schoollists.py to pytest
Comment 10 Felix Botner univentionstaff 2021-09-17 16:54:21 CEST
ok
Comment 11 Jürn Brodersen univentionstaff 2021-11-29 17:20:05 CET
UCS@school 5.0 v1 has been released.

https://docs.software-univention.de/release-notes-ucsschool-5.0v1-de.html

If this error occurs again, please clone this bug.