Bug 52043 - services/univention-s4-connector: Migrate to Python 3
services/univention-s4-connector: Migrate to Python 3
Status: CLOSED FIXED
Product: UCS
Classification: Unclassified
Component: S4 Connector
UCS 5.0
Other Linux
: P5 normal (vote)
: UCS 5.0
Assigned To: Florian Best
Arvid Requate
:
: 31724 (view as bug list)
Depends on:
Blocks: 52358
  Show dependency treegraph
 
Reported: 2020-09-16 14:06 CEST by Florian Best
Modified: 2021-05-25 15:59 CEST (History)
3 users (show)

See Also:
What kind of report is it?: Development Internal
What type of bug is this?: ---
Who will be affected by this bug?: ---
How will those affected feel about the bug?: ---
User Pain:
Enterprise Customer affected?:
School Customer affected?:
ISV affected?:
Waiting Support:
Flags outvoted (downgraded) after PO Review:
Ticket number:
Bug group (optional):
Max CVSS v3 score:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Florian Best univentionstaff 2020-09-16 14:06:25 CEST
The univention-s4-connector should be Python 3 compatible.
As well as the string-changes in UDM have to be integrated.
Comment 2 Florian Best univentionstaff 2020-10-15 21:32:35 CEST
The S4-Connector has been migrated to Python 3.

ucs-test (10.0.1-18)
2f52369ba533 | Bug #52043: migrate univention-s4-connector to Python 3
a0052a5774eb | Bug #52043: migrate 402check_mapping_for_single_value_samba4_attributes to python3
9b0dd7655972 | Bug #52043: fix 175sync_create_dns_msdcs_record_ucs2con in non-Jenkins environment
a715183776c2 | Bug #52043: adjust mapping via localmapping.py
11205828f078 | Bug #52043: fix 100sync_gpo_ntsecurity_descriptor
1aa902311fde | Bug #52043: remove all automatic de/encoding functions
ce8a6d3e6139 | fixup! Bug #52043: Fix test regression
eb9dfb9a0705 | Bug #52043: Fix test regression

univention-s4-connector (14.0.3-1)
8caa80bb3db4 | Bug #52043 s4c: migrate univention-s4-position-sync to python 3
517fd132a5f4 | Bug #52043 s4c: migrate univention-password_sync_ucs_to_s4 to python 3
b3c257029835 | Bug #52043 s4c: migrate sync_krbtgt to python 3
3117302afe6b | Bug #52043 s4c: migrate listener to Python 3
b0286e455a54 | Bug #52043: s4c: remove obsolete configsaver
87f2d6b87467 | fixup! Bug #52043 s4c scripts: python 3 compatibility
4d9516dd0df1 | Bug #52043: s4c mapping: remove unused functions
58917a81aafb | Bug #52043: fix decoding of objectGuid
a927a522fe50 | Bug #52043 s4c: remove legacy code
5140062d62cc | Bug #52043: migrate univention-s4-connector to Python 3
univention-s4-connector (14.0.2-3)
2f52369ba533 | Bug #52043: migrate univention-s4-connector to Python 3
ab0f26487db3 | Bug #52043: Add README.md to preserve Know-How from Joinscript
999ff8302681 | Bug #52043: fix typo in mapping: con_create_extenstions → con_create_extensions
21f6cf119ee2 | Bug #52043: enable UDM option automatically
b08d2578a366 | Bug #52043: flatten set_values
2ac84f50e67c | Bug #52043: flatten _object_mapping
6bfb1c9144c4 | Bug #52043: add docstring for attribute
f300a3614965 | Bug #52043: s4.main: add context manager, fix closing logs
55d3d7771df9 | Bug #52043: s4.main: outsource connection to LDAP into init_ldap_connections()
8daed6093910 | Bug #52043: s4.main: remove init_group_cache parameter
ba416869aa40 | Bug #52043: s4.main: make debug level and logfile configurable via command line options
5a39bc8c867a | Bug #52043: s4.main: unify initialization of S4
7d6c94ed40df | Bug #52043: s4.main: use contextmanager
c51311be9a92 | Bug #52043: fix sync of e.g. con_other_attribute
ff902c21a759 | Bug #52043: fix deepcopying
fef42f5e44d8 | Bug #52043 s4c: encoding of primary_group_sync_from_ucs() / group_members_sync_from_ucs()
78da5351190b | Bug #52043 s4c: manually decode SID
98815b998bb7 | Bug #52043: specify the encoding of the Samba LDAP attributes
6d6bd4330be0 | Bug #52043: filter_match() compare values as bytes
0440f320ef13 | Bug #52043: format_escaped: raise TypeError for bytes
13996856e7c8 | Bug #52043: remove shakespeare logic
e5069f899527 | Bug #52043: remove obsolete self.co
58f41e36d50d | Bug #52043: remove unused variable self.ctrl_show_deleted
786965eff0e1 | Bug #52043: remove deprecated term baseConfig
23c0107cc997 | Bug #52043: remove unused function normalise_userAccountControl
078a795ce0d2 | Bug #52043: remove useless code
73237045bc12 | Bug #52043: remove obsolete setproctitle()
651cfb9c2317 | Bug #52043: remove obsolete print_sid_mapping
1aa902311fde | Bug #52043: remove all automatic de/encoding functions
c66bde6158bf | Bug #52043 s4c: add properly encoding and decoding of attributes
52a4b0cd0c37 | Bug #52043 s4c: initial s4 search: combine filters
7d7630e44288 | Bug #52043: cleanup: log mapping only with debug/level=4
f713613ab16d | Bug #52043: cleanup: fix SQL escaping
fd7d02402f02 | Bug #52043: cleanup: rename variable function name "f"
751e379df57d | Bug #52043: cleanup: fix log message
90567c47a485 | Bug #52043: cleanup: use univention.debug flags instead of integers
b79854b69d52 | Bug #52043: cleanup: simplity con_default_dn adjusting
184a33530b57 | Bug #52043: cleanup: simplify exception handling
f6f057b1d20b | Bug #52043: cleanup: log ignored S4 objects
810772c107f5 | Bug #52043: cleanup: do not ignore fatal exceptions
874812be4237 | Bug #52043: cleanup: log which object is ignored
b229735145b8 | Bug #52043: cleanup: use constant instead of int
eec23d020b08 | Bug #52043 s4c: ntsecurity_descriptor.py: encoding
9f168c5d4e92 | Bug #52043 s4c: lockingdb.py: encoding
032d01ea23f1 | Bug #52043 s4c: lockingdb.py: replace ud.function() with ud.trace()
428b85a0b40a | Bug #52043 s4c: s4cache.py: encoding
2a0c31de570a | Bug #52043 s4c: s4cache.py: replace ud.function() with ud.trace()
c4fc7730483c | Bug #52043 s4c: user.py: encoding
62eccedd1b42 | Bug #52043 s4c: dns.py: encoding
e648df1a642d | Bug #52043 s4c: query_config.py: encoding
c583eddea9c3 | Bug #52043 s4c: dc.py: encoding
2687e05e2254 | Bug #52043 s4c: computer.py: encoding
05c1c6743f01 | Bug #52043 s4c: password.py: fix exception logging
86fe5a1a7746 | Bug #52043 s4c: password.py: encoding
9c168cdb76c5 | Bug #52043 s4c: sid_mapping.py: encoding
74c1c2fc11c3 | Bug #52043 s4c: mapping.py python 3 compatibility
070d43858cb8 | Bug #52043: decode GUID
b2475da6a853 | Bug #52043 s4c: object_mapping: fix iterating over changing dictionary
763b977a5ffb | Bug #52043 s4c: fix loading pickle files
0ded4df4cfca | Bug #52043: Remove obsolete scripts
c5b61155c43d | Bug #52043 s4c: scripts/msgpo.py: migrate to Python 3
b7b8f14f2b02 | Bug #52043: fix DeprecationWarning for imp
f7214244f1ea | Bug #52043 s4c scripts: python 3 compatibility
a58d89c01942 | Bug #52043 s4c scripts: migrate optparse to argparse
37797e7dbe7e | Bug #52043 s4c: fix type comparisions
db0157394b87 | Bug #52043 s4c: add basic python 3 compatibility
c6a88afe7b2c | Bug #52043 s4c: UDM modules: add python3 compatibility
40c9c641387e | Bug #52043: Install python3-univention-connector-s4
d32fc6fd6d2e | Bug #52043: run univention-s4-connector with python3
fb27bd4f7e4b | Bug #52043 s4c: change hashbangs to Python 3
Comment 3 Arvid Requate univentionstaff 2020-10-29 21:22:25 CET
788984f88e | Avoid world writeable /etc/univention/connector/s4/__pycache__

Pushed to 5.0-0. Old source package purged from Repo-NG and rebuilt as 14.0.3-1A~5.0.0.202010292120
Comment 4 Arvid Requate univentionstaff 2020-10-29 21:49:28 CET
f720a04f14 | Follow best daemon practice by explicitly setting a umask.

14.0.3-1A~5.0.0.202010292149
Comment 5 Florian Best univentionstaff 2021-03-24 22:07:22 CET
The mapping UCR template files has been replaced with a real python module: univention.s4connector.s4.mapping.

From now on /etc/univention/connector/s4/localmapping.py may be used to adjust the mapping.

    def mapping_hook(mapping):
        # add new type
        mapping['foo'] = univention.s4connector.property(...)

        # remove type
        mapping.pop('dc', None)

        # add new attribute
        mapping['user'].attributes['foo'] = univention.s4connector.attribute(...)

        return mapping

univention-s4-connector (14.0.6-7)                                                                    
fbf6440d285c | Bug #52043: replace UCR filter based mapping with a real python module                         
64e22b66167e | Bug #52043: remove mapping.py
Comment 6 Florian Best univentionstaff 2021-03-24 22:09:05 CET
*** Bug 31724 has been marked as a duplicate of this bug. ***
Comment 7 Felix Botner univentionstaff 2021-04-09 15:34:01 CEST
(In reply to Florian Best from comment #5)
> The mapping UCR template files has been replaced with a real python module:
> univention.s4connector.s4.mapping.
> 
> From now on /etc/univention/connector/s4/localmapping.py may be used to
> adjust the mapping.
> 
>     def mapping_hook(mapping):
>         # add new type
>         mapping['foo'] = univention.s4connector.property(...)
> 
>         # remove type
>         mapping.pop('dc', None)
> 
>         # add new attribute
>         mapping['user'].attributes['foo'] =
> univention.s4connector.attribute(...)
> 
>         return mapping
> 
> univention-s4-connector (14.0.6-7)                                          
> 
> fbf6440d285c | Bug #52043: replace UCR filter based mapping with a real
> python module                         
> 64e22b66167e | Bug #52043: remove mapping.py

great, one request though

please create a "temporary" file /etc/univention/connector/s4/mapping.current (or wherever you like) with the current mapping (during the startup of the connector after the mapping has been processed)

Seems that with the standard log level the mapping is not logged in connector-s4.log, but anyway i think an extra file is better and i'm afraid support gets crazy if we longer have such a file.
Comment 8 Florian Best univentionstaff 2021-04-14 18:47:02 CEST
univention-s4-connector (14.0.7-1)
05bb33fb76a4 | Bug #52043: Log mapping to /var/log/univention/connector-s4-mapping.log
Comment 9 Arvid Requate univentionstaff 2021-04-20 13:20:14 CEST
Verified:
* The tests look good
* Release changelog
Comment 10 Florian Best univentionstaff 2021-05-25 15:59:55 CEST
UCS 5.0 has been released:
 https://docs.software-univention.de/release-notes-5.0-0-en.html
 https://docs.software-univention.de/release-notes-5.0-0-de.html

If this error occurs again, please use "Clone This Bug".