Univention Bugzilla – Bug 53343
Migrate univention-base-files to Python 3
Last modified: 2021-11-03 17:01:30 CET
univention-base-files should be migrated to be Python 3 compatible. base/univention-base-files/univention-directory-policy/nfsmounts.py:#!/usr/bin/python2.7 base/univention-base-files/univention-directory-policy/univention-policy-update-config-registry.py:#!/usr/bin/python2.7
Migrated univention-policy-update-config-registry.py and nfsmounts.py to Python3. Replaced command parsing with univention.lib.policy_result Replaced file parsing with univention.lib.fstab Applied some refactoring to nfsmounts.py univention-base-files.yaml 079af23ea052 | Bug #53343: YAML updated for univention-base-files 8c65700d7f05 | Bug #53343: Migrated univention-base-files to Python 3 univention-base-files (9.0.3-4) e99f96452577 | Bug #53343: refactor. Created get_nfs_data. 7077e2197f9e | Bug #53343: refactor. Created update_fstab 62f2096f1d97 | Bug #53343: use univention.lib.fstab instead of manual fstab parsing 5cf24ff75261 | Bug #53343: use univention.lib.policy_result() instead of manual subprocess parsing 8c65700d7f05 | Bug #53343: Migrated univention-base-files to Python 3
The school dvd install tests fail with an error from "univention-policy-update-config-registry.py". It looks like the change was introduced here, could you look at that? thanks :) The error happens on the replica during the domain join. The test scenario might still have other problems, so don't be alarmed if it fails with another error after fixing this. Test: https://jenkins.knut.univention.de:8181/job/UCS-5.0/job/UCS-5.0-0/job/Installation%20Tests/mode=school-dev/343/ Error screenshot: https://jenkins.knut.univention.de:8181/job/UCS-5.0/job/UCS-5.0-0/job/Installation%20Tests/mode=school-dev/ws/test/screenshots/error.png Traceback: 2021-11-02 02:18:39.002364475+01:00 (in joinscript_init) Traceback (most recent call last): File "/usr/lib/univention-directory-policy/univention-policy-update-config-registry", line 120, in <module> main() File "/usr/lib/univention-directory-policy/univention-policy-update-config-registry", line 95, in main value = values[0] IndexError: list index out of range run-parts: /usr/lib/univention-directory-policy/univention-policy-update-config-registry exited with return code 1 __JOINERR__:FAILED: /usr/lib/univention-install/20univention-directory-policy.inst
(Pdb) set_list {'univentionPWLength': ['8'], 'univentionPWHistoryLen': ['3'], 'directory/manager/samba3/legacy': ['yes'], 'dhcpd/ldap/base': ['cn=dhcp,ou=school1,dc=test,dc=local'], 'univentionDhcpDomainNameServers': []} # univention-policy-result -D $(ucr get ldap/hostdn) -y /etc/machine.secret $(ucr get ldap/hostdn) Policy: cn=dhcp-dns-clear,cn=policies,ou=school1,dc=test,dc=local Attribute: univentionDhcpDomainNameServers → no "Value" set # univention-ldapsearch -b cn=dhcp-dns-clear,cn=policies,ou=school1,dc=test,dc=local -LLL dn: cn=dhcp-dns-clear,cn=policies,ou=school1,dc=test,dc=local cn: dhcp-dns-clear emptyAttributes: univentionDhcpDomainNameServers objectClass: top objectClass: univentionPolicyDhcpDns objectClass: univentionPolicy objectClass: univentionObject univentionObjectType: policies/dhcp_dns This is not even a UCR policy. The regression is that non UCR policy values are evaluated here. Prior there was: - elif line.startswith(get_policy.ATTR): - key = line[len(get_policy.ATTR):] - key = bytes.fromhex(key).decode('utf-8') get_policy.ATTR = 'Attribute: univentionRegistry;entry-hex-'
Added ucr_policy_result to univention-policy replaced subprocess.Popen with subprocess.call in nfsmounts.py univention-policy (11.0.1-3) 2d1658bd0e8e | Bug #53343: introduce univention.lib.policy_result.ucr_policy_result() univention-base-files.yaml 4923d29bee91 | Bug #53343: Yaml update version univention-base-files (9.0.3-5) 2d1658bd0e8e | Bug #53343: introduce univention.lib.policy_result.ucr_policy_result() 6418957fea79 | Bug #53343: nfsmounts: allow share hosts without FQDNs 0e521673b5d9 | Bug #53343: replaced subprocess.Popen with subprocess.call in nfsmounts.py ucs-test (10.0.6-65) de1c4590cdb0 | Bug #53343: changelog move for ucs-test updated dc5708702a33 | Bug #53343: add test_nfsmount to 16_test_univention_base_files_scripts.py bc66b2fd2c83 | Bug #53343: move 59_udm/15_test_univention_base_files_scripts.py to 01_base f97d85583199 | Bug #53343: add test case 59_udm/15_test_univention_base_files_scripts.py ucs-test (10.0.6-64) b02d00c79264 | Bug #53343: Allow UDP for port redirections
In https://jenkins.knut.univention.de:8181/job/UCS-5.0/job/UCS-5.0-0/view/Default/job/AutotestJoin/619/SambaVersion=s4,Systemrolle=member/testReport/99_end/01_var_log_tracebacks/test_var_log_tracebacks/ we see: 03.11.21 02:15:47.650 LISTENER ( PROCESS ) : updating 'cn=sS.nMVY4,cn=member097.autotest097.local,cn=shares,dc=autotest097,dc=local' command a 03.11.21 02:15:47.660 LISTENER ( PROCESS ) : Writing /etc/exports with 12 lines 03.11.21 02:15:51.723 LISTENER ( PROCESS ) : updating 'cn=S-1-5-21-1741379181-84684004-1730338019-1343,cn=sid,cn=temporary,cn=univention,dc=autotest097,dc=local' command a 03.11.21 02:15:51.735 LISTENER ( PROCESS ) : updating 'uid=izclg9lq,cn=users,dc=autotest097,dc=local' command m 03.11.21 02:15:51.746 LISTENER ( PROCESS ) : updating 'cn=S-1-5-21-1741379181-84684004-1730338019-1343,cn=sid,cn=temporary,cn=univention,dc=autotest097,dc=local' command d Warning: The unit file, source configuration file or drop-ins of nfs-kernel-server.service changed on disk. Run 'systemctl daemon-reload' to reload units. Updating default Portal data updated in 0.13s Updating umc Portal data untouched Traceback (most recent call last): File "/usr/lib/univention-directory-listener/system/quota.py", line 265, in postrun policy_result = univention.lib.policy_result.policy_result(dn)[0] File "/usr/lib/python3/dist-packages/univention/lib/policy_result.py", line 55, in policy_result results, policies = _policy_result(dn, binddn, bindpw, encoding, ldap_server) File "/usr/lib/python3/dist-packages/univention/lib/policy_result.py", line 95, in _policy_result raise PolicyResultFailed("Error getting univention-policy-result for '%(dn)s': %(error)s" % {'dn': dn, 'error': stderr.decode('utf-8', 'replace')}, returncode=p.returncode) univention.lib.policy_result.PolicyResultFailed: Error getting univention-policy-result for 'cn=sS.nMVY4,cn=member097.autotest097.local,cn=shares,dc=autotest097,dc=local': LDAP Error: No such object Reloading smbd. Reloading smbd configuration (via systemctl): smbd.service. 03.11.21 02:16:08.522 LDAP ( PROCESS ) : connecting to ldap://master097.autotest097.local:7389 03.11.21 02:16:08.539 LISTENER ( PROCESS ) : updating 'uid=izclg9lq,cn=users,dc=autotest097,dc=local' command d 03.11.21 02:16:08.543 LISTENER ( PROCESS ) : updating 'cn=Domain Users,cn=groups,dc=autotest097,dc=local' command m 03.11.21 02:16:08.547 LISTENER ( PROCESS ) : updating 'cn=sS.nMVY4,cn=member097.autotest097.local,cn=shares,dc=autotest097,dc=local' command d 03.11.21 02:16:08.549 LISTENER ( PROCESS ) : Writing /etc/exports with 11 lines But I think this is a random race condition, not caused by the changes here.
(In reply to Esteban from comment #5) > Added ucr_policy_result to univention-policy OK: only UCR policies are evaluated by the script. > replaced subprocess.Popen with subprocess.call in nfsmounts.py OK: the script now wait for the mount to succeed/fail.
<https://errata.software-univention.de/#/?erratum=5.0x148>