Univention Bugzilla – Bug 54196
crash at the VERY end of the SYSVOL copy
Last modified: 2022-04-06 17:31:04 CEST
if "versionNumber" in obj: gpcversion = int(obj["versionNumber"][0]) config = configparser.ConfigParser() try: with open(os.path.join(gpo_path, 'GPT.INI')) as f: try: config.readfp(f) fileversion = int(config.get('General', 'version')) if fileversion < gpcversion: log.error("File version %s of GPO %s is lower than GPO container versionNumber (%s)" % (fileversion, name, gpcversion)) raise SysvolGPOVersionTooLow(_("At least one GPO in SYSVOL is not up to date yet.")) if fileversion != gpcversion: log.error("File version %s of GPO %s differs from GPO container versionNumber (%s)" % (fileversion, name, gpcversion)) # TODO: Imrpove error reporting except configparser.Error as ex: log.error(ex.args[0]) except IOError as ex: log.error(ex.args[0]) return True There is a potential error in the code that validates GPT.ini for some reason. a couple of the *.INI files have some 0xd0h bytes (possibly chinese encoded space char). This causes the migration to crash. entering into these files with a text editor allow the char to be cleaned up. and this fixes the error for the next attempt. but sadly you have already corrupted your AD with a shed load of new DNS records and other crap. Traceback (most recent call last): File "/usr/lib/python3/dist-packages/univention/management/console/modules/adtakeover/__init__.py", line 61, in _background result = func(self, request) File "/usr/lib/python3/dist-packages/univention/management/console/modules/adtakeover/__init__.py", line 118, in check_sysvol takeover.check_sysvol(self.progress) File "/usr/lib/python3/dist-packages/univention/management/console/modules/adtakeover/takeover.py", line 406, in check_sysvol check_gpo_presence() File "/usr/lib/python3/dist-packages/univention/management/console/modules/adtakeover/takeover.py", line 1926, in check_gpo_presence config.readfp(f) File "/usr/lib/python3.7/configparser.py", line 762, in readfp self.read_file(fp, source=filename) File "/usr/lib/python3.7/configparser.py", line 717, in read_file self._read(f, source) File "/usr/lib/python3.7/configparser.py", line 1014, in _read for lineno, line in enumerate(fp, start=1): File "/usr/lib/python3.7/codecs.py", line 322, in decode (result, consumed) = self._buffer_decode(data, self.errors, final) UnicodeDecodeError: 'utf-8' codec can't decode byte 0xd0 in position 34: invalid continuation byte
Could you maybe provide an example of a GPT.INI that triggers the issue? Otherwise [MS-GPOL] https://docs.microsoft.com/en-us/openspecs/windows_protocols/ms-gpol/b0e5c9e8-e858-4a7a-a94a-4a3d0a9d87a2 specifies that the file encoding should be ANSI, aka Windows-1252 aka cp1252. So we could open the file using that encoding.
Created attachment 10873 [details] GPT.INT this is an ASIAN GPT.INI file DO NOT translate it is to be treated as BINARY
The wonderful world of Microsoft documentation... meets the real world. I suspect it is a multi byte character set. where "0xd0" is a continuation or signalling byte for multi byte character-sets. this is on a totally working 2008sp2 windows AD, but it's been in service since Dinosaurs roamed the earth...
1. I will get an asian staff to look at this file on an asian AD server. but it seems all the AD servers have similar data. 2.really the workflow for takeover is broken. UCS totally trashes the original system, then crashes out on this sort of thing. Really there are two better ways of doing this. A.Right at the START of AD takeover & BEFORE trashing the original system, you have the admin pw. mount the original sysvol & run your checks against this. and ABORT before trashing the original AD B. mount BOTH the original & replacement AD 1. generate MD5 hashes for ALL the files on the original sysvol 2. do a automated copy between sysvols without "robocopy" 3. run an MD5 check on the replacement AD files 4. don't get into even trying to validate them, if they worked on the original ad then assume they work on the replacement.
Created attachment 10874 [details] translation of hte multi byte characters found in GPT.INI The meaning is: NEW GROUP POLICY OBJECT!!!!!! and it's found on MULTIPLE systems across different locations.
> DO NOT translate it is to be treated as BINARY Perfectly clear, no need to yell. > UCS totally trashes the original system, then crashes out on this sort of thing. The process takes care not to change anything on the original system. You could switch off the UCS system and start your AD again if that's necessary. python3 -c "import chardet; import codecs; print(chardet.detect(codecs.open('./GPT.INI', 'rb').read()))" {'encoding': 'GB2312', 'confidence': 0.99, 'language': 'Chinese'} python3 -c "import codecs; print(codecs.open('./GPT.INI', 'rb').read().decode('GB2312'))" [General] Version=131072 displayName=新建组策略对象 python3 -c "import codecs; from configparser import ConfigParser; config=ConfigParser(); config.read_file(codecs.open('./GPT.INI', 'rb', 'GB2312')); print(config.get('General', 'version'))" 131072
Created attachment 10875 [details] bug54196.patch
Created attachment 10876 [details] bug54196.patch
Cultural differences. It is not yelling it is highlighting & that is surprise and annoyance at MS... because neither myself or the Staff could find any Asian MS documents showing this. But on a fresh CD install it does it. worse, reading the manual the user can put anything in here.. with a rename. we suspect it does it for other Asian languages as well. Also you comment about "takes care to change nothing" is not correct. by this point, UCS has totally stomped all over the master DNS adding in records and secondary "domain controllers" you can do a simple test. add in corruption into the file on a win test system. do a take over, let it fail. now... if you try to do another take over against the same "original" system the "join.py" samba/ucs script is broken in multiple places, because it says, you already have a UCS master domain. this is the result of a re-run, due to the "corrupted" DNS: File "/usr/lib/python3/dist-packages/samba/netcmd/__init__.py", line 186, in _run return self.run(*args, **kwargs) File "/usr/lib/python3/dist-packages/samba/netcmd/domain.py", line 668, in run backend_store_size=backend_store_size) File "/usr/lib/python3/dist-packages/samba/join.py", line 1559, in join_DC ctx.do_join() File "/usr/lib/python3/dist-packages/samba/join.py", line 1447, in do_join ctx.join_add_objects() File "/usr/lib/python3/dist-packages/samba/join.py", line 712, in join_add_objects ctx.samdb.modify(m)
This bug tracker is meant to collect issues to improve the product, not to communicate with users. Particular problems of specific customer environments should be handled via support or in our forum.
sorry.. but this WAS a bug. and a potentially critical one at that.
AD Takeover should work when the original AD has GPO that use non ASCII characters. univention-management-console-module-adtakeover.yaml f9c912fe2232 | Bug #54196: update YAML ec17ed976653 | Bug #54196: adtakeover checks for encoding before reading the GPT.INI univention-management-console-module-adtakeover (7.0.2-7) ec17ed976653 | Bug #54196: adtakeover checks for encoding before reading the GPT.INI
Verified: * Code review * Package update * Functional test * Advisory
<https://errata.software-univention.de/#/?erratum=5.0x284>