Univention Bugzilla – Bug 53860
univention-portal: json.decoder.JSONDecodeError: Incomplete portals.json file
Last modified: 2023-10-18 16:20:26 CEST
1 times in /var/log/univention/listener.log: https://jenkins.knut.univention.de:8181/job/UCS-5.0/job/UCS-5.0-0/job/AutotestJoinReleased/SambaVersion=no-samba,Systemrolle=master-part-II/ws/test/listener.log Traceback (most recent call last): File "/usr/sbin/univention-portal", line 534, in <module> cli() File "/usr/lib/python3/dist-packages/click/core.py", line 764, in __call__ return self.main(*args, **kwargs) File "/usr/lib/python3/dist-packages/click/core.py", line 717, in main rv = self.invoke(ctx) File "/usr/lib/python3/dist-packages/click/core.py", line 1137, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File "/usr/lib/python3/dist-packages/click/core.py", line 956, in invoke return ctx.invoke(self.callback, **ctx.params) File "/usr/lib/python3/dist-packages/click/core.py", line 555, in invoke return callback(*args, **kwargs) File "/usr/sbin/univention-portal", line 450, in update json_content = read_portals_json() File "/usr/sbin/univention-portal", line 54, in read_portals_json return json.load(fd) File "/usr/lib/python3.7/json/__init__.py", line 296, in load parse_constant=parse_constant, object_pairs_hook=object_pairs_hook, **kw) File "/usr/lib/python3.7/json/__init__.py", line 348, in loads return _default_decoder.decode(s) File "/usr/lib/python3.7/json/decoder.py", line 337, in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end()) File "/usr/lib/python3.7/json/decoder.py", line 355, in raw_decode raise JSONDecodeError("Expecting value", s, err.value) from None json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
https://jenkins2022.knut.univention.de/job/UCS-5.0/job/UCS-5.0-5/job/AutotestJoin/SambaVersion=no-samba,Systemrolle=master/ws/test/listener.log Traceback (most recent call last): File "/usr/sbin/univention-portal", line 615, in <module> cli() File "/usr/lib/python3/dist-packages/click/core.py", line 764, in __call__ return self.main(*args, **kwargs) File "/usr/lib/python3/dist-packages/click/core.py", line 717, in main rv = self.invoke(ctx) File "/usr/lib/python3/dist-packages/click/core.py", line 1137, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File "/usr/lib/python3/dist-packages/click/core.py", line 956, in invoke return ctx.invoke(self.callback, **ctx.params) File "/usr/lib/python3/dist-packages/click/core.py", line 555, in invoke return callback(*args, **kwargs) File "/usr/sbin/univention-portal", line 531, in update json_content = read_portals_json() File "/usr/sbin/univention-portal", line 56, in read_portals_json return json.load(fd) File "/usr/lib/python3.7/json/__init__.py", line 296, in load parse_constant=parse_constant, object_pairs_hook=object_pairs_hook, **kw) File "/usr/lib/python3.7/json/__init__.py", line 348, in loads return _default_decoder.decode(s) File "/usr/lib/python3.7/json/decoder.py", line 337, in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end()) File "/usr/lib/python3.7/json/decoder.py", line 353, in raw_decode obj, end = self.scan_once(s, idx) json.decoder.JSONDecodeError: Expecting property name enclosed in double quotes: line 210 column 21 (char 8205) management/univention-portal/univention-portal 61 def write_portals_json(content: Dict): 62 with open(portals_json, "w") as fd: 63 os.chmod(fd.name, 0o660) 64 json.dump(content, fd, sort_keys=True, indent=4) - `open(mode="w") truncates the file before it is written - `write()` is not atomic Change the code to write to a temporary file and only then rename it to the final location to make updating atomic. https://git.knut.univention.de/univention/ucs/-/merge_requests/929
[5.0-5] e41379fd39 fix(portal): Update file portals.json atomically doc/errata/staging/univention-portal.yaml | 5 +++-- management/univention-portal/debian/changelog | 6 ++++++ management/univention-portal/univention-portal | 13 ++++++++++--- 3 files changed, 19 insertions(+), 5 deletions(-) Package: univention-portal Version: 4.0.14-9 Branch: ucs_5.0-0 Scope: errata5.0-5
OK: atomic write, after the upgrade the same results are achieved: # md5sum /usr/share/univention-portal/portals.json 09c04c77ee8627d8a212d5ab430866e5 /usr/share/univention-portal/portals.json # cp /usr/share/univention-portal/portals.json{,.org} # univention-portal remove foo foo does not exist in database foo removed # md5sum /usr/share/univention-portal/portals.json 7fc2dfd257e17669a3a6411d4c228131 /usr/share/univention-portal/portals.json # diff -u /usr/share/univention-portal/portals.json{.org,} … ~OK: Advisory (could be more administrator friendly)
<https://errata.software-univention.de/#/?erratum=5.0x849>