Bug 51680 - open(..., 'rw') is an error in Python 3
open(..., 'rw') is an error in Python 3
Status: CLOSED FIXED
Product: UCS
Classification: Unclassified
Component: UCR
UCS 5.0
Other Linux
: P5 normal (vote)
: UCS 4.4-5-errata
Assigned To: Philipp Hahn
Florian Best
:
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2020-07-16 16:59 CEST by Daniel Tröder
Modified: 2020-07-29 16:50 CEST (History)
2 users (show)

See Also:
What kind of report is it?: Bug Report
What type of bug is this?: 3: Simply Wrong: The implementation doesn't match the docu
Who will be affected by this bug?: 1: Will affect a very few installed domains
How will those affected feel about the bug?: 1: Nuisance – not a big deal but noticeable
User Pain: 0.017
Enterprise Customer affected?:
School Customer affected?:
ISV affected?:
Waiting Support:
Flags outvoted (downgraded) after PO Review:
Ticket number:
Bug group (optional):
Max CVSS v3 score:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Daniel Tröder univentionstaff 2020-07-16 16:59:32 CEST
Trigger für univention-config (14.0.0-17) werden verarbeitet ...
Traceback (most recent call last):
  File "/usr/sbin/univention-config-registry", line 69, in <module>
    ub.main(convertToUtf8(sys.argv[1:]))
  File "/usr/lib/python3/dist-packages/univention/config_registry/frontend.py", line 777, in main
    result = handler_func(args, cmd_opts)
  File "/usr/lib/python3/dist-packages/univention/config_registry/frontend.py", line 271, in handler_update
    handlers.update_divert(cur)
  File "/usr/lib/python3/dist-packages/univention/config_registry/handler.py", line 964, in update_divert
    handler.install_divert()
  File "/usr/lib/python3/dist-packages/univention/config_registry/handler.py", line 326, in install_divert
    self._call_silent('dpkg-divert', '--quiet', '--rename', '--local', '--divert', deb, '--add', self.to_file)
  File "/usr/lib/python3/dist-packages/univention/config_registry/handler.py", line 308, in _call_silent
    null = open(os.path.devnull, 'rw')
ValueError: must have exactly one of create/read/write/append mode
Comment 1 Florian Best univentionstaff 2020-07-16 17:01:36 CEST
Which python version are you using?
Do you know which command is executed there?
Comment 2 Philipp Hahn univentionstaff 2020-07-16 19:08:43 CEST
(In reply to Florian Best from comment #1)
> Which python version are you using?
> Do you know which command is executed there?

How is that information relevant for fixing the bug?
The error message is clear enough to identify the exact problem.
And it is easy enough to reproduce it:

$ python3.5 -c 'open("/dev/null", "rw")'

And there is exactly one case in UCS-4.4-5:

$ git grep 'open(.*\<rw'
base/univention-config-registry/python/univention/config_registry/handler.py:305:               null = open(os.path.devnull, 'rw')

PS: That check is in cpython/Modules/_io/_iomodule.c:343 since ~2009
Comment 3 Florian Best univentionstaff 2020-07-17 12:26:34 CEST
(In reply to Philipp Hahn from comment #2)
> (In reply to Florian Best from comment #1)
> > Which python version are you using?
> > Do you know which command is executed there?
> 
> How is that information relevant for fixing the bug?
Only because UCS@school Team often experiments with Python > 3.7 and docker / debian bullseye. And that it seems this is not detected in our Jenkins test set up.
Comment 4 Philipp Hahn univentionstaff 2020-07-20 07:39:46 CEST
[feature/ucs5] fc5eb9e94c Bug #51680 ucr: Use contextmanager for open()
 .../python/univention-install-config-registry      | 122 ++++++++++-----------
 .../python/univention/config_registry/backend.py   |  68 ++++++------
 .../python/univention/config_registry/frontend.py  |   6 +-
 .../python/univention/config_registry/handler.py   |  50 +++------
 .../python/univention/config_registry_info.py      |  31 +++---
 .../python/univention/service_info.py              |  26 ++---
 6 files changed, 138 insertions(+), 165 deletions(-)

[feature/ucs5] be96c61d31 Bug #51680 ucr: Add unit test for ucr.handler
 base/univention-config-registry/debian/control     |   8 +-
 .../python/univention/config_registry/handler.py   |   5 +-
 .../tests/test_handler.py                          | 444 +++++++++++++++++++++
 base/univention-config-registry/tests/test_info.py |  65 +++
 4 files changed, 518 insertions(+), 4 deletions(-)

[feature/ucs5] 22965c9571 Bug #51680 ucr: Add coverage support
 base/univention-config-registry/debian/control | 2 ++
 base/univention-config-registry/debian/rules   | 4 ++--
 2 files changed, 4 insertions(+), 2 deletions(-)

[feature/ucs5] f1da4d2814 Bug #51680 ucr: Share test setup code
 base/univention-config-registry/tests/conftest.py  | 34 ++++++++++++++++++++++
 .../tests/test_backend.py                          | 23 ---------------
 .../tests/test_backend_threading.py                |  4 +--
 base/univention-config-registry/tests/test_cli.py  |  3 --
 .../tests/test_info_tools.py                       | 15 ++++++++--
 .../tests/test_interfaces.py                       |  7 +++--
 6 files changed, 51 insertions(+), 35 deletions(-)

[feature/ucs5] 06de88981c Bug #51680 ucr: Update documentation
 .../python/univention/config_registry/handler.py                     | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

[feature/ucs5] 4096ac447c Bug #51680 ucr: Fix open(..., "rw") error
 base/univention-config-registry/debian/changelog                    | 6 ++++++
 .../python/univention/config_registry/handler.py                    | 5 +----
 2 files changed, 7 insertions(+), 4 deletions(-)

[feature/ucs5] c3d090b4c7 fixup! Bug #51680 ucr: Add unit test for ucr.handler
 base/univention-config-registry/debian/ucslint.overrides | 5 +++++
 1 file changed, 5 insertions(+)

Package: univention-config-registry
Version: 15.0.0-7A~5.0.0.202007200715
Branch: ucs_5.0-0

----------- coverage: platform linux, python 3.7.3-final-0 -----------
Name                                              Stmts   Miss  Cover   Missing
-------------------------------------------------------------------------------
python/univention/__init__.py                         1      0   100%
python/univention/config_registry/__init__.py        16      7    56%   46-52
python/univention/config_registry/backend.py        334     86    74%   41-42, 51-53, 71-73, 80, 148-151, 163-165, 170-172, 184-186, 193-195, 276, 281, 286, 324, 329, 353, 358, 374-375, 380, 385, 390, 422, 427, 432, 476-477, 515, 539, 557, 567-569, 603-619, 630-631, 636-637, 647-654, 678, 690-693, 699-702, 705-706, 712, 715-716
python/univention/config_registry/filters.py         24     16    33%   36-37, 51-59, 71-74, 86
python/univention/config_registry/frontend.py       347    286    18%   46-47, 78, 81, 95-121, 134-168, 179-191, 202-204, 215-224, 235-244, 255-258, 269-271, 282-287, 298-309, 320-327, 333-335, 346-409, 420-428, 444-482, 493-504, 515-516, 604-606, 612-614, 619-625, 692, 705-707, 715-804, 807-809
python/univention/config_registry/handler.py        635    270    57%   58-59, 106, 119, 195-196, 261, 267, 284-285, 327, 338-339, 369-371, 393, 398-449, 460-461, 470-472, 491-534, 562, 568, 599, 605, 673-685, 701, 784, 849, 872, 876-885, 900-905, 907-909, 921-949, 961-963, 974-1000, 1011-1052, 1062-1078, 1088-1122, 1132-1141
python/univention/config_registry/interfaces.py     186      8    96%   46, 246, 282-283, 297-299, 304
python/univention/config_registry/misc.py            67     42    37%   39-40, 73-93, 115-126, 142-158, 174-177, 182-183
python/univention/config_registry_info.py           196    131    33%   41-42, 66, 116-119, 132-134, 146-148, 158-168, 173-176, 182-183, 191-209, 222-235, 240-241, 253-275, 280-281, 293-315, 325-341, 360-362, 372-381, 391, 401, 408-409
python/univention/info_tools.py                     121     25    79%   39-41, 49, 63, 91, 133, 138, 160, 184, 189, 203-206, 210-220
python/univention/service_info.py                   183    104    43%   57-58, 61, 68-70, 73-79, 83, 87, 91, 95-98, 101-104, 107-112, 115-118, 121-123, 143-144, 159-160, 163, 172-173, 187-190, 202-205, 209-210, 218-223, 227-242, 253-274, 278-287, 291-292, 300, 309, 318-319, 323-324
-------------------------------------------------------------------------------
TOTAL                                              2110    975    54%

Same for 4.4-5 except the cleanup git:fc5eb9e94c, which converted UCR to use `with open()` and git:c3d090b4c7 squashed in to silence ucslint:

[4.4-5] d697b530cf Bug #51680 ucr: Add unit test for ucr.handler
[4.4-5] 77b9fd94ef Bug #51680 ucr: Add coverage support
[4.4-5] 79d0d844c3 Bug #51680 ucr: Share test setup code
[4.4-5] 0863c47f8d Bug #51680 ucr: Update documentation
[4.4-5] 4d713be7d5 Bug #51680 ucr: Fix open(..., "rw") error

Package: univention-config-registry
Version: 14.0.0-18A~4.4.0.202007200731
Branch: ucs_4.4-0
Scope: errata4.4-5

[4.4-5] 0284bbbd31 Bug #51680: univention-config-registry 14.0.0-18A~4.4.0.202007200731
 doc/errata/staging/univention-config-registry.yaml | 10 ++++++++++
 1 file changed, 10 insertions(+)
Comment 5 Florian Best univentionstaff 2020-07-27 15:09:36 CEST
OK: fixes + unittests
OK~: YAML (very technical)