Bug 33522 - UDM module process dies when response contains binary data
UDM module process dies when response contains binary data
Status: CLOSED FIXED
Product: Z_FUKITS
Classification: Unclassified
Component: univention-bittorrent
unstable
Other Linux
: P5 normal
: 3.1 RC1
Assigned To: Felix Botner
Daniel Orrego
:
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2013-11-21 17:10 CET by Alexander Kläser
Modified: 2015-05-07 20:08 CEST (History)
2 users (show)

See Also:
What kind of report is it?: ---
What type of bug is this?: ---
Who will be affected by this bug?: ---
How will those affected feel about the bug?: ---
User Pain:
Enterprise Customer affected?:
School Customer affected?:
ISV affected?:
Waiting Support:
Flags outvoted (downgraded) after PO Review:
Ticket number:
Bug group (optional):
Max CVSS v3 score:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Alexander Kläser univentionstaff 2013-11-21 17:10:13 CET
Due to this problem, we can currently not display a torrent file in UMC.

An ugly but tempting workaround seems to be to set the syntax class of the property "torrent" to userPasswd. This syntax class get filtered out from by the UDM UMC module =) . Downside: from UMC one cannot edit the object anymore.

Another more sensible option would be to adjust the tool chain in order to set/return base64 encoded data on the UDM side which is mapped into binary data.

+++ This bug was initially created as a clone of Bug #33520 +++

I opened a univention-bittorrent file in the LDAP directory tree and the UMC-UDM module process died with the following traceback:

21.11.13 16:14:06.866  MODULE      ( ERROR   ) : Traceback (most recent call last):
  File "/usr/sbin/univention-management-console-module", line 112, in <module>
    notifier.loop()
  File "/usr/lib/pymodules/python2.6/notifier/nf_generic.py", line 284, in loop
    step()
  File "/usr/lib/pymodules/python2.6/notifier/nf_generic.py", line 276, in step
    __min_timer = dispatch.dispatcher_run()
  File "/usr/lib/pymodules/python2.6/notifier/dispatch.py", line 72, in dispatcher_run
    if not disp():
  File "/usr/lib/pymodules/python2.6/notifier/threads.py", line 154, in _simple_threads_dispatcher
    task.announce()
  File "/usr/lib/pymodules/python2.6/notifier/threads.py", line 135, in announce
    self._callback( self, self._result )
  File "/usr/lib/pymodules/python2.6/notifier/__init__.py", line 104, in __call__
    return self._function( *tmp, **self._kwargs )
  File "/usr/lib/pymodules/python2.6/univention/management/console/modules/udm/__init__.py", line 132, in _thread_finished
    self.finished( request.id, result )
  File "/usr/lib/pymodules/python2.6/univention/management/console/modules/__init__.py", line 271, in finished
    self.result( res )
  File "/usr/lib/pymodules/python2.6/univention/management/console/modules/__init__.py", line 278, in result
    self.signal_emit( 'success', response )
  File "/usr/lib/pymodules/python2.6/notifier/signals.py", line 75, in signal_emit
    self.__signals[ signal ].emit( *args )
  File "/usr/lib/pymodules/python2.6/notifier/signals.py", line 41, in emit
    cb( *args )
  File "/usr/lib/pymodules/python2.6/notifier/__init__.py", line 104, in __call__
    return self._function( *tmp, **self._kwargs )
  File "/usr/lib/pymodules/python2.6/univention/management/console/protocol/modserver.py", line 109, in _reply
    self.response( msg )
  File "/usr/lib/pymodules/python2.6/univention/management/console/protocol/modserver.py", line 292, in response
    data = str( msg )
  File "/usr/lib/pymodules/python2.6/univention/management/console/protocol/message.py", line 315, in __str__
    return Message._formattedMessage(self._id, self._type, self.mimetype, self.command, body, self.arguments)
  File "/usr/lib/pymodules/python2.6/univention/management/console/protocol/message.py", line 119, in _formattedMessage
    data = json.dumps( body )
  File "/usr/lib/pymodules/python2.6/simplejson/__init__.py", line 261, in dumps
    return _default_encoder.encode(obj)
  File "/usr/lib/pymodules/python2.6/simplejson/encoder.py", line 214, in encode
    chunks = self.iterencode(o, _one_shot=True)
  File "/usr/lib/pymodules/python2.6/simplejson/encoder.py", line 282, in iterencode
    return _iterencode(o, 0)
UnicodeDecodeError: 'utf8' codec can't decode byte 0xc6 in position 197: invalid continuation byte
Comment 1 Ingo Steuwer univentionstaff 2013-11-21 17:18:19 CET
Janek, how many tools need to be changed if we'd store the hash base64 encoded?
Comment 2 Janek Walkenhorst univentionstaff 2013-11-21 17:42:44 CET
(In reply to Ingo Steuwer from comment #1)
> Janek, how many tools need to be changed if we'd store the hash base64
> encoded?
Complete univention-bittorrent/ - that is the "library" must be changed and every tool tested.

Also 33⅓% space overhead.

I'd much rather see UDM/UMC fixed.
Comment 3 Janek Walkenhorst univentionstaff 2013-11-21 17:50:25 CET
The real problem is
>   File
> "/usr/lib/pymodules/python2.6/univention/management/console/protocol/message.
> py", line 119, in _formattedMessage
>     data = json.dumps( body )
giving trying to encode data of a type (bytes) that JSON cannot store
||
giving a <str> object instead of a <unicode> object to the json library

>   File "/usr/lib/pymodules/python2.6/simplejson/__init__.py", line 261, in
> dumps
>     return _default_encoder.encode(obj)
leading to the json library interpreting it as encoded text
||
leading to the json library having to guess/assume an encoding of some byte string

> UnicodeDecodeError: 'utf8' codec can't decode byte 0xc6 in position 197:
> invalid continuation byte
and thus failing.
Comment 4 Ingo Steuwer univentionstaff 2013-11-22 15:19:54 CET
(In reply to Janek Walkenhorst from comment #2)
> (In reply to Ingo Steuwer from comment #1)
> > Janek, how many tools need to be changed if we'd store the hash base64
> > encoded?
> Complete univention-bittorrent/ - that is the "library" must be changed and
> every tool tested.
> 
> Also 33⅓% space overhead.
> 
> I'd much rather see UDM/UMC fixed.

Yes, that would be the "right" way (I agreed to that in a discussion earlier yesterday with Alex), that's Bug 33520. But we shouldn't fork UDM and UMC to implement a component, because it would generate much more work and maybe problems in the long term.

Felix, please change the handling to base64 encoded binaries in the lib. As long as we don't reach the RC1 we don't need backward compatibility.
Comment 5 Felix Botner univentionstaff 2013-11-25 11:01:20 CET
univention-bittorrent now saves the torrent attriubte as base64 encoded string.
Comment 6 Daniel Orrego univentionstaff 2014-02-11 11:11:45 CET
Verified.