Univention Bugzilla – Bug 53628
TypeError: Object of type bytes is not JSON serializable
Last modified: 2021-08-11 16:37:26 CEST
When creating and editing users. the first noticeable problem is that I always get: ``` An Error Occured; Internal server error: The module process died unexpectedly. ``` inside the UMC when I’m in the users app. then when I try to add or edit an user it tells me: ``` The following properties need to be specified or are invalid: Primary group ``` and there is no way to select a primary group. it also happens when I put it in manually. when I check the group-app it shows me the right users. when I check the log: /var/log/univention/management-console-module-udm.log it tells me: ``` 02.08.21 17:28:40.203 DEBUG_INIT 02.08.21 17:28:40.567 MODULE ( PROCESS ) : Loading python module. 02.08.21 17:28:40.764 MODULE ( PROCESS ) : Imported python module. 02.08.21 17:28:40.764 MODULE ( PROCESS ) : Module instance created. 02.08.21 17:28:40.764 MODULE ( PROCESS ) : Module socket initialized. 02.08.21 17:28:40.787 MODULE ( PROCESS ) : Setting user LDAP DN 'uid=Administrator,cn=myusers,cn=users,dc=domainname,dc=tld' 02.08.21 17:28:40.787 MODULE ( PROCESS ) : Setting auth type to None 02.08.21 17:28:40.787 MODULE ( PROCESS ) : Initializing module. 02.08.21 17:28:40.797 MAIN ( PROCESS ) : LDAP bind for user 'uid=Administrator,cn=myusers,cn=users,dc=domainname,dc=tld'. 02.08.21 17:28:40.814 MAIN ( PROCESS ) : LDAP bind for user 'uid=Administrator,cn=myusers,cn=users,dc=domainname,dc=tld'. 02.08.21 17:28:40.825 MAIN ( PROCESS ) : LDAP bind for user 'uid=Administrator,cn=myusers,cn=users,dc=domainname,dc=tld'. 02.08.21 17:28:40.826 MAIN ( PROCESS ) : LDAP bind for user 'uid=Administrator,cn=myusers,cn=users,dc=domainname,dc=tld'. 02.08.21 17:28:40.829 MAIN ( PROCESS ) : LDAP bind for user 'uid=Administrator,cn=myusers,cn=users,dc=domainname,dc=tld'. 02.08.21 17:28:40.831 MAIN ( PROCESS ) : LDAP bind for user 'uid=Administrator,cn=myusers,cn=users,dc=domainname,dc=tld'. 02.08.21 17:28:40.859 ADMIN ( WARN ) : mapValueDecoded returns a list for service. This is probably not wanted? 02.08.21 17:28:40.860 ADMIN ( WARN ) : mapValueDecoded returns a list for service. This is probably not wanted? 02.08.21 17:28:40.865 ADMIN ( WARN ) : mapValueDecoded returns a list for service. This is probably not wanted? 02.08.21 17:28:40.865 ADMIN ( WARN ) : mapValueDecoded returns a list for service. This is probably not wanted? 02.08.21 17:28:40.877 ADMIN ( WARN ) : mapValueDecoded returns a list for service. This is probably not wanted? 02.08.21 17:28:41.237 PROTOCOL ( ERROR ) : Could not JSON serialize message: {'result': [{'module': 'udm', 'flavor': 'users/user', 'objectType': 'users/user', 'id': '2002', 'label': b'Administrator', 'icon': 'udm-users-user'}, None, 'headers': {'Vary': 'Content-Language', 'Content-Language': 'en-US'}, 'error': None, 'reason': None, 'status': 200} *> removed some content here as it listed all my users like this.* 02.08.21 17:28:41.237 MAIN ( WARN ) : Shutting down all open connections 02.08.21 17:28:41.241 MODULE ( ERROR ) : Traceback (most recent call last): File "/usr/sbin/univention-management-console-module", line 122, in <module> notifier.loop() File "/usr/lib/python3/dist-packages/notifier/nf_generic.py", line 306, in loop step() File "/usr/lib/python3/dist-packages/notifier/nf_generic.py", line 297, in step __min_timer = dispatch.dispatcher_run() File "/usr/lib/python3/dist-packages/notifier/dispatch.py", line 75, in dispatcher_run if not disp(): File "/usr/lib/python3/dist-packages/notifier/threads.py", line 161, in _simple_threads_dispatcher task.announce() File "/usr/lib/python3/dist-packages/notifier/threads.py", line 139, in announce self._callback(self, self._result) File "/usr/lib/python3/dist-packages/notifier/__init__.py", line 105, in __call__ return self._function(*tmp, **self._kwargs) File "/usr/lib/python3/dist-packages/univention/management/console/base.py", line 283, in thread_finished_callback self.finished(request.id, result) File "/usr/lib/python3/dist-packages/univention/management/console/base.py", line 465, in finished self.result(res) File "/usr/lib/python3/dist-packages/univention/management/console/base.py", line 469, in result self.signal_emit('success', response) File "/usr/lib/python3/dist-packages/notifier/signals.py", line 87, in signal_emit self.__signals[signal].emit(*args) File "/usr/lib/python3/dist-packages/notifier/signals.py", line 50, in emit if cb(*args): File "/usr/lib/python3/dist-packages/notifier/__init__.py", line 105, in __call__ return self._function(*tmp, **self._kwargs) File "/usr/lib/python3/dist-packages/univention/management/console/protocol/modserver.py", line 141, in _reply self.response(msg) File "/usr/lib/python3/dist-packages/univention/management/console/protocol/modserver.py", line 376, in response self.__queue += bytes(msg) File "/usr/lib/python3/dist-packages/univention/management/console/protocol/message.py", line 346, in __bytes__ return Message._formattedMessage(self._id, self._type, self.mimetype, self.command, body, self.arguments) File "/usr/lib/python3/dist-packages/univention/management/console/protocol/message.py", line 123, in _formattedMessage data = json.dumps(body) File "/usr/lib/python3.7/json/__init__.py", line 231, in dumps return _default_encoder.encode(obj) File "/usr/lib/python3.7/json/encoder.py", line 199, in encode chunks = self.iterencode(o, _one_shot=True) File "/usr/lib/python3.7/json/encoder.py", line 257, in iterencode return _iterencode(o, 0) File "/usr/lib/python3.7/json/encoder.py", line 179, in default raise TypeError(f'Object of type {o.__class__.__name__} ' TypeError: Object of type bytes is not JSON serializable ``` system diagnostics and these tools don’t show any erros: samba-tool drs kcc samba-tool dbcheck samba-tool dbcheck --cross-ncs this is also happenig on a new “Backup Directory Node”
Created attachment 10793 [details] patch? / hotfix thank you for your bug report. I am currently unsure what caused this bug. Could you list all yours installed apps and extended attributes? univention-app info udm settings/extended_attributes list Do you have any custom UCR setting overwriting some syntax classes or default labels? ucr search directory/manager/web/ Can you open the developer tools in your browser, select the network tab and right click on the request which fails (it's something with syntax/choices) and click copy → copy as curl and paste that request here? I need the specific syntax class which causes the request to fail. Looking into the code I found the potential cause. could you test the attached patch? patch -p0 -i /root/53628.patch -d /
how can I download this patch? and I would actually be happy to setup the domain from scratch.. but I need to know the passwords of my users to create them manually. univention-app info: UCS: 5.0-0 errata65 Installed: letsencrypt=2.0.0-1 radius=5.0 samba4=4.13 self-service=5.0 self-service-backend=5.0 4.4/openid-connect-provider=2.2-konnect-0.33.11-2 udm settings/extended_attributes list: unknown module settings/extended_attributes ucr search directory/manager/web/: directory/manager/web/allow_wildcard_search: <empty> Defines whether wildcards (*) are evaluated when searching. directory/manager/web/auto_substring_search: <empty> Defines if it is searched automatically for substrings. directory/manager/web/language: de_CH.UTF-8 directory/manager/web/modules/.*/.*/display: <empty> This variable configures the name of an object displayed in the grid of the web frontend. Allowed values are properties of the object, e.g. displayName for users. directory/manager/web/modules/.*/.*/search/advanced_on_open: <empty> The initial display of the simple or advanced search (see 'directory/manager/web/modules/search/advanced_on_open') can also be set per module, e.g. 'directory/manager/web/modules/users/user/search/advanced_on_open=true'. directory/manager/web/modules/.*/.*/search/autosearch: <empty> The automatic search of the UMC domain management modules (see 'directory/manager/web/modules/autosearch') can also be disabled per module, e.g. 'directory/manager/web/modules/users/user/search/autosearch=false'. directory/manager/web/modules/.*/.*/search/default: <empty> This variable allows to configure for every module which attribute should be searched for by default, e.g. 'directory/manager/web/modules/users/user/search/default=lastname'. directory/manager/web/modules/.*/.*/show/adnotification: <empty> This variable allows to configure for every module whether a warning should be displayed when creating objects if the system is in Active Directory mode. directory/manager/web/modules/.*/.*/wizard/disabled: <empty> The simplified form for creating new objects (see 'directory/manager/web/modules/wizard/disabled') can be disabled per module, e.g. 'directory/manager/web/modules/users/user/wizard/disabled=true'. directory/manager/web/modules/autosearch: 1 By default an automatic search for all objects is performed by the domain management modules of the Univention Management Console. This behavior can be completely disabled through this option. Alternatively, the autosearch can be configured on a per module basis (see 'directory/manager/web/modules/.*/.*/search/autosearch'). directory/manager/web/modules/computers/computer/add/default: computers/windows directory/manager/web/modules/groups/group/caching/uniqueMember/timeout: 300 Ein group may contain users, computers or further groups. The UMC maintains an internal cache storing the membership type. This variable configures the time frame in seconds the type is cached. Setting it to -1 disables the cache. If the variable is unset, 300 applies. directory/manager/web/modules/groups/group/checks/circular_dependency: yes If this variable is set to 'yes' or unset, cyclic dependencies of nested groups are automatically detected and refused. This check can be disabled with 'no'. directory/manager/web/modules/search/advanced_on_open: false This variable determines whether the simple or advanced search is initially displayed when a domain management module is opened. This setting applies to all modules. In addition, this setting can be specifically overwritten for individual modules, e.g. 'directory/manager/web/modules/users/user/search/advanced_on_open=true'. directory/manager/web/modules/users/user/add/default: <empty> This variable configures the DN or the label of a user template which is preselected when creating a user. directory/manager/web/modules/users/user/properties/homePostalAddress/syntax: postalAddress directory/manager/web/modules/users/user/wizard/property/.*/default: <empty> Defines the default value for the referenced widget in the user create wizard (placeholder are e.g. invite, overridePWLength or pwdChangeNextLogin). Caution: This overwrites the value of all templates! directory/manager/web/modules/users/user/wizard/property/.*/visible: <empty> Defines if the referenced widget is shown in the user create wizard by default (placeholder are e.g. invite, overridePWLength or pwdChangeNextLogin). directory/manager/web/modules/wizard/disabled: <empty> Some modules provide a simplified form for creating objects in the domain management modules of the Univention Management Console. It is shown instead of the full form. This behavior can be disabled through this option. The wizards can also be disabled per module (see 'directory/manager/web/modules/.*/.*/wizard/disabled'). directory/manager/web/modules/wizards/disabled: no directory/manager/web/sizelimit: 2000 This variable configures an upper limit for search results. If more search results have been found, the search is aborted and instead the user is asked to refine the search. If the variable is unset, the limit is 2000. directory/manager/web/widget/.*/default: <empty> Defines the default search value for the widget definition (e.g.: *). directory/manager/web/widget/.*/property/.*: <empty> Defines a differing widget path for the specified property. directory/manager/web/widget/.*/subclasses: <empty> Defines whether this widget is also applied on subclasses of the defined syntax class. directory/manager/web/widget/.*/syntax: <empty> Defines a commaseparated list of syntax classes where this widget will be applied to. directory/manager/web/widget/.*/widget: <empty> Defines a widget for displaying in UMC. The value is the path to the widget. devtools-> network: main.js:1234 Module could not be loaded: umc/modules/reboot (anonymous) @ main.js:1234 main.js:1234 udm:license-import is not a umc.widgets._ModuleMixin! (2} (anonymous) @ main.js:1234 main.js:1234 Module could not be loaded: umc/modules/lib (anonymous) @ main.js:1234 main.js:1234 Module could not be loaded: umc/modules/ipchange (anonymous) @ main.js:1234 main.js:1234 Module could not be loaded: umc/modules/serveroverview (anonymous) @ main.js:1234 /univention/management/?header=try-hide&overview=false&menu=false#module=udm:users/user:0::1 [DOM] Multiple forms should be contained in their own form elements; break up complex forms into ones that represent a single action: (More info: https://goo.gl/9p2vKq) <form data-dojo-attach-point="containerNode" data-dojo-attach-event="onreset:_onReset,onsubmit:_onSubmit" id="umc_widgets_Form_0" class="umcUDMDetailForm" widgetid="umc_widgets_Form_0">…</form> /univention/command/udm/syntax/choices:1 Failed to load resource: the server responded with a status of 510 (Not Extended) /univention/command/udm/syntax/choices:1 Failed to load resource: the server responded with a status of 510 (Not Extended) /univention/command/udm/syntax/choices:1 Failed to load resource: the server responded with a status of 510 (Not Extended)
(In reply to 90j4k89034mj95mj489 from comment #2) > how can I download this patch? wget -O /root/53628.patch 'https://forge.univention.org/bugzilla/attachment.cgi?id=10793&action=diff&context=patch&collapsed=&headers=1&format=raw'
(In reply to 90j4k89034mj95mj489 from comment #2) > and I would actually be happy to setup the domain from scratch.. but I need > to know the passwords of my users to create them manually. Setting up the domain from strech will not fix this problem and is not a good choice. If you still want to, you can backup the ldap database and readd the content - so all passwords are the same. If you do this please ask for help on help.univention.com.
the patch works :)
but this patch will be in an update too, or do I need to add it manually to every new UCS system? like.. is it a fix for my problem but th eproblem shouldn't occur in the first place?
Yes, we will release an upgrade for this problem but I can't guarantee when it will be released. I can reproduce the problem with the following command: curl 'https://Administrator:univention@demo.univention.de/univention/command/udm/syntax/choices' -H 'Content-Type: application/json' --data-raw '{"options":{"syntax":"LDAP_Search","options":{"syntax":"LDAP_Search","filter":"(objectClass=univentionMailDomainname)","viewonly":false,"base":"","value":"mail/domain: name","attributes":["mail/domain: cn"],"empty":false,"empty_end":false}},"flavor":"mail/mail"}'
univention-management-console-module-udm.yaml 1aa30c8f2b80 | Bug #53628: prevent crash when a LDAP_Search syntax uses a LDAP attribute as displayed label. univention-management-console-module-udm (10.0.1-13) 1aa30c8f2b80 | Bug #53628: prevent crash when a LDAP_Search syntax uses a LDAP attribute as displayed label.
OK - reproducer and fixed OK - univention-management-console-module-udm OK - yaml -> curl 'https://Administrator:univention@master.five.local/univention/command/udm/syntax/choices' -H 'Content-Type: application/json' --data-raw '{"options":{"syntax":"LDAP_Search","options":{"syntax":"LDAP_Search","filter":"(objectClass=univentionMailDomainname)","viewonly":false,"base":"","value":"mail/domain: name","attributes":["mail/domain: cn"],"empty":false,"empty_end":false}},"flavor":"mail/mail"}' {"result": [{"module": "udm", "flavor": "mail/mail", "objectType": "mail/domain", "id": "five.local", "label": "five.local", "icon": "udm-mail-domain"}], "message": null, "error": null, "reason": null, "status": 200} couldn't reproduce this in umc, but couldn't find a problem after update either
<https://errata.software-univention.de/#/?erratum=5.0x68>