Bug 54773 - Write UDM properties with UDM Type into appcenter listener JSON files
Write UDM properties with UDM Type into appcenter listener JSON files
Product: UCS
Classification: Unclassified
Component: App Center
UCS 5.0
Other Linux
: P5 normal (vote)
: UCS 5.0-3
Assigned To: Florian Best
Peter Stoll
: 54776 (view as bug list)
Depends on:
Blocks: 54307
  Show dependency treegraph
Reported: 2022-05-18 18:05 CEST by Florian Best
Modified: 2023-02-13 11:41 CET (History)
3 users (show)

See Also:
What kind of report is it?: Development Internal
What type of bug is this?: ---
Who will be affected by this bug?: ---
How will those affected feel about the bug?: ---
User Pain:
Enterprise Customer affected?:
School Customer affected?:
ISV affected?:
Waiting Support:
Flags outvoted (downgraded) after PO Review:
Ticket number:
Bug group (optional): Cleanup
Max CVSS v3 score:
best: Patch_Available+


Note You need to log in before you can comment on or make changes to this bug.
Description Florian Best univentionstaff 2022-05-18 18:05:49 CEST
The univention-appcenter-listener-converter script writes JSON files containing the UDM properties into the data/listener/ directory of docker apps.

The property information are the raw UDM properties i.e. "1" string for boolean values.

We could additionally (to be backwards compatible) also write the JSON type representation from the UDM REST API into it.
Comment 1 Florian Best univentionstaff 2022-05-18 18:07:14 CEST
Patch in MR: https://git.knut.univention.de/univention/ucs/-/merge_requests/395
Comment 2 Daniel Tröder univentionstaff 2022-05-19 09:04:08 CEST
*** Bug 54776 has been marked as a duplicate of this bug. ***
Comment 4 Dirk Wiesenthal univentionstaff 2023-01-04 14:47:35 CET
We should try to do this via the UDM REST API. This way, we are closer to the way we want the UCS ecosystem to work. We would also benefit from anything added to the API.

We would need to adjust it so that the UDM REST API accepts (at least read) connections from managed nodes, though.

One downside may be the performance.

If the App has set a specific attribute, the old python API is used. Otherwise, the UDM REST API is used. The keys in the JSON file stay the same.

I suggest we add an attibute "ListenerUDMLegacy = False" (default). With True, it still uses the python API. And we change all Apps that currently have ListenerUDMModules set.
Comment 5 Daniel Tröder univentionstaff 2023-01-06 09:12:52 CET
+1 for the proposal.

I don't think the performance is an issue during normal operations.
Just an additional "lag" of <100ms per object.
Not relevant, as the "listener_trigger" runs only every 5.000ms!

But in the case of a resync, where maybe >100.000 objects are read, that will become a problem.
The performance loss can be compensated by the "appcenter_converter" service, it it uses multiple async/threaded connections to the UDM REST API to work on its queue.
Comment 7 Florian Best univentionstaff 2023-02-01 20:53:09 CET
The listener converter now writes the data in the UDM REST API representation to the JSON files if the listener_udm_version==2.
The default of listener_udm_version has been set to 2 as requested by Dirk. He adjusted all ini files of all existing apps which used it to set version=1.
The documentation has been adjusted so that the new representation is listed there.

The type information are received by the Python UDM API, not by requesting the UDM REST API. E.g .a memberserver has no access to the UDM REST API and the replication of the DC Master (where the UDM REST API runs) might be on a different replication state.

univention-appcenter (9.0.4-1)
fd53ad3d7caa | Bug #54773: change default of listener_udm_version to 2
d136a5d0c4f6 | Bug #54773: add error handling for global exceptions
4ba7289ea7c2 | Bug #54773: write UDM type information into app JSON listener files

39f1342d2ff3 | docs: Changelog Bug #54773
e7f8f67a5749 | Bug #54773: document "object" with ListenerUDMVersion=2
f4e03079211e | Bug #54773: fix wrong name of dict key in listener integration
Comment 8 Florian Best univentionstaff 2023-02-03 18:12:30 CET
The code introduced an error if the object was not found (probably race condition in tests).
Fixed in:
29c4dcedbf Bug #54773: fix unbound-local variable access

    Traceback (most recent call last):
      File "/usr/share/univention-appcenter-listener-converter", line 217, in <module>
      File "/usr/share/univention-appcenter-listener-converter", line 206, in main
      File "/usr/share/univention-appcenter-listener-converter", line 182, in find_and_convert_files
        if convert(app, dumped, filename, lo, pos):
      File "/usr/share/univention-appcenter-listener-converter", line 129, in convert
        for key, value in attrs.items()
    UnboundLocalError: local variable 'attrs' referenced before assignment
Comment 9 Peter Stoll univentionstaff 2023-02-06 15:00:34 CET

- verified changelog
- verified YAML
- code reviewed
- verfied packet installation
- tested and diffed json files with app.listener_udm_version 2 vs. <2
Comment 11 Florian Best univentionstaff 2023-02-13 11:41:26 CET
UCS 5.0-3 has been released.


If this error occurs again, please clone this bug.