Bug 57431 - univention-directory-reports: traceback with ipmanagedclients
Summary: univention-directory-reports: traceback with ipmanagedclients
Status: CLOSED FIXED
Alias: None
Product: UCS
Classification: Unclassified
Component: UMC - Reports
Version: UCS 5.0
Hardware: Other Linux
: P5 normal
Target Milestone: UCS 5.0-9-errata
Assignee: Johannes Königer
QA Contact: Arvid Requate
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2024-07-05 16:58 CEST by Oliver Friedrich
Modified: 2024-10-16 16:38 CEST (History)
4 users (show)

See Also:
What kind of report is it?: Bug Report
What type of bug is this?: 7: Crash: Bug causes crash or data loss
Who will be affected by this bug?: 1: Will affect a very few installed domains
How will those affected feel about the bug?: 2: A Pain – users won’t like this once they notice it
User Pain: 0.080
Enterprise Customer affected?:
School Customer affected?: Yes
ISV affected?:
Waiting Support:
Flags outvoted (downgraded) after PO Review:
Ticket number:
Bug group (optional): Workaround is available
Customer ID:
Max CVSS v3 score:


Attachments
Proposed Patch (1.98 KB, patch)
2024-09-04 13:43 CEST, Jan-Luca Kiok
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Oliver Friedrich univentionstaff 2024-07-05 16:58:27 CEST
This:

mkdir /etc/univention/directory/reports/custom
ucr set 'directory/reports/templates/csv/ipmanagedclients=computers/ipmanagedclient "ipmanagedclients" /etc/univention/directory/reports/custom/ipmanagedclients.csv'

Leads to a traceback:
univention-directory-reports -m computers/ipmanagedclient -r "ipmanagedclients" $(udm computers/ipmanagedclient list --filter cn=* | sed -ne 's/^DN: //p')
Traceback (most recent call last):
  File "/usr/bin/univention-directory-reports", line 183, in <module>
    main()
  File "/usr/bin/univention-directory-reports", line 165, in main
    filename = report.create(options.module, options.report, args)
  File "/usr/lib/python3/dist-packages/univention/directory/reports/report.py", line 73, in create
    tmpfile = doc.create_source(objects)
  File "/usr/lib/python3/dist-packages/univention/directory/reports/document.py", line 136, in create_source
    interpret.run()
  File "/usr/lib/python3/dist-packages/univention/directory/reports/interpreter.py", line 100, in run
    self.attribute(token, base_objects[0])
  File "/usr/lib/python3/dist-packages/univention/directory/reports/interpreter.py", line 178, in attribute
    token.value = sep.join(value)
TypeError: sequence item 0: expected str instance, list found


I had a look at the code, at least for ipmanagedclients, the fix is easy:


/usr/lib/python3/dist-packages/univention/directory/report/interpreter.py
@@ -175,7 +175,7 @@
                         token.value = token.attrs.get('default', '')
                     else:
                         sep = token.attrs.get('separator', ', ')
-                        token.value = sep.join(value)
+                        token.value = sep.join(value[0])
                 else:
                     token.value = value
             elif 'default' in token.attrs:

With this, the value variable, which is a list in a list, is properly unpacked.
Comment 2 Oliver Friedrich univentionstaff 2024-07-05 17:24:52 CEST
addition: my "fix" leads to scrambled up IP and mac addresses, like this: 1, 0, ., 2, 0, 0, ., 3, 4, ., 2, 5, 2 (should be: 10.200.34.252)

So the str join part is flaky
Comment 3 Oliver Friedrich univentionstaff 2024-08-20 17:14:07 CEST
cat /etc/univention/directory/reports/custom/ipmanagedclients.csv

<@header@>Rechnername;Beschreibung;MAC-Adresse;Netzwerk;IP-Adresse;dhcpEntryZone;dnsEntryZoneForward;dnsEntryZoneReverse;domain;fqdn;networkAccess
<@/header@>
<@attribute name="name"@>;<@attribute name="description"@>;<@attribute name="mac"@>;<@resolve module="networks/network" dn-attribute="network" alternative=""@><@attribute name="name"@><@/resolve@>;<@attribute name="ip"@>;<@attribute name="dhcpEntryZone"@>;<@attribute name="dnsEntryZoneForward"@>;<@attribute name="dnsEntryZoneReverse"@>;<@attribute name="domain"@>;<@attribute name="fqdn"@>;<@attribute name="networkAccess"@>
<@footer@> <@/footer@>

System used:
univention-app info
UCS: 5.0-8 errata1060
Installed: cups=2.2.1 samba4=4.16 self-service=5.0 self-service-backend=5.0 squid=3.5 ucsschool=5.0 v5 ucsschool-veyon-proxy=4.8.3.8 univention-demo-data=2.1.0-1
Comment 4 Julia Bremer univentionstaff 2024-08-21 09:12:45 CEST
THis happens for all properties that are multivalue and have a "complex" syntax in UDM. 
The code can handle lists (multivalue properties) but properties that are multivalue and complex are returned as lists in lists.

This currently creates a traceback for properties like e.g. dnsEntryZoneForward
Comment 7 Robert Heyer univentionstaff 2024-09-04 11:30:18 CEST
I'll modified the interpreter.py based on the attachment the patch and it did work, but i found another thing:

* Source customfile

<@header@>
Rechnername Beschreibung MAC-Adresse Netzwerk IP-Adresse dhcpEntryZone dnsEntryZoneForward dnsEntryZoneReverse domain
<@/header@>
<@attribute name="name"@>;<@attribute name="description"@>;<@attribute name="mac"@>;<@resolve module="networks/network" dn-attribute="network" alternative=""@><@attribute name="name"@><@/resolve@>;<@attribute name="ip"@>;<@attribute name="dhcpEntryZone"@>;<@attribute name="dnsEntryZoneForward"@>;<@attribute name="dnsEntryZoneReverse"@>;<@attribute name="domain"@>
<@footer@>
<@/footer@>

* script: univention-directory-reports -m computers/ipmanagedclient -r "ipmanagedclients" $(udm computers/ipmanagedclient list --filter cn=* | sed -ne 's/^DN: //p')


* Result CSV-file:

Rechnername Beschreibung MAC-Adresse Netzwerk IP-Adresse dhcpEntryZone dnsEntryZoneForward dnsEntryZoneReverse domain
telefon2;test;86:f5:d1:f5:6b:3e, 86:f5:d1:f5:6b:3a;default;10.200.0.3, 10.200.0.4;;zoneName=nostromo.intranet,cn=dns,dc=nostromo,dc=intranet; 10.200.0.3;zoneName=200.10.in-addr.arpa,cn=dns,dc=nostromo,dc=intranet; 10.200.0.3;nostromo.intranet


* Problem:

zoneName=nostromo.intranet,cn=dns,dc=nostromo,dc=intranet; 10.200.0.3 > semicolon between first value and ip address
zoneName=200.10.in-addr.arpa,cn=dns,dc=nostromo,dc=intranet; 10.200.0.3 > semicolon between first value and ip address

Could you please give me an patch?
Comment 8 Robert Heyer univentionstaff 2024-09-04 13:35:43 CEST
JLK gave me the patch. Its works.

def attribute(self, token, base):
        if 'name' in token.attrs:
            if token.attrs['name'] in base.info:
                value = base.info[token.attrs['name']]
                if isinstance(value, (list, tuple)):
                    if not value or (isinstance(value, str) and value.lower() == 'none'):
                        token.value = token.attrs.get('default', '')
                    else:
                        sep = token.attrs.get('separator', ', ')
                        inner_separator = token.attrs.get('inner-separator', ' ')

                        if base.descriptions[token.attrs['name']].multivalue:
                            if all(isinstance(element, (list, tuple)) for element in value):
                                if all(isinstance(v, str) for element in value for v in element):
                                    value = [inner_separator.join(element) for element in value]
                                else:
                                    value = [inner_separator.join([str(v) for v in element]) for element in value]
                            elif not all(isinstance(element, str) for element in value):
                                value = [str(element) for element in value]
                            token.value = sep.join(value)
                        else:
                            if isinstance(value, (list, tuple)):
                                if all(isinstance(v, str) for v in value):
                                    token.value = inner_separator.join(value)
                                else:
                                    token.value = inner_separator.join([str(v) for v in value])

                else:
                    token.value = value
            elif 'default' in token.attrs:
                token.value = token.attrs['default']
            if token.value is None or token.value == '':
                token.value = ''
                if 'default' in token.attrs:
                    token.value = token.attrs['default']
Comment 9 Jan-Luca Kiok univentionstaff 2024-09-04 13:43:52 CEST
Created attachment 11232 [details]
Proposed Patch
Comment 10 Johannes Königer univentionstaff 2024-10-08 13:01:41 CEST
```
Successful build
Package: univention-directory-reports
Version: 12.0.7-3
Branch: 5.0-0
Scope: errata5.0-9
User: jkoeniger
```
Comment 12 Arvid Requate univentionstaff 2024-10-10 16:41:59 CEST
Verified:
* univention-directory-reports 12.0.7-3 installable in UCS 5.0-9
* univention-directory-reports 14.0.4 installable in UCS 5.2-0
Comment 13 Dirk Wiesenthal univentionstaff 2024-10-16 16:38:07 CEST
<https://errata.software-univention.de/#/?erratum=5.0x1143>