Bug 56622 - Bad CSV parsing leading to fix_oxContext_attribute skipping users
Bad CSV parsing leading to fix_oxContext_attribute skipping users
Status: NEW
Product: Z_Internal OX development
Classification: Unclassified
Component: Generic
unspecified
Other Linux
: P5 normal (vote)
: ---
Assigned To: Mail maintainers
:
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2023-09-18 17:37 CEST by Christian Kowarzik
Modified: 2023-09-19 08:35 CEST (History)
2 users (show)

See Also:
What kind of report is it?: Bug Report
What type of bug is this?: 4: Minor Usability: Impairs usability in secondary scenarios
Who will be affected by this bug?: 3: Will affect average number of installed domains
How will those affected feel about the bug?: 2: A Pain – users won’t like this once they notice it
User Pain: 0.137
Enterprise Customer affected?: Yes
School Customer affected?:
ISV affected?:
Waiting Support:
Flags outvoted (downgraded) after PO Review:
Ticket number:
Bug group (optional):
Max CVSS v3 score:
troeder: Patch_Available+


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Christian Kowarzik 2023-09-18 17:37:07 CEST
System-Info:

UCS: 5.0-5 errata804
Installed: fetchmail=6.3.26 mailserver=12.0 ox-connector=2.2.7 oxseforucs=7.10.6-ucs9

univention-ox-12.0.38


Abstract:
The Script /usr/sbin/fix_oxContext_attribute skips ox-users with a comma-character in their OX Display Name (e.g. the format "Surname, Forename") due to incomplete csv-parsing, thus not updating these users ox-context from 'None' to a valid entry if necessary.
Another example of OX Display Names that will trigger this bug would be typing errors with abbreviated middle names (e.g. "John M, Doe"). 

For background why the script 'fix_oxContext_attribute' is needed at all refer to: 
https://help.univention.com/t/recover-broken-environments-with-early-installations-of-ox-app-suite-7-10-6-ucs4-and-ox-connector-2-1-2/21471


Explanation:

The Script /usr/sbin/fix_oxContext_attribute parses the csv formatted output of the command /opt/open-xchange/sbin/listuser to get the users primary e-mail address.
The field separator of this csv output is the comma-character, but the script 'fix_oxContext_attribute' does not take into account that a comma-character might also be part of the field-data thus wrongly splitting the respective csv-record and not getting the primary e-mail address from these users.

Example with part of csv output from command 'listuser' that will trigger this bug:
(with the users email-address in the second field)

   Display_name,PrimaryEmail,MaxQuota
   "Doe, John","jdoe@example.intranet",-1    (this will give ' John"' as data from the second field instead of '"jdoe@example.intranet"')
   "John M, Doe","jmdoe@example.intranet",-1 (this will give ' Doe"' as data from the second field instead of '"jmdoe@example.intranet"')


Suggested solution:

Let command 'cvstool' instead of 'awk' parse the csv formatted output and deal with unexpected input.
The 'csvtool' package is not installed by default and thus needs to be added as dependency to the 'univention-ox' package.

Patch to replace 'awk' with 'csvtool' in script 'fix_oxContext_attribute':

=== code===
root@ucs-2201:~# diff -u fix_oxContext_attribute /usr/sbin/fix_oxContext_attribute 
--- fix_oxContext_attribute	2023-09-18 18:21:41.957509538 +0200
+++ /usr/sbin/fix_oxContext_attribute	2023-09-18 18:23:43.701835477 +0200
@@ -74,14 +74,8 @@
 echo
 /opt/open-xchange/sbin/listcontext -P $(< /etc/ox-secrets/master.secret) -A oxadminmaster | awk 'NR>1 { print $1; }' > ctxs.txt
 cat ctxs.txt | xargs -I{} bash -c '/opt/open-xchange/sbin/listuser -c {} -A $(if [[ {} == 10 ]]; then echo "oxadmin"; else echo "oxadmin-context{}"; fi) -P $(< /etc/ox-secrets/context{}.secret) --csv \
-| awk -F ","  '"'"' \
-NR==1 {
-    for (i=1; i<=NF; i++) {
-        f[$i] = i
-    }
-}
-{ if (NR!=1) print $(f["PrimaryEmail"]) }
-  '"'"' \
+| csvtool namedcols PrimaryEmail - \
+| csvtool drop 1 - \
 | xargs -I{} univention-ldapsearch -xLLL '"'"'(&(&(mailPrimaryAddress={})(isOXUser=OK))(!(oxContextIDNum=*)))'"'"' 1.1 \
 | sed -ne '"'"'s,dn: ,,p'"'"' >> dns_ctx{}.txt '
=== code ===
Comment 1 Daniel Tröder univentionstaff 2023-09-19 08:33:20 CEST
Thank you for the patch!