Bug 48504 - CSV separator (,) of the class lists is not recognized by Excel or columns are not separated
CSV separator (,) of the class lists is not recognized by Excel or columns ar...
Status: CLOSED FIXED
Product: UCS@school
Classification: Unclassified
Component: UMC - Class lists
UCS@school 4.3
Other Mac OS X 10.1
: P5 normal (vote)
: UCS@school 4.4 v3
Assigned To: Jürn Brodersen
Sönke Schwardt-Krummrich
:
Depends on:
Blocks: 48957
  Show dependency treegraph
 
Reported: 2019-01-21 20:58 CET by Michel Smidt
Modified: 2019-06-26 14:19 CEST (History)
1 user (show)

See Also:
What kind of report is it?: Bug Report
What type of bug is this?: 4: Minor Usability: Impairs usability in secondary scenarios
Who will be affected by this bug?: 2: Will only affect a few installed domains
How will those affected feel about the bug?: 2: A Pain – users won’t like this once they notice it
User Pain: 0.091
Enterprise Customer affected?:
School Customer affected?: Yes
ISV affected?:
Waiting Support:
Flags outvoted (downgraded) after PO Review:
Ticket number:
Bug group (optional):
Max CVSS v3 score:


Attachments
Screenshot Excel-Import (173.80 KB, image/jpeg)
2019-01-21 20:58 CET, Michel Smidt
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Michel Smidt 2019-01-21 20:58:23 CET
Created attachment 9809 [details]
Screenshot Excel-Import

The class list module is reported to be a popular choice. Unfortunately the CSV separator (,) of the class lists which will be genereated is not recognized by Excel and the columns are not separated if the file will be opened.
Comment 2 Jürn Brodersen univentionstaff 2019-02-04 16:21:46 CET
I have two ideas:
We add a drop-down to choose the separator, or just change it to semicolons. This might break in some regions because Excel seems to choose the separator according to the systems locale settings.

We export the list as ods. Modern version of excel can read ods and this would probably also solve encoding problems.
See for example unoconv or python-odf for conversion tools
Comment 3 Jürn Brodersen univentionstaff 2019-05-08 12:18:10 CEST
Changing the separator to ";" would basically break the csv import into excel in all regions that use a "." as a decimal separator. That includes US/UK.

I guess we could use the umc language setting to decide which separator to use.
Comment 4 Jürn Brodersen univentionstaff 2019-05-17 17:24:37 CEST
[4.4 f412b5c2a] Bug #48504: Change csv separator according to the current umc language settings
[4.4 760b20378] Bug #48504: yaml
Comment 5 Sönke Schwardt-Krummrich univentionstaff 2019-05-22 13:11:00 CEST
(In reply to Jürn Brodersen from comment #3)
> I guess we could use the umc language setting to decide which separator to
> use.

I don't think so. Using a en_US locale for UMC does not imply, that the Excel version is also a US version and not a DE version.
(I tested this together with Michel: his browser used a english UMC, the locale of Excel was in english but the Excel installation was german, so the delimiter had not been detected correctly.

I think we should add a drop down to select the delimiter with 2 options:

"Delimiter" → "semicolon", "comma"
"Trennzeichen" → "Semikolon", "Komma"

Semicolon should be the default, because it works with open-/libreoffice and for DE installations of Excel.

@Michel: any objections?
Comment 6 Jürn Brodersen univentionstaff 2019-05-22 13:32:40 CEST
Should I store the last choice? I could save it in ldap like favourites (which is afaik atm broken for saml) or in a cookie.
Comment 7 Michel Smidt 2019-05-22 18:05:31 CEST
(In reply to Sönke Schwardt-Krummrich from comment #5)
> (In reply to Jürn Brodersen from comment #3)
> > I guess we could use the umc language setting to decide which separator to
> > use.
> 
> I don't think so. Using a en_US locale for UMC does not imply, that the
> Excel version is also a US version and not a DE version.
> (I tested this together with Michel: his browser used a english UMC, the
> locale of Excel was in english but the Excel installation was german, so the
> delimiter had not been detected correctly.
> 
> I think we should add a drop down to select the delimiter with 2 options:
> 
> "Delimiter" → "semicolon", "comma"
> "Trennzeichen" → "Semikolon", "Komma"
> 
> Semicolon should be the default, because it works with open-/libreoffice and
> for DE installations of Excel.
> 
> @Michel: any objections?

Hi, 
honestly, I find a drop-down for just two options too much. What would that look like? 
Maybe I didn't understand that properly either.
For my terms, that's one click too many if you want to select the option "comma". 

If a teacher wants to download a class list, there should be two buttons labeled "Excel (CSV)" + "Libreoffice (CSV)".
Comment 8 Jürn Brodersen univentionstaff 2019-05-24 14:13:54 CEST
[4.4 d213ec464] Bug #48504: Improve csv export compatibility by using UTF-16 and tabs
[4.4 c78c40d0a] Bug #48504: yaml

Package: ucs-school-umc-lists
Version: 2.0.0-2A~4.4.0.201905241405
Branch: ucs_4.4-0
Scope: ucs-school-4.4
Comment 9 Jürn Brodersen univentionstaff 2019-05-24 14:17:19 CEST
Notes:
UTF-16 with tabs did work with excel both in English and German. It also works with libreoffice (at least on a fresh installation tabs are part of the default pre selected separators). Google docs and Micorsoft live were also able to open the file without problems.
Comment 10 Sönke Schwardt-Krummrich univentionstaff 2019-05-29 23:29:57 CEST
[4.4] 77c1efea9 Bug #48504: update advisory
[4.4] 1857d48d6 Bug #48504: update advisory
[4.4] f1008d751 Bug #48504: fixed some spelling errors

Why do you convert the file within the browser to UTF-16 and not within the backend?

??: code change
OK: installation
??: ucs-test
OK: changelog entry
OK: advisory
OK: changes applied to 4.4 branch
OK: functional change
OK: package built and installable
Comment 11 Sönke Schwardt-Krummrich univentionstaff 2019-05-30 00:27:28 CEST
(In reply to Sönke Schwardt-Krummrich from comment #10)
> ??: code change
> ??: ucs-test

Everything else was ok. So if the question is discussed, the bug may jump to VERIFIED.
Comment 12 Sönke Schwardt-Krummrich univentionstaff 2019-06-03 10:21:22 CEST
As discussed, UMCP does not support downloads efficiently.
Comment 13 Sönke Schwardt-Krummrich univentionstaff 2019-06-26 14:19:21 CEST
UCS@school 4.4 v2 has been released.

https://docs.software-univention.de/changelog-ucsschool-4.4v2-de.html

If this error occurs again, please clone this bug.