Bug 45673 - no statistics, passwords and summary written when import aborts
no statistics, passwords and summary written when import aborts
Product: UCS@school
Classification: Unclassified
Component: Import scripts
UCS@school 4.2
Other Linux
: P5 normal (vote)
: UCS@school 4.3 v6
Assigned To: Daniel Tröder
Ole Schwiegert
: 47399 (view as bug list)
Depends on:
Blocks: 45165
  Show dependency treegraph
Reported: 2017-11-09 09:19 CET by Daniel Tröder
Modified: 2018-11-16 11:48 CET (History)
6 users (show)

See Also:
What kind of report is it?: Bug Report
What type of bug is this?: 3: Simply Wrong: The implementation doesn't match the docu
Who will be affected by this bug?: 2: Will only affect a few installed domains
How will those affected feel about the bug?: 4: A User would return the product
User Pain: 0.137
Enterprise Customer affected?:
School Customer affected?: Yes
ISV affected?:
Waiting Support:
Flags outvoted (downgraded) after PO Review:
Ticket number:
Bug group (optional):
Max CVSS v3 score:


Note You need to log in before you can comment on or make changes to this bug.
Description Daniel Tröder univentionstaff 2017-11-09 09:19:32 CET
When a UcsSchoolImportFatalError is raised (which can happen when more than config["tolerate_errors"] occur) statistics, passwords of new users and the import summary are not written to disk.

This has not been a problem until now for customers, because the usual configurations exit on the first error and the other fatal errors occur mostly before creating the first user (configuration problems).

The problem is, that without at least the summary in the logfile it is difficult to spot such a situation.

The fix could be to move the exception catching code that is in cmdline.main() to mass_import.import_users().
Comment 1 Daniel Tröder univentionstaff 2018-07-25 17:14:00 CEST
*** Bug 47399 has been marked as a duplicate of this bug. ***
Comment 2 Daniel Tröder univentionstaff 2018-09-18 13:03:59 CEST
Fixed in branch dtroeder/45673_summary_when_abort:

b997e83b2: Bug #45673: allow writing of statistics, passwords CSV and summary CSV when aborting import

Known (UcsSchoolImportError) und unknown exceptions are caught before bypassing the result output code, and reraised after that did its work.
Comment 3 Ole Schwiegert univentionstaff 2018-09-25 12:24:45 CEST
I forced an Exception, which was correctly catched by the outer Exception catcher. The summary was logged into my logfile and the exception raised again. Seems to work for me.

After building I can do the final testing and verify

------ End of user import statistics ------
------ Writing user import summary to /tmp/mylog.txt... ------
Searching for hooks of type 'ResultPyHook' in: /usr/share/ucs-school-import/pyhooks...
Found hook classes: 
Loaded hooks: {}.
------ Importing users done. ------
Starting univention-directory-notifier
Starting univention-directory-notifier (via systemctl): univention-directory-notifier.service.

univention-directory-notifier started
Outer Exception catcher: TypeError('expected string or buffer',)
Traceback (most recent call last):
  File "/usr/lib/pymodules/python2.7/ucsschool/importer/frontend/cmdline.py", line 121, in main
  File "/usr/lib/pymodules/python2.7/ucsschool/importer/frontend/cmdline.py", line 98, in do_import
  File "/usr/lib/pymodules/python2.7/ucsschool/importer/mass_import/mass_import.py", line 78, in mass_import
  File "/usr/lib/pymodules/python2.7/ucsschool/importer/mass_import/mass_import.py", line 129, in import_users
    raise exc
TypeError: expected string or buffer
Comment 4 Daniel Tröder univentionstaff 2018-09-25 15:49:33 CEST
Addad a changelog entry, merged, built and created an advisory.

ucs-school-import (16.0.2-50)
Comment 5 Ole Schwiegert univentionstaff 2018-09-26 09:33:39 CEST
Changelog & Advisory: OK
Tested by setting tolerate_errors in config to 2 and putting three errors in the CSV. Exception was thrown and catched by outer Exception catcher. Summary was written as intended.
Comment 6 Sönke Schwardt-Krummrich univentionstaff 2018-11-16 11:48:21 CET
UCS@school 4.3 v6 has been released.


If this error occurs again, please clone this bug.