Bug 47275 - Dry-run import via HTTP API always fails
Dry-run import via HTTP API always fails
Status: CLOSED WORKSFORME
Product: UCS@school
Classification: Unclassified
Component: HTTP-API (Kelvin)
UCS@school 4.3
Other Linux
: P5 major (vote)
: UCS@school 4.3 v4
Assigned To: Daniel Tröder
Sönke Schwardt-Krummrich
:
Depends on:
Blocks: 47277
  Show dependency treegraph
 
Reported: 2018-07-01 23:01 CEST by Sönke Schwardt-Krummrich
Modified: 2018-07-04 18:12 CEST (History)
0 users

See Also:
What kind of report is it?: Bug Report
What type of bug is this?: 5: Major Usability: Impairs usability in key scenarios
Who will be affected by this bug?: 3: Will affect average number of installed domains
How will those affected feel about the bug?: 4: A User would return the product
User Pain: 0.343
Enterprise Customer affected?:
School Customer affected?:
ISV affected?:
Waiting Support:
Flags outvoted (downgraded) after PO Review:
Ticket number:
Bug group (optional):
Max CVSS v3 score:


Attachments
Content of /var/lib/ucs-school-import/jobs/2018/1/ (15.14 KB, application/x-compressed-tar)
2018-07-01 23:01 CEST, Sönke Schwardt-Krummrich
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Sönke Schwardt-Krummrich univentionstaff 2018-07-01 23:01:04 CEST
Created attachment 9580 [details]
Content of /var/lib/ucs-school-import/jobs/2018/1/

Got this error on 2 different systems with latest packages:

2018-06-27 14:11:00 INFO  user_import.read_input:89  ------ Read 10 users from input data. ------
2018-06-27 14:11:00 INFO  user_import.detect_users_to_delete:224  ------ Detecting which users to delete... ------
2018-06-27 14:11:00 DEBUG user_import.detect_users_to_delete:243  Searching with filter=u'(&(&(objectClass=ucsschoolStudent)(!(objectClass=ucsschoolExam)))(ucsschoolSourceUID=gsmitte-student)(ucsschoolRecordUID=*))'
2018-06-27 14:11:00 DEBUG user_import.detect_users_to_delete:258  users_to_delete=[]
2018-06-27 14:11:00 INFO  user_import.delete_users:275  ------ Deleting 0 users... ------
2018-06-27 14:11:00 INFO  user_import.delete_users:311  ------ Deleted 0 users. ------
2018-06-27 14:11:00 INFO  user_import.create_and_modify_users:104  ------ Creating / modifying users... ------
2018-06-27 14:11:00 DEBUG user_import.create_and_modify_users:121  Creating / modifying user 1/10 ImportStudent(name=None, school=None, dn=None)...
2018-06-27 14:11:00 WARNING utils.stopped_notifier:360  Starting univention-directory-notifier
2018-06-27 14:11:00 INFO  utils._run:332  Starting univention-directory-notifier (via systemctl): univention-directory-notifier.service.

2018-06-27 14:11:00 INFO  utils.stopped_notifier:367  univention-directory-notifier started
2018-06-27 14:11:00 ERROR cmdline.main:138  Outer Exception catcher: AttributeError("'NoneType' object has no attribute 'replace'",)
Traceback (most recent call last):
  File "/usr/lib/pymodules/python2.7/ucsschool/importer/frontend/cmdline.py", line 118, in main
    self.do_import()
  File "/usr/lib/pymodules/python2.7/ucsschool/importer/frontend/cmdline.py", line 96, in do_import
    importer.mass_import()
  File "/usr/lib/pymodules/python2.7/ucsschool/importer/mass_import/mass_import.py", line 70, in mass_import
    self.import_users()
  File "/usr/lib/pymodules/python2.7/ucsschool/importer/mass_import/mass_import.py", line 100, in import_users
    user_import.create_and_modify_users(imported_users)  # 90% - 100%
  File "/usr/lib/pymodules/python2.7/ucsschool/importer/mass_import/user_import.py", line 122, in create_and_modify_users
    user = self.determine_add_modify_action(imported_user)
  File "/usr/lib/pymodules/python2.7/ucsschool/importer/mass_import/user_import.py", line 197, in determine_add_modify_action
    user = imported_user.get_by_import_id(self.connection, imported_user.source_uid, imported_user.record_uid)
  File "/usr/lib/pymodules/python2.7/ucsschool/importer/models/import_user.py", line 338, in get_by_import_id
    (source_uid, record_uid)
  File "/usr/lib/python2.7/dist-packages/ldap/filter.py", line 59, in filter_format
    return filter_template % (tuple(map(escape_filter_chars,assertion_values)))
  File "/usr/lib/python2.7/dist-packages/ldap/filter.py", line 43, in escape_filter_chars
    s = assertion_value.replace('\\', r'\5c')
AttributeError: 'NoneType' object has no attribute 'replace'
2018-06-27 14:11:00 INFO  tasks.run_import_job:89  -- Finished import framework main procedure. --


Testfile has been created with this command:
/usr/share/ucs-school-import/scripts/ucs-school-testuser-import --httpapi -c testuser_http_api.csv --students 10 --schools 1 --classes 3 gsmitte
Comment 1 Daniel Tröder univentionstaff 2018-07-02 10:12:43 CEST
The configuration is broken. Both "csv:mapping" and "scheme" are empty.
Thus the recourd_uid was "None" and that led to the AttributeError.
Comment 2 Sönke Schwardt-Krummrich univentionstaff 2018-07-02 12:16:03 CEST
/usr/share/ucs-school-import/scripts/ucs-school-testuser-import --httpapi -c testuser_http_api.csv --students 10 --schools 1 --classes 3 gsmitte

grep -v "Benutzertyp" /usr/share/ucs-school-import/configs/ucs-school-testuser-import.json > /var/lib/ucs-school-import/configs/user_import.json

Then it is possible to import the test user data via HTTP-API.

(The manual 
https://billy.knut.univention.de/~dtroeder/http-api-doc/test-data.html
seems to be outdated at this point. Daniel created a new bug for this issue).
Comment 3 Sönke Schwardt-Krummrich univentionstaff 2018-07-04 18:12:24 CEST
UCS@school 4.3 v4 has been released.

https://docs.software-univention.de/changelog-ucsschool-4.3v4-de.html

If this error occurs again, please clone this bug.