Bug 46533 - ucsschool43 HTTP-API import: celeryd start script doesn't work anymore
ucsschool43 HTTP-API import: celeryd start script doesn't work anymore
Status: CLOSED FIXED
Product: UCS@school
Classification: Unclassified
Component: HTTP-API (Kelvin)
UCS@school 4.3
Other Linux
: P5 normal (vote)
: UCS@school 4.3 v1
Assigned To: Daniel Tröder
Sönke Schwardt-Krummrich
:
Depends on:
Blocks: 47118
  Show dependency treegraph
 
Reported: 2018-03-07 16:07 CET by Daniel Tröder
Modified: 2018-06-04 10:28 CEST (History)
1 user (show)

See Also:
What kind of report is it?: Bug Report
What type of bug is this?: 5: Major Usability: Impairs usability in key scenarios
Who will be affected by this bug?: 1: Will affect a very few installed domains
How will those affected feel about the bug?: 3: A User would likely not purchase the product
User Pain: 0.086
Enterprise Customer affected?:
School Customer affected?:
ISV affected?:
Waiting Support:
Flags outvoted (downgraded) after PO Review:
Ticket number:
Bug group (optional):
Max CVSS v3 score:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Daniel Tröder univentionstaff 2018-03-07 16:07:02 CET
The celeryd workers are not started anymore in 4.3.
Comment 1 Daniel Tröder univentionstaff 2018-03-08 10:21:03 CET
The underlying problem is:

------------------------------------------------------------------------
# /usr/share/pyshared/ucsschool/http_api/manage.py check
SystemCheckError: System check identified some issues:

ERRORS:
import_api.UserImportJob.log_file: (fields.E304) Reverse accessor for 'UserImportJob.log_file' clashes with reverse accessor for 'UserImportJob.password_file'.
	HINT: Add or change a related_name argument to the definition for 'UserImportJob.log_file' or 'UserImportJob.password_file'.
import_api.UserImportJob.log_file: (fields.E304) Reverse accessor for 'UserImportJob.log_file' clashes with reverse accessor for 'UserImportJob.summary_file'.
	HINT: Add or change a related_name argument to the definition for 'UserImportJob.log_file' or 'UserImportJob.summary_file'.
import_api.UserImportJob.log_file: (fields.E305) Reverse query name for 'UserImportJob.log_file' clashes with reverse query name for 'UserImportJob.password_file'.
	HINT: Add or change a related_name argument to the definition for 'UserImportJob.log_file' or 'UserImportJob.password_file'.
import_api.UserImportJob.log_file: (fields.E305) Reverse query name for 'UserImportJob.log_file' clashes with reverse query name for 'UserImportJob.summary_file'.
	HINT: Add or change a related_name argument to the definition for 'UserImportJob.log_file' or 'UserImportJob.summary_file'.
import_api.UserImportJob.password_file: (fields.E304) Reverse accessor for 'UserImportJob.password_file' clashes with reverse accessor for 'UserImportJob.log_file'.
	HINT: Add or change a related_name argument to the definition for 'UserImportJob.password_file' or 'UserImportJob.log_file'.
import_api.UserImportJob.password_file: (fields.E304) Reverse accessor for 'UserImportJob.password_file' clashes with reverse accessor for 'UserImportJob.summary_file'.
	HINT: Add or change a related_name argument to the definition for 'UserImportJob.password_file' or 'UserImportJob.summary_file'.
import_api.UserImportJob.password_file: (fields.E305) Reverse query name for 'UserImportJob.password_file' clashes with reverse query name for 'UserImportJob.log_file'.
	HINT: Add or change a related_name argument to the definition for 'UserImportJob.password_file' or 'UserImportJob.log_file'.
import_api.UserImportJob.password_file: (fields.E305) Reverse query name for 'UserImportJob.password_file' clashes with reverse query name for 'UserImportJob.summary_file'.
	HINT: Add or change a related_name argument to the definition for 'UserImportJob.password_file' or 'UserImportJob.summary_file'.
import_api.UserImportJob.summary_file: (fields.E304) Reverse accessor for 'UserImportJob.summary_file' clashes with reverse accessor for 'UserImportJob.log_file'.
	HINT: Add or change a related_name argument to the definition for 'UserImportJob.summary_file' or 'UserImportJob.log_file'.
import_api.UserImportJob.summary_file: (fields.E304) Reverse accessor for 'UserImportJob.summary_file' clashes with reverse accessor for 'UserImportJob.password_file'.
	HINT: Add or change a related_name argument to the definition for 'UserImportJob.summary_file' or 'UserImportJob.password_file'.
import_api.UserImportJob.summary_file: (fields.E305) Reverse query name for 'UserImportJob.summary_file' clashes with reverse query name for 'UserImportJob.log_file'.
	HINT: Add or change a related_name argument to the definition for 'UserImportJob.summary_file' or 'UserImportJob.log_file'.
import_api.UserImportJob.summary_file: (fields.E305) Reverse query name for 'UserImportJob.summary_file' clashes with reverse query name for 'UserImportJob.password_file'.
	HINT: Add or change a related_name argument to the definition for 'UserImportJob.summary_file' or 'UserImportJob.password_file'.
------------------------------------------------------------------------

Those errors may be introduced by the upgrade of Django from 1.8 to 1.10.
Comment 2 Daniel Tröder univentionstaff 2018-03-08 10:27:37 CET
(In reply to Daniel Tröder from comment #1)
> Those errors may be introduced by the upgrade of Django from 1.8 to 1.10.
Yes: downgrading to 1.8.18-1~bpo8+1A~4.2.0.201712111108 (from 4.2/ucsschool_20180112151618) made the above errors vanish and the celery workers are started by the init script again.

I will now verify if the HTTP-API import works in 4.3 with Django 1.8. If it does, I'll see if it also works with 1.10, if I fix the above errors (required change is simple: did it with other software upgrading to Django 1.10).
Comment 3 Daniel Tröder univentionstaff 2018-03-08 15:17:46 CET
The functionality of the HTTP-API import (server + client) was OK, when running with Django 1.8. Nonetheless even version 1.10 is not supported by the Django Project anymore. So I adapted the code to handle the above error.

For the errors to be fixed, only the "related_name" changes in models.UserImportJob were necessary. But having changed the name of the reverse accessor made more changes necessary. All together are in e8db8dc2.

With 1cd2e7ac the original init.d script in Stretch and ours are as similar as possible.

[4.3 edef5176] Bug #46533: increase startup verbosity
[4.3 1cd2e7ac] Bug #46533: adapt to Stretch init script
[4.3 e8db8dc2] Bug #46533: fix name clashes with reverse accessor (adapt to Django 1.10)

ucs-school-import (16.0.1-5A~4.3.0.201803081512)
Comment 4 Daniel Tröder univentionstaff 2018-03-09 11:08:23 CET
DB migrations are missing.
Comment 5 Daniel Tröder univentionstaff 2018-03-09 11:28:51 CET
[4.3 b7898738] Bug #46533: add DB migrations
[4.3 3feaf30f] Bug #46533: workaround displayName not set when ou==displayName
[4.3 a42b4119] Bug #46533: depend on current postgres
[4.3 9f0119c2] Bug #46533: manage.py now supports a -c argument to run Python code from cmdline

ucs-school-import (16.0.1-7) → uploaded to test appcenter
Comment 6 Sönke Schwardt-Krummrich univentionstaff 2018-03-11 23:06:29 CET
(In reply to Daniel Tröder from comment #5)
> [4.3 9f0119c2] Bug #46533: manage.py now supports a -c argument to run
> Python code from cmdline

I suggest to create a helper script next time, that can be called by the join script (or manually).

OK: code change
OK: functional change
??: tests
    OK: Installation test
    ??: Update test 4.2→4.3
Comment 7 Sönke Schwardt-Krummrich univentionstaff 2018-03-12 22:12:45 CET
(In reply to Sönke Schwardt-Krummrich from comment #6)
> (In reply to Daniel Tröder from comment #5)
> > [4.3 9f0119c2] Bug #46533: manage.py now supports a -c argument to run
> > Python code from cmdline
> 
> I suggest to create a helper script next time, that can be called by the
> join script (or manually).
> 
> OK: code change
> OK: functional change
> ??: tests
>     OK: Installation test
>     ??: Update test 4.2→4.3

OK: tests
     OK: Installation test  
     OK: Update test 4.2→4.3
Comment 8 Daniel Tröder univentionstaff 2018-03-13 08:54:01 CET
(In reply to Sönke Schwardt-Krummrich from comment #7)
> (In reply to Sönke Schwardt-Krummrich from comment #6)
> > (In reply to Daniel Tröder from comment #5)
> > > [4.3 9f0119c2] Bug #46533: manage.py now supports a -c argument to run
> > > Python code from cmdline
> > 
> > I suggest to create a helper script next time, that can be called by the
> > join script (or manually).
There already exists a bug for that: Bug #45163.
Comment 9 Sönke Schwardt-Krummrich univentionstaff 2018-03-14 13:11:57 CET
UCS@school 4.3 v1 has been released.

https://docs.software-univention.de/release-notes-ucsschool-4.3v1-de.html

If this error occurs again, please clone this bug.