Bug 50532 - Clients which can't handle many redirects and Retry-After wait time crash when moving multiple objects
Clients which can't handle many redirects and Retry-After wait time crash whe...
Status: NEW
Product: UCS
Classification: Unclassified
Component: UDM - REST API
UCS 4.4
Other Linux
: P5 normal (vote)
: ---
Assigned To: UMC maintainers
UMC maintainers
:
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2019-11-22 05:21 CET by Daniel Tröder
Modified: 2020-08-03 17:37 CEST (History)
3 users (show)

See Also:
What kind of report is it?: Development Internal
What type of bug is this?: ---
Who will be affected by this bug?: ---
How will those affected feel about the bug?: ---
User Pain:
Enterprise Customer affected?:
School Customer affected?:
ISV affected?:
Waiting Support:
Flags outvoted (downgraded) after PO Review:
Ticket number:
Bug group (optional): API change
Max CVSS v3 score:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Daniel Tröder univentionstaff 2019-11-22 05:21:04 CET
When moving an object (PUT with changed 'position'), the UDM REST API returns a 'Location' header like this: https://HOST/univention/udm/progress/9a07b1a3-5f43-426a-b499-f121380092f2

When the move is quick (like moving a singe users/user object), when following that link the client will be redirected to the object at the new location - good.

But when multiple objects have to be moved, the link leads to another redirect resulting in the client to fail with TooManyRedirects.

From /var/log/apache2/access.log:

172.17.0.1 - - [22/Nov/2019:05:11:05 +0100] "PUT /univention/udm//container/ou/ou=bar,dc=ucs-test,dc=intranet HTTP/1.1" 201 947 "-" "OpenAPI-Generator/1.0.0/python"
172.17.0.1 - - [22/Nov/2019:05:11:05 +0100] "GET /univention/udm/progress/9a07b1a3-5f43-426a-b499-f121380092f2 HTTP/1.1" 301 993 "-" "Python/3.7 aiohttp/3.6.2"
172.17.0.1 - - [22/Nov/2019:05:11:05 +0100] "GET /univention/udm/progress/9a07b1a3-5f43-426a-b499-f121380092f2 HTTP/1.1" 301 1089 "-" "Python/3.7 aiohttp/3.6.2"
172.17.0.1 - - [22/Nov/2019:05:11:05 +0100] "GET /univention/udm/progress/9a07b1a3-5f43-426a-b499-f121380092f2 HTTP/1.1" 301 1185 "-" "Python/3.7 aiohttp/3.6.2"
172.17.0.1 - - [22/Nov/2019:05:11:05 +0100] "GET /univention/udm/progress/9a07b1a3-5f43-426a-b499-f121380092f2 HTTP/1.1" 301 1281 "-" "Python/3.7 aiohttp/3.6.2"
172.17.0.1 - - [22/Nov/2019:05:11:05 +0100] "GET /univention/udm/progress/9a07b1a3-5f43-426a-b499-f121380092f2 HTTP/1.1" 301 1377 "-" "Python/3.7 aiohttp/3.6.2"
172.17.0.1 - - [22/Nov/2019:05:11:05 +0100] "GET /univention/udm/progress/9a07b1a3-5f43-426a-b499-f121380092f2 HTTP/1.1" 301 1473 "-" "Python/3.7 aiohttp/3.6.2"
172.17.0.1 - - [22/Nov/2019:05:11:05 +0100] "GET /univention/udm/progress/9a07b1a3-5f43-426a-b499-f121380092f2 HTTP/1.1" 301 1569 "-" "Python/3.7 aiohttp/3.6.2"
172.17.0.1 - - [22/Nov/2019:05:11:05 +0100] "GET /univention/udm/progress/9a07b1a3-5f43-426a-b499-f121380092f2 HTTP/1.1" 301 1665 "-" "Python/3.7 aiohttp/3.6.2"
172.17.0.1 - - [22/Nov/2019:05:11:05 +0100] "GET /univention/udm/progress/9a07b1a3-5f43-426a-b499-f121380092f2 HTTP/1.1" 301 1762 "-" "Python/3.7 aiohttp/3.6.2"
172.17.0.1 - - [22/Nov/2019:05:11:05 +0100] "GET /univention/udm/progress/9a07b1a3-5f43-426a-b499-f121380092f2 HTTP/1.1" 301 1858 "-" "Python/3.7 aiohttp/3.6.2"

From /var/log/univention/directory-manager-rest.log:

22.11.19 05:11:05        INFO      (      467) : 201 PUT /udm/container/ou/ou=bar,dc=ucs-test,dc=intranet (0.0.0.0) 4.65ms
22.11.19 05:11:05        INFO      (      467) : 301 GET /udm/progress/9a07b1a3-5f43-426a-b499-f121380092f2 (0.0.0.0) 59.76ms
22.11.19 05:11:05        INFO      (      467) : 301 GET /udm/progress/9a07b1a3-5f43-426a-b499-f121380092f2 (0.0.0.0) 6.87ms
22.11.19 05:11:05        INFO      (      467) : 301 GET /udm/progress/9a07b1a3-5f43-426a-b499-f121380092f2 (0.0.0.0) 136.69ms
22.11.19 05:11:05        INFO      (      467) : 301 GET /udm/progress/9a07b1a3-5f43-426a-b499-f121380092f2 (0.0.0.0) 1.62ms
22.11.19 05:11:05        INFO      (      467) : 301 GET /udm/progress/9a07b1a3-5f43-426a-b499-f121380092f2 (0.0.0.0) 4.65ms
22.11.19 05:11:05        INFO      (      467) : 301 GET /udm/progress/9a07b1a3-5f43-426a-b499-f121380092f2 (0.0.0.0) 140.14ms
22.11.19 05:11:05        INFO      (      467) : 301 GET /udm/progress/9a07b1a3-5f43-426a-b499-f121380092f2 (0.0.0.0) 8.67ms
22.11.19 05:11:05        INFO      (      467) : 301 GET /udm/progress/9a07b1a3-5f43-426a-b499-f121380092f2 (0.0.0.0) 151.86ms
22.11.19 05:11:05        INFO      (      467) : 301 GET /udm/progress/9a07b1a3-5f43-426a-b499-f121380092f2 (0.0.0.0) 1.83ms
22.11.19 05:11:05        INFO      (      467) : 301 GET /udm/progress/9a07b1a3-5f43-426a-b499-f121380092f2 (0.0.0.0) 2.63ms


I think this could be solved by returning 200 and a 'Location' header, instead of redirecting (301), until the move has completed.
That would also allow (if possible) a progress report to be in the (200) response.
Comment 1 Daniel Tröder univentionstaff 2019-11-22 08:35:21 CET
I see now, that it is actually a series of 301s until the move is complete and then a 303.

Example:

201 PUT /udm/container/ou/ou=bar,dc=ucs-test,dc=intranet (0.0.0.0) 4.06ms
301 GET /udm/progress/a9f6073d-113d-467c-8f26-0e97960e6aa3 (0.0.0.0) 7.85ms
301 GET /udm/progress/a9f6073d-113d-467c-8f26-0e97960e6aa3 (0.0.0.0) 8.72ms
303 GET /udm/progress/a9f6073d-113d-467c-8f26-0e97960e6aa3 (0.0.0.0) 1.25ms
200 GET /udm/container/ou/ou=bar,ou=foo,dc=ucs-test,dc=intranet (0.0.0.0) 7.68ms

Clients can handle this programatically, but it is rather difficult.
The programmer has to hook into the redirect function of its HTTP client lib and insert idle and timeout code there.
(Idle code to not overwhelm the UDM REST API with 1000s of requests per second (following the 303s) and timeout code to allow to abort if an operation takes to long.)
Comment 2 Florian Best univentionstaff 2019-11-22 08:54:32 CET
The response contains a Retry-After header which specifies the wait time between two requests, which the client must respect.
Comment 3 Daniel Tröder univentionstaff 2019-11-25 08:11:48 CET
(In reply to Florian Best from comment #2)
> The response contains a Retry-After header which specifies the wait time
> between two requests, which the client must respect.
All clients I know will still by default stop to follow redirections after x (usually 10) times.
So if an operation takes more than 10 seconds the client will stop following the redirections.
Clients can be configures to follow more than 10 redirections, but
1. that must be documented.
2. doesn't change the problem of the missing progress report. Clients will still want to timeout, because they cannot know if the server is still working or just stuck in a loop. Without a progress report they'll just have to timeout at a random point in time.
Comment 4 Florian Best univentionstaff 2019-11-25 09:57:15 CET
I don't understand 2.
The server gives progress information in the response body.
Comment 5 Ingo Steuwer univentionstaff 2019-11-26 15:02:33 CET
I think we should add this to the documentation and review it in case we get more feedback from client implementations.