Univention Bugzilla – Bug 50532
Clients which can't handle many redirects and Retry-After wait time crash when moving multiple objects
Last modified: 2020-08-03 17:37:47 CEST
When moving an object (PUT with changed 'position'), the UDM REST API returns a 'Location' header like this: https://HOST/univention/udm/progress/9a07b1a3-5f43-426a-b499-f121380092f2 When the move is quick (like moving a singe users/user object), when following that link the client will be redirected to the object at the new location - good. But when multiple objects have to be moved, the link leads to another redirect resulting in the client to fail with TooManyRedirects. From /var/log/apache2/access.log: 172.17.0.1 - - [22/Nov/2019:05:11:05 +0100] "PUT /univention/udm//container/ou/ou=bar,dc=ucs-test,dc=intranet HTTP/1.1" 201 947 "-" "OpenAPI-Generator/1.0.0/python" 172.17.0.1 - - [22/Nov/2019:05:11:05 +0100] "GET /univention/udm/progress/9a07b1a3-5f43-426a-b499-f121380092f2 HTTP/1.1" 301 993 "-" "Python/3.7 aiohttp/3.6.2" 172.17.0.1 - - [22/Nov/2019:05:11:05 +0100] "GET /univention/udm/progress/9a07b1a3-5f43-426a-b499-f121380092f2 HTTP/1.1" 301 1089 "-" "Python/3.7 aiohttp/3.6.2" 172.17.0.1 - - [22/Nov/2019:05:11:05 +0100] "GET /univention/udm/progress/9a07b1a3-5f43-426a-b499-f121380092f2 HTTP/1.1" 301 1185 "-" "Python/3.7 aiohttp/3.6.2" 172.17.0.1 - - [22/Nov/2019:05:11:05 +0100] "GET /univention/udm/progress/9a07b1a3-5f43-426a-b499-f121380092f2 HTTP/1.1" 301 1281 "-" "Python/3.7 aiohttp/3.6.2" 172.17.0.1 - - [22/Nov/2019:05:11:05 +0100] "GET /univention/udm/progress/9a07b1a3-5f43-426a-b499-f121380092f2 HTTP/1.1" 301 1377 "-" "Python/3.7 aiohttp/3.6.2" 172.17.0.1 - - [22/Nov/2019:05:11:05 +0100] "GET /univention/udm/progress/9a07b1a3-5f43-426a-b499-f121380092f2 HTTP/1.1" 301 1473 "-" "Python/3.7 aiohttp/3.6.2" 172.17.0.1 - - [22/Nov/2019:05:11:05 +0100] "GET /univention/udm/progress/9a07b1a3-5f43-426a-b499-f121380092f2 HTTP/1.1" 301 1569 "-" "Python/3.7 aiohttp/3.6.2" 172.17.0.1 - - [22/Nov/2019:05:11:05 +0100] "GET /univention/udm/progress/9a07b1a3-5f43-426a-b499-f121380092f2 HTTP/1.1" 301 1665 "-" "Python/3.7 aiohttp/3.6.2" 172.17.0.1 - - [22/Nov/2019:05:11:05 +0100] "GET /univention/udm/progress/9a07b1a3-5f43-426a-b499-f121380092f2 HTTP/1.1" 301 1762 "-" "Python/3.7 aiohttp/3.6.2" 172.17.0.1 - - [22/Nov/2019:05:11:05 +0100] "GET /univention/udm/progress/9a07b1a3-5f43-426a-b499-f121380092f2 HTTP/1.1" 301 1858 "-" "Python/3.7 aiohttp/3.6.2" From /var/log/univention/directory-manager-rest.log: 22.11.19 05:11:05 INFO ( 467) : 201 PUT /udm/container/ou/ou=bar,dc=ucs-test,dc=intranet (0.0.0.0) 4.65ms 22.11.19 05:11:05 INFO ( 467) : 301 GET /udm/progress/9a07b1a3-5f43-426a-b499-f121380092f2 (0.0.0.0) 59.76ms 22.11.19 05:11:05 INFO ( 467) : 301 GET /udm/progress/9a07b1a3-5f43-426a-b499-f121380092f2 (0.0.0.0) 6.87ms 22.11.19 05:11:05 INFO ( 467) : 301 GET /udm/progress/9a07b1a3-5f43-426a-b499-f121380092f2 (0.0.0.0) 136.69ms 22.11.19 05:11:05 INFO ( 467) : 301 GET /udm/progress/9a07b1a3-5f43-426a-b499-f121380092f2 (0.0.0.0) 1.62ms 22.11.19 05:11:05 INFO ( 467) : 301 GET /udm/progress/9a07b1a3-5f43-426a-b499-f121380092f2 (0.0.0.0) 4.65ms 22.11.19 05:11:05 INFO ( 467) : 301 GET /udm/progress/9a07b1a3-5f43-426a-b499-f121380092f2 (0.0.0.0) 140.14ms 22.11.19 05:11:05 INFO ( 467) : 301 GET /udm/progress/9a07b1a3-5f43-426a-b499-f121380092f2 (0.0.0.0) 8.67ms 22.11.19 05:11:05 INFO ( 467) : 301 GET /udm/progress/9a07b1a3-5f43-426a-b499-f121380092f2 (0.0.0.0) 151.86ms 22.11.19 05:11:05 INFO ( 467) : 301 GET /udm/progress/9a07b1a3-5f43-426a-b499-f121380092f2 (0.0.0.0) 1.83ms 22.11.19 05:11:05 INFO ( 467) : 301 GET /udm/progress/9a07b1a3-5f43-426a-b499-f121380092f2 (0.0.0.0) 2.63ms I think this could be solved by returning 200 and a 'Location' header, instead of redirecting (301), until the move has completed. That would also allow (if possible) a progress report to be in the (200) response.
I see now, that it is actually a series of 301s until the move is complete and then a 303. Example: 201 PUT /udm/container/ou/ou=bar,dc=ucs-test,dc=intranet (0.0.0.0) 4.06ms 301 GET /udm/progress/a9f6073d-113d-467c-8f26-0e97960e6aa3 (0.0.0.0) 7.85ms 301 GET /udm/progress/a9f6073d-113d-467c-8f26-0e97960e6aa3 (0.0.0.0) 8.72ms 303 GET /udm/progress/a9f6073d-113d-467c-8f26-0e97960e6aa3 (0.0.0.0) 1.25ms 200 GET /udm/container/ou/ou=bar,ou=foo,dc=ucs-test,dc=intranet (0.0.0.0) 7.68ms Clients can handle this programatically, but it is rather difficult. The programmer has to hook into the redirect function of its HTTP client lib and insert idle and timeout code there. (Idle code to not overwhelm the UDM REST API with 1000s of requests per second (following the 303s) and timeout code to allow to abort if an operation takes to long.)
The response contains a Retry-After header which specifies the wait time between two requests, which the client must respect.
(In reply to Florian Best from comment #2) > The response contains a Retry-After header which specifies the wait time > between two requests, which the client must respect. All clients I know will still by default stop to follow redirections after x (usually 10) times. So if an operation takes more than 10 seconds the client will stop following the redirections. Clients can be configures to follow more than 10 redirections, but 1. that must be documented. 2. doesn't change the problem of the missing progress report. Clients will still want to timeout, because they cannot know if the server is still working or just stuck in a loop. Without a progress report they'll just have to timeout at a random point in time.
I don't understand 2. The server gives progress information in the response body.
I think we should add this to the documentation and review it in case we get more feedback from client implementations.