Univention Bugzilla – Bug 53535
Implement provisioning API for the ID Broker
Last modified: 2022-02-08 13:07:27 CET
Als ID Broker-System benötige ich eine Möglichkeit, über den ID Connector vermittelte Daten im ID Broker System in einer Multi-Schulträger-Umgebung anzulegen. As the ID Broker system I need an API that allows a system with the ID Connector to provision user data into the ID Broker. Since data can come from multiple tenants (school authorities), the data has to be encoded with the school authority.
Implemented in oschwieg/53535 - route: users - route: classes - route: schools - restrictions via opa rules - unit tests -> because mocking was not trivial there are no extra tests for exceptions (500), since this is also tested in the integration tests. Having unit tests as an addition to our integration tests is valuable since they are run each time we commit in our pipeline. - integration tests
- Fixed integration tests - adapted jenkins job [4.4-8] 20bed447fa Bug #53535: create provisioning users in scenario - install them in the plugin dir - execute tests in [4.4] 73b6cb271 Bug #53535: add call for id-broker tests
QA: The QA remarks (00_QA) in the following branches - jbaach/qa_53535 - jbaach/53535_qa2 were all handled, and an additional test was added by Tobias in pair programming mode -> no extra QA. - Are all routes implemented? (users, school_classes, schools with POST, PATCH/ PUT) - ok - Are there unittests for each function? - ok - Test the routes manually. - ok - Does only the provisioning-SCHOOL_AUTHORITY user have access? -ok - Are there any tests which prove that? - integration tests, opa, - ok - Test manually. - ok - Are there integration tests (ok) which pass in the new Jenkins tests? (not yet) - Is there documentation on how to setup the scenario? (ok) OK, reopened -> can be merged.
merged to main with [main] 8657c3a Bug #53535: add documentation [main] ea3fb95 Bug #53535: Add school_classes, schools & user route and tests
The jenkins tests fail, this has to be fixed.
The problem was, that our jenkins job installs the plugin from a scope in which the package was not yet build with the new implementations and tests. http://omar.knut.univention.de/build2/ucs_4.4-0-id-broker/all/ This was now fixed with Package: id-broker-plugin Version: 0.0.2A~4.4.0.202109150856 Branch: ucs_4.4-0 Scope: id-broker The Jenkins job runs now: https://jenkins.knut.univention.de:8181/job/UCSschool-4.4/job/ID%20Broker/82/ If it passes we can VERIFY this
I commited a fix for the kelvin-client in Bug #53814 WhenBug #53814 is verified/ published, run the id-broker tests / check jenkins to verify this bug.
Creating a school may take longer than 10s. Currently the Kelvin client times out after 10s: https://jenkins.knut.univention.de:8181/job/UCSschool-5.0/job/ID%20Broker/ws/test/IDBroker-Provisioning/ucsschool-apis/http.log ============================================================================== 2022-02-08 02:51:21 INFO h11_impl.send:461 172.17.42.1:36554 - "POST /ucsschool/apis/auth/token HTTP/1.1" 200 2022-02-08 02:51:31 INFO h11_impl.send:461 172.17.42.1:36432 - "POST /ucsschool/apis/provisioning/v1/TEST/schools HTTP/1.1" 500 2022-02-08 02:51:31 ERROR h11_impl.run_asgi:399 Exception in ASGI application Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/uvicorn/protocols/http/h11_impl.py", line 396, in run_asgi result = await app(self.scope, self.receive, self.send) File "/usr/local/lib/python3.9/site-packages/uvicorn/middleware/proxy_headers.py", line 45, in __call__ return await self.app(scope, receive, send) File "/usr/local/lib/python3.9/site-packages/fastapi/applications.py", line 199, in __call__ await super().__call__(scope, receive, send) File "/usr/local/lib/python3.9/site-packages/starlette/applications.py", line 112, in __call__ await self.middleware_stack(scope, receive, send) File "/usr/local/lib/python3.9/site-packages/starlette/middleware/errors.py", line 181, in __call__ raise exc from None File "/usr/local/lib/python3.9/site-packages/starlette/middleware/errors.py", line 159, in __call__ await self.app(scope, receive, _send) File "/usr/local/lib/python3.9/site-packages/starlette/exceptions.py", line 82, in __call__ raise exc from None File "/usr/local/lib/python3.9/site-packages/starlette/exceptions.py", line 71, in __call__ await self.app(scope, receive, sender) File "/usr/local/lib/python3.9/site-packages/starlette/routing.py", line 580, in __call__ await route.handle(scope, receive, send) File "/usr/local/lib/python3.9/site-packages/starlette/routing.py", line 241, in handle await self.app(scope, receive, send) File "/usr/local/lib/python3.9/site-packages/starlette/routing.py", line 52, in app response = await func(request) File "/usr/local/lib/python3.9/site-packages/fastapi/routing.py", line 216, in app raw_response = await run_endpoint_function( File "/usr/local/lib/python3.9/site-packages/fastapi/routing.py", line 149, in run_endpoint_function return await dependant.call(**values) File "/usr/local/lib/python3.9/site-packages/provisioning_plugin/routes/v1/schools.py", line 62, in post await school.save() File "/usr/local/lib/python3.9/site-packages/ucsschool/kelvin/client/school.py", line 84, in save return await super().save() File "/usr/local/lib/python3.9/site-packages/ucsschool/kelvin/client/base.py", line 108, in save resp_json = await self.session.post( File "/usr/local/lib/python3.9/site-packages/ucsschool/kelvin/client/session.py", line 227, in post return await self.request(self.client.post, url, **kwargs) File "/usr/local/lib/python3.9/site-packages/ucsschool/kelvin/client/session.py", line 172, in request response: httpx.Response = await async_request_method(url, **kwargs) File "/usr/local/lib/python3.9/site-packages/httpx/_client.py", line 1820, in post return await self.request( File "/usr/local/lib/python3.9/site-packages/httpx/_client.py", line 1506, in request return await self.send(request, auth=auth, follow_redirects=follow_redirects) File "/usr/local/lib/python3.9/site-packages/httpx/_client.py", line 1593, in send response = await self._send_handling_auth( File "/usr/local/lib/python3.9/site-packages/httpx/_client.py", line 1621, in _send_handling_auth response = await self._send_handling_redirects( File "/usr/local/lib/python3.9/site-packages/httpx/_client.py", line 1658, in _send_handling_redirects response = await self._send_single_request(request) File "/usr/local/lib/python3.9/site-packages/httpx/_client.py", line 1695, in _send_single_request response = await transport.handle_async_request(request) File "/usr/local/lib/python3.9/site-packages/httpx/_transports/default.py", line 353, in handle_async_request resp = await self._pool.handle_async_request(req) File "/usr/local/lib/python3.9/contextlib.py", line 137, in __exit__ self.gen.throw(typ, value, traceback) File "/usr/local/lib/python3.9/site-packages/httpx/_transports/default.py", line 77, in map_httpcore_exceptions raise mapped_exc(message) from exc httpx.ReadTimeout ============================================================================== Increase the timeout to 20s for the Kelvin client, when creating school objects.
> Increase the timeout to 20s for the Kelvin client, when creating school objects. And then increase the HTTP timeout of the ID connector too...