Description Ingo Jürgensmann 2022-08-22 16:33:55 CEST

Hi!

After setting up ID Broker in a customer environment, I found some pitfalls, inaccuracies and other potential improvements in the ID Broker documentation on https://docs.software-univention.de/idbroker-school-authority-manual/index.html

I'll list them all here. 

Installation
============

* Documentation states:
 
https://[FQDN]/ucsschool-id-connector/api/v1/docs. Replace FQDN with the fully qualified domain name of your UCS system that has the UCS@school ID Connector app installed.

Problem: documentation is not clear about whether or not the API needs to be reachable from outside. There is a note about that below on that page that the API shouldn't be reachable from the Internet. But in the default setup the app will generate Portal entries for IP address and $(hostname -f) FQDN. Those are sometimes not reachable from outside, so it can't be verified if the API is running or not, e.g. when hostname -f reports something like "ucs02.domain.intranet". 

In a later config step during Configuration this can lead to inaccessible or failing call of curl like: 

root@ucs02:~# token=$(curl -X POST \
> https://$(hostname -f)/ucsschool-id-connector/api/token \
> -H "Content-Type:application/x-www-form-urlencoded" \
> --data-urlencode "username=Administrator" \
> --data-urlencode "password=<topsecret>" | \
>  python -c 'import json,sys;print json.load(sys.stdin)["access_token"]')
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
curl: (60) SSL certificate problem: unable to get local issuer certificate
More details here: https://curl.haxx.se/docs/sslcerts.html

curl failed to verify the legitimacy of the server and therefore could not
establish a secure connection to it. To learn more about this situation and
how to fix it, please visit the web page mentioned above.
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/lib/python2.7/json/__init__.py", line 291, in load
    **kw)
  File "/usr/lib/python2.7/json/__init__.py", line 339, in loads
    return _default_decoder.decode(s)
  File "/usr/lib/python2.7/json/decoder.py", line 364, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/lib/python2.7/json/decoder.py", line 382, in raw_decode
    raise ValueError("No JSON object could be decoded")
ValueError: No JSON object could be decoded

Maybe the documentation should be more clear about whether or not the URL of the API should be accessible from the Internet or not (API should, but v1/docs shouldn't) and instruct how to set the correct URL, for example by using something like $(ucr get ucs/server/sso/fqdn) instead or other applicable URLs on that host. 

* Apache virtual hosts
Additional setup steps might be necessary when public FQDN differs from "hostname -f": ID-Broker or ucsschool-id-connector adds a ProxyPass statement in 000-default.conf and ssl-default.conf - but when an additional virtual host is configured, it might happen that this ProxyPass never matches. For example, on a host with internal domain and additional virtual host for SSO the ProxyPass statements in the *-default.confs will never match, because the virtual host will match first. In that case the SSO virtual host config might be expanded by the correct ProxyPass statement for the SSO virtual host. See internal OTRS [Ticket#2022081521000467] for more details on this. 

* Staging Environment
Surely not applicable to all customers, but when asking LINET for credentials, I was asked whether I want to have credentials for the staging or the production environment. Pretty sure a normal school authority should only be given credentials in the production environment, this might be clearified with LINET maybe to not give out the option of staging environment. 
If there is a valid reason for using staging environment it should be documented how to switch from staging to production environment. Maybe only in Service Provider docs. However I came across this issue and mention this for completeness and an error I got was this, when I wanted to configure the token:
 
{"detail":[{"loc":["name"],"msg":"object with \"name\"=\"schulenbremerhaven\" already exists","type":"value_error.object_exists"}]}


* Migration from existing deployments of Service Providers (SP) to ID-Broker
Although the SPs are few and also very few ID-Broker installations are around right now, there might be the case that a school authority/customer has an older service connector to a provider (e.g. Bettermarks Connector) already running in the past and wants to migrate to ID-Broker. The documentation should mention that there are maybe additional tasks need to be done on SP side to use ID-Broker instead of the old connector - otherwise customer will be confronted with a new setup in SP when the Primary Keys are changing, eg. in Bettermarks. 

* Passwords in Code Snippets
The documentation is using code snippets where ADMINPW needs to be entered. This might result in a security risk when the admin password can be found in shell history. 

* Files in Trash - documentation states: 
After handling a file in the trash directory, it should be deleted.
"Handling a file" is quite unspecific. Although it is not straightforward what to do with files in /trash, because it depends on the specific error for that file/user, the documentation could be more clear about this and explain some generic tasks that can be done, e.g. iterating on the files, syncing single user per file and remove that specific json file afterwards as a very basic step. Or to open a support ticket when the problem persists.

* After a sync 
After a sync there are many files in "/var/lib/univention-appcenter/apps/ucsschool-id-connector/data/out_queues/$NAMESPACEID" - the documentation could state here that those files will be moved out of that directory at a later time, because how the listener and the sync works. When does the sync be done? By cron, triggered by a change through listener or something? It can take some time until the files are moved away. Without some hint to the customer, the customer may be uncertain if everything is working as designed or if there is an error. 

* Clarification about dry-runs
Another customer reported back: 
"One of my few problems was topic 4.3 in the documentation where it is not well explained that the first command is a dry-run and the second command is the actual group sync"
So, maybe this could be clearified a little as well? I assume that it's the command with "-d" at the end

* Layout of the code snippets
Sometimes the code snippets are quite long and this can cause copy & paste errors, for example when the $NAMESPACEID is out of visibility. 

Maybe offering an additional install script might be better? The corner cases like FQDN external/internal could be handled by prompting the user for input as well as asking for the passwords. 
I understand that ID-Broker should become an app later, but for the time being this might be a good addition to the install & config documentation. 


If you have question about the above issues, please don't hesitate to contact me directly. 

Best regards, 
Ingo