And AGAIN! The customer removed the cool solution by himself. The slapschema in the preupdate-check did not find, that there are still remnants of the schema. Multifile: /etc/ldap/slapd.conf Backing up /etc/ldap/slapd.conf in /var/backups/slapd-2.4.57+dfsg-3+deb11u1A~5.1.0.202501151832... done. Moving old database directories to /var/backups: - directory cn=internal... done. - directory cn=translog... done. - directory dc=edu,dc=schein,dc=de... done. Loading from /var/backups/slapd-2.4.57+dfsg-3+deb11u1A~5.1.0.202501151832: - directory cn=internal... done. - chowning database directory (openldap:openldap)... done - directory cn=translog... done. - chowning database directory (openldap:openldap)... done - directory dc=edu,dc=schein,dc=de... failed. Loading the database from the LDIF dump failed with the following error while running slapadd: <= str2entry: str2ad(UNIVENTIONCREATEREVOKECERTIFICATE): attribute type undefined slapadd: could not parse entry (line=1631266) Error, entries missing! entry 269: cn=402,cn=users,dc=edu,dc=schein,dc=de entry 270: cn=4021,cn=402,cn=users,dc=edu,dc=schein,dc=de entry 2476: cn=4022,cn=402,cn=users,dc=edu,dc=schein,dc=de entry 4996: cn=4023,cn=402,cn=users,dc=edu,dc=schein,dc=de entry 33152: cn=bind-users,cn=users,dc=edu,dc=schein,dc=de Stopping slapd (via systemctl): slapd.serviceESC[0;1;38;5;185mWarning: The unit file, source configuration file or drop-ins of slapd.service changed on disk. Run 'systemc tl daemon-reload' to reload units.ESC[0m Starting update process, this may take a while. Check /var/log/univention/updater.log for more information. Running postup.sh script:ldap_bind: Invalid credentials (49) done. Starting univention-upgrade. Current UCS version is 5.2-0 errata0 +++ This bug was initially created as a clone of Bug #58164 +++ Happend again! Starting /tmp/tmpk_izuu7m/https:__updates.software-univention.de_dists_ucs510_preup.sh (Do 3. Apr 21:10:13 CEST 2025): HINT: Please check the release notes carefully BEFORE updating to UCS 5.1-0: UCS 5.1-0 is an intermediate release and must not be used in production. After the update to UCS 5.1-0 make sure to immediately update to UCS 5.2-0, the updater will ask you to do so. All the necessary information is therefore in the release notes for UCS 5.2-0. English version: https://docs.software-univention.de/release-notes/5.2-0/en/ German version: https://docs.software-univention.de/release-notes/5.2-0/de/ Please also consider documents of following release updates and 3rd party components. Do you want to continue [Y/n]? Custom preupdate script /var/lib/local-preup.sh not found Checking auth_faillog ... OK Checking blocking_apps ... Starting univention-upgrade. Current UCS version is 5.0-10 errata1240 Unable to cache apps Unable to cache apps OK Checking disk_space ... OK Checking docker_storage_driver ... OK Checking failed_ldif ... OK Checking for_postgresql96 ... OK Checking hold_packages ... OK Checking kernel ... OK Checking keycloak_migration ... OK Checking ldap_connection ... OK Checking ldap_schema ... 67eedd25 UNKNOWN attributeDescription "CLIENTSECRET" inserted. 67eedd25 UNKNOWN attributeDescription "CLIENTID" inserted. 67eedd25 UNKNOWN attributeDescription "APPLICATIONTYPE" inserted. 67eedd25 UNKNOWN attributeDescription "REDIRECTURI" inserted. 67eedd25 UNKNOWN attributeDescription "TRUSTED" inserted. OK Checking legacy_objects ... OK Checking master_version ... OK Checking min_version ... OK Checking minimum_ucs_version_of_all_systems_in_domain ... OK Checking openldap_bdb ... OK Checking overwritten_umc_templates ... OK Checking package_status ... OK Checking role_package_removed ... OK Checking selinux_deactivated ... OK Checking slapd_on_member ... OK Checking ssh ... OK Checking system_date_too_old ... OK Checking term ... OK Checking user_country_mapping ... OK Checking valid_machine_credentials ... OK Checking verify_translog_schema ... OK > Several LDAP objects are no longer supported with UCS 5.2 and are removed automatically. > An LDIF file of removed objects is available: /var/univention-backup/update-to-5.1-0/removed_with_ucs5_2025-04-03-43.ldif > Removing objects with obsolete objectClasses >> (structuralObjectClass=univentionPortalEntry) Deleting object(s) with dn: cn=m23,cn=portal,cn=univention,dc=schein,dc=de cn=OWA,cn=portal,cn=univention,dc=schein,dc=de cn=OTRS,cn=portal,cn=univention,dc=schein,dc=de cn=Slack,cn=portal,cn=univention,dc=schein,dc=de [...] Neue Version der Konfigurationsdatei /etc/ldap/schema/pmi.ldif wird installiert ... Neue Version der Konfigurationsdatei /etc/ldap/schema/pmi.schema wird installiert ... File: /etc/init.d/slapd Multifile: /etc/ldap/slapd.conf Backing up /etc/ldap/slapd.conf in /var/backups/slapd-2.4.57+dfsg-3+deb11u1A~5.1.0.202501151832... done. Moving old database directories to /var/backups: - directory cn=internal... done. - directory cn=translog... done. - directory dc=becon,dc=de... done. Loading from /var/backups/slapd-2.4.57+dfsg-3+deb11u1A~5.1.0.202501151832: - directory cn=internal... done. - chowning database directory (openldap:openldap)... done - directory cn=translog... done. - chowning database directory (openldap:openldap)... done - directory dc=becon,dc=de... failed. Loading the database from the LDIF dump failed with the following error while running slapadd: <= str2entry: str2ad(UNIVENTIONCERTIFICATEDAYS): attribute type undefined slapadd: could not parse entry (line=15467) Error, entries missing! entry 195: ou=disabled,dc=becon,dc=de entry 196: ou=user,ou=disabled,dc=becon,dc=de Stopping slapd (via systemctl): slapd.serviceESC[0;1;38;5;185mWarning: The unit file, source configuration file or drop-ins of slapd.service changed on disk. Run 'systemctl daemon-reload' to reload units.ESC[0m . Removing obsolete conffile /etc/ldap/schema/ppolicy.schema ... Removing obsolete conffile /etc/ldap/schema/ppolicy.ldif ... ============================================================================================================================================================= +++ This bug was initially created as a clone of Bug #58072 +++ More or less similar issue, but with an other root cause, which we should also prevent. This is not as critical as the original bug, but nevertheless with impact. Custom preupdate script /var/lib/local-preup.sh not found Checking disk_space ... OK Checking failed_ldif ... OK Checking hold_packages ... OK Checking kernel ... OK Checking ldap_connection ... OK Checking ldap_schema ... 67d3453f UNKNOWN attributeDescription "XMPPENABLED" inserted. 67d3453f UNKNOWN attributeDescription "XMPPDOMAIN" inserted. 67d3453f UNKNOWN attributeDescription "XMPPDOMAINS" inserted. # (65) Object class violation: unrecognized objectClass 'univentionXMPPAccount' dn: uid=cc1,cn=users,dc=schein,dc=local # (65) Object class violation: unrecognized objectClass 'univentionXMPPAccount' dn: uid=support,cn=users,dc=schein,dc=local # (65) Object class violation: unrecognized objectClass 'univentionXMPPAccount' dn: uid=regi,cn=users,dc=schein,dc=local # (65) Object class violation: unrecognized objectClass 'univentionXMPPAccount' dn: uid=michi,cn=users,dc=schein,dc=local # (65) Object class violation: unrecognized objectClass 'univentionXMPPAccount' dn: uid=alex,cn=users,dc=schein,dc=local # (65) Object class violation: unrecognized objectClass 'univentionXMPPHost' dn: cn=ucs-dc1,cn=dc,cn=computers,dc=schein,dc=local OK Checking master_version ... OK Checking minimum_ucs_version_of_all_systems_in_domain ... OK Checking overwritten_umc_templates ... OK Checking package_status ... OK Checking role_package_removed ... OK Checking slapd_on_member ... OK Checking ssh ... OK Checking system_date_too_old ... OK Checking term ... OK Checking valid_machine_credentials ... OK Paketlisten werden gelesen... wenn das Update dann startet. Mich wundert nicht, dass das ldap anschließend defekt ist: Neue Version der Konfigurationsdatei /etc/ldap/schema/pmi.schema wird installiert ... File: /etc/init.d/slapd Multifile: /etc/ldap/slapd.conf Backing up /etc/ldap/slapd.conf in /var/backups/slapd-2.4.57+dfsg-3+deb11u1A~5.1.0.202501151832... done. Moving old database directories to /var/backups: - directory cn=translog... done. - directory dc=schein,dc=local... done. Loading from /var/backups/slapd-2.4.57+dfsg-3+deb11u1A~5.1.0.202501151832: - directory cn=translog... done. - chowning database directory (openldap:openldap)... done - directory dc=schein,dc=local... failed. Loading the database from the LDIF dump failed with the following error while running slapadd: /usr/lib/python3/dist-packages/requests/__init__.py:87: RequestsDependencyWarning: urllib3 (1.26.5) or chardet (5.1.0) doesn't match a supported version! warnings.warn("urllib3 ({}) or chardet ({}) doesn't match a supported " <= str2entry: str2ad(XMPPENABLED): attribute type undefined slapadd: could not parse entry (line=376) Stopping slapd (via systemctl): slapd.serviceESC[0;1;38;5;185mWarning: The unit file, source configuration file or drop-ins of slapd.service changed on disk. Run 'systemctl daemon-reload' to reload units.ESC[0m . Removing obsolete conffile /etc/ldap/schema/ppolicy.schema ... The primary dc has still this objectClass univention-ldapsearch -LLL cn=ucs-dc1 objectClass dn: cn=ucs-dc1,cn=dc,cn=computers,dc=schein,dc=local objectClass: krb5KDCEntry objectClass: univentionPolicyReference objectClass: person objectClass: univentionXMPPHost And the preup check on the primary has no issue about that: Custom preupdate script /var/lib/local-preup.sh not found Checking disk_space ... OK Checking failed_ldif ... OK Checking hold_packages ... OK Checking kernel ... OK Checking ldap_connection ... OK Checking ldap_schema ... OK Checking master_version ... OK Checking minimum_ucs_version_of_all_systems_in_domain ... OK Checking overwritten_umc_templates ... OK Checking package_status ... OK Checking role_package_removed ... OK Checking slapd_on_member ... OK Checking ssh ... OK Checking system_date_too_old ... OK Checking term ... OK Checking valid_machine_credentials ... OK Paketlisten werden gelesen... ------------------------------------------------------------------------------------------------------------- +++ This bug was initially created as a clone of Bug #58045 +++ When upgrading from UCS 5.1 to 5.2, a slapadd from backup.ldif fails with the following traceback. slapd (2.5.13+dfsg-5A~5.2.0.202501141029) wird eingerichtet ... Neue Version der Konfigurationsdatei /etc/ldap/schema/README wird installiert ... Neue Version der Konfigurationsdatei /etc/ldap/schema/collective.ldif wird installiert ... Neue Version der Konfigurationsdatei /etc/ldap/schema/corba.ldif wird installiert ... Neue Version der Konfigurationsdatei /etc/ldap/schema/core.ldif wird installiert ... Neue Version der Konfigurationsdatei /etc/ldap/schema/core.schema wird installiert ... Neue Version der Konfigurationsdatei /etc/ldap/schema/cosine.ldif wird installiert ... Neue Version der Konfigurationsdatei /etc/ldap/schema/cosine.schema wird installiert ... Neue Version der Konfigurationsdatei /etc/ldap/schema/duaconf.ldif wird installiert ... Neue Version der Konfigurationsdatei /etc/ldap/schema/dyngroup.ldif wird installiert ... Neue Version der Konfigurationsdatei /etc/ldap/schema/dyngroup.schema wird installiert ... Neue Version der Konfigurationsdatei /etc/ldap/schema/inetorgperson.ldif wird installiert ... Neue Version der Konfigurationsdatei /etc/ldap/schema/java.ldif wird installiert ... Neue Version der Konfigurationsdatei /etc/ldap/schema/misc.ldif wird installiert ... Neue Version der Konfigurationsdatei /etc/ldap/schema/misc.schema wird installiert ... Neue Version der Konfigurationsdatei /etc/ldap/schema/nis.ldif wird installiert ... Neue Version der Konfigurationsdatei /etc/ldap/schema/nis.schema wird installiert ... Neue Version der Konfigurationsdatei /etc/ldap/schema/openldap.ldif wird installiert ... Neue Version der Konfigurationsdatei /etc/ldap/schema/openldap.schema wird installiert ... Neue Version der Konfigurationsdatei /etc/ldap/schema/pmi.ldif wird installiert ... Neue Version der Konfigurationsdatei /etc/ldap/schema/pmi.schema wird installiert ... Multifile: /etc/ldap/slapd.conf File: /etc/init.d/slapd Backing up /etc/ldap/slapd.conf in /var/backups/slapd-2.4.57+dfsg-3+deb11u1A~5.1.0.202501151832... done. Moving old database directories to /var/backups: - directory cn=internal... done. - directory cn=translog... done. - directory dc=dde001826,dc=com... done. Loading from /var/backups/slapd-2.4.57+dfsg-3+deb11u1A~5.1.0.202501151832: - directory cn=internal... done. - chowning database directory (openldap:openldap)... done - directory cn=translog... failed. Loading the database from the LDIF dump failed with the following error while running slapadd: /usr/lib/python3/dist-packages/requests/__init__.py:87: RequestsDependencyWarning: urllib3 (1.26.5) or chardet (5.1.0) doesn't match a supported version! warnings.warn("urllib3 ({}) or chardet ({}) doesn't match a supported " <= str2entry NULL (smr_normalize reqDN 21) slapadd: could not parse entry (line=14068) Stopping slapd (via systemctl): slapd.serviceESC[0;1;38;5;185mWarning: The unit file, source configuration file or drop-ins of slapd.service changed on disk. Run 'systemctl daemon-reload' to reload units.ESC[0m . Removing obsolete conffile /etc/ldap/schema/ppolicy.schema ... Removing obsolete conffile /etc/ldap/schema/ppolicy.ldif ... Multifile: /etc/ldap/slapd.conf File: /etc/init.d/slapd As a result, the LDAP cannot be accessed during the upgrade and the respective join scripts cannot be executed due to invalid credentials. The system is thus completely destroyed and the upgrade cannot be continued. I would therefore classify the bug as critical.
So the check seems to find the attributes but that is okay and is moving on! Starting univention-upgrade. Current UCS version is 5.1-0 errata0 Checking for local repository: none Checking for package updates: none Checking for app updates: none Checking for release updates: found: UCS 5.2-0 Starting update to UCS version 5.2-0 at Mon May 5 21:06:48 2025... Starting update to UCS version 5.2-0 05.05.25 21:06:49.408 DEBUG_INIT **** Starting univention-updater 5.1-0 with parameter=['/usr/share/univention-updater/univention-updater', 'net', '--updateto', '5.2-0', '--silent'] --->DBG:update_available(mode=net) Checking network repository Update to = 5.2-0 **** Downloading scripts at Mon May 5 21:06:50 2025 **** Starting actual update at Mon May 5 21:06:51 2025 Starting /tmp/tmpnh1wnonh/https:__updates.software-univention.de_dists_ucs520_preup.sh (Mo 5. Mai 21:06:51 CEST 2025): HINT: Please check the release notes carefully BEFORE updating to UCS 5.2-0: English version: https://docs.software-univention.de/release-notes/5.2-0/en/ German version: https://docs.software-univention.de/release-notes/5.2-0/de/ Please also consider documents of following release updates and 3rd party components. Do you want to continue [Y/n]? Custom preupdate script /var/lib/local-preup.sh not found Checking auth_faillog ... OK Checking blocking_apps ... Unable to cache apps Unable to cache apps OK Checking disk_space ... OK Checking docker_storage_driver ... OK Checking failed_ldif ... OK Checking for_postgresql96 ... OK Checking hold_packages ... OK Checking kernel ... OK Checking keycloak_migration ... OK Checking ldap_connection ... OK Checking ldap_schema ... 68190c55 The first database does not allow slapschema; using the first available one (2) 68190c55 UNKNOWN attributeDescription "UNIVENTIONCREATEREVOKECERTIFICATE" inserted. 68190c55 UNKNOWN attributeDescription "UNIVENTIONCERTIFICATEDAYS" inserted. 68190c55 UNKNOWN attributeDescription "UNIVENTIONRENEWCERTIFICATE" inserted. # (65) Object class violation: unrecognized objectClass 'univentionManageCertificates' dn: cn=pc5149,cn=computers,ou=vh1,dc=edu,dc=schein,dc=de # (65) Object class violation: unrecognized objectClass 'univentionManageCertificates' dn: cn=servicedesk,cn=computers,dc=edu,dc=schein,dc=de OK Checking legacy_objects ... OK Checking master_version ... OK Checking min_version ... OK
Fun fact, that the messages is only in the logfile, on cmdline everything seems fine. Just like this: Checking ldap_schema ... 6819de94 The first database does not allow slapschema; using the first available one (2) ok is only visible in the logfile, but on cmdline it shows: Checking ldap_schema ... OK 
The warning "The first database does not allow slapschema; using the first available one (2)" seems harmless and can be reproduced with ``` ucr set ldap/monitor=yes /usr/sbin/slapschema -f /etc/ldap/slapd.conf -c ``` But adding a `-v` shows that it actually checks DB number 2 (`-n 2`)
That objectClass `univentionManageCertificates` comes from this package: https://git.knut.univention.de/univention/prof-services/cool-solutions/-/blob/ucs-5.0/master/univention-usercert/schema/univention-manage-certificates.schema?ref_type=heads#L29 The updater.log shows that that package was still installed during the update to `5.1-0` and the apt problem-resolver decided to uninstall it: ``` Starting update to UCS version 5.1-0 at Mon May 5 20:30:04 2025... [...] Investigating (0) univention-usercert:amd64 < 5.0.0-4A~5.0.0.202303221056 @ii mK Ib > Broken univention-usercert:amd64 Hängt ab von on python2.7:amd64 < 2.7.16-2+deb10u5 | 2.7.18-8+deb11u1 @ii umR > Considering python2.7:amd64 88 as a solution to univention-usercert:amd64 -1 Removing univention-usercert:amd64 rather than change python2.7:amd64 ``` We might need to improve the pre-up-checks for cases like these. Also, AFAIK that cool-solution is not so cool in the sense, that it doesn't use the schema registration mechanism that is now default in UCS since quite some time. Maybe that also would have helped.
Support also demands that slapschema shall be fixed in 5.1-0 like we did for Bug #58072 in `5.0-10` and `5.2-1`. Not sure how we would build & release that though, as that scope is closed in our buildsystem. But we can find workarounds. But: Then you are stuck in `5.1-0` and need to find your way out again alone. I still think a preup-check for known-problemantic packages is required too.
Support does not want a messed up system, with no ldap in the end, so better stuck in 5.1 with faulty ldap, then lost in 5.2 without ldap at all. → my 2 cents
Ok, for the cause of this particular bug: The predecessor Bug #58164 introduced a check update_check_cool_solutions in http://updates-test.software-univention.de/dists/ucs510/preup.sh But the check looks for the UCR variables that had a problem (leading space) but doesn't look if e.g. `univention-usercert` is still installed. The customer in turn unset UCR variables: ``` /var/log/univention/config-registry.replog.3.gz:2025-04-25 09:32:10: set repository/online/component/cool-solutions=disabled old:yes /var/log/univention/config-registry.replog.3.gz:2025-04-25 09:32:10: set repository/online/component/cool-solutions/server=http:// old:[Previously undefined] /var/log/univention/config-registry.replog.3.gz:2025-04-25 09:32:10: set repository/online/component/cool-solutions/unmaintained=enabled old:yes /var/log/univention/config-registry.replog.2.gz:2025-04-30 07:06:00: unset --ldap-policy 'repository/online/component/cool-solutions' old:yes /var/log/univention/config-registry.replog.2.gz:2025-04-30 07:06:00: unset --ldap-policy 'repository/online/component/cool-solutions/unmaintained' old:yes /var/log/univention/config-registry.replog.2.gz:2025-04-30 07:06:00: unset --ldap-policy 'repository/online/component/cool-solutions/version' old:current /var/log/univention/config-registry.replog.2.gz:2025-04-30 07:35:03: unset 'repository/online/component/cool-solutions' old:disabled /var/log/univention/config-registry.replog.2.gz:2025-04-30 07:35:03: unset 'repository/online/component/cool-solutions/server' old:http:// /var/log/univention/config-registry.replog.2.gz:2025-04-30 07:35:03: unset 'repository/online/component/cool-solutions/version' old:current ``` And thus the preup.sh didn't trigger: ``` Starting /tmp/tmpnwbvi1os/https:__updates.software-univention.de_dists_ucs510_preup.sh (Mo 5. Mai 20:30:07 CEST 2025): [...] Checking cool_solutions ... OK [...] ``` But later apt detects that `univention-usercert` depends on `python2.7` decides to rather uninstall it: ``` Broken univention-usercert:amd64 Hängt ab von on python2.7:amd64 < 2.7.16-2+deb10u5 | 2.7.18-8+deb11u1 @ii umR > Considering python2.7:amd64 88 as a solution to univention-usercert:amd64 -1 Removing univention-usercert:amd64 rather than change python2.7:amd64 ``` As a result, the schema is gone, but since slapschema in 5.1-0 doesn't block on that, the customer is left with a messed up system. I'd prefer to *actually* check for problematic packages like univention-usercert, as proposed as first option in the previous issue: https://git.knut.univention.de/univention/dev/ucs/-/issues/2787#note_434563 Additionally I found that update_check_cool_solutions is not present in pre-update-checks-5.2-0: http://updates.software-univention.de/download/univention-update-checks/pre-update-checks-5.2-0 We should probably add the checks there too, whatever method we choose.
Summary: I think this occurrence of this bug has several contributions * https://github.com/univention/cool-solutions had a broken line wrap in the UCR-variables (Bug #58164) * slapschema returned an exit code of 0 despite problems (5.2-1 Bug #58120, backported to 5.0-10 via Bug #58072, but not fixed in 5.1-0 itself) * The customer seems to have skipped the first step of https://help.univention.com/t/cool-solution-creation-and-management-of-user-and-windows-certificates/11782 so `univention-ldap-usercert` only got installed automatically as dependency of `univention-usercert`, and once the latter got uninstalled due to unavailability of python2.7, apt did consider the schema package as obsolete and autoremoved it during 5.1-0 postup. We now took three measures to address this: 0. Adjust the help article for cool-solutions to more explicitly explain the reasoning of the order of the steps 1. Adjust the preup.sh for 5.1-0 to mark univention-ldap-usercert as manually installed 2. Adjust the preup.sh for 5.2-0 to check the output of slapschema rather than the return code The latter should catch all (still) unknown cases in such a way, that we avoid jumping into the update to 5.2-0 in case a Debian package containing an LDAP schema got autoremoved during the 5.1-0 postup. cb105511369 | 5.1-0 | apt-mark manual univention-ldap-usercert 7732ed9134d | 5.2-0 | Workaround for broken slapschema (Bug #58072)
QA: Code review: OK Script has been updated: OK Manual Upgrade: OK