Univention Bugzilla – Bug 51655
Remove LDAP objects no longer supported by UCS-5
Last modified: 2021-09-07 09:06:48 CEST
For UCS-5 (and earlier) several UDM modules and ACLs for them were removed: fatclient thinclient managedclient policy/thinclient admin-settings … Their data might still be stored in LDAP. When their ACLs are removed, they might become visible to all users and might leak sensitive data. We should write a UMC diagnostic module, which - similar to "univention-object-type-migrate" looks for unhandled LDAP objects - looks for known deprecated objects / containers and provides an option to dump them to a backup file and then delete them. The script should also be callable from command line. The script should also be included into the preup.sh script of UCS-5.0-0. The script should also be included into <https://updates.software-univention.de/download/univention-update-checks/pre-update-checks-5.0> mechanism. +++ This bug was initially created as a clone of Bug #31048 +++
objectClass=univentionSamba4WinsHost Bug #51497
The following commits have been done so far. univention-updater (15.0.1-8) 135e11742726 | Bug #51655 up: Improve delete_legacy_objects 4eadbadd3f43 | Bug #51655: Implement delete_legacy_objects ad2d88025fc2 | Bug #51655 up: Unify preup.sh and pre-check.sh 219d304804c1 | Bug #51655: Bug #51973: add UCC object classes to be removed 911359059199 | Bug #51655: add more objects to be removed ca984b245487 | Bug #51655: add UCR variable to skip check a467f09a49ea | Bug #51497: Bug #51973: Bug #31048: Bug #51655: block upgrade if LDAP contains old objects
* if still missing, please add an entry to release notes
I think this needs improvement in either UCS 4 or in the preup script leading up to UCS 5. Currently the following objects are detected in every UCS domain, which means every update attempt will fail with the following message in updater.log, and manual steps are required to update to UCS 5: legacy_objects: The following objects are no longer supported with UCS-5: dn: cn=default-settings,cn=thinclient,cn=policies,dc=mydomain,dc=intranet dn: cn=OpenStack,cn=CloudType,cn=Virtual Machine Manager,dc=mydomain,dc=intranet They must be removed before the update can be done. See <https://help.univention.com/t/16227> for details.
(In reply to Erik Damrose from comment #5) > I think this needs improvement in either UCS 4 or in the preup script > leading up to UCS 5. Currently the following objects are detected in every > UCS domain, which means every update attempt will fail with the following > message in updater.log, and manual steps are required to update to UCS 5: ... > See <https://help.univention.com/t/16227> for details. Pleas do not delete objects automatically. Similar to Debians policy for conffiles below /etc/ they should only be deleted automatically if they are unmodified: "Your never should loose data touched manually by an user". Maybe add an UCRV to make this configurable: - warn only - remove unmodified - remove always At least make a backup somewhere so they can be restored easily. (with instructions on how to restore them, e.g. "sudo ldapadd -Y GSSAPI -H ldapi:/// -f /var/univention-backup/legacy500.ldif")
Two issuse with the update_check_legacy_objects check * runs on un-joined systems ( * does not check if univention-ldapsearch actually works Checking ldap_schema ... OK Checking legacy_objects ... ldap_start_tls: Can't contact LDAP server (-1) ldap_start_tls: Can't contact LDAP server (-1) ldap_start_tls: Can't contact LDAP server (-1) ld... ldap_start_tls: Can't contact LDAP server (-1) ldap_start_tls: Can't contact LDAP server (-1) ldap_start_tls: Can't contact LDAP server (-1) ldap_start_tls: Can't contact LDAP server (-1) ldap_start_tls: Can't contact LDAP server (-1) OK -> this should not be OK
(In reply to Felix Botner from comment #7) > Two issuse with the update_check_legacy_objects check > * runs on un-joined systems ( > * does not check if univention-ldapsearch actually works ... > -> this should not be OK [5.0-0] c02140eb0f Bug #51655 up: Handle errors in update_check_legacy_objects base/univention-updater/debian/changelog | 6 ++++++ base/univention-updater/script/check.sh | 13 +++++++++++-- 2 files changed, 17 insertions(+), 2 deletions(-) Package: univention-updater Version: 15.0.3-8A~5.0.0.202102161817 Branch: ucs_5.0-0
d3dc28b8eb preup.sh: delete obsolete LDAP objects during update to UCS 5.0 univention-updater 15.0.3-14A~5.0.0.202102221854
3901eb66 Delete obsolete LDAP only on primary dc univention-updater 15.0.3-16A~5.0.0.202102241851
delete_legacy_objects is currently not called in the preup, is that OK? delete_obsolete_objects is called but on all server roles! s4 member Checking term ... OK Checking usr_mountpoint ... OK Checking valid_machine_credentials ... OK preupC4IGSM.sh: Cannot get LDAP credentials from '/etc/ldap.secret' Error: Update aborted by pre-update script of release 5.0-0 exitcode of univention-updater: 1 ERROR: update failed. Please check /var/log
1d467369 postup.sh: wait for listener postrun Also fix check for server role, move to correct function. univention-updater 15.0.3-17A~5.0.0.202102251033
I adapted the help article to reflect the changes to the objectClasses that are detected as blocking the update. https://help.univention.com/t/16227
FAIL - delete_obsolete_objects made some changes, please have a look - 97ac2b94bf770021bae871ddb45b383a0ed71331 * check if updateLogDir is set, abort * timestamp for backupfile (in case update fails and is restarted) * removed >&3 2>&3, if we want to redirect the output we should change the preup delete_obsolete_objects >&3 2>&3 * -a for tee, otherwise every search overwrites the backup * die if ldapdelete fails But there is still a problem: Some of the supposed to be structural objectClass's are in fact auxiliary. legacy_ocs_structural: structuralObjectClass=univentionCorporateClient -> aux obsolete_ocs_structural: structuralObjectClass=univentionThinClient -> aux structuralObjectClass=univentionMobileClient -> aux These filter will never find something and my ucc object is not found # ucc1, four.five dn: cn=ucc1,dc=four,dc=five macAddress: 00:00:00:00:00:00 loginShell: /bin/bash displayName: ucc1 cn: ucc1 krb5PrincipalName: host/ucc1.four.five@FOUR.FIVE objectClass: krb5KDCEntry objectClass: top objectClass: univentionHost objectClass: univentionObject objectClass: krb5Principal objectClass: person objectClass: shadowAccount objectClass: univentionCorporateClient objectClass: sambaSamAccount objectClass: posixAccount univentionCorporateClientBootVariant: overlayfs univentionObjectType: computers/ucc uidNumber: 2011 krb5KDCFlags: 126 sambaAcctFlags: [W ] krb5MaxRenew: 604800 krb5KeyVersionNumber: 1 sn: ucc1 homeDirectory: /dev/null uid: ucc1$ sambaSID: S-1-5-21-2862311440-2092257086-639877991-5022 krb5MaxLife: 86400 univentionCorporateClientBootRepartitioning: FALSE gidNumber: 5007 sambaPrimaryGroupSID: S-1-5-21-2862311440-2092257086-639877991-11015 What do we do about that? update_check_legacy_objects OK - finds str OK - finds aux OK - aborts update OK - bash check.sh delete_legacy_objects
(In reply to Erik Damrose from comment #13) > I adapted the help article to reflect the changes to the objectClasses that > are detected as blocking the update. > > https://help.univention.com/t/16227 The list of object classes is not complete.
During the upgrade the following error message is printed: /tmp/tmp13KZeR/postupjCGYIX.sh: Zeile 151: /var/lib/univention-directory-listener/handlers/udm_extension: Datei oder Verzeichnis nicht gefunden /tmp/tmp13KZeR/postupjCGYIX.sh: Zeile 151: /var/lib/univention-directory-listener/handlers/udm_extension: Datei oder Verzeichnis nicht gefunden
git:ad2d88025fc24aded2181eae98db21b2adec21ba removed the function fail_if_role_package_will_be_removed. A new function update_check_role_package_removed exists. The old function is still called and shows an error message during upgrade: /tmp/tmp8Ri7un/preupS8z8KG.sh: Zeile 845: fail_if_role_package_will_be_removed: Kommando nicht gefunden.
(In reply to Florian Best from comment #15) > (In reply to Erik Damrose from comment #13) > > I adapted the help article to reflect the changes to the objectClasses that > > are detected as blocking the update. > > > > https://help.univention.com/t/16227 > > The list of object classes is not complete. What is missing? (In reply to Florian Best from comment #16) > During the upgrade the following error message is printed: > /tmp/tmp13KZeR/postupjCGYIX.sh: Zeile 151: > /var/lib/univention-directory-listener/handlers/udm_extension: Datei oder > Verzeichnis nicht gefunden > > /tmp/tmp13KZeR/postupjCGYIX.sh: Zeile 151: > /var/lib/univention-directory-listener/handlers/udm_extension: Datei oder > Verzeichnis nicht gefunden yes is see it, updater.log on a replica node Removing univention-postgresql-9.6 (11.0.1-3A~4.4.0.202004161257) ... Processing triggers for man-db (2.8.5-2) ... Processing triggers for libc-bin (2.28-10) ... Traceback (most recent call last): File "/usr/sbin/univention-pkgdb-scan", line 37, in <module> univention.pkgdb.main() File "/usr/lib/python2.7/dist-packages/univention/pkgdb.py", line 582, in main connection = open_database_connection(config_registry, pkgdbu=False) File "/usr/lib/python2.7/dist-packages/univention/pkgdb.py", line 564, in open_database_connection connection = pgdb.connect(database=connectstring) File "/usr/lib/python2.7/dist-packages/pgdb.py", line 1619, in connect cnx = _connect(dbname, dbhost, dbport, dbopt, dbuser, dbpasswd) pg.InternalError: FATAL: kein pg_hba.conf-Eintrag für Host »10.207.61.117«, Benutzer »slave075$«, Datenbank »pkgdb«, SSL an W: --force-yes is deprecated, use one of the options starting with --allow instead. Custom postupdate script /var/lib/local-postup.sh not found Object modified: cn=slave075,cn=dc,cn=computers,dc=autotest075,dc=local listener shutdown done /tmp/tmpyXL_H7/postupawHUnf.sh: line 151: /var/lib/univention-directory-listener/handlers/udm_extension: No such file or directory /tmp/tmpyXL_H7/postupawHUnf.sh: line 151: /var/lib/univention-directory-listener/handlers/udm_extension: No such file or directory /tmp/tmpyXL_H7/postupawHUnf.sh: line 151: /var/lib/univention-directory-listener/handlers/udm_extension: No such file or directory /tmp/tmpyXL_H7/postupawHUnf.sh: line 151: /var/lib/univention-directory-listener/handlers/udm_extension: No such file or directory /tmp/tmpyXL_H7/postupawHUnf.sh: line 151: /var/lib/univention-directory-listener/handlers/udm_extension: No such file or directory /tmp/tmpyXL_H7/postupawHUnf.sh: line 151: /var/lib/univention-directory-listener/handlers/udm_extension: No such file or directory /tmp/tmpyXL_H7/postupawHUnf.sh: line 151: /var/lib/univention-directory-listener/handlers/udm_extension: No such file or directory /tmp/tmpyXL_H7/postupawHUnf.sh: line 151: /var/lib/univention-directory-listener/handlers/udm_extension: No such file or directory /tmp/tmpyXL_H7/postupawHUnf.sh: line 151: /var/lib/univention-directory-listener/handlers/udm_extension: No such file or directory /tmp/tmpyXL_H7/postupawHUnf.sh: line 151: /var/lib/univention-directory-listener/handlers/udm_extension: No such file or directory /tmp/tmpyXL_H7/postupawHUnf.sh: line 151: /var/lib/univention-directory-listener/handlers/udm_extension: No such file or directory repository/online/component/4.4-7-errata-test/description: Preview errata updates for UCS 4.4-7 not sure how this can happen though (In reply to Florian Best from comment #17) > git:ad2d88025fc24aded2181eae98db21b2adec21ba removed the function > fail_if_role_package_will_be_removed. > > A new function update_check_role_package_removed exists. > > The old function is still called and shows an error message during upgrade: > /tmp/tmp8Ri7un/preupS8z8KG.sh: Zeile 845: > fail_if_role_package_will_be_removed: Kommando nicht gefunden. yep, this needs to be fixed
(In reply to Felix Botner from comment #18) > (In reply to Florian Best from comment #15) > > (In reply to Erik Damrose from comment #13) > > > I adapted the help article to reflect the changes to the objectClasses that > > > are detected as blocking the update. > > > > > > https://help.univention.com/t/16227 > > > > The list of object classes is not complete. > > What is missing? The obsolete_ocs_structural from base/univention-updater/script/check.sh.
see also #51531, seems that this "wait for udm_extension ldap_extension" does not work on members
(In reply to Florian Best from comment #16) > During the upgrade the following error message is printed: > /tmp/tmp13KZeR/postupjCGYIX.sh: Zeile 151: > /var/lib/univention-directory-listener/handlers/udm_extension: Datei oder > Verzeichnis nicht gefunden > > /tmp/tmp13KZeR/postupjCGYIX.sh: Zeile 151: > /var/lib/univention-directory-listener/handlers/udm_extension: Datei oder > Verzeichnis nicht gefunden I fixed that in git:b58844130fda. The reason was that the listener removed the file during re-initialization.
cn=default-settings,cn=thinclient,cn=policies,$ldap_base is removed but still referenced in: dn: $ldap_base univentionPolicyReference: cn=default-settings,cn=thinclient,cn=policies,$ldap_base and dn: cn=default containers,cn=univention,$ldap_base univentionPolicyObject: cn=default-settings,cn=thinclient,cn=policies,$ldap_base
(In reply to Felix Botner from comment #18) > (In reply to Florian Best from comment #17) > > git:ad2d88025fc24aded2181eae98db21b2adec21ba removed the function > > fail_if_role_package_will_be_removed. > > > > A new function update_check_role_package_removed exists. > > > > The old function is still called and shows an error message during upgrade: > > /tmp/tmp8Ri7un/preupS8z8KG.sh: Zeile 845: > > fail_if_role_package_will_be_removed: Kommando nicht gefunden. > > yep, this needs to be fixed Fixed: a08abac125 fixup! Bug #51655 up: Unify preup.sh and pre-check.sh
9f8725a932 Bug #51655: Also remove policy references for automatically deleted objects 548ed20355 Bug #51655: preup.sh: Fixes and improvements for delete_obsolete_objects univention-updater 15.0.3-58A~5.0.0.202105111659
(In reply to Erik Damrose from comment #24) > 548ed20355 Bug #51655: preup.sh: Fixes and improvements for delete_obsolete_objects Why? You broke it, especially the backup file if now an invalid LDIF file!
(In reply to Philipp Hahn from comment #25) > You broke it, especially the backup file if now an invalid LDIF file! The comment is not really helpful. Please clarify what has been broken, your wording makes it sound like not only the backup logfile is affected? Regarding the logfile: The file could not be imported directly since the beginning as all attributes are logged, including operational attributes. I suggest we simply rename its ending to avoid disappointed expectations. 2d1fd596 + 4740bd03 preup.sh: Rename logfile created by delete_obsolete_objects univention-updater 15.0.3-60A~5.0.0.202105120936
git 1a7fbb3b Explicitely use structuralObjectClass Backupfile has ending .ldif Insert Note about removed policy references as comment in LDIF univention-updater 15.0.3-61A~5.0.0.202105121204
OK: removal of legacy objects (structural object class, object class) OK: backup ldif file REOPEN: missing changelog entry
[5.0-0 5975203a02] Bug #51655: changelog
OK: changelog entry
UCS 5.0 has been released: https://docs.software-univention.de/release-notes-5.0-0-en.html https://docs.software-univention.de/release-notes-5.0-0-de.html If this error occurs again, please use "Clone This Bug".