Bug 33582 - Checking package version in ucs_register* is too strict
Checking package version in ucs_register* is too strict
Status: CLOSED FIXED
Product: UCS
Classification: Unclassified
Component: univention-lib
UCS 3.2
Other Linux
: P5 normal (vote)
: UCS 3.2-0-errata
Assigned To: Arvid Requate
Dirk Wiesenthal
:
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2013-11-27 09:20 CET by Dirk Wiesenthal
Modified: 2014-01-29 11:16 CET (History)
3 users (show)

See Also:
What kind of report is it?: ---
What type of bug is this?: ---
Who will be affected by this bug?: ---
How will those affected feel about the bug?: ---
User Pain:
Enterprise Customer affected?:
School Customer affected?:
ISV affected?:
Waiting Support:
Flags outvoted (downgraded) after PO Review:
Ticket number:
Bug group (optional):
Max CVSS v3 score:
requate: Patch_Available+


Attachments
ucs_register.patch (1.89 KB, patch)
2013-11-28 11:21 CET, Arvid Requate
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Dirk Wiesenthal univentionstaff 2013-11-27 09:20:41 CET
Not sure whether to fix this generically, i.e. issue a warning, not an error, or to add an option --ignore-if-package-version-is-newer (and change every occurrence in any join script) or do not change the function, instead change "|| die" to "|| true" in any join script.

Situation:

Master installed, appcenter join script run, errata updates installed (including appcenter), appcenter join script run again.

Now: Slave installed, trying to run join script before errata updates are installed.

/var/log/univention/join.log:
RUNNING 35univention-management-console-module-appcenter.inst
[...]
Object exists: cn=ldapschema,cn=univention,dc=dirk32,dc=appcenter,dc=qa
ERROR: Registered package version 3.0.50-10.223.201311261610 is newer, refusing registration.
Comment 1 Arvid Requate univentionstaff 2013-11-27 13:35:31 CET
Updating to the recent package versions and running univention-run-join-scripts should fix the issue?

Relaxing the version restrictions as a workaround for this situation may have domain wide consequences?
Comment 2 Dirk Wiesenthal univentionstaff 2013-11-27 17:00:51 CET
(In reply to Arvid Requate from comment #1)
> Updating to the recent package versions and running
> univention-run-join-scripts should fix the issue?

Yes.

> Relaxing the version restrictions as a workaround for this situation may
> have domain wide consequences?

Hardly. If a newer version of the schema was already registered: Great!

Following situation:
Master upgrades to 3.3, new version of univention-important-package with ucs_registerSchema important/object.schema. Slaves stay at UCS 3.2 for a while.

Shortly after that an errata for UCS 3.2 is released: A critical UDM object was not created by univention-important-package. Update of the joinscript, and "udm important/object create" just after "ucs_registerSchema important/object.schema".

-> Will never work (as long as the Slave stays at UCS 3.2 at least).
Comment 3 Arvid Requate univentionstaff 2013-11-28 11:21:01 CET
Created attachment 5680 [details]
ucs_register.patch

The attached patch may relax the behaviour of the shell wrapper.
Comment 4 Stefan Gohmann univentionstaff 2013-12-17 06:58:21 CET
(In reply to Dirk Wiesenthal from comment #2)
> Following situation:
> Master upgrades to 3.3, new version of univention-important-package with
> ucs_registerSchema important/object.schema. Slaves stay at UCS 3.2 for a
> while.
> 
> Shortly after that an errata for UCS 3.2 is released: A critical UDM object
> was not created by univention-important-package. Update of the joinscript,
> and "udm important/object create" just after "ucs_registerSchema
> important/object.schema".
> 
> -> Will never work (as long as the Slave stays at UCS 3.2 at least).

So, what we need is the possibility to update an UDM module for UCS 3.2 even if a module for UCS 3.3 is already registered, right? So the version check should be UCS version (major / minor) specific.
Comment 5 Arvid Requate univentionstaff 2014-01-06 09:38:48 CET
This happens each time a joinscript tries to register a non-current version of an UMC or LDAP extension, i.e. also if the system doesn't have the latest errata updates installed.

The currently implemented concept of extension replication does not support having several parallel versions of an extension (identified by name), e.g. installed on different subsets of machines in the domain. So, to fix the symptom of failing joins ASAP, the simplest solution would be to let the joinscript continue and just issue the warning.
Comment 6 Arvid Requate univentionstaff 2014-01-08 18:28:46 CET
The python function ucs_registerLDAPExtension now exists with rc==4 in this case and issues a "WARNING" instead of an "ERROR" and the shell wrapper specifically masks this return code and returns 0 instead.

Advisory: 2014-01-08-univention-lib.yaml
Comment 7 Stefan Gohmann univentionstaff 2014-01-09 08:25:05 CET
From the jenkins build:

-----------------------------------------------------------------------------
Configure /usr/lib/univention-install/35univention-management-console-module-appcenter.inst
Object exists: cn=UMC,cn=univention,dc=autotest093,dc=local
Object exists: cn=UMC,cn=policies,dc=autotest093,dc=local
Object exists: cn=operations,cn=UMC,cn=univention,dc=autotest093,dc=local
Object exists: cn=default-umc-all,cn=UMC,cn=policies,dc=autotest093,dc=local
WARNING: cannot append cn=default-umc-all,cn=UMC,cn=policies,dc=autotest093,dc=local to univentionPolicyReference, value exists
No modification: cn=Domain Admins,cn=groups,dc=autotest093,dc=local
Object exists: cn=default-umc-users,cn=UMC,cn=policies,dc=autotest093,dc=local
WARNING: cannot append cn=default-umc-users,cn=UMC,cn=policies,dc=autotest093,dc=local to univentionPolicyReference, value exists
No modification: cn=Domain Users,cn=groups,dc=autotest093,dc=local
Object created: cn=appcenter-all,cn=operations,cn=UMC,cn=univention,dc=autotest093,dc=local
Object modified: cn=default-umc-all,cn=UMC,cn=policies,dc=autotest093,dc=local
Object created: cn=apps,cn=univention,dc=autotest093,dc=local
Object exists: cn=ldapschema,cn=univention,dc=autotest093,dc=local
Object exists: cn=ldapacl,cn=univention,dc=autotest093,dc=local
Object created: cn=udm_module,cn=univention,dc=autotest093,dc=local
Object created: cn=univention-app,cn=ldapschema,cn=univention,dc=autotest093,dc=local

Object created: cn=66univention-appcenter_app,cn=ldapacl,cn=univention,dc=autotest093,dc=local

Object created: cn=appcenter/app,cn=udm_module,cn=univention,dc=autotest093,dc=local

Waiting for activation of the extension object univention-app:.............OK
Waiting for activation of the extension object 66univention-appcenter_app: OK
Waiting for activation of the extension object appcenter/app: OK
Waiting for file /usr/share/pyshared/univention/admin/handlers/appcenter/app.py: OK
Terminating running univention-cli-server processes.
[: 67: 0: unexpected operator
-----------------------------------------------------------------------------

I think it is a bashism. Use = instead of == in univention-lib/shell/ldap.sh.
Comment 8 Arvid Requate univentionstaff 2014-01-09 11:56:52 CET
Fixed, advisory is updated.
Comment 9 Arvid Requate univentionstaff 2014-01-23 18:27:56 CET
ucs-3.2-0/base/univention-lib needs to be built again but dimma is currently down.
Comment 10 Arvid Requate univentionstaff 2014-01-24 12:02:36 CET
Advisory updated.
Comment 11 Dirk Wiesenthal univentionstaff 2014-01-27 12:03:31 CET
Currently, saving an already registered schema a second time gives an error. The problem is (presumably) these lines in udm

if not  apt.apt_pkg.version_compare(self['packageversion'], old_version) > -1:
  # raise

and shell-univention-lib:

if not apt.apt_pkg.version_compare(options.packageversion, registered_package_version) > -1:
  # exit

=> 0 (equal) does not exit but raises => joinscript waits for 180 seconds and then fails.

I would exit if rc != -1 ("%s is not older").
One could also raise only if rc == 1 but I do not know whether this has side effects.
Comment 12 Arvid Requate univentionstaff 2014-01-27 19:09:42 CET
As discussed, I don't follow the logic of the argument and I cannot reproduce it either until now. 

For Bug 33936 I have a test environment with:
################################################################
DN: cn=settings/mswmifilter,cn=udm_module,cn=univention,dc=ar320i2,dc=qa
ARG: None
  ucsversionstart: None
  active: TRUE
  packageversion: 8.0.33-26.451.201401241704
  data: ## base64 encoded stuff
  name: settings/mswmifilter
  umcregistration: None
  package: univention-s4-connector
  filename: settings/mswmifilter.py
  ucsversionend: None
################################################################

If I now call the following commands to mimic the joinscript I don't run into the timeout:
################################################################
root@slave47:~# ucr set \
  tests/ucs_registerLDAP/packagename=univention-s4-connector \
  tests/ucs_registerLDAP/packageversion=8.0.33-26.451.201401241704
Create tests/ucs_registerLDAP/packagename
Create tests/ucs_registerLDAP/packageversion
root@slave47:~# ucs_registerLDAPExtension "$@"  \
  --udm_module /usr/share/pyshared/univention/admin/handlers/settings/mswmifilter.py
Object exists: cn=udm_module,cn=univention,dc=ar320i2,dc=qa
INFO: No change of core data of object settings/mswmifilter.
No modification: cn=settings/mswmifilter,cn=udm_module,cn=univention,dc=ar320i2,dc=qa

Waiting for activation of the extension object settings/mswmifilter: OK
Waiting for file /usr/share/pyshared/univention/admin/handlers/settings/mswmifilter.py: OK
Terminating running univention-cli-server processes.
################################################################

When I change the content of the file, the module gets replaced ASAP:
################################################################
root@slave47:~# echo "## Foo" >>  /usr/share/pyshared/univention/admin/handlers/settings/mswmifilter.py
ucs_registerLDAPExtension \
  --binddn uid=Administrator,cn=users,dc=ar320i2,dc=qa \
  --bindpwd=univention \
  --udm_module /usr/share/pyshared/univention/admin/handlers/settings/mswmifilter.py
Object exists: cn=udm_module,cn=univention,dc=ar320i2,dc=qa
Object modified: cn=settings/mswmifilter,cn=udm_module,cn=univention,dc=ar320i2,dc=qa

Waiting for activation of the extension object settings/mswmifilter:.OK
Waiting for file /usr/share/pyshared/univention/admin/handlers/settings/mswmifilter.py: OK
Terminating running univention-cli-server processes.
################################################################
Comment 13 Dirk Wiesenthal univentionstaff 2014-01-28 13:13:32 CET
Yes, checked a second time: Backup server was somehow unable to connect to the master. This was the problem of the timeout.

Reinstalled: Everything works fine.

YAML OK
Comment 14 Moritz Muehlenhoff univentionstaff 2014-01-29 11:16:12 CET
http://errata.univention.de/ucs/3.2/29.html