Univention Bugzilla – Bug 51686
ox-context listener fails on existing context
Last modified: 2020-07-23 08:49:07 CEST
The ox-context listener module cannot handle finding an existing context, when it wants to create a new one: ----------------------------------------------------------------- ox-context: transaction 1594730805.4214169979 deferred: dn: cn=context10,cn=open-xchange,dc=ldap,dc=base "['/opt/open-xchange/sbin/createcontext', '--adminuser', 'oxadminmaster', '--contextid', '10', '--contextname', 'context10' [..] returned 1: context 10 could not be created: Server response: Context 10 already exists! ----------------------------------------------------------------- As the initial installation of the OX App Suite creates a context using ox-create-context directly, this bug happens on every new installation!
Added ticketnumber. A customer has reported this issue.
for new installations: what are the consequences beside an ugly log message?
(In reply to Ingo Steuwer from comment #2) > for new installations: what are the consequences beside an ugly log message? For new installations it is not possible to log in to OX. This means that the AppSuite cannot be used. In general, no new installation with OX can be realized with this.
Customer reports all new installations with OX 7.10.3 concerned. Listener Logfile in ticket comment 8
The error situation occurred because the listener module ox-context.py immediately overwrote the file /etc/ox-secrets/context$CONTEXTID.secret with a new password when a new OX context object appeared in LDAP and then tried to create the "new" OX context. For new installations the default context 10 is created by the installation scripts (and not by the listener module). This a) overwrote the credentials of Context 10 and b) the createcontext call did not work because the context already exists in OX. Because an exception is thrown in the listener module, the listener module or the corresponding cronjob tries again once per minute (→ again new PW → again failure in context creation). Thus, the error state could not be corrected manually on the easy way. PLEASE NOTE: This error situation also occurs when the listener module is resynced or the UCS system is rejoined! Several adjustments have now been made in the listener module: 1) The listener module first checks in create_context() whether the context already exists in the OX. If this is the case, the modify_context() function is called instead and create_context() is terminated. 2) if there is still an old .secret file, a backup of this file is created 3) if an error message "Context X already exists!" is reported when calling the "createcontext" CLI tool, this error is ignored and the listener module continues (this is only the fail-safe, "actually" this case should not occur anymore) Also the script /usr/share/univention-ox/fix-admin-context-passwords was added. It checks if the passwords in the /etc/ox-secrets/contextX.secret files match the OX contexts. If this is not the case, an error message is displayed for each affected context and the script exits with exitcode 1 (this allows you to reuse it for e.g. a UMC diagnostic module; see also --only-errors). If the parameter --fix is specified, the script sets the passwords in the files for the respective OX contexts. The OX option MASTER_ACCOUNT_OVERRIDE in the file /top/open-xchange/etc/AdminDaemon.properties is set to true and OX is restated. Thus the oxadminmaster can now modify the OX contexts with his account and, for example, reset the password. After all passwords are corrected, the original configuration is restored and OX is restarted. A ucs-test script, which checks fix-admin-context-passwords, I have added under 51_fix_broken_context_admin_passwords in ucs-test-oxseforucs. In univention-ox.postinst it now automatically checks with every package update whether the credentials are correct for all OX contexts. If this is not the case, they are corrected automatically. This can be deactivated via the UCR variable "ox/fix-context-admin-passwords/skip=yes".
Package: univention-ox Version: 11.0.0-63A~4.4.0.202007211651 Branch: ucs_4.4-0 Scope: oxse4ucs Package: ucs-test-oxseforucs Version: 5.0.3A~4.4.0.202007211651 Branch: ucs_4.4-0 Scope: oxse4ucs [4.4] 3715697 Bug #51686: add changelog entry [4.4] 93dabcc Bug #51686: do not shuffle CR + LF [4.4] f6298bf Bug #51686: add changelog entry [4.4] 6fb5220 Bug #51686: install new fix script [4.4] 8d56108 Bug #51686: use a context manager [4.4] 695a89c Bug #51686: add advisory [4.4] 552f427 Bug #51686: add changelog entry [4.4] 760cc99 Bug #51686: add changelog entry [4.4] 72da4d4 Bug #51686: add ucs-test script that checks fix-context-admin-passwords [4.4] 46a8479 Bug #51686: automatically fix broken context admin passwords during updates [4.4] 82d96c5 Bug #51686: add a script that is able to fix broken context admin passwords [4.4] b3d7a42 Bug #51686: just in case, if the context exists, do not die [4.4] 21f4ae8 Bug #51686: before overwriting an existing .secret-file create a backup file [4.4] 12d29ab Bug #51686: do not try to recreate the context if it already exists [4.4] a4d48c8 Bug #51686: add some mypy hints
- README_UPDATE_* has been updated - packages have been uploaded to new app version 7.10.3-ucs6
Module: ox-config File: /etc/dovecot/conf.d/30-ox-push-notifications.conf Checking and fixing broken context admin passwords... Traceback (most recent call last): File "/usr/share/univention-ox/fix-context-admin-passwords", line 240, in <module> main() File "/usr/share/univention-ox/fix-context-admin-passwords", line 236, in main sys.exit(check_and_fix(args.do_fix, args.only_errors, args.verbose, args.language)) File "/usr/share/univention-ox/fix-context-admin-passwords", line 143, in check_and_fix ox_master_admin, ox_master_pass = get_master_credentials() File "/usr/lib/pymodules/python2.7/univention/ox/listener_tools.py", line 905, in get_master_credentials __master_password = get_credentials(OX_MASTER_SECRET) File "/usr/lib/pymodules/python2.7/univention/ox/listener_tools.py", line 918, in get_credentials raise ListenerTools_RuntimeError(str(ex)) univention.ox.listener_tools.ListenerTools_RuntimeError: [Errno 2] No such file or directory: '/etc/ox-secrets/master.secret' Fixing broken context admin passwords. This might take some time due to restarts of OX...
The OX admin master credentials are now checked first. The script fails with exitcode 2 if this is the case. The postinst runs the script to fix broken context passwords only during package updates - not on initial installations. [4.4] 74dfe30 Bug #51686: check OX admin master credentials first [4.4] 59364d7 Bug #51686: add changelog entry [4.4] 6d5f5c4 Bug #51686: do not try to fix passwords on initial installation [4.4] 633c9c3 Bug #51686: catch exception if OX admin master credentials cannot be read Package: univention-ox Version: 11.0.0-65A~4.4.0.202007221041 Branch: ucs_4.4-0 Scope: oxse4ucs
During the update from 7.10.3-ucs[45] → 7.10.3-ucs6, the following messages are printed to appcenter.log: ---[cut]--- Restarting open-xchange-documentconverter-server... Restarting open-xchange (this might take a while)... Waiting 300s for OX server.........290.........280.. users in context 10 could not be listed: Server response: Authentication failed at com.openexchange.admin.rmi.impl.BasicAuthenticator.doAuthentication(BasicAuthenticator.java:259) at com.openexchange.admin.rmi.impl.OXUser.list(OXUser.java:2332) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:357) at sun.rmi.transport.Transport$1.run(Transport.java:200) at sun.rmi.transport.Transport$1.run(Transport.java:197) at java.security.AccessController.doPrivileged(Native Method) at sun.rmi.transport.Transport.serviceCall(Transport.java:196) at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:573) at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:834) at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.lambda$run$0(TCPTransport.java:688) at java.security.AccessController.doPrivileged(Native Method) at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:687) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) at sun.rmi.transport.StreamRemoteCall.exceptionReceivedFromServer(StreamRemoteCall.java:303) at sun.rmi.transport.StreamRemoteCall.executeCall(StreamRemoteCall.java:279) at sun.rmi.server.UnicastRef.invoke(UnicastRef.java:164) at java.rmi.server.RemoteObjectInvocationHandler.invokeRemoteMethod(RemoteObjectInvocationHandler.java:235) at java.rmi.server.RemoteObjectInvocationHandler.invoke(RemoteObjectInvocationHandler.java:180) at com.sun.proxy.$Proxy0.list(Unknown Source) at com.openexchange.admin.console.user.List.maincall(List.java:87) at com.openexchange.admin.console.user.ListCoreExtended.maincall(ListCoreExtended.java:98) at com.openexchange.admin.console.user.ListCore.commonfunctions(ListCore.java:107) at com.openexchange.admin.console.user.List.<init>(List.java:77) at com.openexchange.admin.console.user.List.main(List.java:70) enabled Restarting open-xchange-mobile-api-facade... ---[cut]--- This message is issued by the script wait_for_ox_server, which actually waits for the successful *complete* restart of the OX system. At the time of the restart, however, the context passwords have not yet been corrected. Directly after the restart, the context passwords are corrected and the accumulated queue items are processed automatically. It was then possible to log on to the OX. The order was deliberately not reversed here to prevent the misleading error message, because for the fix-admin-context-passwords it must be ensured that the OX is really running, which is achieved here by the ox_restart. The message of wait_for_ox_server is rather informative and shows that the OX-Server is fully accessible again (but the context passwords are still wrong).
I tested the installation of 7.10.3-ucs6, created a user and logged in into OX successfully. Also an update from …-ucs5 to …-ucs6 has been tested. The state of ucs5 was a fresh installation which was broken. A login with a OX user was not possible. After the update to ucs6, the context admin passwords have been fixed and all deferred queue items were processed, so a login with the previously created user was possible.
QA -> all ok -> VERIFY for production release Code -> all ok Changelogs -> all ok (univention-ox & ucs-test-oxseforucs) yaml -> ok fix-context-admin-passwords script I changed the password for context 10 and got the following (~ diagnose-module) Authentication error for context 10! The password in /etc/ox-secrets/context10.secret is incorrect! The secrets file for context 12 is correct. --only-erros only prints the erros as expected. Adding the --fix parameter lead to fixing the contexts: /fix-context-admin-passwords --fix Fixing broken context admin passwords. This might take some time due to restarts of OX... The secrets file for context 10 has been fixed successfully. The secrets file for context 12 is correct. --verbose also prints listuser output and error-trace of users of contexts which could not be listed because of the authentification failed. During the installation, the fix-script is called and successfully fixed contexts, which I broke before. -> ok univention-directory-listener-ctrl resync ox-context -> secret files stay correct Test -> passes README_UPDATE_* -> up-to-date I used the fix-context-admin-password script to test this. New installation fix-script is not called as described in comment 9 Update - broken contexts -> OK (raises error, finds broken contexts & fixes them) - no broken contexts -> OK (finds correct contexts, passes)
OX App Suite 7.10.3-ucs6 has been released. If this error occurs again, please clone this bug.
Updated to OX 7.10.3-ucs6. New added users can now login correctly. Thank you for fixing this.