Bug 37223 - Block UMC while update is running
Block UMC while update is running
Status: CLOSED FIXED
Product: UCS
Classification: Unclassified
Component: UMC (Generic)
UCS 4.3
Other Linux
: P5 normal (vote)
: UCS 4.3-1-errata
Assigned To: Dirk Wiesenthal
Ole Schwiegert
:
Depends on: 47592
Blocks:
  Show dependency treegraph
 
Reported: 2014-12-08 09:41 CET by Janis Meybohm
Modified: 2020-04-28 16:47 CEST (History)
7 users (show)

See Also:
What kind of report is it?: Feature Request
What type of bug is this?: ---
Who will be affected by this bug?: ---
How will those affected feel about the bug?: ---
User Pain:
Enterprise Customer affected?:
School Customer affected?:
ISV affected?:
Waiting Support:
Flags outvoted (downgraded) after PO Review:
Ticket number:
Bug group (optional): External feedback, Troubleshooting, Usability
Max CVSS v3 score:


Attachments
module load error notifications (144.25 KB, image/png)
2018-07-24 17:31 CEST, Ole Schwiegert
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Janis Meybohm univentionstaff 2014-12-08 09:41:14 CET
While an update is running, all other UMC modules should be blocked and logging in to UMC should auto-redirect to the updater module. This would prevent some variations of "reboot while updating"
Comment 1 Alexander Kläser univentionstaff 2015-05-07 19:48:21 CEST
... i.e., something like a maintenance mode for UMC.
Comment 2 Jannik Ahlers univentionstaff 2018-07-12 15:33:54 CEST
I adjusted the styling some in the branch. We still need to find a good way to include images (progressBarAnim.gif and univention.svg).
I also included a link to directly load the roboto font from google. Is this ok, or are there some license issues prohibiting this?
There are no additional features/links on the page right now.
Comment 3 Dirk Wiesenthal univentionstaff 2018-07-24 12:19:32 CEST
Fixed in
  univention-management-console 10.0.6-8A~4.3.0.201807241153
  univention-updater 13.0.1-54A~4.3.0.201807200128
with all new
  univention-maintenance-mode 1.0.0-3A~4.3.0.201807241151

The updater now sets
  updater/maintenance=true
as long as a release update takes when clicking on the button in UMC. The UMC module also redirects to /univention/login/.

The management console's apache templates were extended so that
  /univention/login/
  /univention/management/
are serving the new /univention/maintenance/index.html instead if updater/maintenance=true.

univention-maintenance-mode starts a service that constantly watches /var/lib/univention-updater/univention-updater.status.details and writes a simple progress json file.

The maintenance/index.html keeps polling this progress json and updates its content.
This site also polls "itself" (which is /univention/login/). If this changes (because apache no longer serves maintenance/index.html due to a finished update), it reloads itself.

To test:

1) On 4.3-0
Add
  http://omar.knut.univention.de/build2 ucs_4.3-0-errata4.3-1/all/
  http://omar.knut.univention.de/build2 ucs_4.3-0-errata4.3-1/$(ARCH)/
to /etc/apt/sources.list

and install the binary packages from the source packages listed above.

2) On 4.3-1
Same, but do
  ucr set version/patchlevel=0
and edit /var/lib/univention-updater/univention-updater.status
(should work with version/version=4.2 version/patchlevel=3, too).
You would also need packages to be updated. At least try the correct one of
  apt-cache policy univention-errata-level

1 + 2)
Click on Release Update on the UMC module.

(I highly recommend taking a snapshot or two)
Comment 4 Ole Schwiegert univentionstaff 2018-07-24 17:31:00 CEST
Changelog: OK
Advisory: REOP (C0)
package build: OK
package installs: OK (C1)
functional: REOP (C2)


C0: Advisories are missing!
C1: The package installs fine and is automatically installed on a normal update.
Should we maybe enter the dependencies for the other two packages in the right version to univention-maintenance-mode to prevent any installation of that package without the correct versions on the other packages?
C2: The maintenance mode is working, enables and disables correctly. The service runs, files are written and UCR variable switches off after update is done.

But I encountered a few problems, that are hard to replicate since they focus on timing:
After the update was done and I logged in again, I got a Internal Server Error message. I guess the server was not up again yet when I already logged in.

When you start the update you can still navigate around in umc because of the browser caching. Only if you press F5 you actually get the maintenance mode site. In the screenshot you can see the ugly result of that (dozens of module load error notifications)

When I started the update and navigated to another module in umc and then pressed F5 I got into the maintenance mode. After the update was done I got redirected to the module I refreshed before. When I tried doing sth then I got a Permission denied error since I was actually not logged in anymore. We should try to prevent that.
Comment 5 Ole Schwiegert univentionstaff 2018-07-24 17:31:21 CEST
Created attachment 9610 [details]
module load error notifications
Comment 6 Erik Damrose univentionstaff 2018-07-25 12:33:54 CEST
(In reply to Ole Schwiegert from comment #4)
> Advisory: REOP (C0)
> 
> C0: Advisories are missing!

If anyone comes across such an issue, add the bugnumber to an existing YAML or create a new one for the package(s) immediately!

Right now, e.g. the univention-updater package could be released, without this change being verified properly! I will now add dummy advisories, so this bug at least gets tracked.
Comment 7 Dirk Wiesenthal univentionstaff 2018-07-28 12:21:18 CEST
Fixed in
  univention-maintenance-mode 1.0.0-4A~4.3.0.201807271836
  univention-management-console 10.0.6-8A~4.3.0.201807241153
  univention-updater 13.0.1-53A~4.3.0.201807031200

After clicking on "Release Update", the maintenance page should load automatically. This may take a second or two (some UCR templates and one Apache reload), but it should work. If not, this is a problem.

In the meantime, it may give you errors. In my opinion, this is "good enough" - as long as this only holds for a couple of seconds.

If you manage to reproduce the "Permission denied" error, I would like to investigate any traceback in the logs. It would make the feeling of this feature a lot better if this does not happen. But I _think_ that one F5 is more or less forgivable after a release update. Again, I think we should fix it, if we know the exact reason.
Comment 9 Dirk Wiesenthal univentionstaff 2018-07-30 22:53:24 CEST
Hopefully fixed in
  ucs-test 8.0.28-153A~4.3.0.201807302253
(Not tested)
Comment 10 Ole Schwiegert univentionstaff 2018-08-02 10:07:35 CEST
I did not find any problems anymore. The test seems to pass now. I will set this to Verfified.
Comment 11 Erik Damrose univentionstaff 2018-08-02 13:00:00 CEST
reopen: In my test i could not get it to work. Is there a missing dependency?

I installed a 4.3-0, updated to the mentioned packages from comment 3 and started the release update.

* I saw the old updater module with the scrolling logfile
* I opened a second tab and tried to access UMC, and saw the maintenance mode. The interface looked clean and nice. 
* After the update finished, i was not redirected back to UMC.

# ls /var/lib/univention-updater/
reboot.at  univention-updater.status  univention-upgrade.status

# cat /var/lib/univention-updater/univention-upgrade.status
status=DONE

# cat /var/lib/univention-updater/univention-updater.status
status=DONE
current_version=4.3-1
type=NET

# cat /var/lib/univention-updater/reboot.at 
/sbin/reboot
Comment 12 Erik Damrose univentionstaff 2018-08-03 13:25:54 CEST
In my case it seems to be a browser or browser cache issue. I accessed the UMC before updating the package and installed u-maintenance-mode. Then, after reloading and logging into UMC and starting the release update, the old updater module was shown. When i accessed the UMC with a new private browser session, i was redirected to univention/login/ and the maintenance mode was shown.

One other thing: when accessing the portal while an update is running, a blank portal without any text is shown. Is that intended?
Comment 13 Erik Damrose univentionstaff 2018-08-09 12:43:14 CEST
There seems to be no way to detect and present preup-update errors to the user. 

In a test i created a preup error and started the release update. Maintenance mode started and i saw the error in the updater.log. Maintenance mode ended itself a few seconds later without showing any indication that the update was not started due to an error. I was redirected to the UMC login.

Quck way to produce something that triggers a preup error: /var/lib/univention-directory-replication; touch /var/lib/univention-directory-replication/failed.ldif
Comment 14 Dirk Wiesenthal univentionstaff 2018-08-15 13:55:10 CEST
(In reply to Erik Damrose from comment #13)
> There seems to be no way to detect and present preup-update errors to the
> user. 
> 
> In a test i created a preup error and started the release update.
> Maintenance mode started and i saw the error in the updater.log. Maintenance
> mode ended itself a few seconds later without showing any indication that
> the update was not started due to an error. I was redirected to the UMC
> login.
> 
> Quck way to produce something that triggers a preup error:
> /var/lib/univention-directory-replication; touch
> /var/lib/univention-directory-replication/failed.ldif

I have decided to open another bug for that: Bug#47592. The exact reason of a PREUP abort should not be shown on the maintenance page. But it would be nice if UMC would show a warning upon login and allow reading the log file.
Comment 15 Dirk Wiesenthal univentionstaff 2018-08-16 00:01:45 CEST
(In reply to Erik Damrose from comment #12)
> In my case it seems to be a browser or browser cache issue. I accessed the
> UMC before updating the package and installed u-maintenance-mode. Then,
> after reloading and logging into UMC and starting the release update, the
> old updater module was shown. When i accessed the UMC with a new private
> browser session, i was redirected to univention/login/ and the maintenance
> mode was shown.

I am going to test a few more times tomorrow. But until now, I could only reproduce it when updating the packages on the command line while being logged in in UMC and starting the release update afterwards from there. This seems rather uncommon.
After package updates via UMC, the frontend asks me to restart the Web Server. I did not test everything when refusing to do this. But problems after such a decisions feel "unsupported".

I tried a few things to cope with frontend and backend not matching but none felt bullet proof.

I will set this to RESOLVED to start the QA phase at least for the next point.

> One other thing: when accessing the portal while an update is running, a
> blank portal without any text is shown. Is that intended?

No. Fixed in
  univention-management-console 10.0.6-11A~4.3.0.201808151705
Comment 16 Ole Schwiegert univentionstaff 2018-08-22 10:28:01 CEST
Tested with 1.0.0-5A~4.3.0.201808211355

Packages installed: OK
Advisories and Changelog: OK
Normal portal displayed while in maintenance mode: OK
Maintenance Mode functioning: OK (*)

(*) As discussed with Dirk the problems described here are probably some kind of caching problems caused by installing and then using UMC to upgrade. The tests were thus performed by upgrading in a private browser window to prevent any caching.
Comment 18 Florian Best univentionstaff 2020-04-28 16:47:25 CEST
This bug changed: git:d154c1ea6e43b140620e75955fd3e73c27c1f04f
+++ management/univention-management-console/conffiles/etc/apache2/sites-available/univention.conf
-print '''\tHeader always setifempty "Content-Security-Policy" "default-src 'self' 'unsafe-inline' 'unsafe-eval' %s;"''' % (piwik,)
+maintenance = 'https://updates.software-univention.de https://fonts.googleapis.com' if configRegistry.is_true('updater/maintenance', False) else ''
+print '''\tHeader always setifempty "Content-Security-Policy" "default-src 'self' 'unsafe-inline' 'unsafe-eval' %s %s;"''' % (piwik, maintenance)


→ Why does the login URL needs this?