Bug 49333 - Handle storage space problems with the newly implemented backup feature
Handle storage space problems with the newly implemented backup feature
Status: CLOSED FIXED
Product: UCS@school
Classification: Unclassified
Component: UMC - Exam mode
UCS@school 4.4
Other Linux
: P5 normal (vote)
: UCS@school 4.4 v3
Assigned To: Ole Schwiegert
Daniel Tröder
:
Depends on: 49227
Blocks:
  Show dependency treegraph
 
Reported: 2019-04-23 11:12 CEST by Ole Schwiegert
Modified: 2019-05-29 14:04 CEST (History)
4 users (show)

See Also:
What kind of report is it?: Feature Request
What type of bug is this?: ---
Who will be affected by this bug?: ---
How will those affected feel about the bug?: ---
User Pain:
Enterprise Customer affected?:
School Customer affected?:
ISV affected?:
Waiting Support:
Flags outvoted (downgraded) after PO Review:
Ticket number:
Bug group (optional):
Max CVSS v3 score:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Ole Schwiegert univentionstaff 2019-04-23 11:12:18 CEST
+++ This bug was initially created as a clone of Bug #49227 +++

User Story from Trello Board:

Als Lehrkraft möchte ich, dass Zwischenstände der Ergebnisse einer Klassenarbeit zwischengespeichert werden, um Datenverlust zu verhindern.
- Das Zeitintervall (Standard 5 min) sollte per UCR-Variable konfigurierbar sein.
- In einen Unterordner Zwischenergebnisse
Comment 1 Ole Schwiegert univentionstaff 2019-04-23 11:16:56 CEST
For the definition of the problem read on from comment #5 of the cloned bug.
Comment 2 Daniel Tröder univentionstaff 2019-04-23 11:31:51 CEST
tar supports incrementally growing archives (add 1st backup, then later add 2nd backup, ...).
Comment 3 Ole Schwiegert univentionstaff 2019-05-06 09:25:11 CEST
Technically a good idea which would help with the problem. Though the process of recovering the data afterwards was deemed too complicated for a teacher after discussing it with Michel.

Zipping the backups (the archive natively supported by windows) is an option though.
Comment 4 Ole Schwiegert univentionstaff 2019-05-06 12:18:32 CEST
The feature is developed in oschwieg/4.4/49333 (subject to force pushes)

As a first step the backups are zip compressed. It can be configured with UCR ucsschool/exam/backup/compress which defaults to True and thus compression happens.
Comment 5 Ole Schwiegert univentionstaff 2019-05-07 12:15:12 CEST
The current status is as follows:

- Backups are compressed into a zip by default (configurable via UCR)
- There is a limit for the number of backups per user per exam. The default is 40 (configurable via UCR and can be deactivated)

- Before attempting a backup !or manual collection! it is now checked if there is enough space in the teachers home to copy over a students exam folder (since zip <= uncompressed this test should suffice). If not it is checked if enough space would be freed, by removing the single oldest collected result/backup. If yes then the old result is deleted and the process continues. If not the copy fails and continues with the next attempt to collect a students files.


Please QA in the feature branch and give feedback regarding the taken measures. If accepted I will add tests and extend the manual as well as merge the feature into 4.4

Especially regarding the automated deletion in point three I would appreciate feedback. It can be done in many different ways (try to delete more than only one result, always delete versions for all students). I chose this way in the end to avoid the possibility of shrinking numbers of collected results (in case the students folder grows much bigger during the exam) and to keep the fingerprint of this automated deletion as small as possible.
Comment 6 Ole Schwiegert univentionstaff 2019-05-09 13:07:31 CEST
Package: ucs-school-umc-exam
Version: 9.0.0-9A~4.4.0.201905091301
Branch: ucs_4.4-0
Scope: ucs-school-4.4

Package: ucs-school-umc-distribution
Version: 17.0.1-0A~4.4.0.201905091304
Branch: ucs_4.4-0
Scope: ucs-school-4.4

Further Changes:

- Results are no longer deleted if the space runs out. Only a log message is created

- The backup limit does not cause rotation of the backup. If the limit of backups is reached there are simply no more backups created.
Comment 7 Ole Schwiegert univentionstaff 2019-05-10 14:50:38 CEST
Package: ucs-school-umc-distribution
Version: 17.0.1-1A~4.4.0.201905101448
Branch: ucs_4.4-0
Scope: ucs-school-4.4

Fix error in num_results property if no results were collected yet.
Comment 8 Ole Schwiegert univentionstaff 2019-05-13 10:22:52 CEST
Package: ucs-school-umc-exam
Version: 9.0.1-1A~4.4.0.201905131022
Branch: ucs_4.4-0
Scope: ucs-school-4.4


Improved logging for exam backup
Comment 9 Florian Best univentionstaff 2019-05-13 10:44:34 CEST
Fixed the test cases in:

ucs-test-ucsschool (6.0.0-55)
0a5321ec46d8 | Bug #49333: fix distribution test cases
Comment 10 Daniel Tröder univentionstaff 2019-05-13 11:30:18 CEST
OK: compression
OK: number of backups limit handling
OK: cron job generation
OK: advisory
OK: documentation for UCRV and in manual
OK: 18_* ucs-tests
Comment 11 Sönke Schwardt-Krummrich univentionstaff 2019-05-29 14:04:50 CEST
UCS@school 4.4 v2 has been released.

https://docs.software-univention.de/changelog-ucsschool-4.4v2-de.html

If this error occurs again, please clone this bug.