Bug 35859 - Free space on root file system less than 10 %
Free space on root file system less than 10 %
Status: CLOSED FIXED
Product: UCS
Classification: Unclassified
Component: UMC - System diagnostic
UCS 4.0
Other Linux
: P5 enhancement (vote)
: UCS 4.2-2-errata
Assigned To: Lukas Oyen
Felix Botner
:
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2014-09-10 08:28 CEST by Stefan Gohmann
Modified: 2017-09-20 15:03 CEST (History)
4 users (show)

See Also:
What kind of report is it?: ---
What type of bug is this?: ---
Who will be affected by this bug?: ---
How will those affected feel about the bug?: ---
User Pain:
Enterprise Customer affected?:
School Customer affected?:
ISV affected?:
Waiting Support:
Flags outvoted (downgraded) after PO Review:
Ticket number:
Bug group (optional):
Max CVSS v3 score:
oyen: Patch_Available+


Attachments
35859-diagnostic-disk-space-420.patch (9.77 KB, patch)
2017-06-06 17:16 CEST, Lukas Oyen
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Stefan Gohmann univentionstaff 2014-09-10 08:28:40 CEST
We should add a system diagnostic plugin which checks free space on /. If less than 10 % are free, the module should add some possibilities:
 - Uninstall unused software (link to App Center)
 - Check /var/log and recommend to move them to another server / storage
 - If an local repository is installed recommend to move it to a different server
 - change the hard drive
Comment 1 Janis Meybohm univentionstaff 2014-09-10 08:46:30 CEST
(In reply to Stefan Gohmann from comment #0)
> If less than 10 % are free, the module should add some possibilities:
- Lower Debug levels (if /var/log is on the root filesystem)
Comment 2 Alexander Kläser univentionstaff 2014-09-24 15:06:34 CEST
A check for /boot could also be done as this might prohibit an update due to too many installed kernels (cf. Bug 35947 as an example).
Comment 3 Lukas Oyen univentionstaff 2017-06-06 17:16:23 CEST
Created attachment 8902 [details]
35859-diagnostic-disk-space-420.patch

A new diagnostic check `disk_usage.py`. This checks every mounted device and warns if less than 10% space is available. If the root partition has less than 10% space left, a critical warning is issued.

All solutions as described in comment 0 are implemented. Additionally, if /var/log is on the root fs or its own partition and no space is left, UCR variables for logging are inspected and a proposal to reset them via the UCR-module is shown, if high debug/logging levels are configured.
Comment 4 Lukas Oyen univentionstaff 2017-08-01 16:23:19 CEST
Committed in r81600 - r81601 (advisory r81649).
Comment 5 Felix Botner univentionstaff 2017-08-07 11:27:02 CEST
> Einige Festplatten sind beinahe voll: 
> - Festplatte für Moint Point / ist 96.0% voll. 
> - Festplatte für Moint Point /var/lib/docker/overlay ist 96.0% voll.
> Sie können per Modul "App Center" Software entfernen. 
> Sie können /var/log auf eine andere Festplatte oder Speicher verschieben. 
> Sie können die Festplatte austauschen.

Why /var/lib/docker/overlay?

> Sie können die Festplatte austauschen.
I don't like this. Just replacing the disk is not all, you have to copy the data, align the partitions and filesystems ... people who know all this stuff already know what to do when disk space is running out
Comment 6 Lukas Oyen univentionstaff 2017-08-07 11:39:03 CEST
(In reply to Felix Botner from comment #5)
> Why /var/lib/docker/overlay?

Because it is a mount point as recognized by `psutil.disk_partitions()` [1].

We could filter for the same device, but that could mean, that `/` is removed instead of `/var/lib/docker/overlay`. 


> > Sie können die Festplatte austauschen.
> I don't like this. Just replacing the disk is not all, you have to copy the
> data, align the partitions and filesystems ... people who know all this
> stuff already know what to do when disk space is running out

Of course. This is as proposed in comment c0. I can remove that if you want.

[1]: https://pythonhosted.org/psutil/#psutil.disk_partitions
Comment 7 Felix Botner univentionstaff 2017-08-07 12:13:37 CEST
(In reply to Lukas Oyen from comment #6)
> (In reply to Felix Botner from comment #5)
> > Why /var/lib/docker/overlay?
> 
> Because it is a mount point as recognized by `psutil.disk_partitions()` [1].
> 
> We could filter for the same device, but that could mean, that `/` is
> removed instead of `/var/lib/docker/overlay`.

filter out docker/overlay

> 
> 
> > > Sie können die Festplatte austauschen.
> > I don't like this. Just replacing the disk is not all, you have to copy the
> > data, align the partitions and filesystems ... people who know all this
> > stuff already know what to do when disk space is running out
> 
> Of course. This is as proposed in comment c0. I can remove that if you want.
yes
Comment 8 Philipp Hahn univentionstaff 2017-08-07 12:29:48 CEST
(In reply to Felix Botner from comment #7)
> (In reply to Lukas Oyen from comment #6)
> > (In reply to Felix Botner from comment #5)
> > > Why /var/lib/docker/overlay?
> > 
> > Because it is a mount point as recognized by `psutil.disk_partitions()` [1].
> > 
> > We could filter for the same device, but that could mean, that `/` is
> > removed instead of `/var/lib/docker/overlay`.
> 
> filter out docker/overlay

# df -T
Dateisystem    Typ      1K-Blöcke Benutzt Verfügbar Verw% Eingehängt auf
udev           devtmpfs     10240       0     10240    0% /dev
tmpfs          tmpfs       204100   20892    183208   11% /run
/dev/dm-0      ext4      48135984 2188764  43439444    5% /
tmpfs          tmpfs       510248       0    510248    0% /dev/shm
tmpfs          tmpfs         5120       0      5120    0% /run/lock
tmpfs          tmpfs       510248       0    510248    0% /sys/fs/cgroup
/dev/vda1      ext2        482922   99077    358911   22% /boot
overlay        overlay   48135984 2188764  43439444    5% /var/lib/docker/overlay/b5aefe33e468c98ec76f489470b77f6b79280b4c35f124a95376f99733185ee6/merged
shm            tmpfs        65536       0     65536    0% /var/lib/docker/containers/e839c86942b2784bc2a5111c80be385dfd8c48f0573ad74f9d2914f5d2294621/shm

# man 1 df
       -x, --exclude-type=TYPE
              limit listing to file systems not of type TYPE
Comment 9 Lukas Oyen univentionstaff 2017-08-08 14:41:38 CEST
(In reply to Felix Botner from comment #7)
> filter out docker/overlay

Done in r81893.

> > Of course. This is as proposed in comment c0. I can remove that if you want.
> yes

Done in r81894.
Comment 10 Felix Botner univentionstaff 2017-08-09 17:04:40 CEST
OK
Comment 11 Erik Damrose univentionstaff 2017-09-20 15:03:34 CEST
<http://errata.software-univention.de/ucs/4.2/166.html>