Bug 50126 - Do not do double logging on disk with journald
Summary: Do not do double logging on disk with journald
Status: RESOLVED WONTFIX
Alias: None
Product: UCS
Classification: Unclassified
Component: logrotate
Version: UCS 4.4
Hardware: Other Linux
: P5 normal
Target Milestone: ---
Assignee: UCS maintainers
QA Contact: UCS maintainers
URL: https://help.univention.com/t/problem...
Keywords:
Depends on:
Blocks:
 
Reported: 2019-09-06 15:12 CEST by Christian Völker
Modified: 2025-09-10 15:20 CEST (History)
0 users

See Also:
What kind of report is it?: Bug Report
What type of bug is this?: 3: Simply Wrong: The implementation doesn't match the docu
Who will be affected by this bug?: 2: Will only affect a few installed domains
How will those affected feel about the bug?: 2: A Pain – users won’t like this once they notice it
User Pain: 0.069
Enterprise Customer affected?: Yes
School Customer affected?: Yes
ISV affected?:
Waiting Support:
Flags outvoted (downgraded) after PO Review:
Ticket number: 2019090521000525
Bug group (optional):
Customer ID: 00009
Max CVSS v3 score:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Christian Völker univentionstaff 2019-09-06 15:12:05 CEST
It happens frequently the filesystem /var getting full

Logging itself is important and can be configured for rsyslogd through UCRV and works together with logrotate. Unfortunately, logging is done in a double way with journald as part of systemd does logging, too.

The configurable limits usually do not really help as journald takes only the widest limitation into account (which usually is to keep around 4GB free and fill the rest with journald-logs).

But frequently 4GB is not enough for a busy slave to get logrotate clean them up on a daily base. So the filesystem gets full, because journald only leaves 4GB of free space.

Citing from man journald:
"If the file system is nearly full and either SystemKeepFree= or RuntimeKeepFree= are violated when systemd-journald is started, the limit will be raised to the percentage that is actually free. This means that if there was enough free space before and journal files were created, and subsequently something else causes the file system to fill up, journald will stop using more space, but it will not be removing existing files to reduce the footprint again, either."


Overall it is quite confusing and customers have several gigabytes of journald aging back two or three years.

I would suggest to configure journald to use only "volatile" logging under /var/run.
There should be no loss in case of server breakdown because rsyslog is still logging persistently to disk /var/log.
Comment 1 Jan-Luca Kiok univentionstaff 2024-06-27 12:09:50 CEST
This issue has been filed against UCS 4.4.

UCS 4.4 is out of general maintenance and components may have vastly changed in later releases. Thus, this issue is now being closed.

If this issue still occurs in newer versions, please use "Clone this bug" or reopen this issue. In this case please provide detailed information on how this issue is affecting you.