Bug 49037 - use best practice python logging
use best practice python logging
Status: CLOSED FIXED
Product: UCS@school
Classification: Unclassified
Component: General
UCS@school 4.4
Other Linux
: P5 normal (vote)
: UCS@school 4.4 v2
Assigned To: Daniel Tröder
Ole Schwiegert
:
Depends on:
Blocks: 50515
  Show dependency treegraph
 
Reported: 2019-03-19 14:50 CET by Daniel Tröder
Modified: 2019-11-18 17:11 CET (History)
0 users

See Also:
What kind of report is it?: Development Internal
What type of bug is this?: ---
Who will be affected by this bug?: ---
How will those affected feel about the bug?: ---
User Pain:
Enterprise Customer affected?:
School Customer affected?:
ISV affected?:
Waiting Support:
Flags outvoted (downgraded) after PO Review:
Ticket number:
Bug group (optional):
Max CVSS v3 score:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Daniel Tröder univentionstaff 2019-03-19 14:50:58 CET
A lot of UCS@school lib and import modules create handlers when they should only be creating loggers.

Also: loggers should not be created at module level, because they might be imported and created before a logger further up in the hierarchy is configured using logging.config.fileConfig() or logging.config.dictConfig(). They disable previous loggers by default.
Comment 1 Daniel Tröder univentionstaff 2019-03-19 22:01:39 CET
The ucsschool.libs and the imports get_logger() functions have been deprecated.
Please use everywhere just:
---
logger = logging.getLogger(__name__)
---

Avoid creating the Logger instance on the module level.

To add a file handler use: ucsschool.lib.models.utils.get_file_handler()
To add a stream (console) handler use: ucsschool.lib.models.utils.get_stream_handler()

All console output is now colorized. The default colors from colorlog (https://github.com/borntyping/python-colorlog) are used atm. We should discuss them as a team. python-colorlog is currently in 4.3-unmaintained. I have added it to the ucs_4.4-0-ucs-school-4.4 scope.

[dtroeder/improve_logging] 4f09b47a6 Bug #49037: provide handlers separately from loggers, colorize console output
[dtroeder/improve_logging] 202254c52 Bug #49037: provide logger to all UCS@school lib objects
[dtroeder/improve_logging] abbe50ff7 Bug #49037: use UCSSchoolHelperAbstractClass logger
[dtroeder/improve_logging] 39e16057c Bug #49037: use explicit relative imports for modules of own package or direct parent/child package
[dtroeder/improve_logging] 7007a5569 Bug #49037: create a logger per module, use provided handlers, don't create loggers at module level


Please reopen, when starting QA (missing merge, changelogs and package builds).
Comment 2 Daniel Tröder univentionstaff 2019-03-20 09:46:56 CET
Uncluttered utils.py by moving the logging configuration into a config file.
(Removed unnecessary symbols from shell.py.)

[dtroeder/improve_logging 5c3e66c49] Bug #49037: move constants to config file
Comment 3 Ole Schwiegert univentionstaff 2019-03-21 12:57:37 CET
looks good so far. Please build and add advisory for final QA
Comment 4 Daniel Tröder univentionstaff 2019-03-21 21:35:06 CET
[4.4] 4e0e3b57d Bug #49037: Merge branch 'dtroeder/improve_logging' into 4.4
[4.4] 87cb71117 Bug #49037: remove now superfluous custom ColorFormatter
[4.4] 33d21fb36 Bug #49037: changelogs
[4.4] f46186577 Bug #49037: update dependencies
[4.4] 2802b5dc0 Bug #49037: advisories

ucs-school-lib (12.1.0-15)
ucs-school-import (17.0.5-7)
ucs-test-ucsschool (6.0.0-32)
Comment 5 Daniel Tröder univentionstaff 2019-03-25 07:53:48 CET
Reintroduce a switch like ColorFormatter used, to not have control symbols for cmdline colors in logfiles like jenkins runs.
Comment 6 Daniel Tröder univentionstaff 2019-03-25 09:44:27 CET
[4.4] df5f9100c Bug #49037: don't colorize output that's redirected away from a terminal
[4.4] f0fa0e1f0 Bug #49037: adjust handlers loglevel after reading configuration
[4.4] ba55ab487 Bug #49037: adjust test to pyhook test and logging
[4.4] e6c07e00b Bug #49037: advisory updates

ucs-school-lib (12.1.0-16)
ucs-school-import (17.0.5-8)
ucs-test-ucsschool (6.0.0-33)
Comment 7 Ole Schwiegert univentionstaff 2019-03-26 10:52:46 CET
Changelog&Advisory: OK
Package installs: OK
Jenkins tests: OK
Changes effective: OK
Comment 8 Daniel Tröder univentionstaff 2019-03-26 15:57:36 CET
* The INFO level is now white, it was grey/light_white like DEBUG.
* Color configuration is stored in the same YAML file as the format configuration.
* The handler that prevents colorizing output in case of a missing TTY, was preventing colorization if run by ucs-test. Users of ImportTestbase._get_logger() now get colored output anyway.
* The get_stream_handler() function now allows to set a custom Formatter class.
* The get_stream_handler() function now allows to force colorization by setting the environment variable UCSSCHOOL_FORCE_COLOR_TERM.

[4.4] 0cd121db3 Bug #49037: allow to force colorized output, load colors from configuration file
[4.4] e6f1270bb Bug #49037: prevent adding 2nd stream handler
[4.4] 73067ee51 Bug #49037: force colorized output if run interactively by ucs-test but not when run from jenkins
[4.4] b0cf3c96f Bug #49037: advisory updates
Comment 9 Daniel Tröder univentionstaff 2019-03-27 07:46:44 CET
[4.4] 784d62a9b Bug #49037: simplify TTY testing
[4.4] d52e9e583 Bug #49037: make get_ucsschool_logger() accessable for all tests
[4.4] dc961311f Bug #49037: advisory update
Comment 10 Ole Schwiegert univentionstaff 2019-03-29 11:05:14 CET
bChangelog&Advisory: OK
Package installs: OK
Jenkins tests: OK
Changes effective: OK
Comment 11 Sönke Schwardt-Krummrich univentionstaff 2019-04-01 00:44:10 CEST
UCS@school 4.4 v2 has been released.

https://docs.software-univention.de/changelog-ucsschool-4.4v2-de.html

If this error occurs again, please clone this bug.