Comment 1 Florian Best 2023-02-14 08:59:00 CET

probably there is a debian/ucslint.overrides which ignores the 0020 flake8 module?

anyway we are going to remove the 0020-flake8 plugin soon → then those extra reports are gone.

Comment 2 Lukas Rettler 2023-02-14 09:08:57 CET

Hi Florian

No, unfortunately this is not the case. Even if it is, the log output should be the same in my opinion.

```
# LMZ ticket numbers do not follow the UCS ticket number scheme. Hence, we ignore "latest changelog
# entry does not contain bug/ticket/issue number" warnings:
0007-2
# Do not complain about changelog dates being "not strict-monotonically increasing by time", neither
# on the latest entry:
0007-3
# ... nor on the older entries:
0007-5

# 'contains no copyright block' really means 'no Univention copyright block'
0010-2
# usr/sbin/opsi-upgrade.py: file specifies wrong python version in hashbang
0009-3
```

Comment 3 Florian Best 2023-02-27 18:53:46 CET

Situation improved in ucslint 9.0.3-2 for UCS 5.0-3-errata (unreleased):

9e5ad212fc chore(ucslint): update debian/changelog
30f9bbfbb7 fix(ucslint): remove 0011-18 definition: no longer relevant
c418a3fb83 fix(ucslint): remove 0011-17 definition: no longer relevant
dd450d4db8 fix(ucslint): remove 0011-12 definition: no longer relevant
8d667682c9 fix(ucslint): remove 0011-11 definition: no longer relevant
0733808df3 fix(ucslint): remove 0011-7 definition: is not checked anymore
48e34b9d55 fix(ucslint): remove 0011-6 definition: is not checked anymore
2247cf7a74 fix(ucslint): update to Python 3 syntax
ac69e72581 fix(ucslint): remove 0011-8 definition: is not checked anymore
98e3d508d1 fix(ucslint): remove 0020-flake8 as pre-commit replaces this
fadc5a4007 fix(ucslint): remove 0009-13 internal-python-keyword by ruff A001
938a2a8f6b fix(ucslint): remove 0009-12 reserved-python-keyword by ruff A001
94a976e692 fix(ucslint): remove 0009-7 comparision None covered by flake8/ruff E711
43796158ab fix(ucslint): remove 0009-6 raise "text" covered by ruff B016
7d187d5e13 fix(ucslint): remove 0009-5 dict.has_key covered by flake8 W601

The 0020-flake8 plugin has been removed as nowadays git "pre-commit" hooks should check such things.

The root cause remains:
> I have a different ucslint report while running in a package or by giving the packagepath.
This is because ucslint operates on directories and not on files.
We could argue that the invocation is wrong as it should then be:
find -name debian -printf '%h\0' | xargs -0 ucslint | ucslint-sort-output.py -g -s

Comment 4 Philipp Hahn 2023-11-24 18:23:17 CET

(In reply to Florian Best from comment #3)
> The root cause remains:
> > I have a different ucslint report while running in a package or by giving the packagepath.
>
> This is because ucslint operates on directories and not on files.
> We could argue that the invocation is wrong as it should then be:
> find -name debian -printf '%h\0' | xargs -0 ucslint | ucslint-sort-output.py -g -s

Fixed in <https://git.knut.univention.de/univention/ucs/-/merge_requests/973>: paths are not normalized properly.

[phahn/ucslint] 50ca0bd599 style: Fix shell quoting
[phahn/ucslint] baa033da5a fixup! doc(bind): Fix changelog entry spelling
[phahn/ucslint] 3f02b9bea8 style(test): Fix shell quoting
[phahn/ucslint] 2506404235 doc(bind): Fix changelog entry spelling
[phahn/ucslint] eab17ab7f9 refactor(ucslint): Convert to pathlib
[phahn/ucslint] 5cc567a94f refactor(ucslint): re.Match[index]
[phahn/ucslint] 32458cb383 refactor(ucslint): Deprecate univention-(admin|baseconf)
[phahn/ucslint] a633ce3155 perf(ucslint): Load plugins only once
[phahn/ucslint] a954a7200d style(ucslint): Sort argparse.ArgumentParser.add_argument() arguments
[phahn/ucslint] 0c35538877 style(ucslint): Convert to f-strings
[phahn/ucslint] e2bee3742b style(ucslint): Upgrade to Python 3.7+ style
[phahn/ucslint] 02ab265e39 test(ucslint): package via argument or CWD
[phahn/ucslint] f8abbdc9d6 fix(ucslint/sort): Ignore unmatching lines
[phahn/ucslint] 533044476a fix(ucslint): normalize path
[phahn/ucslint] bd09081711 refactor(ucslint): Use enum.IntFlag to track join script usage
[phahn/ucslint] 551e36a64f test(ucslint): Fix testframework
[phahn/ucslint] 0d2feac023 test(ucslint): Fix wrongly named join script
[phahn/ucslint] 327e21c481 test(ucslint): Strip trailing slash from test name
[phahn/ucslint] 92bb178f73 doc(ucslint): exit codes
[phahn/ucslint] c9140c3bbe ci(ucslint): Fix wrong exclude

Comment 5 Philipp Hahn 2023-12-08 12:03:11 CET

[5.0-6] e47d2f589b fix(udm): Remove deprecated univention-admin
 doc/changelog/index.rst                                                                  | 3 +++
 management/univention-directory-manager-modules/debian/changelog                         | 5 +++--
 .../univention-directory-manager-modules/debian/univention-directory-manager-tools.links | 1 -
 .../etc/bash_completion.d/univention-directory-manager                                   | 1 -
 management/univention-directory-manager-modules/univention-cli-server                    | 2 +-
 5 files changed, 7 insertions(+), 5 deletions(-)

[5.0-6] 83d11377a2 fix(ldap): Remove broken test code
 management/univention-ldap/test/listner-notifier-test | 795 --------------------------------------
 1 file changed, 795 deletions(-)

[5.0-6] 60ec672744 style: Fix shell quoting
 base/univention-quota/files/univention-group-quota              |   9 +-
 management/univention-ldap/debian/changelog                     |   5 +-
 .../univention-ldap/debian/univention-ldap-server.postinst      |   3 +-
 management/univention-ldap/univention-backup2master             | 187 ++++++++++++----------------
 services/univention-samba4/debian/changelog                     |   5 +-
 services/univention-samba4/sysvol-sync-scripts/sysvol-sync.sh   |  41 +++---
 test/ucs-test/debian/changelog                                  |   5 +-
 7 files changed, 117 insertions(+), 138 deletions(-)

[5.0-6] a037fe18f0 style(test): Fix shell quoting
 test/ucs-test/lib/base.sh                                       | 22 ++++----------
 test/ucs-test/lib/computer.sh                                   |  9 ++----
 test/ucs-test/lib/container.sh                                  | 14 +++------
 test/ucs-test/lib/group.sh                                      | 47 +++++++++++------------------
 test/ucs-test/lib/shares.sh                                     | 15 +++------
 test/ucs-test/lib/user.sh                                       | 20 +++++-------
 test/ucs-test/tests/01_base/52proofuniquemember                 |  4 +--
 test/ucs-test/tests/01_base/52proofuniquemember_empty_attribute |  4 +--
 8 files changed, 48 insertions(+), 87 deletions(-)

[5.0-6] 3c924bad10 doc(bind): Fix changelog entry spelling
 packaging/ucslint/univention/ucslint/0015-FuzzyNames.py | 1 +
 services/univention-bind/debian/changelog               | 2 +-
 test/ucs-test/debian/ucslint.overrides                  | 2 +-
 3 files changed, 3 insertions(+), 2 deletions(-)

[5.0-6] e0cb6a1a4e refactor(ucslint): Convert to pathlib
 packaging/ucslint/ucslint                                    |  91 +++---
 packaging/ucslint/univention/ucslint/0001-CheckJoinScript.py |  88 ++---
 packaging/ucslint/univention/ucslint/0002-CopyPasteErrors.py |   8 +-
 packaging/ucslint/univention/ucslint/0004-CheckUCR.py        | 536 +++++++++++++++----------------
 packaging/ucslint/univention/ucslint/0006-CheckPostinst.py   |  15 +-
 packaging/ucslint/univention/ucslint/0007-Changelog.py       |   8 +-
 packaging/ucslint/univention/ucslint/0008-Translations.py    |  15 +-
 packaging/ucslint/univention/ucslint/0009-Python.py          |   3 +-
 packaging/ucslint/univention/ucslint/0010-Copyright.py       |  31 +-
 packaging/ucslint/univention/ucslint/0011-Control.py         |  43 +--
 ...
 18 files changed, 567 insertions(+), 653 deletions(-)

[5.0-6] d2267f3809 refactor(ucslint): re.Match[index]
 packaging/ucslint/ucslint-sort-output.py                     |  2 +-
 packaging/ucslint/univention/ucslint/0001-CheckJoinScript.py |  6 +++---
 packaging/ucslint/univention/ucslint/0004-CheckUCR.py        | 10 +++++-----
 packaging/ucslint/univention/ucslint/0008-Translations.py    |  6 +++---
 packaging/ucslint/univention/ucslint/0009-Python.py          |  2 +-
 packaging/ucslint/univention/ucslint/0010-Copyright.py       |  2 +-
 packaging/ucslint/univention/ucslint/0011-Control.py         |  4 ++--
 packaging/ucslint/univention/ucslint/0013-bashism.py         |  6 +++---
 packaging/ucslint/univention/ucslint/0014-Depends.py         |  2 +-
 packaging/ucslint/univention/ucslint/0015-FuzzyNames.py      |  2 +-
 ...
 12 files changed, 24 insertions(+), 24 deletions(-)

[5.0-6] 43cd3690c6 refactor(ucslint): Deprecate univention-(admin|baseconf)
 packaging/ucslint/testframework/0016-2-3.correct        |  26 ++++----
 packaging/ucslint/testframework/0017-1.correct          | 116 +++++++-----------------------------
 packaging/ucslint/testframework/0017-1/0017-1-1         |  18 ------
 packaging/ucslint/univention/ucslint/0004-CheckUCR.py   |   4 +-
 packaging/ucslint/univention/ucslint/0016-Deprecated.py |  10 ++--
 packaging/ucslint/univention/ucslint/0017-Shell.py      |   2 +-
 6 files changed, 43 insertions(+), 133 deletions(-)

[5.0-6] 5f6913ce1b perf(ucslint): Load plugins only once
 packaging/ucslint/ucslint | 98 +++++++++++++++++++++++++++++++++++++------------------------------
 1 file changed, 54 insertions(+), 44 deletions(-)

[5.0-6] fd6f49a63e style(ucslint): Sort argparse.ArgumentParser.add_argument() arguments
 packaging/ucslint/ucslint                | 72 +++++++++++++++++++++++++++++++++++++---------------
 packaging/ucslint/ucslint-sort-output.py |  2 +-
 2 files changed, 53 insertions(+), 21 deletions(-)

[5.0-6] 217b2491dc style(ucslint): Convert to f-strings
 packaging/ucslint/ucslint                                    | 16 +++----
 packaging/ucslint/ucslint-sort-output.py                     |  2 +-
 packaging/ucslint/univention/ucslint/0001-CheckJoinScript.py | 12 ++---
 packaging/ucslint/univention/ucslint/0004-CheckUCR.py        | 78 ++++++++++++++++----------------
 packaging/ucslint/univention/ucslint/0006-CheckPostinst.py   | 12 ++---
 packaging/ucslint/univention/ucslint/0007-Changelog.py       |  2 +-
 packaging/ucslint/univention/ucslint/0008-Translations.py    |  8 ++--
 packaging/ucslint/univention/ucslint/0010-Copyright.py       |  4 +-
 packaging/ucslint/univention/ucslint/0011-Control.py         |  4 +-
 packaging/ucslint/univention/ucslint/0013-bashism.py         |  5 +-
 ...
 15 files changed, 108 insertions(+), 108 deletions(-)

[5.0-6] c115c9da50 style(ucslint): Upgrade to Python 3.7+ style
 packaging/ucslint/ucslint                                    | 35 ++++++++++++-------------
 packaging/ucslint/ucslint-sort-output.py                     | 11 ++++----
 packaging/ucslint/univention/ucslint/0001-CheckJoinScript.py |  5 ++--
 packaging/ucslint/univention/ucslint/0004-CheckUCR.py        | 44 ++++++++++++++++----------------
 packaging/ucslint/univention/ucslint/0006-CheckPostinst.py   |  4 +--
 packaging/ucslint/univention/ucslint/0010-Copyright.py       |  3 +--
 packaging/ucslint/univention/ucslint/0014-Depends.py         |  8 +++---
 packaging/ucslint/univention/ucslint/0015-FuzzyNames.py      | 13 +++++-----
 packaging/ucslint/univention/ucslint/0018-Debian.py          | 14 +++++-----
 packaging/ucslint/univention/ucslint/__init__.py             |  1 -
 ...
 12 files changed, 87 insertions(+), 93 deletions(-)

[5.0-6] 57ea90fd49 test(ucslint): package via argument or CWD
 packaging/ucslint/testucslint.sh | 42 +++++++++++++++++++++++++++++-------------
 1 file changed, 29 insertions(+), 13 deletions(-)

[5.0-6] 30742e1ab0 fix(ucslint/sort): Ignore unmatching lines
 packaging/ucslint/ucslint-sort-output.py | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

[5.0-6] 214b83ea92 fix(ucslint): normalize path
 doc/changelog/index.rst                                      |  6 ++++++
 packaging/ucslint/debian/changelog                           |  5 +++--
 packaging/ucslint/testframework/0004-24.correct              |  2 +-
 packaging/ucslint/testframework/0004-34.correct              |  2 +-
 packaging/ucslint/testframework/0004-62-63.correct           |  2 +-
 packaging/ucslint/ucslint                                    | 10 +++++-----
 packaging/ucslint/univention/ucslint/0001-CheckJoinScript.py | 25 +++++++++++++------------
 packaging/ucslint/univention/ucslint/0002-CopyPasteErrors.py |  6 +++---
 packaging/ucslint/univention/ucslint/0004-CheckUCR.py        | 25 +++++++++++++------------
 packaging/ucslint/univention/ucslint/0006-CheckPostinst.py   |  7 ++++---
 ...
 17 files changed, 98 insertions(+), 88 deletions(-)

[5.0-6] cb4844fd43 refactor(ucslint): Use enum.IntFlag to track join script usage
 packaging/ucslint/univention/ucslint/0001-CheckJoinScript.py | 28 ++++++++++++++++------------
 1 file changed, 16 insertions(+), 12 deletions(-)

Package: ucslint
Version: 9.0.4-1
Branch: ucs_5.0-0
Scope: ucs5.0-6

Package: univention-directory-manager-modules
Version: 15.0.25-1

Package: univention-ldap
Version: 16.0.14-1

Package: univention-samba4
Version: 9.0.15-1

Comment 6 Florian Best 2023-12-08 13:20:39 CET

OK: code review
OK: changelog entry
OK: basic ucslint tests

Comment 7 Florian Best 2023-12-09 11:47:20 CET

Additions, which enable ucslint to run on selected files instead of against selected debian directories:
This is the basis to run ucslint via pre-commit, which acts on files specified in the .pre-commit-hooks definition.

ucslint (9.0.4-2)
2447d3ba7192 | chore(ucslint): add changelog entry
b2047c31c0f8 | build(ucslint): build ucslint in docker via setuptools
599f88462f6c | build(ucslint): integrate pytest to test doctests
e5e27022bede | feat(ucslint): provide entrypoint script
bcf63f30355d | fix(ucslint): handle files which are not part of a debian package
6ceed2eac2cc | feat(ucslint): allow to execute ucslint plugins for specific given files
77fc49827886 | fixup! refactor(ucslint): Convert to pathlib

FYI: When executing ucslint via the main.py instead of the entrypoint script, it raises (before my adaptions):
    FAILED[arg]
    Traceback (most recent call last):
      File "packaging/ucslint/univention/ucslint/main.py", line 437, in <module>
        main()
      File "packaging/ucslint/univention/ucslint/main.py", line 429, in main
        incident_cnt, exitcode_cnt = chk.printResult(ignore_IDs, display_only_IDs, options.display_only_categories, options.exitcode_categories, options.junit_xml)
      File "packaging/ucslint/univention/ucslint/main.py", line 216, in printResult
        if self.in_overrides(msg):
      File "packaging/ucslint/univention/ucslint/main.py", line 178, in in_overrides
        filepath = msg.filename.relative_to(self.path) if msg.filename else Path("")
    AttributeError: 'str' object has no attribute 'relative_to'

Seems something has not been migrated to pathlib. I couldn't figure it out, it behaves strange, as if filename is magically replaced without initializing the class. Afaik, I don't have local plugins.

If that is fixed, we can think about reverting e5e27022bede.

Comment 8 Philipp Hahn 2023-12-12 06:46:52 CET

(In reply to Florian Best from comment #7)
> FYI: When executing ucslint via the main.py instead of the entrypoint
> script, it raises (before my adaptions):
>     FAILED[arg]
>     Traceback (most recent call last):
...
>     AttributeError: 'str' object has no attribute 'relative_to'
...
> Seems something has not been migrated to pathlib. I couldn't figure it out,
> it behaves strange, as if filename is magically replaced without
> initializing the class. Afaik, I don't have local plugins.

There is one big difference between these methods to run a program:

- With `./ucslint` you basically do `PYTHONPATH=.:…` first, so `import univention.ucslint` imports `./univention/ucslint/__init__.py`

- With your `./univention/ucslint/main.py` this becomes `PYTHONPATH=univention/ucslint:…`, so the `import univention.ucslint` imports from somewhere else where-ever it finds it on you following `$PYTHONPATH`s: There is no `./univention/ucslint/univention/ucslint/__init__.py` to import.

- With `python3 -m univention.ucslint.main` you again get `PYTHONPATH=.:…` prepended, which again works as expected.

See <https://docs.python.org/3/library/sys.html#sys.path>

Directly invoking a script via `python3 ./univention/ucslint/main.py` — which basically is the same as `./univention/ucslint/main.py` due to its hash-bash `#!/usr/bin/python3` — has the draw-back, that you sometimes import modules twice: Once as `univention.ucslint.main` and once as `__main__`, which for example breaks "identity tests": `__main__ is not univention.ucslint.main`.

So for backward compatibility (and ease of use) we should have a "./ucslint" wrapper with just `#!/bin/sh\nexec env python3 -m univention.ucslint.main "$@"` or your `#!/usr/bin/env python3\nfrom univention.ucslint.main import main\nmain()`.


> If that is fixed, we can think about reverting e5e27022bede.

Please check your `$PYTHONPATH` for any other reference to an older version of ucslint first. Then decide on how to proceed from there.