Bug 40461 - CLI tool to check basic system health
CLI tool to check basic system health
Status: VERIFIED FIXED
Product: UCS
Classification: Unclassified
Component: Sysinfo
UNSTABLE
Other Linux
: P5 enhancement (vote)
: UCS 4.1-x
Assigned To: Felix Botner
Arvid Requate
:
Depends on: 34765
Blocks: 46005 45340 47215
  Show dependency treegraph
 
Reported: 2016-01-19 20:11 CET by Arvid Requate
Modified: 2018-06-20 11:19 CEST (History)
7 users (show)

See Also:
What kind of report is it?: Development Internal
What type of bug is this?: ---
Who will be affected by this bug?: ---
How will those affected feel about the bug?: ---
User Pain:
Enterprise Customer affected?:
School Customer affected?:
ISV affected?:
Waiting Support:
Ticket number:
Bug group (optional): Troubleshooting
Max CVSS v3 score:


Attachments
univention-check-basic-system-health (1.09 KB, application/x-shellscript)
2016-01-19 20:11 CET, Arvid Requate
Details
univention-check-basic-system-health (2.73 KB, application/x-shellscript)
2016-01-21 21:17 CET, Arvid Requate
Details
univention-check-basic-system-health (4.19 KB, application/x-shellscript)
2016-01-28 18:14 CET, Arvid Requate
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Arvid Requate univentionstaff 2016-01-19 20:11:13 CET
Created attachment 7420 [details]
univention-check-basic-system-health

While UMC offers a system diagnostic module it would be good to

* Cover additional issues

* Make the tool callable via CLI

* The CLI tool could offer to encrypt the information for univention support
  (into a file)


I guess one could use the plugin infrastructure

/usr/share/pyshared/univention/management/console/modules/diagnostic/plugins/

A collection of simple tests is attached as a shell script.
Comment 1 Arvid Requate univentionstaff 2016-01-21 21:17:42 CET
Created attachment 7423 [details]
univention-check-basic-system-health

New Version of univention-check-basic-system-health:

* check_samba_processes
* check_s4_connector_listener_active
* check_for_temporary_udm_sids
Comment 2 Arvid Requate univentionstaff 2016-01-28 16:38:15 CET
Ideally this tool would also pull and run all 00_checks from the package ucs-test-checks.
Comment 3 Arvid Requate univentionstaff 2016-01-28 17:29:52 CET
When this tool has been released, it needs to be added to this SDB article:

 http://sdb.univention.de/1235 - Samba 4 Troubleshooting Guide
Comment 4 Arvid Requate univentionstaff 2016-01-28 18:14:08 CET
Created attachment 7436 [details]
univention-check-basic-system-health

New features:

* check Kerberos auth for DDNS update
* check Kerberos auth via krb5.keytab against local smbd
* now also works fur UCS 4.0
* improved error reporting
Comment 5 Arvid Requate univentionstaff 2017-05-16 21:03:21 CEST
I've added an example CLI script here:

svn/dev/trunk/internal/univention-system-check

By default it runs all executables below ./univention-system-check.d

There is an option to specify a glob pattern of sections like:
  ./univention-system-check -s basic*

By default it only logs errors to stdout but it logs all script output to ./univention-system-check.log

Now we should add more check scripts into suitable sections.


Once it becomes useful enough we should create a tarball from it, sign it with the support key and make it downloadable somewhere for customers. That way it should be easily usable in support cases as well as in product tests.
Comment 6 Felix Botner univentionstaff 2017-06-21 16:41:17 CEST
added 

-> more 01_univention_system_check
#!/usr/share/ucs-test/runner bash
## desc: Check system status with univention-system-check
## tags:
##  - basic
## exposure: safe

. "$TESTLIBPATH/base.sh" || exit 137

RETVAL=100

curl -OOs https://updates.software-univention.de/download/univention-system-check/univention-system-check.tar.gz{,.gpg}
gpgv \
  --keyring /usr/share/keyrings/univention-archive-key-ucs-4x.gpg \
  univention-system-check.tar.gz.gpg \
  univention-system-check.tar.gz \
  && tar -xzf  univention-system-check.tar.gz \
  && python ./univention-system-check
if [ $? != 0 ]; then
	RETVAL=110
fi

exit $RETVAL


to ucs-test-checks
Comment 7 Stefan Gohmann univentionstaff 2017-06-28 08:23:53 CEST
r80540:

samba/check_s4_connector_rejects.sh: Don't check for list-rejected on a DC Backup which don't have an active S4 connector instance. This results in the following error message: 
[2017-06-28 00:45:52.657607] Test failed: univention-system-check.d/samba/check_s4_connector_rejects.sh
[2017-06-28 00:45:52.657618] + '[' -e /usr/lib/nagios/plugins/check_univention_s4_connector ']'
[2017-06-28 00:45:52.657628] + univention-s4connector-list-rejected
[2017-06-28 00:45:52.657637] Traceback (most recent call last):
[2017-06-28 00:45:52.657646] File /usr/sbin/univention-s4connector-list-rejected, line 73, in <module>
[2017-06-28 00:45:52.657656] import mapping
[2017-06-28 00:45:52.657666] ImportError: No module named mapping

(Bug #40461)
Comment 8 Stefan Gohmann univentionstaff 2017-06-28 08:38:04 CEST
r80541: 
 samba/check_smbclient_via_krb5_keytab.sh: Limit this test to Samba 4 DCs (Bug #40461)
Comment 9 Stefan Gohmann univentionstaff 2017-06-28 10:56:17 CEST
r80542:

samba/check_s4_connector_autostart.sh: Added a check if the S4 Connector service entry matches the daemon status (Bug #40461 and Ticket #2017062721000761)
Comment 10 Stefan Gohmann univentionstaff 2017-06-29 07:19:40 CEST
(In reply to Arvid Requate from comment #4)
> Created attachment 7436 [details]
> univention-check-basic-system-health
> 
> New features:
> 
> * check Kerberos auth for DDNS update

At least in Jenkins the DDNS test fails on a member server which is joined into a S4 domain:

-----------------------------------------------------------------------------
[2017-06-29 00:21:41.111123] Test failed: univention-system-check.d/samba/check_ddns_update.sh
[2017-06-29 00:21:41.111131] 	+ . /usr/share/univention-lib/ucr.sh
[2017-06-29 00:21:41.111139] 	++ ucr shell hostname domainname samba4/role
[2017-06-29 00:21:41.111147] 	+ eval 'domainname=autotest097.local
[2017-06-29 00:21:41.111156] 	hostname=member097'
[2017-06-29 00:21:41.111163] 	++ domainname=autotest097.local
[2017-06-29 00:21:41.111171] 	++ hostname=member097
[2017-06-29 00:21:41.111179] 	+ grep -q '^dn:'
[2017-06-29 00:21:41.111187] 	+ univention-ldapsearch 'univentionService=S4 Connector' dn
[2017-06-29 00:21:41.111195] 	+ trivial_ddns_update_by_machine_principal
[2017-06-29 00:21:41.111203] 	+ local rc
[2017-06-29 00:21:41.111210] 	+ kdestroy
[2017-06-29 00:21:41.111218] 	+ kinit --password-file=/etc/machine.secret 'member097$'
[2017-06-29 00:21:41.111225] 	+ nsupdate -g
[2017-06-29 00:21:41.111234] 	+ echo -e 'server member097.autotest097.local\nprereq yxdomain autotest097.local\n'
[2017-06-29 00:21:41.111262] 	; Communication with 10.210.47.48#53 failed: timed out
[2017-06-29 00:21:41.111272] 	could not talk to specified name server
-----------------------------------------------------------------------------
Comment 11 Tobias Birkefeld univentionstaff 2017-07-15 12:11:41 CEST
r81194:
check_for_temporary_udm_sids.sh: Limit the test to check only relevant objects for temporary sids
Comment 12 Stefan Gohmann univentionstaff 2017-08-30 14:33:16 CEST
r82540:
 Added replication check (Bug #40461)
Comment 13 Arvid Requate univentionstaff 2018-04-16 19:58:19 CEST
I guess we have that, so let's close this bug?
Comment 14 Felix Botner univentionstaff 2018-04-24 17:08:47 CEST
done
Comment 15 Arvid Requate univentionstaff 2018-04-26 17:07:49 CEST
Ok, works.