Bug 43218 - Add automated tests for the installation procedure
Add automated tests for the installation procedure
Status: REOPENED
Product: UCS Test
Classification: Unclassified
Component: UCS Installer
unspecified
Other Linux
: P5 normal (vote)
: ---
Assigned To: UCS maintainers
:
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2016-12-19 17:37 CET by Alexander Kläser
Modified: 2018-04-14 14:02 CEST (History)
4 users (show)

See Also:
What kind of report is it?: Development Internal
What type of bug is this?: ---
Who will be affected by this bug?: ---
How will those affected feel about the bug?: ---
User Pain:
Enterprise Customer affected?:
School Customer affected?:
ISV affected?:
Waiting Support:
Flags outvoted (downgraded) after PO Review:
Ticket number:
Bug group (optional):
Max CVSS v3 score:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Alexander Kläser univentionstaff 2016-12-19 17:37:22 CET
It would be helpful to provide a test framework for testing the graphical Debian installer + UMC Setup.
Comment 1 Alexander Kläser univentionstaff 2016-12-20 19:39:40 CET
I added a test script to ucs-4.1/ucs-4.1-4/test/ucs-gui-tests/installation:

r75449 | Bug #43218: add installation test

(I unintentionally made a change in univention-appcenter [r75449] which I reverted with [r75450].)


I added to /dev/trunk/jenkins the package vnc-automate which provides all VNC and OCR functionality needed in the above mentioned test script:

vnc-automate (0.0.1-3):
r75415 | Bug #43218: initial release of python-vnc-automate
r75444 | Bug #43218: fix some import errors
r75443 | Bug #43218: added '--with python_support'
r75441 | Bug #43218: rename package

The package has been built in the 4.1 internal scope.


I also imported and built the pillow package for python (which is needed for image processing functions) into the 4.1 internal scope. I needed to supply a patch in order to make the package compile correctly:

r17036 | Bug #43218: move patch for setup.py into .quilt file
r17035 | Bug #43218: fix directories in patch
r17034 | Bug #43218: add patch for build of package pillow


In order to run a test script, one needs to install python-vnc-automate from the internal 4.1 scope and have a VNC connection ready. Execute the test script with --help for more information. Here is a sample call:

> test-installation-master-DE.py --dump-dir dump-images/  vnc.server.local:1
Comment 2 Florian Best univentionstaff 2017-08-17 15:53:39 CEST
I copied this to UCS 4.2-1:
r82358 | Bug #43218: copy to UCS 4.2-1
Comment 3 Florian Best univentionstaff 2017-08-17 19:13:08 CEST
I created a Jenkins Job for UCS 4.2-1:
http://jenkins.knut.univention.de:8080/job/UCS-4.2/job/UCS-4.2-1/job/Installer%20Test/

The jenkins job has a matrix of the languages fr-FR, de-DE, en-US and parameter for the ISO-Image, an IP address range, and the KVM server to use.

The jenkins job starts a wrapped py.test which executes all the tests from UCS 4.1-4 sequentially.

I migrated the base system test to py-test.

TODO: all the other tests need to be adjusted in that similar manner.

r82366 | Bug #43218: create directory only if not existant
r82365 | Bug #43218: fix missing with statement
r82364 | Bug #43218: Create random IP Address; Use ISO Image from parameter
r82363 | Bug #43218: Migrate 40_base-system-*.py to pytest
r82361 | Bug #43218: outsource jenkins jobs into pytest files
r82360 | Bug #43218: rename file because it conflicts with pytest naming scheme
r82359 | Bug #43218: add pytest skeleton
Comment 4 Florian Best univentionstaff 2017-08-17 19:25:50 CEST
I migrated another test:
r82367 | Bug #43218: Migrate test_master_with_different_languages.py
Comment 5 Florian Best univentionstaff 2017-08-18 16:25:17 CEST
r82375 | Bug #43218: Migrate test_master_with_multiple_hdds_and_nics
r82374 | Bug #43218: Migrate test_master_with_multiple_partitions
r82373 | Bug #43218: Migrate test_master_with_hard_password
r82372 | Bug #43218: Migrate test_master_with_additional_software_components