This charm will deploy a basic environment to run DEP8 tests (with
autopkgtest) on the target machine, VM or container. Basic idea is to
reuse the tests already available in the packages and run them, using
Juju to automate target selection and deployment. It is assumed the
target environment is "disposable", i.e. it will released or recreated
once the tests are finished.
This charm is a thin automation layer to run DEP8 tests in a target environment.
This charm will deploy a basic environment to run DEP8 tests (with autopkgtest) on the target MAAS-provisioned machine, VM or container. Packages will be build and run in the target itself to improve test coverage.
Basic idea is to reuse the tests already available in the Debian packages to validate new hardware configurations and run these tests in existing infrastructure while using Juju to abstract the nature of the target environment. It will not matter, for the charm, if the test is running in a VM or in bare metal, but the intention is run them on metal to minimize the interference of virtualization providers on test results.
The test run directly on the target environment using the "null" autopkgtest virtualization provider and it is assumed the environment is "disposable", i.e. it will released or recreated once the tests are finished.
This charm is intended to be deployed and used through test automation but, in development enviroments, the deployment procedure is:
Install the charm with
$ juju deploy cs:~ce-hyperscale/autopkgtest testbed1.
Notice that this example assumes a testbed named "testbed1".
Run a test with the Juju action
run-test, passing the Debian package name in argument
$ juju run-action testbed1/0 run-test pkgname=nginx Action queued with id: d13d1822-724f-4719-8ed1-35dd0d124474
Use the action-id to check if the execution is already finished:
$ juju show-action-status d13d1822-724f-4719-8ed1-35dd0d124474 actions: - action: run-test completed at: n/a id: d13d1822-724f-4719-8ed1-35dd0d124474 status: running unit: testbed1/0
juju debug-logto see the outputs of the test (stdout and stderr) in realtime;
Once the test is finished, grab the output with:
$ juju show-action-output d13d1822-724f-4719-8ed1-35dd0d124474 results: output-file: /tmp/test-nginx-kz6TT/output.txt retcode: "0" summary-file: /tmp/test-nginx-kz6TT/summary.txt status: completed timing: completed: 2019-10-22 21:54:33 +0000 UTC enqueued: 2019-10-22 21:48:01 +0000 UTC started: 2019-10-22 21:48:06 +0000 UTC
Optionally, run the action with the "--wait" command line switch. This spares the user from polling the test state, but blocks the Juju client process.
The exit code returned by
autopkgtest will be saved in variable
retcode (see autpkgtest manpage for more information on it meaning). This only happens if
autopkgtest gets a chance to run and won't be created if the test script fails before this step.
output will have paths to files containing the summary of the test results and a copy of stdout and stderr of the test execution. Use
juju ssh to see their contents or copy them to the host machine.
$ juju ssh testbed1/0 cat /tmp/test-nginx-kz6TT/summary.txt light-simple PASS core-simple PASS lua PASS full-simple PASS extras-simple PASS full-module-deps PASS light-module-deps PASS core-module-deps PASS extras-module-deps PASS Connection to 10.214.19.195 closed.
Notice that file output.txt can be pretty large.
Known Limitations and Issues
Some tests may refuse to run in containers;
Autopkgtest has a protection against running tests that can break the testbed (restriction
breaks-testbed). It does not matter for us as the target will be recreated anyway, but these tests will require patching to force them to run.