autopkgtest #1
Description
This charm will deploy a basic environment to run DEP8 tests (with autopkgtest) on the target machine, VM or container. Basic idea is to reuse the tests already available in the packages and run them, using Juju to automate target selection and deployment. It is assumed the target environment is "disposable", i.e. it will released or recreated once the tests are finished.
Overview
This charm is a thin automation layer to run DEP8 tests in a target environment.
This charm will deploy a basic environment to run DEP8 tests (with autopkgtest) on the target MAAS-provisioned machine, VM or container. Packages will be build and run in the target itself to improve test coverage.
Basic idea is to reuse the tests already available in the Debian packages to validate new hardware configurations and run these tests in existing infrastructure while using Juju to abstract the nature of the target environment. It will not matter, for the charm, if the test is running in a VM or in bare metal, but the intention is run them on metal to minimize the interference of virtualization providers on test results.
The test run directly on the target environment using the "null" autopkgtest virtualization provider and it is assumed the environment is "disposable", i.e. it will released or recreated once the tests are finished.
Usage
This charm is intended to be deployed and used through test automation but, in development enviroments, the deployment procedure is:
-
Install the charm with
$ juju deploy cs:~ce-hyperscale/autopkgtest testbed1
. Notice that this example assumes a testbed named "testbed1". -
Run a test with the Juju action
run-test
, passing the Debian package name in argumentpkgname
:$ juju run-action testbed1/0 run-test pkgname=nginx Action queued with id: d13d1822-724f-4719-8ed1-35dd0d124474
-
Use the action-id to check if the execution is already finished:
$ juju show-action-status d13d1822-724f-4719-8ed1-35dd0d124474 actions: - action: run-test completed at: n/a id: d13d1822-724f-4719-8ed1-35dd0d124474 status: running unit: testbed1/0
-
Use
juju debug-log
to see the outputs of the test (stdout and stderr) in realtime; -
Once the test is finished, grab the output with:
$ juju show-action-output d13d1822-724f-4719-8ed1-35dd0d124474 results: output-file: /tmp/test-nginx-kz6TT/output.txt retcode: "0" summary-file: /tmp/test-nginx-kz6TT/summary.txt status: completed timing: completed: 2019-10-22 21:54:33 +0000 UTC enqueued: 2019-10-22 21:48:01 +0000 UTC started: 2019-10-22 21:48:06 +0000 UTC
Optionally, run the action with the "--wait" command line switch. This spares the user from polling the test state, but blocks the Juju client process.
Interpreting outputs
The exit code returned by autopkgtest
will be saved in variable retcode
(see autpkgtest manpage for more information on it meaning). This only happens if autopkgtest
gets a chance to run and won't be created if the test script fails before this step.
Variables summary
and output
will have paths to files containing the summary of the test results and a copy of stdout and stderr of the test execution. Use juju ssh
to see their contents or copy them to the host machine.
$ juju ssh testbed1/0 cat /tmp/test-nginx-kz6TT/summary.txt
light-simple PASS
core-simple PASS
lua PASS
full-simple PASS
extras-simple PASS
full-module-deps PASS
light-module-deps PASS
core-module-deps PASS
extras-module-deps PASS
Connection to 10.214.19.195 closed.
Notice that file output.txt can be pretty large.
Known Limitations and Issues
-
Some tests may refuse to run in containers;
-
Autopkgtest has a protection against running tests that can break the testbed (restriction
breaks-testbed
). It does not matter for us as the target will be recreated anyway, but these tests will require patching to force them to run.