Automates jobs in a ScaleBot Lab.
- ops ›
What is ScaleBot?
ScaleBot is a framework for running scheduled jobs against a cloud. It is built upon the Jenkins CI and Juju modeling tools.
QuickStart (using Amazon Web Services)
Clone a local copy of the sample ScaleBot configuration repository:
$ git clone git://git.launchpad.net/~scalebot-team/+git/scalebot-config
Make a copy of the aws credentials template, and add your account keys:
$ cd scalebot-config $ cp labs/aws/credentials.yaml.template labs/aws/credentials.yaml $ vi labs/aws/credentials.yaml
Deploy the ScaleBot bundle. In this example, we'll deploy to local LXD containers:
$ juju bootstrap localhost Creating Juju controller "localhost-localhost" on localhost/localhost Looking for packaged Juju agent version 2.2.4 for amd64 To configure your system to better support LXD containers, please see: https://github.com/lxc/lxd/blob/master/doc/production-setup.md Launching controller instance(s) on localhost/localhost... - juju-689661-0 (arch=amd64) Fetching Juju GUI 2.9.2 Waiting for address Attempting to connect to 10.98.200.35:22 Bootstrap agent now started Contacting Juju controller at 10.98.200.35 to verify accessibility... Bootstrap complete, "localhost-localhost" controller now available. Controller machines are in the "controller" model. Initial model "default" added. $ juju deploy bundle.yaml Deploying charm "cs:xenial/jenkins-4" Deploying charm "cs:~ce-hyperscale/scalebot-jenkins-3" Related "jenkins:extension" and "scalebot:extension" Deploy of bundle completed.
Once deployment has completed, use
juju status to find the IP of your scalebot jenkins server. You can connect to it at http://ip-of-server:8080. The sample configuration includes an example "Hello World" job. Building this job will deploy a node in the Lab cloud and run
echo "Hello World" on it.
ScaleBot is deployed from a Juju charm that builds a running instance of Jenkins CI, initialized with a git repository that provides job definitions and a target cloud to run those jobs against. We refer to this target cloud as a "lab". ScaleBot will use the target cloud ("lab") configuration to automatically setup an internal Juju controller which Jenkins jobs can use to deploy systems and workloads as necessary within the lab.
Of course, since scalebot-jenkins is a charm itself, it also is deployed in a cloud. This can just be a LXD cloud though, as the controller and Jenkins server can easily run in containers. The important thing is that the jenkins server in the ScaleBot cloud is on a network that can access the Lab cloud.
The scalebot-jenkins charm accepts configuration that tells ScaleBot where to fetch Jenkins job and Lab Cloud definitions. This data must be in a git repository with the following layout:
jobs/<jobfoo>.yaml jobs/<jobbar>.yaml ... jobs/<jobbaz>.yaml labs/<labname>/<labname>.yaml labs/<labname>/clouds.yaml labs/<labname>/credentials.yaml labs/<labname>/model-defaults.yaml scalebot.d/init scalebot.d/refresh
Jobs must be in the YAML format used by Jenkins Job Builder, as that is the tool used to load them into Jenkins. The clouds.yaml, credentials.yaml, and model-defaults.yaml files are passed directly to Juju when bootsrapping the Lab controller, so see Juju documentation for details on their contents.
You may also want to store code in this repository and call it from your jobs.
ScaleBot will automatically create an empty model prior to building a jenkins job. This model will have a unique name, based on the $BUILD_ID. Jobs can assume that this model exists and is in-focus prior to executing, so your build code can just begin deploying charms immediately on startup.
Since it is expected that builds will deploy systems using Juju, the python bindings for juju are pre-installed on the system.
In addition to the environment variables set by Jenkins, the following variable is also available:
$SCALEBOT_REPO will be set to the path of the locally cloned copy of the Scalebot configuration repository.
If it exists,
$SCALEBOT_REPO/scalebot.d/init will be executed each time the configuration repository is refreshed. You can use this to install any additional dependencies needed for your jobs.
If it exists,
$SCALEBOT_REPO/scalebot.d/refresh will be executed after the repository is updated, and before loading jobs. This is useful in case something job yaml files need to be dynamically generated.
- (string) Space separated list of extra deb packages to install.
- (string) List of signing keys for install_sources package sources, per charmhelpers standard format (a yaml list of strings encoded as a string). The keys should be the full ASCII armoured GPG public keys. While GPG key ids are also supported and looked up on a keyserver, operators should be aware that this mechanism is insecure. null can be used if a standard package signing key is used that will already be installed on the machine, and for PPA sources where the package signing key is securely retrieved from Launchpad.
- - null
- (string) List of extra apt sources, per charm-helpers standard format (a yaml list of strings encoded as a string). Each source may be either a line that can be added directly to sources.list(5), or in the form ppa:<user>/<ppa-name> for adding Personal Package Archives, or a distribution component to enable.
- - ppa:juju/stable
- (string) The status of service-affecting packages will be set to this value in the dpkg database. Valid values are "install" and "hold".
- (string) The branch of $scalebot_repo to checkout
- (string) URI to a ScaleBot config git repository
- (string) Desired jobset to import jenkins jobs from
- (string) Bootstrap constraints for Juju controller
- (string) Juju cloud config YAML for target cloud
- (string) Credentials YAML for Juju cloud defined in scalebot_juju_clouds
- (string) Juju model defaults configuration YAML
- (string) A null-passphrase SSH private key with access to $scalebot_repo
- (boolean) Is this a production (vs. a development) deployment? When set to true, tests will find SCALEBOT_PRODUCTION=1 in their environment. This gives tests an option to behave differently in production vs. development environments.