Using Kubernetes with Juju

Kubernetes (“K8s”) provides a flexible architecture for managing containerised applications at scale. Juju can be used to create a Kubernetes cluster and to manage workloads on top of a live cluster.

This page gives an overview of how an existing Kubernetes cluster can be integrated with Juju.

Although this page is not about showing how to install Kubernetes itself, we do give pointers to a few approaches for doing so.

Juju Kubernetes-specific workflow

Essentially, Juju is able to treat the added cluster as it does any other of its known clouds (i.e. create models and deploy charms). There are some differences to working with such a cloud and they are called out in this section.

The k8s-specific Juju commands are add-k8s, remove-k8s, and scale-application. All other concepts and commands are applied in the traditional manner.

The add-k8s command makes the usual combination of add-cloud and add-credential unnecessary.

In v.2.6.1, the add-k8s command can be used to add the Kubernetes cluster and its credentials to the client’s local cache (option --local) or to add these things directly to a running controller (the default behaviour). There are also options --aks and --gke for streamlining the addition of AKS and GKE clusters.

In v.2.5.0, only local behaviour is possible (there is no --local option). The cluster configuration file will also first need to be copied to ~/.kube/config.

User credentials can still be added by way of the add-credential or autoload-credentials commands. Also, at any time, the k8s CLI can be used to add a new user to the cluster.

The add-k8s command can repeatedly set up different clusters as long as the contents of the configuration file has been changed accordingly. The KUBECONFIG environment variable is useful here as it will be honoured by Juju when finding the file to load.

The remove-k8s command is used to remove a Kubernetes cluster from Juju’s list of known clouds.

The scale-application command is used to scale a Kubernetes cluster. The add-unit and remove-unit commands do not apply to a Kubernetes model.

Charms need to be written specifically for Kubernetes workloads. See tutorial Understanding Kubernetes charms.

Running Kubernetes workloads

First off, a Kubernetes cluster will be required. Essentially, you will use it as you would any other cloud that Juju interacts with: the cluster becomes the backing cloud.

The following steps describe the general approach for using Kubernetes with Juju:

  1. Obtain a Kubernetes cluster
  2. Add the cluster
  3. Add a model
  4. Create persistent static storage (if necessary)
  5. Create storage pools
  6. Deploy a Kubernetes charm

Obtain a Kubernetes cluster

There are many ways to obtain a Kubernetes cluster. Here is a list of suggestions:

Kubernetes bundles do not work well on a LXD cloud at this time. Refer to Deploying on LXD for details.

Add the cluster

Juju needs information about the cluster in order to add it. This information is part of the main Kubernetes configuration file which can be found on the Kubernetes master node. This is the same config file that kubectl uses. Copy this configuration file to your local machine and save it as ~/.kube/config. The add-k8s command will parse that file and add the cluster as a cloud.

In v.2.6.1 clusters based on AKS and GKE can be added without the cluster configuration file. These types have special helper modes that enable their respective, independently installed, CLI toolkits and are accessible via options --aks and --gke.

In v.2.6.1 the add-k8s command can apply to the client’s local cache of clouds or to an existing controller. This works similar to the add-cloud command; see the Adding clouds section.

The conjure-up installer adds the cluster automatically.

Add a model

After having added the cluster to the controller, you can create k8s models on that controller using juju add-model <k8s-cloud-name>. This will create a Kubernetes namespace in the cluster which is named after the model name. This namespace will host all of the pods and other resources of that model. The model also has a default storage pool called “kubernetes”.

Create persistent static storage

Create persistent static volumes for operator storage if your chosen cloud’s storage is not supported natively by Kubernetes. You will need to do the same for workload storage if your charm has storage requirements. This is done with the Kubernetes tool kubectl.

Create storage pools

Create storage pools for operator storage and, if needed, workload storage. This is done in the usual way, with the create-storage-pool command.

Deploy a Kubernetes charm

A Kubernetes-specific charm is deployed in standard fashion, with the deploy command. If the charm has storage requirements you will need to specify them, as you do with a normal charm.

Configuration

The below table lists configuration keys supported by Kubernetes charms that are set at deploy time. The corresponding Kubernetes meaning can be obtained from the Kubernetes documentation for Services and Ingress.

Key Type Default
kubernetes-service-type string ClusterIP
kubernetes-service-external-ips string []
kubernetes-service-target-port string <container port>
kubernetes-service-loadbalancer-ip string ""
kubernetes-service-loadbalancer-sourceranges string "[]"
kubernetes-service-externalname string ""
kubernetes-ingress-class string nginx
kubernetes-ingress-ssl-redirect boolean false
kubernetes-ingress-ssl-passthrough boolean false
kubernetes-ingress-allow-http boolean false

For example:

juju deploy mariadb-k8s --config kubernetes-service-loadbalancer-ip=10.1.1.1

There are two other keys that are not Kubernetes-specific:

Key Type Default
juju-external-hostname string ""
juju-application-path string "/"

Keys ‘juju-external-hostname’ and ‘juju-application-path’ control how the application is exposed externally using a Kubernetes Ingress Resource in conjunction with the configured ingress controller (default: nginx).

Storage theory

Refer to the Persistent storage and Kubernetes page for the theory on how Juju works with Kubernetes storage.

Tutorials and in-depth guides

The following practical guides are available:

Last updated a month ago. Help improve this document in the forum.