kubernetes worker #3
Description
Kubernetes is an open-source platform for deploying, scaling, and operations of application containers across a cluster of hosts. Kubernetes is portable in that it works with public, private, and hybrid clouds. Extensible through a pluggable infrastructure. Self healing in that it will automatically restart and place containers on healthy nodes if a node ever goes away.
- Tags:
- misc ›
- containers ›
- layer ›
Kubernetes Worker
Usage
This charm deploys a container runtime, and additionally stands up the Kubernetes worker applications: kubelet, and kube-proxy.
In order for this charm to be useful, it should be deployed with its companion charm kubernetes-master and linked with an SDN-Plugin.
This charm has also been bundled up for your convenience so you can skip the above steps, and deploy it with a single command:
juju deploy canonical-kubernetes
For more information about Canonical Kubernetes
consult the bundle README.md
file.
Scale out
To add additional compute capacity to your Kubernetes workers, you may
juju add-unit
scale the cluster of applications. They will automatically
join any related kubernetes-master, and enlist themselves as ready once the
deployment is complete.
Operational actions
The kubernetes-worker charm supports the following Operational Actions:
Pause
Pausing the workload enables administrators to both drain and cordon a unit for maintenance.
Resume
Resuming the workload will uncordon a paused unit. Workloads will automatically migrate unless otherwise directed via their application declaration.
Private registry
With the "registry" action that is part for the kubernetes-worker charm, you can very easily create a private docker registry, with authentication, and available over TLS. Please note that the registry deployed with the action is not HA, and uses storage tied to the kubernetes node where the pod is running. So if the registry pod changes is migrated from one node to another for whatever reason, you will need to re-publish the images.
Example usage
Create the relevant authentication files. Let's say you want user userA
to authenticate with the password passwordA
. Then you'll do :
echo -n "userA:passwordA" > htpasswd-plain
htpasswd -c -b -B htpasswd userA passwordA
(the htpasswd
program comes with the apache2-utils
package)
Supposing your registry will be reachable at myregistry.company.com
, and that you already have your TLS key in the registry.key
file, and your TLS certificate (with myregistry.company.com
as Common Name) in the registry.crt
file, you would then run :
juju run-action kubernetes-worker/0 registry domain=myregistry.company.com htpasswd="$(base64 -w0 htpasswd)" htpasswd-plain="$(base64 -w0 htpasswd-plain)" tlscert="$(base64 -w0 registry.crt)" tlskey="$(base64 -w0 registry.key)" ingress=true
If you then decide that you want do delete the registry, just run :
juju run-action kubernetes-worker/0 registry delete=true ingress=true
Known Limitations
Kubernetes workers currently only support 'phaux' HA scenarios. Even when configured with an HA cluster string, they will only ever contact the first unit in the cluster map. To enable a proper HA story, kubernetes-worker units are encouraged to proxy through a kubeapi-load-balancer application. This enables a HA deployment without the need to re-render configuration and disrupt the worker services.
External access to pods must be performed through a Kubernetes Ingress Resource.
When using NodePort type networking, there is no automation in exposing the ports selected by kubernetes or chosen by the user. They will need to be opened manually and can be performed across an entire worker pool.
If your NodePort service port selected is 30510
you can open this across all
members of a worker pool named kubernetes-worker
like so:
juju run --application kubernetes-worker open-port 30510/tcp
Don't forget to expose the kubernetes-worker application if its not already exposed, as this can cause confusion once the port has been opened and the service is not reachable.
Note: When debugging connection issues with NodePort services, its important to first check the kube-proxy service on the worker units. If kube-proxy is not running, the associated port-mapping will not be configured in the iptables rulechains.
If you need to close the NodePort once a workload has been terminated, you can follow the same steps inversely.
juju run --application kubernetes-worker close-port 30510
Configuration
- allow-privileged
- (string) Allow privileged containers to run on worker nodes. Supported values are "true", "false", and "auto". If "true", kubelet will run in privileged mode by default. If "false", kubelet will never run in privileged mode. If "auto", kubelet will not run in privileged mode by default, but will switch to privileged mode if gpu hardware is detected.
- auto
- channel
- (string) Snap channel to install Kubernetes worker services from
- 1.10/stable
- cuda_repo
- (string) The cuda-repo package version to install.
- 9.1.85-1
- default-backend-image
- (string) Docker image to use for the default backend. Auto will select an image based on architecture.
- auto
- docker-ce-package
- (string) The pined version of docker-ce package installed with nvidia-docker.
- docker-ce=17.12.1~ce-0~ubuntu
- docker-logins
- (string) Docker login credentials. Setting this config allows Kubelet to pull images from registries where auth is required. The value for this config must be a JSON array of credential objects, like this: [{"server": "my.registry", "username": "myUser", "password": "myPass"}]
- []
- docker-opts
- (string) Extra options to pass to the docker daemon. e.g. --insecure-registry
- docker_runtime
- (string) docker runtime to install valid values are "upstream" (docker PPA), "nvidia" (nvidia PPA), "apt" (ubuntu archive), or "auto" (nvidia PPA or ubuntu archive, based on your hardware)
- auto
- enable-cgroups
- (boolean) Enable GRUB cgroup overrides cgroup_enable=memory swapaccount=1. WARNING changing this option will reboot the host - use with caution on production services
- http_proxy
- (string) URL to use for HTTP_PROXY to be used by Docker. Only useful in closed environments where a proxy is the only option for routing to the registry to pull images
- https_proxy
- (string) URL to use for HTTPS_PROXY to be used by Docker. Only useful in closed environments where a proxy is the only option for routing to the registry to pull images
- ingress
- (boolean) Deploy the default http backend and ingress controller to handle ingress requests.
- True
- kubelet-extra-args
- (string) Space separated list of flags and key=value pairs that will be passed as arguments to kubelet. For example a value like this: runtime-config=batch/v2alpha1=true profiling=true will result in kube-apiserver being run with the following options: --runtime-config=batch/v2alpha1=true --profiling=true
- labels
- (string) Labels can be used to organize and to select subsets of nodes in the cluster. Declare node labels in key=value format, separated by spaces.
- nagios_context
- (string) Used by the nrpe subordinate charms. A string that will be prepended to instance name to set the host name in nagios. So for instance the hostname would be something like: juju-myservice-0 If you're running multiple environments with the same services in them this allows you to differentiate between them.
- juju
- nagios_servicegroups
- (string) A comma-separated list of nagios servicegroups. If left empty, the nagios_context will be used as the servicegroup
- nginx-image
- (string) Docker image to use for the nginx ingress controller. Auto will select an image based on architecture.
- auto
- no_proxy
- (string) Comma-separated list of destinations (either domain names or IP addresses) that should be directly accessed, by opposition of going through the proxy defined above. Must be less than 2023 characters long
- nvidia-container-runtime-package
- (string) The pined version of nvidia-container-runtime package.
- nvidia-container-runtime=2.0.0+docker17.12.1-1
- nvidia-docker-package
- (string) The pined version of nvidia-docker2 package.
- nvidia-docker2=2.0.3+docker17.12.1-1
- proxy-extra-args
- (string) Space separated list of flags and key=value pairs that will be passed as arguments to kube-proxy. For example a value like this: runtime-config=batch/v2alpha1=true profiling=true will result in kube-apiserver being run with the following options: --runtime-config=batch/v2alpha1=true --profiling=true
- require-manual-upgrade
- (boolean) When true, worker services will not be upgraded until the user triggers it manually by running the upgrade action.
- True
- snap_proxy
- (string) HTTP/HTTPS web proxy for Snappy to use when accessing the snap store.
- snap_proxy_url
- (string) The address of a Snappy Enterprise Proxy to use for snaps e.g. http://snap-proxy.example.com