Elasticsearch is a search server based on Lucene. It provides a distributed, multitenant-capable full-text search engine with a RESTful web interface and schema-free JSON documents. This package contains the infrastructure needed to an ElasticSearch node.
This ElasticSearch charm is for 99% based on the charm of James Beedy.
What we have changed is the
extra_bindings settings and which IP the charm now
distribute over its relation. It only uses the machine's private IP. We noticed
problems while deploying the ELK stack on GCE. It uses the FAN IP addresses to
pass over the client relation, however the client (Kibana in this case) couldn't
Elasticsearch is a flexible and powerful open source, distributed, real-time search and analytics engine. Architected from the ground up for use in distributed environments where reliability and scalability are must haves, Elasticsearch gives you the ability to move easily beyond simple full-text search. Through its robust set of APIs and query DSLs, plus clients for the most popular programming languages, Elasticsearch delivers on the near limitless promises of search technology.
Excerpt from elasticsearch.org
This charm can be used to deploy Elasticsearch in any way that is suppored by upstream Elasticsearch, and then some.
The charm configuration exposes an option called
node-type, which is used by the charm to know what type of
Elasticsearch node to configure. In order to orchestrate this charm to facilitate more complex deployments,
you must configure the node-type on a per application basis.
The options for the
node-type config can be explained as follows:
all- The node will assume all roles of Elasticsearch, there will be no difference in configuration from one node to the next.
master- The node will assume the 'master' node-type. Master nodes will wait for the number of peers to be >= the charm configuration option
minn-master-count(this defaults to 1) before bootstrapping the cluster.
coordinator- The node will assume the 'coordinator' node-type. Coordinator nodes will wait until they have a relation to the master before joining the cluster.
data- The node will assume the 'data' node-type. Data nodes will wait until they have a relation to the master before joining the cluster.
ingest- The node will assume the 'ingest' node-type. Ingest nodes will wait until they have a relation to the master before joining the cluster.
This charm supports Juju storage (as of Juju 2.3).
To deploy this charm using Juju storage (most common for data nodes)
juju deploy elasticsearch --storage data=ebs,10G
Following deployment, we can see the attached volume, and that is being used for Elasticsearch data:
$ df -h | grep elasticsearch /dev/xvdf1 9.8G 23M 9.2G 1% /srv/elasticsearch-data $ ls -la /srv/elasticsearch-data total 28 drwxr-xr-x 4 elasticsearch elasticsearch 4096 Oct 30 21:53 . drwxr-xr-x 3 root root 4096 Oct 30 21:49 .. drwx------ 2 elasticsearch elasticsearch 16384 Oct 30 21:49 lost+found drwxr-xr-x 3 elasticsearch elasticsearch 4096 Oct 30 21:53 nodes
Deploying this charm with the defaults will get you Elasticsearch installed from the elastic.co apt sources, and an all-in-one node-type, where each unit is every node type.
juju deploy elasticsearch -n 3
The above command would deploy 3 elasticsearch nodes, of which all are master, data, ingest, and coordinator. This functionality mirrors that of the legacy Juju Elasticsearch charm, you can deploy and scale a cluster without worrying about node-types because every node is all node-types.
Desparate Node Types
The extended functionality of this charm lends itself to the configuration of Elasticsearch clusters with non-uniform node-types.
# config.yaml es-master: node-type: "master" es-data: node-type: "data" es-coordinator: node-type: "coordinator" es-ingest: node-type: "ingest"
# Deploy 3 units of each node-type juju deploy elasticsearch es-master -n 3 --config config.yaml juju deploy elasticsearch es-data -n 3 --config config.yaml juju deploy elasticsearch es-ingest -n 3 --config config.yaml juju deploy elasticsearch es-coordinator -n 3 --config config.yaml # Make the relations between the components of the cluster # and the master node juju relate es-master:provide-master es-data:request-master juju relate es-master:provide-master es-ingest:request-master juju relate es-master:provide-master es-coordinator:request-master
Managed Configuration MGMT
This charm manages three files primarily, they are:
If you wish to make a customization to any of the aforementioned files, please do so through the charm configuration options.
Only file based discovery is currently supported (pull requests welcome!).
- AGPLv3 (see
copyrightfile in this directory)
- James Beedy email@example.com
- Sebastien Pattyn firstname.lastname@example.org
- Gregory Van Seghbroeck email@example.com
- (string) This sets the elasticsearch cluster name. This name is then added to the elasticsearch.yml file
- (string) String representation of custom elasticsearch.yml configs
- (string) Space separated list of extra deb packages to install.
- (boolean) this option will enable or disable the ufw. By default this is set to 'false' since the firewall is never enabled when a machine or container is created with juju. Set to 'true' to enable firewall. Once enabled rules will be added when another charm has a relation with elasticsearch. This charm will be able to access elasticsearch on port 9200, while all the other traffice to that port will be blocked. Enabling the firewall will also allow port 22 from anywhere.
- (string) list of signing keys for install_sources package sources, per charmhelpers standard format (a yaml list of strings encoded as a string). the keys should be the full ascii armoured gpg public keys. while gpg key ids are also supported and looked up on a keyserver, operators should be aware that this mechanism is insecure. null can be used if a standard package signing key is used that will already be installed on the machine, and for ppa sources where the package signing key is securely retrieved from launchpad.
- - | -----BEGIN PGP PUBLIC KEY BLOCK----- Version: GnuPG v2.0.14 (GNU/Linux) mQENBFI3HsoBCADXDtbNJnxbPqB1vDNtCsqhe49vFYsZN9IOZsZXgp7aHjh6CJBD A+bGFOwyhbd7at35jQjWAw1O3cfYsKAmFy+Ar3LHCMkV3oZspJACTIgCrwnkic/9 CUliQe324qvObU2QRtP4Fl0zWcfb/S8UYzWXWIFuJqMvE9MaRY1bwUBvzoqavLGZ j3SF1SPO+TB5QrHkrQHBsmX+Jda6d4Ylt8/t6CvMwgQNlrlzIO9WT+YN6zS+sqHd 1YK/aY5qhoLNhp9G/HxhcSVCkLq8SStj1ZZ1S9juBPoXV1ZWNbxFNGwOh/NYGldD 2kmBf3YgCqeLzHahsAEpvAm8TBa7Q9W21C8vABEBAAG0RUVsYXN0aWNzZWFyY2gg KEVsYXN0aWNzZWFyY2ggU2lnbmluZyBLZXkpIDxkZXZfb3BzQGVsYXN0aWNzZWFy Y2gub3JnPokBOAQTAQIAIgUCUjceygIbAwYLCQgHAwIGFQgCCQoLBBYCAwECHgEC F4AACgkQ0n1mbNiOQrRzjAgAlTUQ1mgo3nK6BGXbj4XAJvuZDG0HILiUt+pPnz75 nsf0NWhqR4yGFlmpuctgCmTD+HzYtV9fp9qW/bwVuJCNtKXk3sdzYABY+Yl0Cez/ 7C2GuGCOlbn0luCNT9BxJnh4mC9h/cKI3y5jvZ7wavwe41teqG14V+EoFSn3NPKm TxcDTFrV7SmVPxCBcQze00cJhprKxkuZMPPVqpBS+JfDQtzUQD/LSFfhHj9eD+Xe 8d7sw+XvxB2aN4gnTlRzjL1nTRp0h2/IOGkqYfIG9rWmSLNlxhB2t+c0RsjdGM4/ eRlPWylFbVMc5pmDpItrkWSnzBfkmXL3vO2X3WvwmSFiQbkBDQRSNx7KAQgA5JUl zcMW5/cuyZR8alSacKqhSbvoSqqbzHKcUQZmlzNMKGTABFG1yRx9r+wa/fvqP6OT RzRDvVS/cycws8YX7Ddum7x8uI95b9ye1/Xy5noPEm8cD+hplnpU+PBQZJ5XJ2I+ 1l9Nixx47wPGXeClLqcdn0ayd+v+Rwf3/XUJrvccG2YZUiQ4jWZkoxsA07xx7Bj+ Lt8/FKG7sHRFvePFU0ZS6JFx9GJqjSBbHRRkam+4emW3uWgVfZxuwcUCn1ayNgRt KiFv9jQrg2TIWEvzYx9tywTCxc+FFMWAlbCzi+m4WD+QUWWfDQ009U/WM0ks0Kww EwSk/UDuToxGnKU2dQARAQABiQEfBBgBAgAJBQJSNx7KAhsMAAoJENJ9ZmzYjkK0 c3MIAIE9hAR20mqJWLcsxLtrRs6uNF1VrpB+4n/55QU7oxA1iVBO6IFu4qgsF12J TavnJ5MLaETlggXY+zDef9syTPXoQctpzcaNVDmedwo1SiL03uMoblOvWpMR/Y0j 6rm7IgrMWUDXDPvoPGjMl2q1iTeyHkMZEyUJ8SKsaHh4jV9wp9KmC8C+9CwMukL7 vM5w8cgvJoAwsp3Fn59AxWthN3XJYcnMfStkIuWgR7U2r+a210W6vnUxU4oN0PmM cursYPyeV0NX/KQeUeNMwGTFB6QHS/anRaGQewijkrYYoTNtfllxIu9XYmiBERQ/ qPDlGRlOgVTd9xUfHFkzB52c70E= =92oX -----END PGP PUBLIC KEY BLOCK-----
- (string) list of extra apt sources, per charm-helpers standard format (a yaml list of strings encoded as a string). each source may be either a line that can be added directly to sources.list(5), or in the form ppa:<user>/<ppa-name> for adding personal package archives, or a distribution component to enable.
- - 'deb https://artifacts.elastic.co/packages/5.x/apt stable main'
- (string) Java Opts
- (int) Minimum number of master nodes
- (string) Type of elasticsearch node. Choices are 'data', 'master', 'ingest', 'coordinating', 'all'.
- (string) The status of service-affecting packages will be set to this value in the dpkg database. Valid values are "install" and "hold".