spark #41

Supports: bionic
Add to new model

Description

Apache Spark is a unified analytics engine for big data processing,
with built-in modules for streaming, SQL, machine learning and graph processing.


Charm Spark

This charm provides an opinionated yet minimal spark configuration.

Usage

To deploy this charm:

juju deploy cs:~omnivector/spark

This will configure a single unit to be a master, worker, and history-server.

To add worker nodes to the spark deploy use juju add-unit.

juju add-unit spark -n 3
License
  • GPLv3 (see LICENSE file in this directory)
Copyright

Configuration

aws-access-key
(string) AWS access key
aws-secret-key
(string) AWS secret key
extra_packages
(string) Space separated list of extra deb packages to install.
install_keys
(string) List of signing keys for install_sources package sources, per charmhelpers standard format (a yaml list of strings encoded as a string). The keys should be the full ASCII armoured GPG public keys. While GPG key ids are also supported and looked up on a keyserver, operators should be aware that this mechanism is insecure. null can be used if a standard package signing key is used that will already be installed on the machine, and for PPA sources where the package signing key is securely retrieved from Launchpad.
install_sources
(string) List of extra apt sources, per charm-helpers standard format (a yaml list of strings encoded as a string). Each source may be either a line that can be added directly to sources.list(5), or in the form ppa:<user>/<ppa-name> for adding Personal Package Archives, or a distribution component to enable.
object-storage-gateway
(string) The HTTP or HTTPS object storage gateway
package_status
(string) The status of service-affecting packages will be set to this value in the dpkg database. Valid values are "install" and "hold".
install
s3-ssl-enabled
(string) Whether or not to enable ssl s3 connection.
true
shuffle-enabled
(boolean) Enable the spark shuffle service.
True