ceph radosgw #66

Supports: xenial bionic cosmic trusty
Add to new model


Ceph is a distributed storage and network file system designed to provide
excellent performance, reliability, and scalability.
This charm provides the RADOS HTTP gateway supporting S3 and Swift protocols
for object storage.


Ceph is a distributed storage and network file system designed to provide
excellent performance, reliability and scalability.

This charm deploys the RADOS Gateway, a S3 and Swift compatible HTTP gateway
for online object storage on-top of a ceph cluster.


In order to use this charm, it is assumed that you have already deployed a ceph
storage cluster using the 'ceph' charm with something like this::

juju deploy -n 3 --config ceph.yaml ceph

To deploy the RADOS gateway simple do::

juju deploy ceph-radosgw
juju add-relation ceph-radosgw ceph

You can then directly access the RADOS gateway by exposing the service::

juju expose ceph-radosgw

The gateway can be accessed over port 80 (as show in juju status exposed


Note that you will need to login to one of the service units supporting the
ceph charm to generate some access credentials::

juju ssh ceph/0 \
  'sudo radosgw-admin user create --uid="ubuntu" --display-name="Ubuntu Ceph"'

For security reasons the ceph-radosgw charm is not set up with appropriate
permissions to administer the ceph cluster.

Keystone Integration

Ceph >= 0.55 integrates with Openstack Keystone for authentication of Swift requests.

This is enabled by relating the ceph-radosgw service with keystone::

juju deploy keystone
juju add-relation keystone ceph-radosgw

If you try to relate the radosgw to keystone with an earlier version of ceph the hook
will error out to let you know.


There are two mutually exclusive high availability options: using virtual
IP(s) or DNS. In both cases, a relationship to hacluster is required which
provides the corosync back end HA functionality.

To use virtual IP(s) the clustered nodes must be on the same subnet such that
the VIP is a valid IP on the subnet for one of the node's interfaces and each
node has an interface in said subnet. The VIP becomes a highly-available API

At a minimum, the config option 'vip' must be set in order to use virtual IP
HA. If multiple networks are being used, a VIP should be provided for each
network, separated by spaces. Optionally, vip_iface or vip_cidr may be

To use DNS high availability there are several prerequisites. However, DNS HA
does not require the clustered nodes to be on the same subnet.
Currently the DNS HA feature is only available for MAAS 2.0 or greater
environments. MAAS 2.0 requires Juju 2.0 or greater. The clustered nodes must
have static or "reserved" IP addresses registered in MAAS. The DNS hostname(s)
must be pre-registered in MAAS before use with DNS HA.

At a minimum, the config option 'dns-ha' must be set to true and at least one
of 'os-public-hostname', 'os-internal-hostname' or 'os-internal-hostname' must
be set in order to use DNS HA. One or more of the above hostnames may be set.

The charm will throw an exception in the following circumstances:
If neither 'vip' nor 'dns-ha' is set and the charm is related to hacluster
If both 'vip' and 'dns-ha' are set as they are mutually exclusive
If 'dns-ha' is set and none of the os-{admin,internal,public}-hostname(s) are

Network Space support

This charm supports the use of Juju Network Spaces, allowing the charm to be bound to network space configurations managed directly by Juju. This is only supported with Juju 2.0 and above.

API endpoints can be bound to distinct network spaces supporting the network separation of public, internal and admin endpoints.

To use this feature, use the --bind option when deploying the charm:

juju deploy ceph-radosgw --bind "public=public-space internal=internal-space admin=admin-space"

alternatively these can also be provided as part of a juju native bundle configuration:

  charm: cs:ceph-radosgw
  num_units: 1
    public: public-space
    admin: admin-space
    internal: internal-space

NOTE: Spaces must be configured in the underlying provider prior to attempting to use them.

NOTE: Existing deployments using os-*-network configuration options will continue to function; these options are preferred over any network space binding provided if set.

Multi-Site replication


This charm supports configuration of native replication between Ceph RADOS
gateway deployments.

This is supported both within a single model and between different models
using cross-model relations.

By default either ceph-radosgw deployment will accept write operations.


NOTE: example bundles for the us-west and us-east models can be found
in the bundles subdirectory of the ceph-radosgw charm.

To deploy in this configuration ensure that the following configuration
options are set on the ceph-radosgw charm deployments - in this example
rgw-us-east and rgw-us-west are both instances of the ceph-radosgw charm:

  realm: replicated
  zonegroup: us
  zone: us-east
  realm: replicated
  zonegroup: us
  zone: us-west

When deploying with this configuration the ceph-radosgw applications will
deploy into a blocked state until the master/slave (cross-model) relation
is added.

Typically each ceph-radosgw deployment will be associated with a separate
ceph cluster at different physical locations - in this example the deployments
are in different models ('us-east' and 'us-west').

One ceph-radosgw application acts as the initial master for the deployment -
setup the master relation endpoint as the provider of the offer for the
cross-model relation:

juju offer -m us-east rgw-us-east:master

The cross-model relation offer can then be consumed in the other model and
related to the slave ceph-radosgw application:

juju consume -m us-west admin/us-east.rgw-us-east
juju add-relation -m us-west rgw-us-west:slave rgw-us-east:master

Once the relation has been added the realm, zonegroup and zone configuration
will be created in the master deployment and then synced to the slave

The current sync status can be validated from either model:

juju ssh -m us-east ceph-mon/0
sudo radosgw-admin sync status
          realm 142eb39c-67c4-42b3-9116-1f4ffca23964 (replicated)
      zonegroup 7b69f059-425b-44f5-8a21-ade63c2034bd (us)
           zone 4ee3bc39-b526-4ac9-a233-64ebeacc4574 (us-east)
  metadata sync no sync (zone is master)
      data sync source: db876cf0-62a8-4b95-88f4-d0f543136a07 (us-west)
                        full sync: 0/128 shards
                        incremental sync: 128/128 shards
                        data is caught up with source


In the event that the site hosting the zone which is the master for metadata
(in this example us-east) has an outage, the master metadata zone must be
failed over to the slave site; this operation is performed using the 'promote'

juju run-action -m us-west --wait rgw-us-west/0 promote

Once this action has completed, the slave site will be the master for metadata
updates and the deployment will accept new uploads of data.

Once the failed site has been recovered it will resync and resume as a slave
to the promoted master site (us-west in this example).

The master metadata zone can be failed back to its original location once resync
has completed using the 'promote' action:

juju run-action -m us-east --wait rgw-us-east/0 promote

The now non-master site can also be marked as read only:

juju run-action -m us-west --wait rgw-us-west/0 readonly


(int) Number of keystone tokens to hold in local cache.
(int) This value dictates the number of replicas ceph must make of any object it stores within RGW pools. Note that once the RGW pools have been created, changing this value will not have any effect (although it can be changed in ceph by manually configuring your ceph cluster).
(string) User provided Ceph configuration. Supports a string representation of a python dictionary where each top-level key represents a section in the ceph.conf template. You may only use sections supported in the template. . WARNING: this is not the recommended way to configure the underlying services that this charm installs and is used at the user's own risk. This option is mainly provided as a stop-gap for users that either want to test the effect of modifying some config or who have found a critical bug in the way the charm has configured their services and need it fixed immediately. We ask that whenever this is used, that the user consider opening a bug on this charm at http://bugs.launchpad.net/charms providing an explanation of why the config was needed so that we may consider it for inclusion as a natively supported config in the charm.
(boolean) Use DNS HA with MAAS 2.0. Note if this is set do not set vip settings below.
(string) Default network interface on which HA cluster will bind to communication with the other members of the HA Cluster.
(int) Default multicast port number that will be used to communicate between HA Cluster nodes.
(int) Client timeout configuration in ms for haproxy, used in HA configurations. If not provided, default value of 90000ms is used.
(int) Connect timeout configuration in ms for haproxy, used in HA configurations. If not provided, default value of 9000ms is used.
(int) Queue timeout configuration in ms for haproxy, used in HA configurations. If not provided, default value of 9000ms is used.
(int) Server timeout configuration in ms for haproxy, used in HA configurations. If not provided, default value of 90000ms is used.
(string) Apply system hardening. Supports a space-delimited list of modules to run. Supported modules currently include os, ssh, apache and mysql.
(string) Key ID to import to the apt keyring to support use with arbitary source configuration from outside of Launchpad archives or PPA's.
(int) RadosGW debug level. Max is 20.
(string) Used by the nrpe-external-master subordinate charm. A string that will be prepended to instance name to set the host name in nagios. So for instance the hostname would be something like: . juju-myservice-0 . If you're running multiple environments with the same services in them this allows you to differentiate between them.
(string) A comma-separated list of nagios servicegroups. If left empty, the nagios_context will be used as the servicegroup
(string) Comma-separated list of Swift operator roles; used when integrating with OpenStack Keystone.
(string) The hostname or address of the admin endpoints created for ceph-radosgw in the keystone identity provider. . This value will be used for admin endpoints. For example, an os-admin-hostname set to 'files.admin.example.com' with will create the following admin endpoint for the ceph-radosgw: . https://files.admin.example.com:80/swift/v1
(string) The IP address and netmask of the OpenStack Admin network (e.g. . This network will be used for admin endpoints.
(string) The hostname or address of the internal endpoints created for ceph-radosgw in the keystone identity provider. . This value will be used for internal endpoints. For example, an os-internal-hostname set to 'files.internal.example.com' with will create the following internal endpoint for the ceph-radosgw: . https://files.internal.example.com:80/swift/v1
(string) The IP address and netmask of the OpenStack Internal network (e.g. . This network will be used for internal endpoints.
(string) The hostname or address of the public endpoints created for ceph-radosgw in the keystone identity provider. . This value will be used for public endpoints. For example, an os-public-hostname set to 'files.example.com' with will create the following public endpoint for the ceph-radosgw: . https://files.example.com:80/swift/v1
(string) The IP address and netmask of the OpenStack Public network (e.g. . This network will be used for public endpoints.
(string) The rados gateway stores objects in many different pools. If you would like to have multiple rados gateways each pointing to a separate set of pools set this prefix. The charm will then set up a new set of pools. If your prefix has a dash in it that will be used to split the prefix into region and zone. Please read the documentation on federated rados gateways for more information on region and zone.
(int) The port that the RADOS Gateway will listen on.
(boolean) If True enables IPv6 support. The charm will expect network interfaces to be configured with an IPv6 address. If set to False (default) IPv4 is expected. . NOTE: these charms do not currently support IPv6 privacy extension. In order for this charm to function correctly, the privacy extension must be disabled and a non-temporary address must be configured/available on your network interface.
(string) Name of RADOS Gateway Realm to create for multi-site replication. Setting this option will enable support for multi-site replication, at which point the zonegroup and zone options must also be provided.
(string) OpenStack region that the RADOS gateway supports; used when integrating with OpenStack Keystone.
(boolean) Optionally restrict Ceph key permissions to access pools as required.
(int) Interval between revocation checks to keystone.
(int) Defines a relative weighting of the pool as a percentage of the total amount of data in the Ceph cluster. This effectively weights the number of placement groups for the pool created to be appropriately portioned to the amount of data expected. For example, if the amount of data loaded into the RADOS Gateway/S3 interface is expected to be reserved for or consume 20% of the data in the Ceph cluster, then this value would be specified as 20.
(int) When the Rados Gatway is installed it, by default, creates pools with pg_num 8 which, in the majority of cases is suboptimal. A few rgw pools tend to carry more data than others e.g. .rgw.buckets tends to be larger than most. So, for pools with greater requirements than others the charm will apply the optimal value i.e. corresponding to the number of OSDs up+in the cluster at the time the pool is created. For others it will use this value which can be altered depending on how big you cluster is. Note that once a pool has been created, changes to this setting will be ignored. Setting this value to -1, enables the number of placement groups to be calculated based on the Ceph placement group calculator.
(string) Optional repository from which to install. May be one of the following: distro (default), ppa:somecustom/ppa, a deb url sources entry, or a supported Ubuntu Cloud Archive e.g. . cloud:<series>-<openstack-release> cloud:<series>-<openstack-release>/updates cloud:<series>-<openstack-release>/staging cloud:<series>-<openstack-release>/proposed . See https://wiki.ubuntu.com/OpenStack/CloudArchive for info on which cloud archives are available and supported. . Note that a minimum ceph version of 0.48.2 is required for use with this charm which is NOT provided by the packages in the main Ubuntu archive for precise but is provided in the Ubuntu cloud archive.
(string) SSL CA to use with the certificate and key provided - this is only required if you are providing a privately signed ssl_cert and ssl_key.
(string) SSL certificate to install and use for API ports. Setting this value and ssl_key will enable reverse proxying, point Glance's entry in the Keystone catalog to use https, and override any certificate and key issued by Keystone (if it is configured to do so).
(string) SSL key to use with certificate specified as ssl_cert.
(boolean) If set to True, supporting services will log to syslog.
(string) Virtual IP(s) to use to front API services in HA configuration. . If multiple networks are being used, a VIP should be provided for each network, separated by spaces.
(string) Name of RADOS Gateway Zone to create for multi-site replication. This option must be specific to the local site e.g. us-west or us-east.
(string) Name of RADOS Gateway Zone Group to create for multi-site replication.