duplicity #1

Supports: focal bionic xenial
Add to new model

Description

Duplicity backs directories by producing encrypted tar-format volumes and uploading them to a remote or local file server. Because duplicity uses librsync, the incremental archives are space efficient and only record the parts of files that have changed since the last backup. Because duplicity uses GnuPG to encrypt and/or sign these archives, they will be safe from spying and/or modification by the server.


Duplicity Charm

Overview

The Duplicity charm provides functionality for both manual and automatic backups for a deployed application. As the name suggests, it utilizes the Duplicity tool and acts as an easy-to-use-and-configure interface for operators to set up backups.

After relating the Duplicity to another charm, you can backup a directory to either the local unit, a remote host, or even an AWS S3 bucket. All it takes is a bit of configuration and remote destination preparation.

The following backends are currently supported: - File (local) - S3 - SCP - Rsync - FTP/SFTP

Usage

Simple deployment

This will get duplicity deployed on whatever deployed charm you want. Here, we see it being related to the ubuntu charm.

juju deploy ubuntu
juju deploy duplicity
juju add-relation duplicity ubuntu

However, we will need to fill out various other, required configs, depending on the backend type selected.

Local file backups

This will backup a selected directory to the local unit.

juju config duplicity \
    backend=file \
    remote_backup_url=file:///home/me/backups
    aux_backup_dir=/path/to/back/up

SCP/Rsync/SFTP Backups

Using the backends scp, rsync, and sftp require, at minimum, the following options to be set.

juju config duplicity \
    backend=scp \
    remote_backup_url=my.host:22/my_backups
    known_host_key='my.host,10.10.10.2 ssh-rsa AAABBBCCC' \
    private_ssh_key="$(base64 my_priv_id)"

Alternatively, you can use remote_password=password instead of the private_ssh_key option if you prefer password authentication.

S3 Backups

The following will backup to S3 buckets. This configuration requires an IAM account access and secret key to be passed into the config.

juju config duplicity \
    backend=s3 \
    remote_backup_url=s3:my.aws.com/bucket_name/prefix \
    aws_access_key_id=my_aws_key \
    aws_secret_access_key=my_aws_secret

Encryption

To encrypt your backups, you can use symmetric encryption using a passed in password or encrypt the backup with a GPG key. Alternative to these methods, you can ignore encryption entirely.

# Symmetric password encryption
juju config duplicity encryption_passphrase=my_passphrase

# Asymmetric GPG encryption
juju config duplicity gpg_public_key=MY_GPG_KEY

# Disable encryption (not recommended)
juju config duplicity disable_encryption=True

Setting Periodic Backups

The big draw of this charm is being able to periodically backup a directory. By default, the charm will only backup manually, i.e. through the do-backup action. To enable periodic backups, set backup_frequency to any of the following:

  • hourly
  • daily
  • weekly
  • monthly
  • any valid cron schedule string

Adding NRPE Checks for alerting

Adding NRPE checks allows for alerting when a periodic backup fails to complete.

juju deploy nrpe
juju add-relation nrpe ubuntu       # required on host 
juju add-relation nrpe duplicity

Known Limitations and Future Features

This charm is currently still under development. The only supported Duplicity action right now is full backups (through both an action and periodic backups). The following is the list of future Duplicity functionality:

  • incremental backups
  • restoring backups
  • verifying backups
  • listing backed-up files
  • cleaning up backed files
  • additional supported backends

Upstream and Bugs

The repository can be found here.

Please report bugs or feature requests on Launchpad.


Configuration

aux_backup_directory
(string) Specifies an additional directory paths which duplicity will monitor on all units for backup.
/tmp/duplicity
aws_access_key_id
(string) Access key id for the AWS IMA user. The user must have a policy that grants it privileges to upload to the S3 bucket. This value is required when backend='s3'.
aws_secret_access_key
(string) Secret access key for the AWS IMA user. The user must have a policy that grants it privileges to upload to the S3 bucket. This value is required when backend='s3'.
backend
(string) Accepted values are s3 | ssh | scp | ftp | rsync | file An empty string will disable backups.
backup_frequency
(string) Sets the crontab backup frequency to a valid cron string or one of the following: hourly|daily|weekly|monthly|manual If set to manual, crontab backup will not run.
manual
disable_encryption
(boolean) By default, duplicity uses symmetric encryption on backup, requiring a simple password. Duplicity also supports asymmetric encryption, via GPG keys. Setting this value to True disables encryption across the entire application.
encryption_passphrase
(string) Set a passphrase required to perform symmetric encryption.
extra_packages
(string) Space separated list of extra deb packages to install.
gpg_public_key
(string) Sets the GPG Public Key used for asymmetrical encryption. When set, this becomes the primary method for encryption.
install_keys
(string) List of signing keys for install_sources package sources, per charmhelpers standard format (a yaml list of strings encoded as a string). The keys should be the full ASCII armoured GPG public keys. While GPG key ids are also supported and looked up on a keyserver, operators should be aware that this mechanism is insecure. null can be used if a standard package signing key is used that will already be installed on the machine, and for PPA sources where the package signing key is securely retrieved from Launchpad.
install_sources
(string) List of extra apt sources, per charm-helpers standard format (a yaml list of strings encoded as a string). Each source may be either a line that can be added directly to sources.list(5), or in the form ppa:<user>/<ppa-name> for adding Personal Package Archives, or a distribution component to enable.
known_host_key
(string) Host key for remote backup host when using scp, rsync, and sftp backends. Valid host key required when using these backends. The format is: hostname[,ip] algo public_key ex: example.com,10.0.0.0 ssh-rsa AAABBBCCC...
nagios_context
(string) Used by the nrpe subordinate charms. A string that will be prepended to instance name to set the host name in nagios. So for instance the hostname would be something like: juju-myservice-0 If you're running multiple environments with the same services in them this allows you to differentiate between them.
juju
nagios_servicegroups
(string) A comma-separated list of nagios servicegroups. If left empty, the nagios_context will be used as the servicegroup
package_status
(string) The status of service-affecting packages will be set to this value in the dpkg database. Valid values are "install" and "hold".
install
private_ssh_key
(string) base64 encoded private SSH key for SSH authentication from duplicity application unit and the remote backup host.
remote_backup_url
(string) URL to the remote server and its local path to be used as the backup destination. Backends and their URL formats: file: 'file:///some_dir' ftp & sftp: 'remote.host[:port]/some_dir' rsync: 'other.host[:port]::/module/some_dir' 'other.host[:port]/relative_path' 'other.host[:port]//absolute_path' s3: 's3:other.host[:port]/bucket_name[/prefix]' 's3+http://bucket_name[/prefix]' scp: 'other.host[:port]/some_dir' ssh: 'other.host[:port]/some_dir'
remote_password
(string) This value sets the remote server's password to be used for ssh or ftp backups. This is required for ftp backups and optional for ssh, which if unset may still be able to authenticate via trusted host keys.
remote_user
(string) This value sets the remote host username for ssh or ftp backups. This is required for ftp type backups and optional for ssh, which if unset it will default to using the local hosts username.