Duplicity backs directories by producing encrypted tar-format volumes and uploading them to a remote or local file server. Because duplicity uses librsync, the incremental archives are space efficient and only record the parts of files that have changed since the last backup. Because duplicity uses GnuPG to encrypt and/or sign these archives, they will be safe from spying and/or modification by the server.
The Duplicity charm provides functionality for both manual and automatic backups for a deployed application. As the name suggests, it utilizes the Duplicity tool and acts as an easy-to-use-and-configure interface for operators to set up backups.
After relating the Duplicity to another charm, you can backup a directory to either the local unit, a remote host, or even an AWS S3 bucket. All it takes is a bit of configuration and remote destination preparation.
The following backends are currently supported: - File (local) - S3 - SCP - Rsync - FTP/SFTP
This will get duplicity deployed on whatever deployed charm you want. Here, we see it being related to the ubuntu charm.
juju deploy ubuntu juju deploy duplicity juju add-relation duplicity ubuntu
However, we will need to fill out various other, required configs, depending on the backend type selected.
Local file backups
This will backup a selected directory to the local unit.
juju config duplicity \ backend=file \ remote_backup_url=file:///home/me/backups aux_backup_dir=/path/to/back/up
Using the backends scp, rsync, and sftp require, at minimum, the following options to be set.
juju config duplicity \ backend=scp \ remote_backup_url=my.host:22/my_backups known_host_key='my.host,10.10.10.2 ssh-rsa AAABBBCCC' \ private_ssh_key="$(base64 my_priv_id)"
Alternatively, you can use
remote_password=password instead of the
private_ssh_key option if you prefer
The following will backup to S3 buckets. This configuration requires an IAM account access and secret key to be passed into the config.
juju config duplicity \ backend=s3 \ remote_backup_url=s3:my.aws.com/bucket_name/prefix \ aws_access_key_id=my_aws_key \ aws_secret_access_key=my_aws_secret
To encrypt your backups, you can use symmetric encryption using a passed in password or encrypt the backup with a GPG key. Alternative to these methods, you can ignore encryption entirely.
# Symmetric password encryption juju config duplicity encryption_passphrase=my_passphrase # Asymmetric GPG encryption juju config duplicity gpg_public_key=MY_GPG_KEY # Disable encryption (not recommended) juju config duplicity disable_encryption=True
Setting Periodic Backups
The big draw of this charm is being able to periodically backup a directory. By default,
the charm will only backup manually, i.e. through the
do-backup action. To enable
periodic backups, set
backup_frequency to any of the following:
- any valid cron schedule string
Adding NRPE Checks for alerting
Adding NRPE checks allows for alerting when a periodic backup fails to complete.
juju deploy nrpe juju add-relation nrpe ubuntu # required on host juju add-relation nrpe duplicity
Known Limitations and Future Features
This charm is currently still under development. The only supported Duplicity action right now is full backups (through both an action and periodic backups). The following is the list of future Duplicity functionality:
- incremental backups
- restoring backups
- verifying backups
- listing backed-up files
- cleaning up backed files
- additional supported backends
Upstream and Bugs
The repository can be found here.
Please report bugs or feature requests on Launchpad.
- (string) Specifies an additional directory paths which duplicity will monitor on all units for backup.
- (string) Access key id for the AWS IMA user. The user must have a policy that grants it privileges to upload to the S3 bucket. This value is required when backend='s3'.
- (string) Secret access key for the AWS IMA user. The user must have a policy that grants it privileges to upload to the S3 bucket. This value is required when backend='s3'.
- (string) Accepted values are s3 | ssh | scp | ftp | rsync | file An empty string will disable backups.
- (string) Sets the crontab backup frequency to a valid cron string or one of the following: hourly|daily|weekly|monthly|manual If set to manual, crontab backup will not run.
- (boolean) By default, duplicity uses symmetric encryption on backup, requiring a simple password. Duplicity also supports asymmetric encryption, via GPG keys. Setting this value to True disables encryption across the entire application.
- (string) Set a passphrase required to perform symmetric encryption.
- (string) Space separated list of extra deb packages to install.
- (string) Sets the GPG Public Key used for asymmetrical encryption. When set, this becomes the primary method for encryption.
- (string) List of signing keys for install_sources package sources, per charmhelpers standard format (a yaml list of strings encoded as a string). The keys should be the full ASCII armoured GPG public keys. While GPG key ids are also supported and looked up on a keyserver, operators should be aware that this mechanism is insecure. null can be used if a standard package signing key is used that will already be installed on the machine, and for PPA sources where the package signing key is securely retrieved from Launchpad.
- (string) List of extra apt sources, per charm-helpers standard format (a yaml list of strings encoded as a string). Each source may be either a line that can be added directly to sources.list(5), or in the form ppa:<user>/<ppa-name> for adding Personal Package Archives, or a distribution component to enable.
- (string) Host key for remote backup host when using scp, rsync, and sftp backends. Valid host key required when using these backends. The format is: hostname[,ip] algo public_key ex: example.com,10.0.0.0 ssh-rsa AAABBBCCC...
- (string) Used by the nrpe subordinate charms. A string that will be prepended to instance name to set the host name in nagios. So for instance the hostname would be something like: juju-myservice-0 If you're running multiple environments with the same services in them this allows you to differentiate between them.
- (string) A comma-separated list of nagios servicegroups. If left empty, the nagios_context will be used as the servicegroup
- (string) The status of service-affecting packages will be set to this value in the dpkg database. Valid values are "install" and "hold".
- (string) base64 encoded private SSH key for SSH authentication from duplicity application unit and the remote backup host.
- (string) URL to the remote server and its local path to be used as the backup destination. Backends and their URL formats: file: 'file:///some_dir' ftp & sftp: 'remote.host[:port]/some_dir' rsync: 'other.host[:port]::/module/some_dir' 'other.host[:port]/relative_path' 'other.host[:port]//absolute_path' s3: 's3:other.host[:port]/bucket_name[/prefix]' 's3+http://bucket_name[/prefix]' scp: 'other.host[:port]/some_dir' ssh: 'other.host[:port]/some_dir'
- (string) This value sets the remote server's password to be used for ssh or ftp backups. This is required for ftp backups and optional for ssh, which if unset may still be able to authenticate via trusted host keys.
- (string) This value sets the remote host username for ssh or ftp backups. This is required for ftp type backups and optional for ssh, which if unset it will default to using the local hosts username.