ubuntu mirror #3
Description
Provides an Ubuntu archive mirror and keeps it up-to-date using rsync
- Tags:
- web_server ›
Overview
This charm provides a complete Ubuntu mirror service, handling fetching the data via rsync from a master server (updates either triggered on-demand using ssh or by cron), and making it available through http, ftp and rsync.
It can work alongside a log-archive subordinate charm to transfer logs to a suitable storage host.
Usage
Assuming you already have a bootstrapped Juju environment, the charm can be deployed using:
juju deploy ubuntu-mirror
Once deployed, you will need to configure it for your local environment. An Ubuntu archive mirror can be created using the following commands:
juju set ubuntu-mirror mirror_ubuntu_name="myhost.mydomain.com"
juju set ubuntu-mirror mirror_ubuntu_sync_time="0 1 * * *"
juju set ubuntu-mirror role_map='{"myhost":["ubuntu"]}'
Replace "myhost" with the hostname of your server and "myhost.mydomain.com" with its fully qualified domain name (FQDN).
The "role_map" variable tells the charm what type of mirror you want to create (currently, this can be any of "ubuntu" (a mirror of archive.ubuntu.com), "releases" (a mirror of releases.ubuntu.com), "cdimage" (a mirror of cdimage.ubuntu.com" or "ports" (a mirror of ports.ubuntu.com) If you have a sufficiently large server, multiple roles can be assigned to one server (role_map='{"myhost":["ubuntu","cdimage"]}' for example)
If you have more than one server, you should list each one (e.g. role_map='{"host1":["cdimage","ubuntu"],"host2":["releases"]}')
The "mirror_ubuntu_name" variable is the name the webserver will use for the Ubuntu mirror. In the above example, it would answer requests for http://myhost.mydomain.com
The "mirror_ubuntu_sync_time" variable tells the charm what time it should update. In the above example, the update would happen every day at 1am.
There are a number of variables that can be set on a per-role basis. They
all follow the pattern mirror_
To cater for rsync servers that require authentication, there is a per-role
variable, mirror_
juju set ubuntu-mirror mirror_cdimage_rsync_auth='{"myhost":{"user":"eric", "password":"YXBwbGVzCg=="}}'
The password value is just a Base64-encoded string, just in case it needs to contain any special characters (it also helps avoid people casually reading passwords over your shoulder when you run juju get).
As with role_map (above), if you have multiple servers they should all be listed (e.g. mirror_cdimage_rsync_auth='{"host1":{"user":"eric","password":"YXBwbGVzCg=="},"host2":{"user":"frank","password":"b3Jhbmdlcwo="}}')
For simplicity, if all of your servers authenticate with the same username
and password, you can use: '{"<
Configuration
- apache_delete_logs_after
- (int) Delete apache logs after n days.
- 14
- apache_logdir
- (string) Where Apache logfiles should be written
- /var/log/apache2
- apache_maxclients
- (int) Maximum number of simultaneous client connections
- 4096
- apache_maxrequestsperchild
- (int) Maximum number of requests a server process serves
- apache_maxsparethreads
- (int) Maximum number of Apache worker threads which are kept spare
- 100
- apache_minsparethreads
- (int) Minimum number of Apache worker threads which are kept spare
- 50
- apache_modules
- (string) A space separated list of extra apache modules to enable
- apache_serverlimit
- (int) Maximum number of Apache server processes allowed
- 256
- apache_startservers
- (int) Initial number of Apache server processes to start
- 2
- apache_status_allowed_from
- (string) A list of IP addresses allowed to access the Apache server-status page
- []
- apache_threadlimit
- (int) Upper limit of ThreadsPerChild allowed in a graceful restart
- 64
- apache_threadsperchild
- (int) Constant number of worker threads in each Apache server process
- 64
- application_name
- (string) The name of this charm
- ubuntu-mirror
- archive_logs_after
- (int) Archive the logs after n days. This is only used when a suitable log-archive subordinate charm is provided
- 1
- archive_logs_apache
- (boolean) Enable log archiving for Apache logs
- True
- archive_logs_ftp
- (boolean) Enable log archiving for FTP logs
- archive_logs_rsync
- (boolean) Enable log archiving for Rsync logs
- disk_inode_crit
- (string) Disk inode critical threshold
- 5%
- disk_space_crit
- (string) Disk space critical threshold
- 20%
- disk_space_warn
- (string) Disk space warning threshold
- 25%
- ftp_delete_logs_after
- (int) Delete FTP logs after n days.
- 28
- ftp_logdir
- (string) Where ftpd logfiles should be written
- /var/log/ftp
- ftp_max_instances
- (int) Maximum number of FTP connections allowed
- 200
- ftp_max_per_source
- (int) Maximum number of concurrent FTP connections allowed
- 10
- (string) FTP welcome message
- FTP server (vsftpd)
- mirror_cdimage_aliases
- (string) List of Apache aliases for the cdimage mirror
- []
- mirror_cdimage_apache_early_extra
- (string) A base64 string containing apache configuration options to be included early in the config file
- mirror_cdimage_apache_late_extra
- (string) A base64 string containing apache configuration options to be included late in the config file
- mirror_cdimage_command
- (string) The command to use to sync the cdimage mirror
- mirror-1stage.sh
- mirror_cdimage_description
- (string) A brief welcome message for the cdimage mirror
- Ubuntu CD Images
- mirror_cdimage_name
- (string) FQDN of the cdimage mirror
- cdimage.ubuntu.com
- mirror_cdimage_path
- (string) Root location of mirrored files for the cdimage mirror
- /srv/ftp.root/cdimage
- mirror_cdimage_rsync_auth
- (string) Optional rsync authentication details for mirror_cdimage_source_url
- {}
- mirror_cdimage_rsync_log
- (boolean) Whether to log rsync requests for the cdimage mirror
- True
- mirror_cdimage_rsync_module
- (string) The name of the rsync module for this mirror role
- cdimage
- mirror_cdimage_source_url
- (string) The URL the cdimage mirror will be fetched from
- rsync://cdimage.ubuntu.com/cdimage
- mirror_cdimage_sync_time
- (string) When mirror updates should be run. Either a cron(5) format time specification or 'ondemand' for ssh triggering
- ondemand
- mirror_cdimage_trigger
- (string) A base64 string containing the ssh trigger public key
- mirror_cloud-images_aliases
- (string) List of Apache aliases for the cloud-images mirror
- []
- mirror_cloud-images_apache_early_extra
- (string) A base64 string containing apache configuration options to be included early in the config file
- mirror_cloud-images_apache_late_extra
- (string) A base64 string containing apache configuration options to be included late in the config file
- mirror_cloud-images_command
- (string) The command to use to sync the cloud-images mirror
- mirror-1stage.sh
- mirror_cloud-images_description
- (string) A brief welcome message for the cloud-images mirror
- Ubuntu Cloud Images
- mirror_cloud-images_name
- (string) FQDN of the cloud-images mirror
- cloud-images.ubuntu.com
- mirror_cloud-images_path
- (string) Root location of mirrored files for the cloud-images mirror
- /srv/ftp.root/cloud-images
- mirror_cloud-images_rsync_auth
- (string) Optional rsync authentication details for mirror_cloud-images_source_url
- {}
- mirror_cloud-images_rsync_log
- (boolean) Whether to log rsync requests for the cloud-images mirror
- True
- mirror_cloud-images_rsync_module
- (string) The name of the rsync module for this mirror role
- cloud-images
- mirror_cloud-images_source_url
- (string) The URL the cloud-images mirror will be fetched from
- rsync://strix.canonical.com/cloud-images
- mirror_cloud-images_sync_time
- (string) When mirror updates should be run. Either a cron(5) format time specification or 'ondemand' for ssh triggering
- ondemand
- mirror_cloud-images_trigger
- (string) A base64 string containing the ssh trigger public key
- mirror_old-releases_aliases
- (string) List of Apache aliases for the old Ubuntu releases mirror
- []
- mirror_old-releases_apache_early_extra
- (string) A base64 string containing apache configuration options to be included early in the config file
- mirror_old-releases_apache_late_extra
- (string) A base64 string containing apache configuration options to be included late in the config file
- mirror_old-releases_command
- (string) The command to use to sync the old Ubuntu releases mirror
- mirror-2stage.sh
- mirror_old-releases_description
- (string) A brief welcome message for the old Ubuntu releases mirror
- Ubuntu Old Releases
- mirror_old-releases_name
- (string) FQDN of the old Ubuntu releases mirror
- old-releases.ubuntu.com
- mirror_old-releases_path
- (string) Root location of mirrored files for the old Ubuntu releases mirror
- /srv/ftp.root/old-images
- mirror_old-releases_rsync_auth
- (string) Optional rsync authentication details for mirror_old-releases_source_url
- {}
- mirror_old-releases_rsync_log
- (boolean) Whether to log rsync requests for the old Ubuntu releases mirror
- mirror_old-releases_rsync_module
- (string) The name of the rsync module for this mirror role
- old-releases
- mirror_old-releases_source_url
- (string) The URL the old Ubuntu releases mirror will be fetched from
- rsync://old-releases.ubuntu.com/old-releases
- mirror_old-releases_sync_time
- (string) When mirror updates should be run. Either a cron(5) format time specification or 'ondemand' for ssh triggering
- ondemand
- mirror_old-releases_trigger
- (string) A base64 string containing the ssh trigger public key
- mirror_ports_aliases
- (string) List of Apache aliases for the ports mirror
- []
- mirror_ports_apache_early_extra
- (string) A base64 string containing apache configuration options to be included early in the config file
- mirror_ports_apache_late_extra
- (string) A base64 string containing apache configuration options to be included late in the config file
- mirror_ports_command
- (string) The command to use to sync the ports mirror
- mirror-2stage.sh
- mirror_ports_description
- (string) A brief welcome message for the ports mirror
- Ubuntu Ports Archive
- mirror_ports_name
- (string) FQDN of the ports mirror
- ports.ubuntu.com
- mirror_ports_path
- (string) Root location of mirrored files for the ports mirror
- /srv/ftp.root/ubuntu-ports
- mirror_ports_rsync_auth
- (string) Optional rsync authentication details for mirror_ports_source_url
- {}
- mirror_ports_rsync_log
- (boolean) Whether to log rsync requests for the ports mirror
- mirror_ports_rsync_module
- (string) The name of the rsync module for this mirror role
- ubuntu-ports
- mirror_ports_source_url
- (string) The URL the ports mirror will be fetched from
- rsync://ports.ubuntu.com/ubuntu-ports
- mirror_ports_sync_time
- (string) When mirror updates should be run. Either a cron(5) format time specification or 'ondemand' for ssh triggering
- ondemand
- mirror_ports_trigger
- (string) A base64 string containing the ssh trigger public key
- mirror_releases_aliases
- (string) List of Apache aliases for the releases mirror
- []
- mirror_releases_apache_early_extra
- (string) A base64 string containing apache configuration options to be included early in the config file
- mirror_releases_apache_late_extra
- (string) A base64 string containing apache configuration options to be included late in the config file
- mirror_releases_command
- (string) The command to use to sync the releases mirror
- mirror-1stage.sh
- mirror_releases_description
- (string) A brief welcome message for the releases mirror
- Ubuntu Releases CD Images
- mirror_releases_name
- (string) FQDN of the releases mirror
- releases.ubuntu.com
- mirror_releases_path
- (string) Root location of mirrored files for the releases mirror
- /srv/ftp.root/releases
- mirror_releases_rsync_auth
- (string) Optional rsync authentication details for mirror_releases_source_url
- {}
- mirror_releases_rsync_log
- (boolean) Whether to log rsync requests for the releases mirror
- True
- mirror_releases_rsync_module
- (string) The name of the rsync module for this mirror role
- releases
- mirror_releases_source_url
- (string) The URL the releases mirror will be fetched from
- rsync://rsync.releases.ubuntu.com/releases
- mirror_releases_sync_time
- (string) When mirror updates should be run. Either a cron(5) format time specification or 'ondemand' for ssh triggering
- ondemand
- mirror_releases_trigger
- (string) A base64 string containing the ssh trigger public key
- mirror_simple-streams_aliases
- (string) List of Apache aliases for the simple-streams mirror
- []
- mirror_simple-streams_apache_early_extra
- (string) A base64 string containing apache configuration options to be included early in the config file
- mirror_simple-streams_apache_late_extra
- (string) A base64 string containing apache configuration options to be included late in the config file
- mirror_simple-streams_command
- (string) The command to use to sync the simple-streams mirror
- mirror-1stage.sh
- mirror_simple-streams_description
- (string) A brief welcome message for the simple-streams mirror
- Juju Simple Streams Mirror
- mirror_simple-streams_name
- (string) FQDN of the simple-streams mirror
- streams.canonical.com
- mirror_simple-streams_path
- (string) Root location of mirrored files for the simple-streams mirror
- /srv/ftp.root/simple-streams
- mirror_simple-streams_rsync_auth
- (string) Optional rsync authentication details for mirror_simple-streams_source_url
- {}
- mirror_simple-streams_rsync_log
- (boolean) Whether to log rsync requests for the simple-streams mirror
- True
- mirror_simple-streams_rsync_module
- (string) The name of the rsync module for this mirror role
- simple-streams
- mirror_simple-streams_source_url
- (string) The URL the simple-streams mirror will be fetched from
- rsync://strix.canonical.com/simple-streams
- mirror_simple-streams_sync_time
- (string) When mirror updates should be run. Either a cron(5) format time specification or 'ondemand' for ssh triggering
- ondemand
- mirror_simple-streams_trigger
- (string) A base64 string containing the ssh trigger public key
- mirror_ubuntu_aliases
- (string) List of Apache aliases for the Ubuntu mirror
- []
- mirror_ubuntu_apache_early_extra
- (string) A base64 string containing apache configuration options to be included early in the config file
- mirror_ubuntu_apache_late_extra
- (string) A base64 string containing apache configuration options to be included late in the config file
- mirror_ubuntu_command
- (string) The command to use to sync the Ubuntu mirror
- mirror-2stage.sh
- mirror_ubuntu_description
- (string) A brief welcome message for the Ubuntu mirror
- Ubuntu Archive
- mirror_ubuntu_name
- (string) FQDN of the Ubuntu mirror
- archive.ubuntu.com
- mirror_ubuntu_path
- (string) Root location of mirrored files for the Ubuntu mirror
- /srv/ftp.root/ubuntu
- mirror_ubuntu_rsync_auth
- (string) Optional rsync authentication details for mirror_ubuntu_source_url
- {}
- mirror_ubuntu_rsync_log
- (boolean) Whether to log rsync requests for the Ubuntu mirror
- mirror_ubuntu_rsync_module
- (string) The name of the rsync module for this mirror role
- ubuntu
- mirror_ubuntu_source_url
- (string) The URL the Ubuntu mirror will be fetched from
- rsync://archive.ubuntu.com/ubuntu
- mirror_ubuntu_sync_time
- (string) When mirror updates should be run. Either a cron(5) format time specification or 'ondemand' for ssh triggering
- ondemand
- mirror_ubuntu_trigger
- (string) A base64 string containing the ssh trigger public key
- mirror_user
- (string) Mirror system user id
- archvsync
- nagios_servicegroup
- (string) If set, this is the Nagios servicegroup for alerts. If unset, an appropriate one will be chosen
- package_status
- (string) The status of service-affecting packages will be set to this value in the dpkg database. Useful valid values are "install" and "hold".
- install
- role_map
- (string) JSON document describing which mirror is enabled on which machine. There are two formats. The first format is simple: {"hostname1": ["cdimage", "ubuntu"], "hostname2": ["releases"]} In the second more complex format each hostname's value is a dictionary rather than a list of roles. For example: { "hostname1": { "releases": { "addresses": ["1.2.3.4"], "https": true }, "ubuntu": null }, "hostname2": [ "releases", "ubuntu" ] } Here, hostname1's ubuntu role will result in an HTTP VirtualHost that listens on *:80 and serves the Ubuntu archive. The releases role will result in HTTP and HTTPS VirtualHosts that listen on 1.2.3.4:80 and 1.2.3.4:443 respectively.
- rsync_logdir
- (string) Rsync log directory
- /var/log/rsyncd
- rsync_max_connections
- (int) Maximum number of rsync connections allowed
- 65
- rsync_max_connections_per_source
- (int) Maximum number of rsync connections allowed per source/IP
- 5
- rsync_motd
- (string) Rsync server welcome message text (base64 encoded)
- VGhpcyBpcyBhbiBVYnVudHUgbWlycm9yIC0gdHJlYXQgaXQga2luZGx5Cg==
- rsync_motdfile
- (string) Rsync server welcome message file
- /etc/rsyncd/motd
- script_dir
- (string) Install location for any mirror-related scripts
- /srv/ubuntu-mirror/bin