Here is my solution to maintaining Development, Staging, and Production apt mirrors on a “deployment” server with Puppet. I assume the reader is familiar with Puppet and has a working Puppet infrastructure.
Let’s start with the site.pp file and work our way down to the actual module. I’ll include the whole site.pp file since it’ll give you an idea of the directory layout. Generally, each Puppet node has an individual configuration file under node/ with their IP configuration, and includes the OS distribution type and role(s).
import "apps/*.pp"
import "dist/*.pp"
import "role/*.pp"
import "user/*.pp"
import "misc.pp"
import "users.pp"
import "hosts.pp"
# include stages last
import "stages.pp"
# default top variables
$realm = "production"
$ppaprefix = "ppa.DOMAINNAME.com/prd/mirror/"
$puppetmaster = "puppetmaster.DOMAINNAME.com"
# specific configs for each node (ip addresses, for example)
import "node/*.DOMAINNAME.com.pp"
# generic dev servers
node /^.*\.dev\.DOMAINNAME\.com$/ {
$realm = "development"
$ppaprefix = "ppa.DOMAINNAME.com/dev/mirror/"
include ubuntu::vmware
}
# generic staging servers
node /^.*\.stg\.DOMAINNAME\.com$/ {
$realm = "staging"
$ppaprefix = "ppa.DOMAINNAME.com/stg/mirror/"
include ubuntu::vmware
}
# catch-all for production servers
node /^[^\.]*\.DOMAINNAME\.com$/ {
include ubuntu::vmware
}
node default {
include ubuntu::vmware
}
The deployment server, which hosts the Development, Staging, and Production apt mirrors, has a node configuration that includes the following (ip and dhcp server information has been removed).
node "deploy.DOMAINNAME.com" {
include ubuntu::vmware,
DOMAINNAME::deploy
}The included ubuntu::vmware class configures this node as an Ubuntu server running under vmware (installs the open-vm-tools, for example), and the DOMAINNAME::deploy class configures this node as a “deployment” server offering apt mirrors (among other infrastructure services).
Here’s an example of the role/DOMAINNAME-deploy.pp file. I’ve left out the dhcp server, squid proxy, apt and yum repository configurations, etc., leaving just the relevant apt mirror sections. The apt::mirror class (from the module) creates the /etc/apt/mirror.list.d/ directory, while the apt::mirror::* definitions create the apt-mirror configuration files within.
class DOMAINNAME::deploy {
include DOMAINNAME::deploy::apt::mirror
}
class DOMAINNAME::deploy::apt::mirror {
include apt::mirror
apt::mirror::ubuntu::precise { [ "prd", "stg", "dev" ]: }
apt::mirror::ubuntu::raring { [ "prd", "stg", "dev" ]: }
apt::mirror::puppet { [ "prd", "stg", "dev" ]: }
apt::mirror::gluster { [ "prd", "stg", "dev" ]: }
apt::mirror::mysql56 { [ "dev" ]: }
}
The apt::mirror definitions reside in the apps/apt-mirror.pp file, and each definition within that file calls apt::mirror::repository from the apt mirror module itself. The uri, dist, and components arguments are essential, and you may also include source (true/false) to add a dev-src to the mirror, ensure (present/absent) to add/remove the apt-mirror configuration file, and sched (daily|hourly|monthly|weekly) to update the mirror on a schedule (default is no schedule).
define apt::mirror::ubuntu::precise {
$uri = "http://archive.ubuntu.com/ubuntu"
$dist = [ "precise", "precise-security", "precise-updates", "precise-backports" ]
$components = "main restricted universe multiverse"
apt::mirror::repository { "${name}-ubuntu-precise": subdir => $name, uri => $uri, dist => $dist, components => $components; }
}
define apt::mirror::ubuntu::raring {
$uri = "http://archive.ubuntu.com/ubuntu"
$dist = [ "raring", "raring-security", "raring-updates", "raring-backports" ]
$components = "main restricted universe multiverse"
apt::mirror::repository { "${name}-ubuntu-raring": subdir => $name, uri => $uri, dist => $dist, components => $components; }
}
define apt::mirror::puppet {
$uri = "http://apt.puppetlabs.com"
$dist = [ "precise", "raring", ]
$components = "main dependencies"
apt::mirror::repository { "${name}-puppetlabs": subdir => $name, uri => $uri, dist => $dist, components => $components; }
}
define apt::mirror::gluster {
$dist = [ "precise", "raring", ]
$components = "main"
apt::mirror::repository { "${name}-gluster33": subdir => $name,
uri => "http://ppa.launchpad.net/semiosis/ubuntu-glusterfs-3.3/ubuntu", dist => $dist, components => $components; }
apt::mirror::repository { "${name}-gluster34": subdir => $name,
uri => "http://ppa.launchpad.net/semiosis/ubuntu-glusterfs-3.4/ubuntu", dist => $dist, components => $components; }
}
define apt::mirror::mysql56 {
$uri = "http://ppa.launchpad.net/ondrej/mysql-experimental/ubuntu"
$dist = [ "precise", "raring", ]
$components = "main"
apt::mirror::repository { "${name}-mysql56": subdir => $name, uri => $uri, dist => $dist, components => $components; }
}
The apt mirror module resides in modules/apt/manifests/mirror.pp and provides the apt::mirror definition, which creates the /etc/apt/mirror.list.d/ directory, and the apt::mirror::repository definition, which creates the individual apt-mirror configuration files within.
class apt::mirror {
file { "/etc/apt/mirror.list.d":
ensure => directory,
owner => "root",
group => "root",
mode => "0755",
recurse => true,
purge => true,
}
}
define apt::mirror::repository (
$subdir = "", # required for the template
$uri = "",
$dist = "",
$components = "",
$source = false,
$ensure = present,
$sched = "",
) {
file { "/etc/apt/mirror.list.d/${name}.list":
ensure => $ensure,
owner => "root",
group => "root",
mode => "0444",
content => template("apt/mirror.list.erb")
}
exec { "apt-mirror-${name}":
command => "/usr/bin/apt-mirror /etc/apt/mirror.list.d/${name}.list",
timeout => 3600,
returns => [ 0 ],
subscribe => File["/etc/apt/mirror.list.d/${name}.list"],
refreshonly => true
}
case $sched {
/^(daily|hourly|monthly|weekly)$/: {
file { "/etc/cron.${sched}/apt-mirror-${name}":
ensure => $ensure,
owner => "root",
group => "root",
mode => "0744",
content => "[ -f /etc/apt/mirror.list.d/${name}.list ] && /usr/bin/apt-mirror /etc/apt/mirror.list.d/${name}.list\n",
}
}
default: {
file { "/etc/cron.daily/apt-mirror-${name}": ensure => absent }
file { "/etc/cron.hourly/apt-mirror-${name}": ensure => absent }
file { "/etc/cron.monthly/apt-mirror-${name}": ensure => absent }
file { "/etc/cron.weekly/apt-mirror-${name}": ensure => absent }
}
}
}
The apt::mirror::repository definition uses the templates/apt/mirror.list.erb template.
# ____ _ _____ _ _
# | _ \ _ _ _ __ _ __ ___| |_ | ___(_) | ___
# | |_) | | | | '_ \| '_ \ / _ \ __| | |_ | | |/ _ \
# | __/| |_| | |_) | |_) | __/ |_ | _| | | | __/
# |_| \__,_| .__/| .__/ \___|\__| |_| |_|_|\___|
# |_| |_|
<% if @subdir -%>
set base_path /var/spool/apt-mirror/<%= @subdir %>
<% else -%>
set base_path /var/spool/apt-mirror
<% end -%>
set mirror_path $base_path/mirror
set skel_path $base_path/skel
set var_path $base_path/var
set cleanscript $var_path/clean.sh
set defaultarch amd64
set postmirror_script $var_path/postmirror.sh
set run_postmirror 0
set nthreads 10
set _tilde 0
<% @dist.each do |d| %>
deb <%= @uri %> <%= d %> <%= @components %>
<% if @source == true -%>deb-src <%= @uri %> <%= d %> <%= @components %><% end -%>
<% end %>
I chose to keep the template outside the module directory since the base_path and other parameters may differ for each installation.
