Adding the nginx-plus Repository to apt-mirror and Puppet

Nginx Inc. provides access to the nginx-plus package and repository using SSL certificates. Their instructions include the configuration of apt for Ubuntu, but for people using apt-mirror and Puppet to manage their internal servers, additional custom configurations are required.

The standard apt configuration for nginx-plus might look like this:

$ cat /etc/apt/apt.conf.d/90nginx 
Acquire::https::plus-pkgs.nginx.com::Verify-Peer "true";
Acquire::https::plus-pkgs.nginx.com::Verify-Host "true";
Acquire::https::plus-pkgs.nginx.com::CaInfo      "/etc/ssl/nginx/CA.crt";
Acquire::https::plus-pkgs.nginx.com::SslCert     "/etc/ssl/nginx/nginx-repo.crt";
Acquire::https::plus-pkgs.nginx.com::SslKey      "/etc/ssl/nginx/nginx-repo.key";

The connection to the nginx-plus repository must be made using HTTPS and authentication is handled by client certificates. As provided, apt-mirror is not able to manage SSL certificates, so two sections in the apt-mirror script must be modified. The %config_variables array defines the settings read from its configuration files. We will add the ‘certificate’, ‘private_key’, and ‘ca_certificate’ settings to the array.

my %config_variables = (
    "defaultarch" => `dpkg --print-installation-architecture 2>/dev/null` || 'i386',
    "nthreads"    => 20,
    "base_path"   => '/var/spool/apt-mirror',
    "mirror_path" => '$base_path/mirror',
    "skel_path"   => '$base_path/skel',
    "var_path"    => '$base_path/var',
    "cleanscript" => '$var_path/clean.sh',
    "_contents"   => 1,
    "_autoclean"  => 0,
    "_tilde"      => 0,
    "limit_rate"  => '100m',
    "run_postmirror" => 1,
    "postmirror_script" => '$var_path/postmirror.sh',
    "certificate" => '',
    "private_key" => '',
    "ca_certificate" => ''
);

If these configuration settings are found, we must pass them to wget.

if($pid == 0) { 
        my $ca = get_variable("ca_certificate") ? '--ca-certificate='.get_variable("ca_certificate") : '';
        my $cert = get_variable("certificate") ? '--certificate='.get_variable("certificate") : '';
        my $key = get_variable("private_key") ? '--private-key='.get_variable("private_key") : '';
        exec '/usr/bin/wget', $ca, $cert, $key, '--no-cache', '--limit-rate='.get_variable("limit_rate"), 
                '-t', '5', '-r', '-N', '-l', 'inf', '-o', get_variable("var_path")."/$stage-log.$i", '-i', 
                get_variable("var_path")."/$stage-urls.$i";
        die("\n\nCould not run wget, please make sure its installed and in your path\n\n");
}

Here is an example template for Puppet to create the apt-mirror configuration files, including the optional SSL certificate paths.

$ cat /etc/puppet/modules/apt/templates/mirror.list.erb 
#        ____                         _     _____ _ _      
#       |  _ \ _   _ _ __  _ __   ___| |_  |  ___(_) | ___ 
#       | |_) | | | | '_ \| '_ \ / _ \ __| | |_  | | |/ _ \
#       |  __/| |_| | |_) | |_) |  __/ |_  |  _| | | |  __/
#       |_|    \__,_| .__/| .__/ \___|\__| |_|   |_|_|\___|
#                   |_|   |_|                              

<% if @subdir -%>set base_path  /var/spool/apt-mirror/<%= @subdir %>
<% else -%>set base_path        /var/spool/apt-mirror
<% end -%>
set mirror_path $base_path/mirror
set skel_path   $base_path/skel
set var_path    $base_path/var
set cleanscript $var_path/clean.sh
set defaultarch amd64
set postmirror_script   $var_path/postmirror.sh
set run_postmirror      0
set nthreads    10
set _tilde      0
<% if @ca_certificate -%>set ca_certificate     <%= @ca_certificate %>
<% end -%>
<% if @certificate -%>set certificate   <%= @certificate %>
<% end -%>
<% if @private_key -%>set private_key   <%= @private_key %>
<% end -%>

<% @dist.each do |d| %>
deb <%= @uri %> <%= d %> <%= @components %>
<% if @source == true -%>deb-src <%= @uri %> <%= d %> <%= @components %><% end -%>
<% end %>

And here is an example Puppet module, with an apt::mirror class and apt::mirror::repository definition, to generate the configuration files from that template. This module also includes a scheduling feature to automate mirror updates, though this feature may not be useful for most production environments (where more stringent processes may be required for controlled mirror updates).

$ cat /etc/puppet/modules/apt/manifests/mirror.pp 

class apt::mirror {
        file { "/etc/apt/mirror.list.d":
                ensure  => directory,
                owner   => "root",
                group   => "root",
                mode    => "0755",
                recurse => true,
                purge   => true,
        }
}

define apt::mirror::repository (
        $subdir = "",   # required for the template
        $uri = "",
        $dist = "",
        $components = "",
        $ca_certificate = false,
        $certificate = false,
        $private_key = false,
        $source = false,
        $ensure = present,
        $sched = "",
) {
        file { "/etc/apt/mirror.list.d/${name}.list":
                ensure  => $ensure,
                owner   => "root",
                group   => "root",
                mode    => "0444",
                content => template("apt/mirror.list.erb");
        }

        # only run the apt-mirror command when the apt-mirror config file is first created
        exec { "apt-mirror-${name}":
                command => "/usr/bin/apt-mirror /etc/apt/mirror.list.d/${name}.list && touch /root/.apt-mirror-${name}",
                timeout => 3600,
                returns => [ 0 ],
                require => File[ "/usr/bin/apt-mirror", "/etc/apt/mirror.list.d/${name}.list" ],
                creates => "/root/.apt-mirror-${name}",
                unless => "/usr/bin/test -f /root/.apt-mirror-${name}";
        }

        # optionally create cronjobs to automate the apt-mirror updates
        case $sched {
                /^(daily|hourly|monthly|weekly)$/: {
                        file { "/etc/cron.${sched}/apt-mirror-${name}":
                                ensure  => $ensure,
                                owner   => "root",
                                group   => "root",
                                mode    => "0744",
                                content => "[ -f /etc/apt/mirror.list.d/${name}.list ] && /usr/bin/apt-mirror /etc/apt/mirror.list.d/${name}.list\n",
                        }
                }
                default: {
                        file { "/etc/cron.daily/apt-mirror-${name}": ensure => absent }
                        file { "/etc/cron.hourly/apt-mirror-${name}": ensure => absent }
                        file { "/etc/cron.monthly/apt-mirror-${name}": ensure => absent }
                        file { "/etc/cron.weekly/apt-mirror-${name}": ensure => absent }
                }
        }
}

A practical example, using the apt-mirror Puppet module above, may look like this (assuming your Puppet configurations use ‘role’ and ‘application’ based classes).

The ‘deploy’ role includes the role::deploy::apt::mirror class, which then includes the apt::mirror module class, and calls the apps::apt::mirror::nginx::plus definition for all three supported environments — production, staging, and development (each environment has its own mirror, which can be updated and tested independently).

$ cat /etc/puppet/manifests/role/deploy.pp
class role::deploy {
        include role::deploy::apt::mirror
}

class role::deploy::apt::mirror {
        include apt::mirror

        apps::apt::mirror::nginx::plus { [ "prd", "stg", "dev" ]: }
        # 
        # ... rinse and repeat for all mirror definitions ...
        #
}

The apps::apt::mirror::nginx::plus definition is located within an apt-mirror ‘application’ manifest. It executes the apt::mirror::repository module definition for each of the environment names we provide.

$ cat /etc/puppet/manifests/apps/apt-mirror.pp

define apps::apt::mirror::nginx::plus {
        $uri = "https://plus-pkgs.nginx.com/ubuntu"
        $dist = [ "precise", "raring", ]
        $components = "nginx-plus"
        apt::mirror::repository { 
                "${name}-nginx-plus": 
                        subdir => $name, 
                        uri => $uri, 
                        dist => $dist, 
                        components => $components,
                        ca_certificate => "/etc/ssl/nginx/CA.crt",
                        certificate => "/etc/ssl/nginx/nginx-repo.crt",
                        private_key => "/etc/ssl/nginx/nginx-repo.key",
        }
}

The resulting apt-mirror configuration files may look like this.

$ ls -al /etc/apt/mirror.list.d/*-nginx-plus.list
-r--r--r-- 1 root root 898 Feb  5 15:18 /etc/apt/mirror.list.d/dev-nginx-plus.list
-r--r--r-- 1 root root 898 Feb  5 15:18 /etc/apt/mirror.list.d/prd-nginx-plus.list
-r--r--r-- 1 root root 898 Feb  5 15:18 /etc/apt/mirror.list.d/stg-nginx-plus.list

root@deploy:~$ cat /etc/apt/mirror.list.d/dev-nginx-plus.list
#        ____                         _     _____ _ _      
#       |  _ \ _   _ _ __  _ __   ___| |_  |  ___(_) | ___ 
#       | |_) | | | | '_ \| '_ \ / _ \ __| | |_  | | |/ _ \
#       |  __/| |_| | |_) | |_) |  __/ |_  |  _| | | |  __/
#       |_|    \__,_| .__/| .__/ \___|\__| |_|   |_|_|\___|
#                   |_|   |_|                              

set base_path   /var/spool/apt-mirror/dev
set mirror_path $base_path/mirror
set skel_path   $base_path/skel
set var_path    $base_path/var
set cleanscript $var_path/clean.sh
set defaultarch amd64
set postmirror_script   $var_path/postmirror.sh
set run_postmirror      0
set nthreads    10
set _tilde      0
set ca_certificate      /etc/ssl/nginx/CA.crt
set certificate /etc/ssl/nginx/nginx-repo.crt
set private_key /etc/ssl/nginx/nginx-repo.key

deb https://plus-pkgs.nginx.com/ubuntu precise nginx-plus
deb https://plus-pkgs.nginx.com/ubuntu raring nginx-plus
Find this content useful? Share it with your friends!