How to create a local Ubuntu mirror

If you need to update more than one Ubuntu based Workstation or Server on a regular base, it can save you a good amount of bandwidth if you provide new packages and updates through a local Ubuntu mirror. In this document i write down the steps i’ve used to create one, based on Ubuntu 10.04 LTS, apt-mirror and the NGINX webserver.

Please read carefully through this tutorial as i will not always mention to adapt the settings to your own environment (hostnames and ports)

• Base Installation
• Network Time Protocol Installation
• Postfix Installation
• Install apt-mirror
• Install Nginx
• Client configuration

Base Installation

1. Install the Server system (or use an existing one which you would/could use for this purpose). Configure a /mirror mount point which has plenty of space available. At the date of writing, one distribution for both architectures (x86 and x64) eats up about 100 GiB of data.

2. Configure the server to use the proxy for apt operations by creating /etc/apt/apt.conf.d/09proxy and inserting the following line:

Acquire::http::proxy "http://yourproxy:8080/";

3. Perform a initial server update to get the latest and greatest packages available:

apt-get update
apt-get dist-upgrade

Network Time Protocol Installation

We want to make sure that our server has the correct time.

1. Install the Package

apt-get install ntp

2. Configure your time sources by editing /etc/ntp.conf (this step is only necessary if you have a local NTP server and can be skipped if you’re using Internet based timeservers)

server 10.138.15.160 version 3 prefer
server 10.154.15.160 version 3

3. Set the system time initially by executing:

ntpdate -b 10.138.15.160

4. Restart the NTP Daemon:

/etc/init.d/ntp restart

Postfix Installation

Our server should be able to send us emails (with status updates, etc.)

1. Install the Package

apt-get install postfix

2. During the installation you should choose “Satellite System” and configure your upstream SMTP Server. If you already have installed Postfix, do a

dpkg-reconfigure --configure postfix

to start the reconfiguration process again

3. Test your mail setup by executing:

sendmail me@mydomain.com << EOF
Hello
EOF

4. Make mails going to root and apt-mirror (this use does not exist already, but will when we are finished) forwarded to your real address. Edit /etc/aliases and append/replace:

root: me@mydomain.com
apt-mirror: me@mydomain.com

Afterwards, create the aliases database by executing:

newaliases

5. Restart the postifx Daemon:

/etc/init.d/postfix restart

Install apt-mirror

This is our real workhorse which will do the actual retrievement of packages and updates:

1. Install the package

apt-get install apt-mirror

2. Configure which distributions you want to mirror by editing /etc/apt/mirror.list and inserting the following content (adopt to your needs):

############# config ##################
#
# set base_path    /var/spool/apt-mirror
#
# set mirror_path  $base_path/mirror
# set skel_path    $base_path/skel
# set var_path     $base_path/var
# set cleanscript $var_path/clean.sh
# set defaultarch  <running host architecture>
# set postmirror_script $var_path/postmirror.sh
# set run_postmirror 0
 
set base_path /mirror
set mirror_path /mirror/www
set nthreads     8
set defaultarch i386
set _tilde 0
 
# lucid lynx (10.04 LTS) for i386 architecture
deb http://at.archive.ubuntu.com/ubuntu lucid main restricted universe multiverse
deb http://at.archive.ubuntu.com/ubuntu lucid-security main restricted universe multiverse
deb http://at.archive.ubuntu.com/ubuntu lucid-updates main restricted universe multiverse
deb http://at.archive.ubuntu.com/ubuntu lucid-proposed main restricted universe multiverse
deb http://at.archive.ubuntu.com/ubuntu lucid-backports main restricted universe multiverse
 
# lucidy lynx (10.04 LTS) for amd64 architecture
deb-amd64 http://at.archive.ubuntu.com/ubuntu lucid main restricted universe multiverse
deb-amd64 http://at.archive.ubuntu.com/ubuntu lucid-security main restricted universe multiverse
deb-amd64 http://at.archive.ubuntu.com/ubuntu lucid-updates main restricted universe multiverse
deb-amd64 http://at.archive.ubuntu.com/ubuntu lucid-proposed main restricted universe multiverse
deb-amd64 http://at.archive.ubuntu.com/ubuntu lucid-backports main restricted universe multiverse
 
 
# oneiric ocelot (11.10) for i386 architecture
deb http://at.archive.ubuntu.com/ubuntu oneiric main restricted universe multiverse
deb http://at.archive.ubuntu.com/ubuntu oneiric-security main restricted universe multiverse
deb http://at.archive.ubuntu.com/ubuntu oneiric-updates main restricted universe multiverse
deb http://at.archive.ubuntu.com/ubuntu oneiric-proposed main restricted universe multiverse
deb http://at.archive.ubuntu.com/ubuntu oneiric-backports main restricted universe multiverse
 
# oneiric ocelot (11.10) for amd64 architecture
deb-amd64 http://at.archive.ubuntu.com/ubuntu oneiric main restricted universe multiverse
deb-amd64 http://at.archive.ubuntu.com/ubuntu oneiric-security main restricted universe multiverse
deb-amd64 http://at.archive.ubuntu.com/ubuntu oneiric-updates main restricted universe multiverse
deb-amd64 http://at.archive.ubuntu.com/ubuntu oneiric-proposed main restricted universe multiverse
deb-amd64 http://at.archive.ubuntu.com/ubuntu oneiric-backports main restricted universe multiverse
 
# security patches for lucid
deb-i386 http://security.ubuntu.com/ubuntu lucid-security main restricted
deb-i386 http://security.ubuntu.com/ubuntu lucid-security universe
deb-i386 http://security.ubuntu.com/ubuntu lucid-security multiverse
deb-amd64 http://security.ubuntu.com/ubuntu lucid-security main restricted
deb-amd64 http://security.ubuntu.com/ubuntu lucid-security universe
deb-amd64 http://security.ubuntu.com/ubuntu lucid-security multiverse
 
# security patches for oneiric
deb-i386 http://security.ubuntu.com/ubuntu oneiric-security main restricted
deb-i386 http://security.ubuntu.com/ubuntu oneiric-security universe
deb-i386 http://security.ubuntu.com/ubuntu oneiric-security multiverse
deb-amd64 http://security.ubuntu.com/ubuntu oneiric-security main restricted
deb-amd64 http://security.ubuntu.com/ubuntu oneiric-security universe
deb-amd64 http://security.ubuntu.com/ubuntu oneiric-security multiverse
 
## remove the comments if you want to get the sources too
#deb-src http://at.archive.ubuntu.com/ubuntu lucid main restricted universe multiverse
#deb-src http://at.archive.ubuntu.com/ubuntu lucid-security main restricted universe multiverse
#deb-src http://at.archive.ubuntu.com/ubuntu lucid-updates main restricted universe multiverse
#deb-src http://at.archive.ubuntu.com/ubuntu lucid-proposed main restricted universe multiverse
#deb-src http://at.archive.ubuntu.com/ubuntu lucid-backports main restricted universe multiverse
 
clean http://at.archive.ubuntu.com/ubuntu
clean http://security.ubuntu.com

In my case i am mirroring 10.04 LTS and 11.10, both architectures available and the security updates

3. Create the necessary directories:

mkdir /mirror/var /mirror/www /mirror/skel

4. Configure wget (which is used by apt-mirror) to go through the http proxy by editing /etc/wgetrc and enabling the following options (skip this one if you aren’t using a http proxy to connect to the internet)

http_proxy
use_proxy

5. Enable the cron entry for apt-mirror by editing /etc/cron.d/apt-mirror and removing the hash at the beginning of the line. Also adopt the time when to run the updater. In my case, the file looks like:

30 02   * * *   apt-mirror      /usr/bin/apt-mirror 2>&1 1>/mirror/apt-mirror.log

This one will run apt-mirror every night at 02:30 am

6. Make apt-mirror send you a status report of it’s last run by editing /mirror/var/postmirror.sh (this one is executed by apt-mirror after it has finished) and inserting the following script:

#!/bin/bash
# environment
export TEMPFILE=/tmp/apt-mirror-mail.tmp.${RANDOM}
export LOGFILE=/mirror/apt-mirror.log
export SENDMAIL=/usr/sbin/sendmail
 
# run clean script
CS=/mirror/var/clean.sh
chmod 755 ${CS}
${CS} 2>&1 >> ${LOGFILE}
 
# send status mail to apt-mirror
echo From: `whoami`@`hostname` >> ${TEMPFILE}
echo To: `whoami` >> ${TEMPFILE}
echo Subject: apt-mirror fetch status from `date` >> ${TEMPFILE}
echo >> ${TEMPFILE}
cat ${LOGFILE} >> ${TEMPFILE}
 
${SENDMAIL} `whoami` < ${TEMPFILE}
rm ${TEMPFILE}

This one will simply run the cleanup script (which in turn removes obsolete files) and send us the logfile

7. Do a initial mirror run

su --command apt-mirror apt-mirror

This will probably take a few hours to complete.

Install nginx

We serve our updates to the clients using the nginx Webserver

1. Install the Package

apt-get install nginx

2. Remove the default configuration file

rm /etc/nginx/sites-enabled/default

3. Create a new configuration by creating /etc/nginx/sites-available/ubuntu-mirror with the following content:

server {
        listen   80 default;
        server_name susrtumi1.health.local susrtumi1;
        gzip off;
 
        access_log  /var/log/nginx/ubuntu-mirror-access.log;
 
        location /ubuntu {
                alias  /mirror/www/at.archive.ubuntu.com;
                autoindex on;
        }
 
        location /ubuntu-security {
                alias /mirror/www/security.ubuntu.com;
                autoindex on;
        }
}

4. Relink the new configuration file to be used by nginx:

cd /etc/nginx/sites-enabled
ln -s ../sites-available/ubuntu-mirror

5. Set the correct permissions:

chown -R apt-mirror:www-data /mirror
chmod -R 775 /mirror

6. Restart nginx

/etc/init.d/nginx restart

Client configuration

1. Edit /etc/apt/sources.list to point to our local server. Below is a example for 10.04 LTS:

deb http://susrtumi1.health.local/ubuntu/ubuntu lucid main restricted
deb http://susrtumi1.health.local/ubuntu/ubuntu lucid-updates main restricted
deb http://susrtumi1.health.local/ubuntu/ubuntu lucid universe
deb http://susrtumi1.health.local/ubuntu/ubuntu lucid-updates universe
deb http://susrtumi1.health.local/ubuntu/ubuntu lucid multiverse
deb http://susrtumi1.health.local/ubuntu/ubuntu lucid-updates multiverse
deb http://susrtumi1.health.local/ubuntu-security/ubuntu lucid-security main restricted
deb http://susrtumi1.health.local/ubuntu-security/ubuntu lucid-security universe
deb http://susrtumi1.health.local/ubuntu-security/ubuntu lucid-security multiverse

2. If you are using non mirrored repositories (like google’s chrome repository) and your http proxy is not able to resolve your local mirror, use CNTLM to access your proxy for everything except the local mirror:

2.1. Install the CNTLM package

apt-get install cntlm

2.2. Configure CNTLM to use no proxy for the local update server by inserting the following line to it’s configuration:

NoProxy susrtumi1*

2.3. Configure APT to use the proxy by creating /etc/apt/apt.conf.d/09proxy with the following content:

Acquire::http::proxy "http://localhost:5865/";

3. Update your system

apt-get update
apt-get dist-upgrade

You should notice that apt is now using your local mirror. Congratulations!