Saving extra-network bandwidth with an Apt Cache

Bandwidth it getting cheaper, but if you have even a moderatly sized network either in a self managed rack, in a colo or at Amazon's EC2 there is a good chance that bandwidth is a good bit of your monthly bill.  Maybe you have a lab in your office or home with several servers, and would like to either save on bandwidth costs or just want faster updates.  Both of these secenarios can benefit from an Apt Cache/Proxy.  There are a few options that you have one is to create a local mirror of one or two of the common package repositories, this is best if you have a wide range of systems (desktops, latptops and servers) which often have changing package requirements.  This solution requires you to host a large amount of data (several hundred GB last time I checked, I'm sure its much larger now), this basicly entails you downloading the entire repository and updating your /etc/apt/sources.list to use that server as the repository.  For server infrastructer this isn't usualy the best route to take.  With servers what is most common is that you have a relative few packages (Apache, PHP, MySQL, SSH on a typical LAMP), these packages exist on all (or most) of your servers and they are one of the few things to actually change from time to time.  In this case, and Apt Cache/Proxy is the best route, this is what I'm going to demonstrate today.  We will be creating a proxy for apt-get which will cache the packages it sees.

Required Tools:

1. Server running Debian or a Debian based distribution such as Ubuntu, with ~50GB of free space[1].
2. Internet Connection

The first thing we need to do is to install the apt-cacher package

apt-get install apt-cacher

Now the easy work is over ;). Lets get the config file setup.

Edit /etc/apt-cacher/apt-cacher.conf

The config file should contain a few important directives first is cache_dir this is the directory that apt-cacher will use to store the cached packages, you can place it anywhere you like, I don't put it my /home folder just to make sure it doesn't end up in my backups.

cache_dir=/var/apt-cache
#The port apt-cacher will use, don't forget to open it in the firewall if need be
daemon=3142
admin_email=root@localhost #email address displayed on error pages
group=www-data #group to use for file permissions
user=www-data #user to use for file permissions
#ip masks that are allowed (or denied) to use use this cache
#I keep it set to * which allows any host, but I bind my cache to an interface that is only available to my VPN or internal network
#192.168.0.2, 192.168.0.3
#192.168.0/16
allowed_hosts=*
denied_hosts=
#same as above, but with IPv6
allowed_hosts_6=fec0::/16
denied_hosts_6=
#apt-cacher can generate reports about cache usage, size and hits/misses.
#The report is accesible via http://[host]/apt-cacher/report
#set to 0 to disable
generate_reports=1
#apt-cache will clean the cache every 24 hours of packages that no longer exist in the repositories
#set to 0 to disable
clean_cache=1
#Set this to 1 to prohibit apt-cacher from fetching new files (it will only provide files that are cached)
offline_mode=0
#log file
logdir=/var/log/apt-cacher
#The number of hours to keep a file in cache before it is requried to be downloaded again
#0 means that apt-cacher will check to see if the file in the repository is newer than the cached one (based on HTTP headers)
expire_hours=0
#if you need to proxy your outbound connections set use_proxy and use_proxy_auth accordingly

#http_proxy=proxy.example.com:8080

use_proxy=0
#http_proxy_auth=proxyuser:proxypass
use_proxy_auth=0

#use this to bind apt-cacher to a specific interface for the outboud connection (for fetching packages)
interface=

# Use 'k' or 'm' to use kilobytes or megabytes / second: eg, 'limit=25k'.
# Use 0 or a negative value for no rate limiting.
limit=0

#if you want a lot more data in your logs set this to 1
debug=0

That wasn't actually too hard was it.  Lets set apt-cacher to auto start with the server; edit /etc/default/apt-cacher set

AUTOSTART=1

Restart apt-cacher

/etc/init.d/apt-cacher restart

And were done with the server.  Now we just need to configure the other systems to use the cache/proxy.

So now on all of our systems that are going to use this cache edit (or create) the file /etc/apt/apt.conf.d/90-apt-proxy.conf, add just one line.

Acquire::http::proxy "http://[ip or hostname]:3142";

Edit the port if you did so above ("daemon=" line of the config file), and you should be good to go!

 The amount of free space will depend on how many packages you use.  For my typical test server running a LAMP config, I use about 500MB of disk space for the cache.