Home » Blogs » acrollet's blog

Using squid as a caching server for drush module downloads

25 Feb 2010
Posted by acrollet

Intro

Due to popular request, I've decided to quickly document how we at UNT Web support use squid as a proxy caching server for drush. The main goal was to speed up module updates, and reduce the load on drupal.org and UNT's internet connection. The instructions are based on my memory, and are for Debian, so there may be gaps here and there depending on your experience/OS.

Squid Setup

Installation was super-simple. (gotta love apt)
  1. apt-get install squid3
Configuration was also quite simple. Basically, the debian package for squid is setup fairly nicely as a proxy caching server out of the box, and you need only configure access. First, add an acl in /etc/squid3/squid.conf for your local network, something like the following:
  1. acl localnet src 192.168.0.0/16 # RFC1918 possible internal network
(make sure to change the subnet to match your network) Next, add a rule allowing access from the local network you just defined:
  1. http_access allow localnet
(If your squid server is on the same box as your drush installation, you just need to allow localhost) Restart squid, and away you go.
  1. /etc/init.d/squid3 restart

Using drush with squid

To make drush use squid, simply do the following:
  1. http_proxy="http://squid-host.example.com:3128/" php /usr/local/drush/drush.php dl cck
(You'll probably want to add an alias)

Is it working?

Let's try and download the inlinetags module.
  1. drush-host # http_proxy="http://squid-host.example.com:3128/" php /usr/local/drush/drush.php dl inlinetags
  2. squid-host # tail -f /var/log/squid3
  3. 1267120330.674    273 192.168.208.29 TCP_MISS/200 8952 GET http://ftp.drupal.org/files/projects/inlinetags-6.x-1.1.tar.gz - DIRECT/140.211.166.142 application/x-gzip
So, we can tell from the log that the request came into squid, but did not find the object cached. So far so good. Let's try again:
  1. drush-host # http_proxy="http://squid-host.example.com:3128/" php /usr/local/drush/drush.php dl inlinetags
  2. squid-host # tail -f /var/log/squid3
  3. 1267121152.216      0 192.168.208.29 TCP_HIT/200 8960 GET http://ftp.drupal.org/files/projects/inlinetags-6.x-1.1.tar.gz - NONE/- application/x-gzip
Good news! This time the request never went out to drupal.org. That's all I have for now, hope it helps someone. Please feel free to comment if you have difficulties with these instructions. (or success!)
Tags:

Thanks for that.

Thanks for taking the time to write this. I just did the setup on ubuntu 9.10, the only thing different was the log files, I had to do sudo tail -f /var/log/squid3/access.log.

Seems to be working. Now to try it with drush. :)

Posted by adrinux on Apr 29th, 2010 at 4:24 pm.