Varnish : use multiple backends depending on host / URL

With the lack of public IPv4 address and the growth of virtualization, one of the commons solutions consists in using revese proxies to distribute the traffic of a physical host, having a public IPv4 address, between multiple virtual machines (VM), as shown in the schema below :

In this example, we will assume that each VM hosts a HTTP server (Apache, lighttpd, nginx, etc.) listening on port 80, and that :

• www.myhost1.com should use VM1
• www.myhost2.com should use VM2
• www.myhost3.com should use VM3

Varnish

Varnish, one of the best known reverse proxy engines (actually a caching HTTP reverse proxy), can be very useful in such a situation, by setting specific backends (ie. a VM) depending on the required hostname.

I'm not going to explain in this post how to build the whole Varnish VCL file, but you'll find very useful resources on Varnish's wiki :  Varnish Default VCL Example.

Defining multiple backends

First of all, you will need to define the different backends that Varnish will rely on. In the example above, we have 3 VMs with each a private IPv4 in the 10.0.0.0/24 range. At the beggining of your VCL, set your backends :

backend vm1 {
    .host = "10.0.0.11";
    .port = "80";
    .connect_timeout = 6000s;
    .first_byte_timeout = 6000s;
    .between_bytes_timeout = 6000s;
}

backend vm2 {
    .host = "10.0.0.12";
    .port = "80";
    .connect_timeout = 6000s;
    .first_byte_timeout = 6000s;
    .between_bytes_timeout = 6000s;
}

backend vm3 {
    .host = "10.0.0.13";
    .port = "80";
    .connect_timeout = 6000s;
    .first_byte_timeout = 6000s;
    .between_bytes_timeout = 6000s;
}

Using the appropriate backend

To define which backend (local HTTP server) should be used by Varnish to respond HTTP requests, we will set a few custom rules in the vcl_recv section of our VCL config file :

# Default backend is set to VM1
set req.backend = vm1;

if (req.http.host == "www.myhost2.com") {
    set req.backend = vm2;
}

if (req.http.host == "www.myhost3.com") {
    set req.backend = vm3;
}

Now restart Varnish and try to connect to one of the 3 hostnames : you should be forwarded to the appropriate backend.

NB : this post only covers HTTP reverse proxying, as Varnish does not handle HTTPS. If you need to use HTTPS, there are 2 options : using Apache as HTTPS reverse proxy (in French), or combining Varnish with another server such as Pound, which will handle the SSL communcation establishment.