Michael BOUVY
CTO E-commerce

Varnish : use multiple backends depending on host / URL

varnish http cache Proxy
Published on 2013/11/01

With the lack of public IPv4 address and the growth of virtualization, one of the commons solutions consists in using revese proxies to distribute the traffic of a physical host, having a public IPv4 address, between multiple virtual machines (VM), as shown in the schema below :

Varnish_front

In this example, we will assume that each VM hosts a HTTP server (Apache, lighttpd, nginx, etc.) listening on port 80, and that :

  • www.myhost1.com should use VM1
  • www.myhost2.com should use VM2
  • www.myhost3.com should use VM3

Varnish

Varnish, one of the best known reverse proxy engines (actually a caching HTTP reverse proxy), can be very useful in such a situation, by setting specific backends (ie. a VM) depending on the required hostname.

Varnish

 

I'm not going to explain in this post how to build the whole Varnish VCL file, but you'll find very useful resources on Varnish's wiki :  Varnish Default VCL Example.

Defining multiple backends

First of all, you will need to define the different backends that Varnish will rely on. In the example above, we have 3 VMs with each a private IPv4 in the 10.0.0.0/24 range. At the beggining of your VCL, set your backends :

backend vm1 {
    .host = "10.0.0.11";
    .port = "80";
    .connect_timeout = 6000s;
    .first_byte_timeout = 6000s;
    .between_bytes_timeout = 6000s;
}

backend vm2 {
    .host = "10.0.0.12";
    .port = "80";
    .connect_timeout = 6000s;
    .first_byte_timeout = 6000s;
    .between_bytes_timeout = 6000s;
}

backend vm3 {
    .host = "10.0.0.13";
    .port = "80";
    .connect_timeout = 6000s;
    .first_byte_timeout = 6000s;
    .between_bytes_timeout = 6000s;
}

Using the appropriate backend

To define which backend (local HTTP server) should be used by Varnish to respond HTTP requests, we will set a few custom rules in the vcl_recv section of our VCL config file :

# Default backend is set to VM1
set req.backend = vm1;

if (req.http.host == "www.myhost2.com") {
    set req.backend = vm2;
}

if (req.http.host == "www.myhost3.com") {
    set req.backend = vm3;
}

Now restart Varnish and try to connect to one of the 3 hostnames : you should be forwarded to the appropriate backend.

NB : this post only covers HTTP reverse proxying, as Varnish does not handle HTTPS. If you need to use HTTPS, there are 2 options : using Apache as HTTPS reverse proxy (in French), or combining Varnish with another server such as Pound, which will handle the SSL communcation establishment.
Michael BOUVY

I'm Michael BOUVY, CTO and co-founder of Click&Mortar, a digital agency based in Paris, France, specialized in e-commerce.

Over the last years, I've worked as an Engineering Manager and CTO for brands like Zadig&Voltaire and Maisons du Monde.

With more than 10 years experience in e-commerce platforms, I'm always looking for new challenges, feel free to get in touch!