MK

Varnish

Today I would like to configure Varnish for Django usage. Now we have Nginx, Gunicorn and we would like to test Nginx,Varnish,Gunicorn eventually HA Proxy instead of nginx for more Varnish instances. After small deep dive, I must say that all configs are for Varnish 4.0 + because it's significantly different from > 3.0.

Installation

Ubuntu 14.04.

  1. apt-get install apt-transport-https
  2. curl https://repo.varnish-cache.org/GPG-key.txt | apt-key add -
  3. echo "deb https://repo.varnish-cache.org/ubuntu/ trusty varnish-4.1" >> /etc/apt/sources.list.d/varnish-cache.list
  4. apt-get update
  5. apt-get install varnish

 

Note: If you want use Varnish 3.0, then you can skip to last step.

Configuration

First TIP about configuration sounds ,,keep it simple as much as possible"

vim /etc/varnish/default.vcl

backend default { .host = "localhost"; .port = "8080"; }

sub vcl_recv { set req.backend = default; }

 

Much complicated version

backend default {
    .host = "backend1";
    .port = "8080";
}

sub vcl_recv {
  set req.backend = default;
  # unless sessionid/csrftoken is in the request, don't pass ANY cookies (referral_source, utm, etc)  
  if (req.request == "GET" && (req.url ~ "^/static" || (req.http.cookie !~ "sessionid" && req.http.cookie !~ "csrftoken" && req.http.cookie !~ "AUTHENTICATION"))) {  
    unset req.http.Cookie;  
  }  

    #normalize accept-encoding to account for different browsers  
    if (req.http.Accept-Encoding) {
        if (req.url ~ "\.(jpg|png|gif|gz|tgz|bz2|tbz|mp3|ogg)$") {
            # No point in compressing these
            remove req.http.Accept-Encoding;
        } elsif (req.http.Accept-Encoding ~ "gzip") {
            set req.http.Accept-Encoding = "gzip";
        } elsif (req.http.Accept-Encoding ~ "deflate") {
            set req.http.Accept-Encoding = "deflate";
        } else {
            # unknown algorithm
            remove req.http.Accept-Encoding;
        }
    }  

}  

sub vcl_backend_response {  

  # /static and /media files always cached  
  if (req.url ~ "^/static" || req.url ~ "^/media") {  
       unset beresp.http.set-cookie; 
  }

  if (beresp.http.set-cookie !~ "sessionid" && beresp.http.set-cookie !~ "csrftoken" && beresp.http.set-cookie !~ "AUTHENTICATION") {  
    unset beresp.http.set-cookie;
  }

}

 

Testing Varnish on 0.0.0.0:8080

varnishd -P /var/run/varnish.pid -a 0.0.0.0:8080 -f /etc/varnish/default.vcl -T localhost:6082 -t 120

 

Supervisor config

[program:varnish_default]
command=varnishd -F -P /var/run/varnish.pid -a 0.0.0.0:8080 -f /etc/varnish/default.vcl -T localhost:6082 -t 120
user=root
autostart=true
autorestart=true

 

Note: On Varnish 4 was vcl_fetch changed to vcl_backend_response

Salt

Make things generic using SaltStack.

Note: We use reclass instead plain pillar declaration.

varnish:
  server:
    enabled: true
    version: 4.0
    lookup:
      varnish_leonardo_majklk:
        type: leonardo
        name: leonardo_majklk
        bind:
          port: 7000
          host: 0.0.0.0
        backend:
          gunicorn1:
            host: localhost
            port: 80

 

Links