Languages
Pages
Legal
Varnish Cache PlusRandom notes for wise web developers
Carlos Abalde, Roberto Moreda {cabalde, moreda}@allenta.com October 2014
Agenda
1. Introduction
2. Varnish 101
3. Invalidations
4. HTTP headers
5. Content composition
6. VAC
7. VCS
8. Device detection
9. Varnish Plus 4.x
10. Q&A
1. Introduction
Disclaimer๏ General understanding of ‘The Varnish Book’ is assumed
‣ This is not the official Varnish Cache training
‣ This is not a Varnish Cache internals course
‣ This is not a Varnish module development course
‣ This is a collection of random notes for web developers willing to make the most of Varnish Cache Plus
๏ OSS Varnish Cache vs. Varnish Cache Plus
‣ 3.x vs. 4.x
Varnish Cache 3.x
๏ The Varnish Book
‣ https://www.varnish-software.com/static/book/
๏ The Varnish Reference Manual
‣ https://www.varnish-cache.org/docs/.../index.html
๏ Default VCL
‣ https://www.varnish-cache.org/trac/.../default.vcl
What everybody should know
Varnish Cache Plus 3.x
๏ Support, advise & training
๏ Varnish Enhanced Cache Invalidation
‣ Hash Two, Hash Ninja…
๏ Varnish Administration Console (VAC)
๏ Varnish Custom Statistics (VCS)
๏ Device detection
Components I
Varnish Cache Plus 3.x
๏ Varnish Tuner
๏ Enhanced HTTP streaming
๏ Packaged binary VMODs
๏ Varnish Paywall
๏ … and more to come shortly!
Components II
Varnish Cache Plus 3.x
๏ 64 bits
๏ Distributions
‣ RedHat Enterprise Linux 5 & 6
‣ Ubuntu Linux 12.04 LTS (precise)
‣ Ubuntu Linux 14.04 LTS (trusty)
‣ Debian Linux 7 (wheezy)
Supported platforms
2. Varnish 101
Caching policy
๏ Varnish Cache Plus would require zero configuration in a perfect world with perfect HTTP citizens
‣ Correct HTTP caching headers
‣ Vary HTTP header used wisely
‣ HTTP cookies used conservatively
๏ By default Varnish Cache Plus will not cache anything marked as private, carrying a cookie or including a '*' Vary HTTP header
VCL
๏ Varnish Configuration Language
‣ Domain specific state engine
‣ No loops, variables, functions…
‣ Command line configuration & Tunable parameters
๏ Translated to C code
๏ Loaded as a dynamically generated shared library
‣ Zero downtime & Blazingly fast
Overview
VCL
๏ Normalize client-input
๏ Pick a backend / director
๏ Re-write / extend client-input
๏ Decide caching policy based on client-input
๏ Access control
๏ Security barriers
vcl_recv I
VCLvcl_recv II
sub vcl_recv { # Backend selection & URL normalization. if (req.http.host ~ "^blogs\.") { set req.backend = blogs; set req.http.host = regsub(req.http.host,"^blogs\.", ""); set req.url = regsub(req.url, "^", "/blogs"); } else { set req.backend = default; } # Poor man's device detection. if (req.http.User-‐Agent ~ "(iPad|iPhone|Android)") { set req.http.X-‐Device = "mobile"; } else { set req.http.X-‐Device = "desktop"; }}
VCL
๏ Sanitize / extend backend response
๏ Override cache duration
‣ beresp.ttl
- s-‐maxage & maxage in Cache-‐Control HTTP header
- Expires HTTP header
- Default TTL
‣ Beware with TTL of hitpass objects
vcl_fetch I
VCLvcl_fetch II
sub vcl_fetch { # Override caching TTL. if (beresp.http.Cache-‐Control !~ "s-‐maxage") { set beresp.ttl = 0; if (bereq.url ~ "\.jpg(\?|$)") { set beresp.ttl = 30s; } } # Never cache a Set-‐Cookie header. if (beresp.ttl > 0s) { unset beresp.http.Set-‐Cookie; } # Create ban-‐lurker friendly objects. set beresp.http.X-‐Url = bereq.url;}
VCLRequest flow I
VCLRequest flow II
Process architecture
VMODs
๏ Shared libraries extending the VCL core
‣ std VMOD
- std.toupper(), std.log(), std.fileread()…
‣ ABI (Application Binary Interface) mismatches
๏ cookie, header, var, curl, digest, geoip, boltsort, memcached, redis, dns…
๏ https://www.varnish-cache.org/vmods
Backends
๏ Multiple backends
‣ Selected at request time based on any request property
๏ Probes
‣ Per-backend periodic health checks
- Interval, timeout, expected response…
๏ Directors
‣ Load balanced backend groups
Error handling
๏ Some backend may be sick for a particular object
‣ Other objects from the same backend can still be accessed
- Unless more than a set amount of objects are added to the saint mode blacklist for a specific backend
๏ Do not request again the object to that backend for a period of time
‣ Grace mode is used when all possible backends for the requested object have been blacklisted
๏ Complement backend probes
Saint mode
Error handling
๏ A graced object is an object that has expired, but is still kept in cache
‣ beresp.ttl vs. beresp.grace
๏ Graced objects are used to
‣ Serve outdated content if the backend is down
- Probes or saint mode is required for this
‣ Serve sightly staled content while fresh versions are fetched
Grace mode
Beyond caching policy
๏ Why restricting VCL / VMODs to implement the caching policy?
๏ Any logic modeled in VCL / VMODs is compiled, embedded & executed in the caching edger layer
‣ 1000x times faster than typical Java / PHP apps
- Strong restrictions
‣ Accounting, paywalling, A/B testing…
varnishtest
๏ Powerful Varnish-specific testing tool
‣ Mocked clients & backends executing / processing HTTP requests against real Varnish Cache Plus instances
‣ http://www.clock.co.uk/...varnishtest
๏ Essential when implementing complex VCL logic
๏ Easily integrable in any CI infrastructure
FAQ๏ When SSL support will be implemented?
‣ "[...] huge waste of time and effort to even think about it."
๏ When SPDY support will be implemented?
‣ "[...] Varnish is not speedy, Varnish is fast! [...]"
๏ What is the recommended value for this bizarre kernel / varnishd parameter I found in some random blog?
‣ Use Varnish Tuner + Fine tune based on necessity
‣ Pay attention to workspaces & syslog messages
3. Invalidations
Overview
๏ Updated objects may be available before TTL expiration
‣ Purges
‣ Forced misses
‣ Bans
‣ Hash Two / Hash Ninja / …
Purges
๏ VCL
๏ Eagerly discards an object along with all its variants
Overview
acl internal { "localhost"; "192.168.55.0"/24;}
sub vcl_recv { if (req.request == "PURGE") { if (client.ip !~ internal) { error 405 "Not allowed."; } return (lookup); }}
sub vcl_hit { if (req.request == "PURGE") { purge; error 200 "Purged."; }}
sub vcl_miss { if (req.request == "PURGE") { purge; error 200 "Purged."; }}
Purges
๏ What if the new object cannot be fetched after the invalidation?
‣ Soft-purges VMOD
‣ Forces misses
๏ What if multiple objects need to be invalidated? What if objects need to be invalidated too frequently?
‣ Bans
‣ Hash Two
Downsides I
Purges
๏ How to invalidate hitpass objects?
‣ Not possible in Varnish Cache Plus 3.x
- Redesigned in Varnish Cache Plus 4.x
- https://www.varnish-cache.org/trac/.../1033
‣ return(pass); during vcl_recv is preferred when possible
Downsides II
Forced misses
๏ VCL
๏ Forces a cache miss for the request
‣ Useful for cache priming scripts
Overview
sub vcl_recv { if (req.http.X-‐Priming-‐Script) { ... set req.hash_always_miss = true; } ...}
Forced misses
๏ Object will always be (re)fetched from the backend
๏ New object is put into cache and used from that point onward
‣ Old object is not evicted until it’s safe to do so
‣ Controls who takes the penalty of waiting for an updated object
๏ Old objects are not freed up until expiration
‣ This is considered a flaw and a fix is expected
Behavior
Bans
๏ VCL or CLI
๏ Lazily discards multiple objects matching an expression
‣ Logical operators + Object attributes + Regular expressions
‣ Only works on objects already in the cache
๏ Ban lurker
‣ Frees up memory + Keeps the ban list at a manageable size
‣ obj.* based expressions
Overview
BansExample
sub vcl_recv { if (req.request == "BAN") { ... if (!req.http.X-‐Ban-‐Url-‐Regexp) { error 400 "Empty URL regexp."; } ban("obj.http.X-‐Url ~ " + req.http.X-‐Ban-‐Url-‐Regexp); }}
sub vcl_fetch { set beresp.http.X-‐Url = req.url;}
sub vcl_deliver { unset resp.http.X-‐Url;}
Hash Two
๏ VCL + VMOD
๏ Workarounds bans scalability
Overview
HTTP/1.x 200 OKTransfer-‐Encoding: chunked...X-‐Tags: C10 P42 P236 P857...
ban obj.http.X-‐Tags ~ "(\s|^)P42(\s|$)"
Hash TwoExample
import hashtwo;
sub vcl_recv { if (req.request == "PURGE") { ... if (hashtwo.purge(req.http.X-‐Tag) != 0) { error 200 "Purged."; } else { error 404 "Not found."; } }}
sub vcl_fetch { set beresp.http.X-‐HashTwo = beresp.http.X-‐Tags; }
4. HTTP headers
Cache related headers
๏ Expires
๏ Cache-Control
๏ Last-Modified
๏ If-Modified-Since
๏ If-None-Match
๏ Etag
๏ Pragma
๏ Vary
๏ Age
Cache-Control
๏ Specifies directives that must be applied by all caching mechanisms (from Varnish Cache Plus to browser cache)
Overview
‣ public | private
‣ no-‐store
‣ no-‐cache
‣ max-‐age
‣ s-‐maxage
‣ must-‐revalidate
‣ no-‐transform
‣ …
Cache-Control
๏ Ignored in incoming client HTTP requests
๏ Only s-‐maxage & max-‐age used in backend HTTP responses to calculate default TTL
‣ Always overrides Expires header
‣ Beware of Age header in client responses
- Objects not cached client side
- https://www.varnish-cache.org/...Caching
beresp.ttl
Vary
๏ Indicates the response returned by the backend server may vary depending on headers received in the request
๏ Object variants & Hit ratio
‣ Vary: Accept-‐Encoding
- Normalization of Accept-‐Encoding header is not required
‣ Vary: User-‐Agent
5. Content composition
Overview๏ Break objects into smaller fragments
‣ Separate cache policy for each fragment
‣ Increase hit ratio
๏ Tools
‣ Edge Side Includes (ESI)
‣ AJAX
- Beware of RTT & Cross domain policy
Edge Side Includes
๏ Subset of ESI Language Specification 1.0
‣ <esi:include src="<URL> " />
‣ <esi:remove>...</esi:remove>
‣ <!-‐-‐esi ...—>
๏ set beresp.do_esi = true;
‣ Separate Varnish requests
๏ Testing ESI in dev environment
6. VAC
Overview
๏ Central control of Varnish Cache Plus servers
‣ Web UI + RESTful API
- Super Fast Purger
๏ Cache group management
‣ Real time statistics, VCL editor, ban submission…
๏ Varnish Agent 2
Super Fast Purger
๏ High performance intermediary distributing invalidation requests to groups of Varnish Cache Plus servers
‣ Leverages speed & flexibility of VCL
‣ Keep-alive workaround
๏ Part of the VAC RESTful API
‣ Trivially integrable in existing applications
Change management
๏ Easily integrable using the VAC RESTful API
‣ git, Mercurial… hooks
‣ Jenkins, Travis, GitLab… CI scripts
๏ Manual VCL bundle generation
๏ Orchestrated / programmed deployments, rollbacks, etc.
7. VCS
Overview
๏ Real-time aggregated statistics
‣ Multiple vstatdprobe daemons
‣ One vstatd daemon
‣ JSON + Time series API
๏ VSM log based
‣ Efficient circular in-memory data structure
‣ std.log("vcs-‐key:" + <key suffix>);
Some ideas
๏ Trending articles or sale products
๏ Cache hits and cache misses
๏ URLs with long load times
๏ URLs with the most 5xx response codes
๏ Where traffic is coming from
๏ …
Example
sub vcl_deliver { std.log("vcs-‐key:" + req.http.host); std.log("vcs-‐key:" + req.http.host + req.url); std.log("vcs-‐key:TOTAL"); if (obj.hits == 0) { std.log("vcs-‐key:MISS"); } }
API I๏ Stats (#requests, #misses, avg ttfb, acc body bytes, #2xx,
#3xx…) for key named “example.com" during the last time windows
‣ GET /key/example.com
๏ Keys that produced the most 5xx responses during the last time window
‣ GET /all/top_5xx
๏ Top 5 requested keys during the last time window
‣ GET /all/top/5?verbose=1
API II
๏ Top 10 most requested keys ending with ‘.gif' during the last time window
‣ GET /match/(.*)%5C.gif$/top
๏ Top 50 slowest backend requests aggregating the last 20 time windows
‣ GET /all/top_ttfb/50?b=20
8. Device detection
Overview๏ VMOD
๏ DeviceAtlas
‣ https://deviceatlas.com
‣ Database locally deployed & Daily updated
๏ OSS alternatives
‣ https://github.com/serbanghita/Mobile-Detect
‣ …
Example
import deviceatlas;
sub vcl_recv { if (deviceatlas.lookup(req.http.User-‐Agent, "isMobilePhone") == "1") { set req.http.X-‐Device = "mobile"; } elsif (deviceatlas.lookup(req.http.User-‐Agent, "isTablet") == "1") { set req.http.X-‐Device = "tablet"; } else { set req.http.X-‐Device = "desktop"; }}
Some ideas
๏ Redirections based on device properties
๏ Backend selection based on device properties
๏ Normalization of the UA header
‣ Caching different versions (i.e. Vary header) of the same object based on normalized UAs
๏ …
9. Varnish Plus 4.x
Highlights๏ Client / backend thread split
‣ Background content refreshing
๏ Redesigned purges
‣ return(purge); during vcl_recv
๏ Directors implemented as VMODs
‣ Consistent hashing director
๏ Distinction between error & synthetic responses
10. Q&A
Top Related