Become Happier by using Varnish Cache
-
Upload
stefan-maurer -
Category
Technology
-
view
1.614 -
download
5
description
Transcript of Become Happier by using Varnish Cache
![Page 1: Become Happier by using Varnish Cache](https://reader035.fdocuments.in/reader035/viewer/2022081403/554fb30fb4c90586258b529a/html5/thumbnails/1.jpg)
APACHE SLING & FRIENDS TECH MEETUP BERLIN, 23-25 SEPTEMBER 2013
Become happier by using Varnish Cache Stefan Maurer
Senior Software Engineer, Namics AG
![Page 2: Become Happier by using Varnish Cache](https://reader035.fdocuments.in/reader035/viewer/2022081403/554fb30fb4c90586258b529a/html5/thumbnails/2.jpg)
Agenda
adaptTo() 2013 2
Varnish Cache at a glance Use Cases and Architectures Limitations and Pitfalls
![Page 3: Become Happier by using Varnish Cache](https://reader035.fdocuments.in/reader035/viewer/2022081403/554fb30fb4c90586258b529a/html5/thumbnails/3.jpg)
Varnish Cache at a glance
adaptTo() 2013 3
Reverse Proxy Cache 1.0.0 in 2006 Current version is 3.0.4 Free under BSD License Available for various Linux Distributions,
Max OS X, Open Solaris, Solaris 10, FreeBSD and Windows (with Cygwin)
![Page 4: Become Happier by using Varnish Cache](https://reader035.fdocuments.in/reader035/viewer/2022081403/554fb30fb4c90586258b529a/html5/thumbnails/4.jpg)
Varnish Cache at a glance
adaptTo() 2013 4
Designed as HTTP accelerator – not more, not less “Varnish is a modern program, designed
and written for modern operating systems” https://www.varnish-cache.org/trac/wiki/VarnishFeatures
Storage malloc or file
Load balancing ESI (subset)
![Page 5: Become Happier by using Varnish Cache](https://reader035.fdocuments.in/reader035/viewer/2022081403/554fb30fb4c90586258b529a/html5/thumbnails/5.jpg)
Varnish Cache at a glance
adaptTo() 2013 5
Process architecture Management process and child process Configuration change without restart
Source: www.varnish-software.com/static/book/Tuning.html
![Page 6: Become Happier by using Varnish Cache](https://reader035.fdocuments.in/reader035/viewer/2022081403/554fb30fb4c90586258b529a/html5/thumbnails/6.jpg)
Varnish Cache at a glance
adaptTo() 2013 6
Configuration Startup parameters Varnish Configuration Language (VCL)
sub vcl_fetch {
if (req.url ~ "\.jpg$") {
set beresp.ttl = 60s;
}
}
![Page 7: Become Happier by using Varnish Cache](https://reader035.fdocuments.in/reader035/viewer/2022081403/554fb30fb4c90586258b529a/html5/thumbnails/7.jpg)
Varnish Cache at a glance
adaptTo() 2013
Source: www.varnish-software.com/static/book/VCL_Basics.html
![Page 8: Become Happier by using Varnish Cache](https://reader035.fdocuments.in/reader035/viewer/2022081403/554fb30fb4c90586258b529a/html5/thumbnails/8.jpg)
Varnish Cache at a glance
adaptTo() 2013 8
Logging Logs to shared memory Separate application for display and write
back
![Page 9: Become Happier by using Varnish Cache](https://reader035.fdocuments.in/reader035/viewer/2022081403/554fb30fb4c90586258b529a/html5/thumbnails/9.jpg)
Agenda
adaptTo() 2013 9
Varnish Cache at a glance
Use Cases and Architectures Limitations and Pitfalls
![Page 10: Become Happier by using Varnish Cache](https://reader035.fdocuments.in/reader035/viewer/2022081403/554fb30fb4c90586258b529a/html5/thumbnails/10.jpg)
Use Cases and Architectures
adaptTo() 2013 10
Use case 1: Static resources Use case 2: REST API calls Use case 3: Full website
![Page 11: Become Happier by using Varnish Cache](https://reader035.fdocuments.in/reader035/viewer/2022081403/554fb30fb4c90586258b529a/html5/thumbnails/11.jpg)
Use Case 1: Static Resources
adaptTo() 2013 11
Use Case Cache resources like CSS / JS / static
images “forever”
Benefit with Varnish Internal network traffic decreases
significant Do not bore backend systems ;-)
Solution Cookie-less domain sc.customer.com
served by Varnish
![Page 12: Become Happier by using Varnish Cache](https://reader035.fdocuments.in/reader035/viewer/2022081403/554fb30fb4c90586258b529a/html5/thumbnails/12.jpg)
CQ5 Publisher
Dispatcher www.customer.com
Varnish Cache sc.customer.com
Use Case 1: Static Resources
adaptTo() 2013 12
Architecture Sample 1 Browser
Varnish Cache sc.customer.com
Dispatcher www.customer.com
Load Balancer
CQ5 Publisher
![Page 13: Become Happier by using Varnish Cache](https://reader035.fdocuments.in/reader035/viewer/2022081403/554fb30fb4c90586258b529a/html5/thumbnails/13.jpg)
Use Case 1: Static Resources
adaptTo() 2013 13
Architecture Sample 2 Browser
Varnish Cache sc.customer.com
Dispatcher www.customer.com
Load Balancer www.customer.com
CQ5 Publisher
Dispatcher www.customer.com
CQ5 Publisher
![Page 14: Become Happier by using Varnish Cache](https://reader035.fdocuments.in/reader035/viewer/2022081403/554fb30fb4c90586258b529a/html5/thumbnails/14.jpg)
Use Case 1: Static Resources
adaptTo() 2013 14
Why not directly connect Varnish with CQ5 Publisher? Possible. But requires configuration for
restricted paths like /system/console
Changes in CQ Load resources from static domain with a
timestamp: <script
type="text/javascript"
src="http://sc.customer.com/etc/designs/customer/clientlib.201309231000.js"
></script>
<img src="http://sc.customer.com/content/dam/logo.201309231000.png" />
![Page 15: Become Happier by using Varnish Cache](https://reader035.fdocuments.in/reader035/viewer/2022081403/554fb30fb4c90586258b529a/html5/thumbnails/15.jpg)
Use Case 1: Static Resources
adaptTo() 2013 15
backend default {
.host = "dispatcher-hostname";
.port = "80";
}
sub vcl_recv {
if (req.url !~ "^/etc/designs/" && req.url !~ "^/content/dam/"){
error 405 "Not allowed.";
}
unset req.http.cookie;
}
sub vcl_fetch {
unset beresp.http.set-cookie;
unset beresp.http.expires;
unset beresp.http.Etag;
set beresp.http.Cache-Control = "max-age=86400";
set beresp.ttl = 4w;
set beresp.http.magicmarker = "1";
}
sub vcl_deliver {
if (resp.http.magicmarker) {
unset resp.http.magicmarker;
set resp.http.age = "0";
}
}
![Page 16: Become Happier by using Varnish Cache](https://reader035.fdocuments.in/reader035/viewer/2022081403/554fb30fb4c90586258b529a/html5/thumbnails/16.jpg)
Use Cases and Architectures
adaptTo() 2013 16
Use case 1: Static resources
Use case 2: REST API calls Use case 3: Full website
![Page 17: Become Happier by using Varnish Cache](https://reader035.fdocuments.in/reader035/viewer/2022081403/554fb30fb4c90586258b529a/html5/thumbnails/17.jpg)
Use Case 2: REST API Calls
adaptTo() 2013 17
Use Case Cache requests to backend APIs
Benefit with Varnish Reduce load on backend systems Reduce response time Handle backend downtime Custom time to life for each backend
Solution Call backend through Varnish Cache
![Page 18: Become Happier by using Varnish Cache](https://reader035.fdocuments.in/reader035/viewer/2022081403/554fb30fb4c90586258b529a/html5/thumbnails/18.jpg)
Use Case 2: REST API Calls
adaptTo() 2013 18
Sample: Yahoo Finance API http://finance.yahoo.com/d/quotes.csv?s=ADBE&f=snd1l1c1p2
Last trade date, price, change and percents But: has limit of requests/day
CQ Publisher calls Varnish with prefix “finance” http://varnish/finance/d/quotes.csv?s=ADBE&f=snd1l1c1p2
Varnish executes API Call and caches the response for one hour
![Page 19: Become Happier by using Varnish Cache](https://reader035.fdocuments.in/reader035/viewer/2022081403/554fb30fb4c90586258b529a/html5/thumbnails/19.jpg)
Use Case 2: REST API Calls
adaptTo() 2013 19
Architecture
Varnish Cache
Yahoo Finance API
CQ5 Publisher
Backend 2..n
![Page 20: Become Happier by using Varnish Cache](https://reader035.fdocuments.in/reader035/viewer/2022081403/554fb30fb4c90586258b529a/html5/thumbnails/20.jpg)
Use Case 2: REST API Calls
adaptTo() 2013 20
backend yahoofinance {
.host = "finance.yahoo.com";
.port = "80";
}
sub vcl_recv {
if (req.url ~ "^/finance") {
set req.backend = yahoofinance;
set req.url = regsub(req.url, "^/finance", "");
unset req.http.cookie;
}
}
sub vcl_fetch {
if(req.backend == yahoofinance) {
set beresp.ttl = 1h;
unset beresp.http.set-cookie;
return(deliver);
}
}
![Page 21: Become Happier by using Varnish Cache](https://reader035.fdocuments.in/reader035/viewer/2022081403/554fb30fb4c90586258b529a/html5/thumbnails/21.jpg)
Use Cases and Architectures
adaptTo() 2013 21
Use case 1: Static resources Use case 2: REST API calls
Use case 3: Full website
![Page 22: Become Happier by using Varnish Cache](https://reader035.fdocuments.in/reader035/viewer/2022081403/554fb30fb4c90586258b529a/html5/thumbnails/22.jpg)
Use Case 3: Full Website
adaptTo() 2013 22
Use Case Cache generated HTML and related
resources
Benefit with Varnish Reduce load on backend systems Reduce response time Granular cache invalidation
![Page 23: Become Happier by using Varnish Cache](https://reader035.fdocuments.in/reader035/viewer/2022081403/554fb30fb4c90586258b529a/html5/thumbnails/23.jpg)
CQ5 Publisher
Dispatcher
Use Case 3: Full Website
adaptTo() 2013 23
Architecture Sample 1
Browser
Varnish Cache
Dispatcher
CQ5 Publisher
![Page 24: Become Happier by using Varnish Cache](https://reader035.fdocuments.in/reader035/viewer/2022081403/554fb30fb4c90586258b529a/html5/thumbnails/24.jpg)
CQ5 Publisher
Varnish Cache
Use Case 3: Full Website
Dispatcher
adaptTo() 2013 24
Architecture Sample 2
Browser
Varnish Cache
Load Balancer
CQ5 Publisher
Dispatcher
![Page 25: Become Happier by using Varnish Cache](https://reader035.fdocuments.in/reader035/viewer/2022081403/554fb30fb4c90586258b529a/html5/thumbnails/25.jpg)
Use Case 3: Full Website
adaptTo() 2013 25
Cache Invalidation
Short TTL (1 minute) Long TTL (1 month)
Frequently called pages + +
Seldom called pages - +
Instant invalidation - +
Low complexity + -
![Page 26: Become Happier by using Varnish Cache](https://reader035.fdocuments.in/reader035/viewer/2022081403/554fb30fb4c90586258b529a/html5/thumbnails/26.jpg)
Use Case 3: Full Website
adaptTo() 2013 26
Excludes from cache sub vcl_recv {
if (req.url ~ "\?"
|| req.url ~ "\/cug_") {
return(pass);
}
}
sub vcl_fetch {
if (beresp.http.Dispatcher ~ "no-cache"
|| beresp.http.cache-control ~ "(no-cache|private)"
|| beresp.http.pragma ~ "no-cache") {
set beresp.ttl = 0s;
}
else {
set beresp.ttl = 4w;
}
}
![Page 27: Become Happier by using Varnish Cache](https://reader035.fdocuments.in/reader035/viewer/2022081403/554fb30fb4c90586258b529a/html5/thumbnails/27.jpg)
Use Case 3: Full Website
adaptTo() 2013 28
Invalidate a page with “smart bans” acl purgers { "127.0.0.1"; "192.168.0.0"/24; } sub vcl_recv { if (req.http.X-Purge-URL) { if (!client.ip ~ purgers) { error 405 "Method not allowed"; } ban("obj.http.x-url == " + req.http.X-Purge-URL); error 200 "Banned URL"; } } sub vcl_fetch { set beresp.http.x-url = req.url; }
![Page 28: Become Happier by using Varnish Cache](https://reader035.fdocuments.in/reader035/viewer/2022081403/554fb30fb4c90586258b529a/html5/thumbnails/28.jpg)
Use Case 3: Full Website
adaptTo() 2013 29
Invalidate from CQ Publisher with Modification Listener Modify: /content/customer/en/news/adaptto
Risky: Purge only page and sub pages
/content/customer/en/news/adaptto.*
Safe: Purge whole language tree /content/customer/en.*
![Page 29: Become Happier by using Varnish Cache](https://reader035.fdocuments.in/reader035/viewer/2022081403/554fb30fb4c90586258b529a/html5/thumbnails/29.jpg)
Agenda
adaptTo() 2013 30
Varnish Cache at a glance Use Cases and Architectures
Limitations and Pitfalls
![Page 30: Become Happier by using Varnish Cache](https://reader035.fdocuments.in/reader035/viewer/2022081403/554fb30fb4c90586258b529a/html5/thumbnails/30.jpg)
Limitations and Pitfalls
adaptTo() 2013 31
SSL termination Problem
Varnish has no SSL termination Justification
Varnish source code: 80’000 lines of code OpenSSL: 340’000 lines of code
Solution Use another tier in front of Varnish (nginx, pond, …)
![Page 31: Become Happier by using Varnish Cache](https://reader035.fdocuments.in/reader035/viewer/2022081403/554fb30fb4c90586258b529a/html5/thumbnails/31.jpg)
Limitations and Pitfalls
adaptTo() 2013 32
Avoid “double purge” Assumed situation
2 Publisher and 1 Varnish Problem
Modification listener is executed on both Publishers 2 purge requests
Solution Define “master” publisher and handle failover
![Page 32: Become Happier by using Varnish Cache](https://reader035.fdocuments.in/reader035/viewer/2022081403/554fb30fb4c90586258b529a/html5/thumbnails/32.jpg)
Limitations and Pitfalls
adaptTo() 2013 33
Proxy Assumed situation
- Varnish is used to cache REST API Calls - Your customer force to use proxy for all external requests
Problem Varnish has no configuration for backend through proxy
Solution Set proxy as backend and tweak URL
![Page 33: Become Happier by using Varnish Cache](https://reader035.fdocuments.in/reader035/viewer/2022081403/554fb30fb4c90586258b529a/html5/thumbnails/33.jpg)
Limitations and Pitfalls
adaptTo() 2013 34
Proxy solution backend proxy {
.host = "corporate-proxy-hostname";
.port = "8080";
}
sub vcl_recv {
set req.backend = proxy;
set req.http.X-Forwarded-For = client.ip;
set req.http.host = "dispatcher-hostname";
set req.http.port = 80;
set req.url = "http://" + req.http.host + ":" +
req.http.port + req.url;
}
![Page 34: Become Happier by using Varnish Cache](https://reader035.fdocuments.in/reader035/viewer/2022081403/554fb30fb4c90586258b529a/html5/thumbnails/34.jpg)
adaptTo() 2013 35
Thank you
![Page 35: Become Happier by using Varnish Cache](https://reader035.fdocuments.in/reader035/viewer/2022081403/554fb30fb4c90586258b529a/html5/thumbnails/35.jpg)
Namics
adaptTo() 2012 36
5 Locations in Germany and Switzerland Frankfurt, Hamburg, München, Zürich, St. Gallen
Many interesting projects with CQ5 and other technologies
400 + 1 (?) Employees www.namics.com/jobs
![Page 36: Become Happier by using Varnish Cache](https://reader035.fdocuments.in/reader035/viewer/2022081403/554fb30fb4c90586258b529a/html5/thumbnails/36.jpg)
Resources and Further Links
adaptTo() 2013 37
https://www.varnish-cache.org/trac/wiki/VCLExampleLongerCaching
https://www.varnish-software.com/static/book/Tuning.html https://www.varnish-
software.com/static/book/_images/vcl.png http://blog.namics.com/2013/01/www-zh-ch-ist-50x-schneller-
und-stabiler.html http://blog.namics.com/2010/12/varnish-cache.html
![Page 37: Become Happier by using Varnish Cache](https://reader035.fdocuments.in/reader035/viewer/2022081403/554fb30fb4c90586258b529a/html5/thumbnails/37.jpg)
Variable availability in VCL
adaptTo() 2012 38