Post on 06-Jul-2020
Caching and tuning funfor high scalability
Wim GoddenCu.be Solutions
Who am I ?Wim Godden (@wimgtr)Owner of Cu.be Solutions (http://cu.be)Open Source developer since 1997Developer of OpenXZend Certified EngineerZend Framework Certified EngineerMySQL Certified Developer
Who are you ?Developers ?System/network engineers ?Managers ?
Caching experience ?
Goals of this tutorialEverything about caching and tuningA few techniques
How-to
How-NOT-to
→ Increase reliability, performance and scalability
5 visitors/day → 5 million visitors/day
(Don't expect miracle cure !)
LAMP
Architecture
Test page3 DB-queries
select firstname, lastname, email from user where user_id = 5;
select title, createddate, body from article order by createddate desc limit 5;
select title, createddate, body from article order by score desc limit 5;
Page just outputs result
Our base benchmarkApachebench = useful enoughResult ?
Single webserver ProxyStatic PHP Static PHP
Apache + PHP 3900 17.5 6700 17.5
Limit :CPU, network
or disk
Limit :database
CachingCaching
What is caching ?
CACHECACHE
What is caching ?
x = 5, y = 2n = 50 Same result
CACHECACHE
select*
fromarticle
join useron article.user_id = user.id
order bycreated desc
limit10
Doesn't changeall the time
Theory of caching
DB
Cache
$data = get('key')
false
GET /pagePage
select data from table
$data = returned result
set('key', $data)
if ($data == false)
Theory of caching
DB
Cache
HIT
Caching techniques#1 : Store entire pages
#2 : Store part of a page (block)#3 : Store data retrieval (SQL ?)
#4 : Store complex processing result#? : Your call !
When you have data, think :Creating time ?Modification frequency ?Retrieval frequency ?
How to find cacheable dataNew projects : start from 'cache everything'Existing projects :
Look at MySQL slow query log
Make a complete query log (don't forget to turn it off !)
Check page loading times
Caching storage - DiskData with few updates : goodCaching SQL queries : preferably not
DON'T use NFS or other network file systemsespecially for sessions
locking issues !
high latency
Caching storage - Disk / ramdiskLocal
5 Webservers → 5 local caches
How will you keep them synchronized ?→ Don't say NFS or rsync !
Caching storage - Memcache(d)Facebook, Twitter, YouTube, … → need we say more ?Distributed memory caching systemMultiple machines ↔ 1 big memory-based hash-tableKey-value storage system
Keys - max. 250bytes
Values - max. 1Mbyte
Caching storage - Memcache(d)Facebook, Twitter, YouTube, … → need we say more ?Distributed memory caching systemKey-value storage system
Keys - max. 250bytes
Values - max. 1Mbyte
Extremely fast... non-blocking, UDP (!)
Memcache - where to install
Memcache - where to install
Memcache - installation & running itInstallation
Distribution package
PECL
Windows : binaries
RunningNo config-files
memcached -d -m <mem> -l <ip> -p <port>
ex. : memcached -d -m 2048 -l 172.16.1.91 -p 11211
Caching storage - Memcache - some notesNot fault-tolerant
It's a cache !
Lose session data
Lose shopping cart data
...
Caching storage - Memcache - some notesNot fault-tolerant
It's a cache !
Lose session data
Lose shopping cart data
…
Firewall your Memcache port !
Memcache in code
<?php$memcache = new Memcache();$memcache->addServer('172.16.0.1', 11211);$memcache->addServer('172.16.0.2', 11211);
$myData = $memcache->get('myKey');if ($myData === false) { $myData = GetMyDataFromDB(); // Put it in Memcache as 'myKey', without compression, with no expiration $memcache->set('myKey', $myData, false, 0);}echo $myData;
Benchmark with Memcache
Single webserver ProxyStatic PHP Static PHP
Apache + PHP 3900 17.5 6700 17.5Apache + PHP + MC 3900 55 6700 108
Memcache slabs(or why Memcache says it's full when it's not)
Multiple slabs of different sizes :Slab 1 : 400 bytes
Slab 2 : 480 bytes (400 * 1.2)
Slab 3 : 576 bytes (480 * 1.2) (and so on...)
Multiplier (1.2 here) can be configuredStore a lot of very large objects→ Large slabs : full→ Rest : free→ Eviction of data !
Memcache - Is it working ?Connect to it using telnet
"stats" command →
Use Cacti or other monitoring tools
STAT pid 2941STAT uptime 10878STAT time 1296074240STAT version 1.4.5STAT pointer_size 64STAT rusage_user 20.089945STAT rusage_system 58.499106STAT curr_connections 16STAT total_connections 276950STAT connection_structures 96STAT cmd_get 276931STAT cmd_set 584148STAT cmd_flush 0STAT get_hits 211106STAT get_misses 65825STAT delete_misses 101STAT delete_hits 276829STAT incr_misses 0STAT incr_hits 0STAT decr_misses 0STAT decr_hits 0STAT cas_misses 0STAT cas_hits 0STAT cas_badval 0STAT auth_cmds 0STAT auth_errors 0STAT bytes_read 613193860STAT bytes_written 553991373STAT limit_maxbytes 268435456STAT accepting_conns 1STAT listen_disabled_num 0STAT threads 4STAT conn_yields 0STAT bytes 20418140STAT curr_items 65826STAT total_items 553856STAT evictions 0STAT reclaimed 0
Memcache - backing up
Memcache - tipPage with multiple blocks ?→ use Memcached::getMulti()
But : what if you get some hits and some misses ?
getMulti($array) Hashingalgorithm
Updating data
Updating data
LCD_Popular_Product_List
Adding/updating data
$memcache->delete('LCD_Popular_Product_List');
Adding/updating data
Adding/updating data - Why it crashed
Adding/updating data - Why it crashed
Adding/updating data - Why it crashed
Cache stampeding
Cache stampeding
Memcache code ?
DB
Visitor interface Admin interface
Memcache code
Cache warmup scriptsUsed to fill your cache when it's emptyRun it before starting Webserver !
2 ways :Visit all URLs
Error-proneHard to maintain
Call all cache-updating methods
Make sure you have a warmup script !
Cache stampeding - what about locking ?Seems like a nice idea, but...While lock in placeWhat if the process that created the lock fails ?
LAMP...
→ LAMMP
→ LNMMP
NginxWeb serverReverse proxyLightweight, fast12.2% of all Websites
NginxNo threads, event-drivenUses epoll / kqueueLow memory footprint
10000 active connections = normal
Nginx - Configuration
server { listen 80; server_name www.domain.ext *.domain.ext; index index.html; root /home/domain.ext/www;} server { listen 80; server_name photo.domain.ext; index index.html; root /home/domain.ext/photo;}
Nginx with PHP-FPMSince PHP 5.3.3Runs on port 9000Nginx connects using fastcgi method
location / { fastcgi_pass 127.0.0.1:9000; fastcgi_index index.php; include fastcgi_params; fastcgi_param SCRIPT_NAME $fastcgi_script_name; fastcgi_param SCRIPT_FILENAME /home/www.4developers.pl/$fastcgi_script_name; fastcgi_param SERVER_NAME $host; fastcgi_intercept_errors on;
}
Nginx + PHP-FPM featuresGraceful upgradeSpawn new processes under high loadChrootSlow request log !
Nginx + PHP-FPM featuresGraceful upgradeSpawn new processes under high loadChrootSlow request log !fastcgi_finish_request() → offline processing
Nginx + PHP-FPM - performance ?
Single webserver ProxyStatic PHP Static PHP
Apache + PHP 3900 17.5 6700 17.5Apache + PHP + MC 3900 55 6700 108Nginx + PHP-FPM + MC 11700 57 11200 112
Limit :single-threadedApachebench
Reverse proxy time...
VarnishNot just a load balancerReverse proxy cache / http accelerator / …Caches (parts of) pages in memoryCareful :
uses threads (like Apache)
Nginx usually scales better (but doesn't have VCL)
Varnish - backends + load balancing
backend server1 {.host = "192.168.0.10";
}backend server2 {
.host = "192.168.0.11";}
director example_director round-robin {{ .backend = server1;}{
.backend = server2;}
}
Varnish - VCLVarnish Configuration LanguageDSL (Domain Specific Language)
→ compiled to C
Hooks into each requestDefines :
Backends (web servers)
ACLs
Load balancing strategy
Can be reloaded while running
Varnish - whatever you wantReal-time statistics (varnishtop, varnishhist, ...)ESI
Article content page
Article content (TTL : 15 min)/article/732
Varnish - ESIPerfect for caching pages
Header (TTL : 60 min)/top
Latest news (TTL : 2 min) /news
Navigation(TTL :
60 min)/nav
In your article page output :<esi:include src="/top"/><esi:include src="/nav"/><esi:include src="/news"/><esi:include src="/article/732"/>
In your Varnish config :sub vcl_fetch { if (req.url == "/news") { esi; /* Do ESI processing */ set obj.ttl = 2m; } elseif (req.url == "/nav") { esi; set obj.ttl = 1m; } elseif ….….}
Varnish with ESI - hold on tight !
Single webserver ProxyStatic PHP Static PHP
Apache + PHP 3900 17.5 6700 17.5Apache + PHP + MC 3900 55 6700 108Nginx + PHP-FPM + MC 11700 57 11200 112Varnish - - 11200 4200
Varnish - what can/can't be cached ?Can :
Static pages
Images, js, css
Pages or parts of pages that don't change often (ESI)
Can't :POST requests
Very large files (it's not a file server !)
Requests with Set-Cookie
User-specific content
ESI → no caching on user-specific content ?
Logged in as : Wim Godden
5 messages
TTL = 5minTTL=1h
TTL = 0s ?
Coming soon...Based on NginxReduces load by 50 – 95%
Requires code changes !
Well-built project → few changes
Effect on webservers and database servers
What's the result ?
What's the result ?
FiguresFirst customer :
No. of web servers : 18 → 4
No. of db servers : 6 → 2
Total : 24 → 6 (75% reduction !)
Second customer (already using Nginx + Memcache) :No. of web servers : 72 → 8
No. of db servers : 15 → 4
Total : 87 → 12 (86% reduction !)
AvailabilityStable at 2 customersStill under heavy developmentBeta : July 2012Final : Sep 2012
PHP speed - some tipsUpgrade PHP - every minor release has 5-15% speed gain !Use an opcode cache (APC, eAccelerator, XCache)
DB speed - some tipsUse same types for joins
i.e. don't join decimal with int
RAND() is evil !count(*) is evil in InnoDB without a where clause !Persistent connect is sort-of evil
Caching & Tuning @ frontend
http://www.websiteoptimization.com/speed/tweak/average-web-page/
Frontend tuning1. You optimize backend2. Frontend engineers messes up → havoc on backend3. Don't forget : frontend sends requests to backend !
SO...
Care about frontendTest frontendCheck what requests frontend sends to backend
Tuning frontendMinimize requests
Combine CSS/JavaScript files
Tuning frontendMinimize requests
Combine CSS/JavaScript files
Use CSS Sprites
CSS Sprites
Tuning content - CSS sprites
Tuning content - CSS sprites
11 images11 HTTP requests24KByte
1 image1 HTTP requests14KByte
Tuning frontendMinimize requests
Combine CSS/JavaScript files
Use CSS Sprites (horizontally if possible)
Put CSS at topPut JavaScript at bottom
Max. no connections
Especially if JavaScript does Ajax (advertising-scripts, …) !
Avoid iFramesAgain : max no. of connections
Don't scale images in HTMLHave a favicon.ico (don't 404 it !)
→ see my blog
What else can kill your site ?Redirect loops
Multiple requestsMore load on WebserverMore PHP to process
Additional latency for visitor
Try to avoid redirects anyway
→ In ZF : use $this->_forward instead of $this->_redirect
Watch your logs, but equally important...Watch the logging process →Logging = disk I/O → can kill your server !
Above all else... be prepared !Have a monitoring systemUse a cache abstraction layer (disk → Memcache)Don't install for the worst → prepare for the worstHave a test-setupHave fallbacks
→ Turn off non-critical functionality
So...Cache
But : never delete, always push !
Have a warmup script
Monitor your cache
Have an abstraction layer
Apache = fine, Nginx = betterStatic pages ? Use VarnishTune your frontend → impact on backend !
Questions ?
Questions ?
We're hiring !Lots of challengesWork with cutting-edge technologyVaried :
PHP developmentSystem / network architectureScalability servicesBuild our own servicesWork on Open Source
→ mail us : info@cu.be
ContactTwitter @wimgtrWeb http://techblog.wimgodden.beSlides http://www.slideshare.net/wimgE-mail wim.godden@cu.be
Please...Rate my talk : http://joind.in/6327
Thanks !
Please...Rate my talk : http://joind.in/6327