Simple Steps to Win the Site Performance Battle

(and other hopefully interesting stuff)

CHRIS WELLS – CEO – NEXCESS.NET LLCSOUTHFIELD, MI USA

Quick Facts About Detroit, MI• It’s the potato chip capital of the world• based on consumption

• It had the 1st mile of concrete road (1909)• Between 6 and 7 mile roads on M-1 (Woodward)

• Home to the only floating Post Office in the U.S.• J.W. Westcott Co (Zip code: 48222)

• It has the most registered bowlers nationwide

• Biggest municipal bankruptcy in the U.S.• Which makes us National Bankruptcy Champions!!

• Eat your heart out Jefferson County, AL!! We did 4 x better than you!!

Notes About This Talk• Lessons learned from MagentoLive Australia• If masses walk out mid-talk I’m being:

A) WAY too technical

B) Ridiculously boring

• Lessons learned from MagentoLive UK• If I sweat profusely:

A) DO NOT BE ALARMED

B) Assume the technical folks located at the back of the room simply swapped out the real talk I had prepared with something they thought was more fun and I’m simply dying a slow 35 minute death on stage before your very eyes.

The Basics NOT On Today’s Agenda• OS level set up / configuration

• Apache or Nginx

• Magento caching

• Magento EE’s FPC

• Varnish / Turpentine or other reverse proxy caching techniques

• Other performance extensions

• At this point we’re all well aware…

• So let’s dig a little deeper into some fun stuff!

*ON* Today’s Agenda• PHP – 92%

• MySQL – 3% (if time permits)

• Drives – 5% (if time permits)

• If you leave here today with some food for thought then I’m happy!

PHP – Personal Home Page• Yes, I know.. It’s now “PHP: Hypertext preprocessor”

• Magento used to be version locked to PHP <= 5.3

• As of January Magento officially supports PHP 5.4! Woohoo!• You may need to patch your Magento for it to work with 5.4

• So…?

• PHP 5.4 is a good deal faster than 5.3• Empty hash table optimizations

• Literal tables

• Interned strings

• Zend Engine VM tuning

• But what does this mean for Magento?

Notes On Magento• Magento is very CPU heavy• The bulk of this heaviness is within PHP code

• MySQL CPU usage is typically bound by PHP’s CPU usage (speaking in single server terms)

• Magento is not very I/O heavy (contrary to popular belief)• Again, I/O is bound by PHP’s CPU usage

• More on I/O later

• Quantitative notes to follow

Let’s Test Us Some PHP• 1-hour Siege runs (meant to tax PHP, not MySQL)

• EE tests• Siege of 20,000 SKUs hitting ~20,000 random URLs

• FPC enabled (Redis)

• CE tests• Siege of 120 SKUs hitting ~120 URLs.

• Bare metal configuration of:• Dell R420 – Dual E5-2420 6-core

• H710 512MB NV + 4 x 15K 300GB RAID 10

• 24G RAM

• Ok ok, let’s see what we’ve got!

PHP 5.3.24 (CE Front-end Baseline)

~42 t/sec

PHP 5.3.24 vs. PHP 5.4.28 (CE Front-end)

~52 t/sec(~23%

increase!)

Magento cache

population

PHP 5.3.24 (EE Front-end Baseline)

~550 t/sec

GO GO FPC!

PHP 5.3.24 vs. PHP 5.4.28 (EE Front-end)

~615 t/sec(~12%

increase!!)FPC / Magento

cache population amplifies base PHP version

improvements

PHP 5.3.24 vs. PHP 5.4.28/.14 (EE Front-end)

Even upgrading from 5.4.14 shows

gains of ~4%

PHP 5.3.24 vs. PHP 5.4.28 (Re-index from CLI)

Time In Seconds

~8% improvement

Let’s Push Things a Little Further...

PHP 5.4.28 vs. PHP 5.5.12 (EE/Front-end)

~781 t/sec~25% better

than 5.4~41% better

than 5.3WOW

FPC / Magento cache population again amplifies

base PHP version improvements

Notes On PHP 5.5.x+• !! Warning – Magento does not yet officially support 5.5+ !!

• APC goes out• Magento configs may need to be tweaked if you’re using APC’s key/val

caching

• Some extensions will need to be updated / re-coded for sure

• OPcache is introduced• OPcache is the name of the bundled ZendOptimizer+ opcode caching system

• Seems to work out of the box without too much fuss

• More research is needed here – was very surprised with the performance results

• !! Warning – Magento does not yet officially support 5.5+ !!

PHP 5.4.28 vs. PHP 5.5.12 (Re-index from CLI)

Time In Seconds

~4% improvement

Let’s Push Things a Little Further…

PHP 5.5.12 vs. PHP 5.6.0-BETA2 (EE/Front-end)

Essentially equal within margin of

error

PHP 5.5.12 vs. PHP 5.6.0-BETA2 (Re-index from CLI)

Time In Seconds

BUT, we got ~11% improvement on the

re-index!

~21.5% better than 5.3.24!

Let’s…

PHP 5.5.12 vs. PHP-NG (EE/Front-end)

Notes on PHP-NG• Removes numerous heap allocations (and de-allocations)

• Stores more native data directly on the stack

• Removes the need to garbage collect basic primitives (bool, long, etc)

• PHP’s reported Wordpress benchmarks show very good results• 26.75 sec -> 18.95 sec (36% improvement)

• 9.5M instructions -> 5M internal instructions executed (90% reduction)

• Take some comfort in knowing that more gains are on the way from PHP folks directly

Simple Step #1 – Use PHP 5.4.28 (for now)

**But watch out for future PHP releases!

MySQL – “My SQL”• My is the Michael Widinius’s daughter’s name (this was

news to me)

• Lots of “brand” options on the MySQL front unlike PHP• MariaDB

• Percona Server / Percona Cluster

• Oracle’s MySQL

• Equally many options on the storage engine front• MyISAM

• InnoDB

• XtraDB

Let’s Test Us Some MySQL• ~1 hour re-index runs (ideally meant to put some load on

MySQL)

• Full CLI-based re-index

• Tested latest EE with various versions/brands of MySQL

• DB contained 800,000 SKUs (thanks Magento performance team)

• Bare metal configuration of:• Dell R420 – Dual E5-2420 6-core

• H710 512MB NV + 4 x 15K 300GB RAID 10

• 24G RAM

• Ok ok, let’s see what we’ve got!

Magento CLI Re-index (EE)

Time In Seconds

Magento CLI Re-index (EE)

Time In Seconds

Thoughts On The Tests• Re-index runs, while DB intensive, are still PHP heavy

• Oracle’s MySQL did surprisingly well• If concurrency were higher I’d expect Percona/MariaDB to excel

• DB bound writes (beyond the re-index) were missing• This would tax the DB servers considerably

• TCP (and other) overhead for Percona Cluster doesn’t seem to kill performance

Simple Step #2 – Use a modern version of MySQL (ideally XtraDB based)

Hard and Solid State Disk Drives• Hard drive vs. disk drive vs. solid state drive vs. SAS,

etc.

• IOPs (I/O operations / sec)• More is better, but don’t throw money away

• All SSDs not created equal• Don’t believe quoted specs!

• Don’t buy consumer grade SSDs!

• Quality SSDs are ~2x the cost of quality HDD of same size

• Know your workload profile!

Live Client Case Study• Magento EE 1.12.0.1

• ~700,000 SKUs

• 350+ orders / hour

• 15 x web servers (bare metal)• Dell R420, Dual E5-2450 8-core, 24GB RAM

• 2 x DB servers (replicated) – peak @ ~20,000 qps

• 1 x file server (shared storage for media)

• FPC / Magento cache in Redis

• ~1 week sample size

Database Server IO Requirements

Read Write

Average IOPs: 3.66 471

Min IOPs: 0.0 6.87

Max IOPs: 830 2250

Standard deviation:

34.5 197.0

95% below: 60.4 795

99.999% below: 151 1310

• Percona Server 5.5.35

• 128GB RAM / 60GB MySQL DB (innodb_buffer_pool = 75G)

• 8 x 300GB 15k HDD/SAS RAID 10 - H700/1G RAID card

• Replicating to a hot-standby DB server (unused by Magento)

???

File Server IO Requirements

Read Write

Average IOPs: 13.7 7.65

Min IOPs: 5.58 2.97

Max IOPs: 85.8 186

Standard deviation:

6.72 11.0

95% below: 24.7 25.8

99.999% below: 42.3 54.6

• Serving /media and /var only over NFS

• 24GB RAM - 600GB data on disk

• 4 x 600GB 15k HDD/SAS (RAID 10) - H700/512M RAID card

Notes On Drives• Get quality drives regardless of technology• No consumer SSDs!

• Caches are very good at relieving disk-bound I/O• MySQL query cache

• Redis/memcaches

• RAID card caches

• On-disk controller caches

• Invest in CPU over IOPs unless you can afford both

• Magento is typically bound by CPU, not I/O – Know your workload profile

Simple Step #3 – Use HDD & SSD drives as needs/costs dictate

Thank you!

** Thanks for not walking out while I sweat. Come collect your free bag of chips!!

CHRIS WELLS – CLWELLS -AT- NEXCESS.NET