Web Performance

John McCaffreyRailsperformance.blogspot.com

John McCaffrey: Its all about me

• Doing Java since 2000, Rails since 2007• Presented at WindyCityRails

– 2008: Advanced Firebug and JS unit testing– 2009: PDF Generation in Rails– 2010: Rails Performance Tuning

• Addicted to Performance Tuning• railsperformance.blogspot.com

• Feedback: http://spkr8.com/t/4961

@J_McCaffrey

railsperformance@gmail.com

(the slides will be available, with additional references)

What are we gonna cover?

• Performance touches a lot of layers• We have a very diverse audience

Measuring Page Load

Fixing page load problems

Analyzing Server Performance

Database issues

Ruby issues Q&A

Starting off on the right foot

It all started...

def organization_branches get_organizations.flatten.uniq.select{|org| org.children.size > 0}end

def organization_leaves get_organizations.flatten.uniq.select{|org| org.children.size == 0}end

def load_organizations branches = current_user.organization_branches leaves = current_user.organization_leaves @branches = branches.to_json(:only => [:id, :name]) @leaves = leaves.to_json(:only => [:id, :name]) @organizations = @criteria.branch ? leaves.collect {|o| [ o.name.titleize, o.id ] }.sort : branches.collect {|o| [ o.name.titleize, o.id ] }.sort end

def organization_branches get_organizations.flatten.uniq.select{|org| org.children.size > 0}end

def organization_leaves get_organizations.flatten.uniq.select{|org| org.children.size == 0}end

def load_organizations branches = current_user.organization_branches leaves = current_user.organization_leaves @branches = branches.to_json(:only => [:id, :name]) @leaves = leaves.to_json(:only => [:id, :name]) @organizations = @criteria.branch ? leaves.collect {|o| [ o.name.titleize, o.id ] }.sort : branches.collect {|o| [ o.name.titleize, o.id ] }.sort end

It all started...

2811 queries!

Down to 17!!

And then...

Invoicing issue: If a customer resubmits an order with 2 years, they should be charged $9 but were only being charged $4.50

How common are ROI calculations on software projects?

53626 * $4.50 = $241,317 (just in the last year)

Our theme of the day: Leverage

to achieve the greatest value

Performance and Business metrics: Bing.com

Slow site = revenue drop1 sec = 2.8% 2 sec = 4.3%

Performance and Business metrics: Speed matters

Amazon: 100 ms delay caused a 1% drop in revenue.

Google: 400 ms delay caused a 0.59% decrease in search requests per user.

Yahoo!: 400 ms delay caused a 5-9% decrease in traffic.

Bing: 2 sec delay caused a 4.3% drop in revenue per user.

Mozilla made their download page 2.2 seconds faster and saw an increase of 15.4% in downloads.

Google Maps reduced the file volume by 30% and observed a 30% increase in map requests.

Netflix enabled gzip on the server; pages became 13-25% faster and saved 50% of traffic volume!

Performance and Business metrics: Shopzilla.com

Went from 6 secdown to 1.2sec

http://www.phpied.com/the-performance-business-pitch/

Fred Wilson: Speed is the most important feature

10 Golden Principles of Successful Web Apps Fred Wilson, Venture Capitalist www.avc.com

•Speed• Instant Utility• Software is media• Less is More• Make it programmable• Make it personal• Make it RESTful• Discoverability• Clean• Playful

OK, so where do we begin?

Give yourself a fighting chance, make a plan

Alois Reitbauer, dynatrace

Anti-patterns• Thinking Scalability something you just sprinkle on later• Guessing, not testing• Ad-hoc, unstructured• Waiting until it hurts, not proactively planning• Thinking that any early planning is Premature optimization

Video on Parleys.com

Premature optimization

"Premature optimization is the root of all evil"

-Donald Knuth

Premature optimization

Some people really like Donald Knuth

Premature optimization, with DJ DK

like, really, really like him

Premature optimization

"Premature optimization is the root of all evil'

-Donald Knuth

"Quoting Knuth, as a way to avoid planning for performance issues, is a cop-out"@J_McCaffrey

Don't waste time on making optimization changes until you KNOW they are necessary

If you can not measure it, you can not improve it.

— Lord Kelvin

Let's do this the right way

Measure:• Understand the tools to use, at each layer, to measure

and monitor performance

Create repeatable tests:• Be able to invoke the system and easily test• A/B, before/after tests, long-term monitoring

Isolate your changes:• One thing at a time

That's it: Measure, Test, and Isolate your changes

Terminology

Load:How much work is there?

Utilization:How much of the resources are in use?

Scalability:How well can we handle the load?

Throughput:How many tasks can be done per unit of time?

Concurrency:How many tasks can we do at once?

Capacity:How big can our throughput go before things fall apart?

Response time (avg) == Latency

Requests per second == Throughput

Terminology

Performance:• The resources and Time for a single request• Measured in Response Time

Throughput:• The number requests that can be handled per/time• Measured in Requests per second

Scalability:• Ability to grow and handle more requests, without

significantly degrading the experience

Performance != Scalability

Terminology

Let's say: 1 worker that can compute a job in .5 sec

Throughput = 2 jobs per sec Latency = .5 sec per job

Performance != Scalability

Adding more workers will not improve the latency

But not having enough workers can increase the response time, due to Queuing and delay

There are performance opportunities at each layer

Waterfall: Twitter.com

Waterfall

Total time: 3815 ms Dynamic generation: 250 ms (~7%) Static: 3369 ms (~89%) * Will not add to 100%

When the html is downloaded

When the rest of it is loaded

Waterfall

When the backend is actually doing work

Walmart: Waterfall

Waterfall

Total time: 5520 ms Dynamic: 388 ms (7%)Static: 4580 ms (~82%)

Firebug Netpanel

Simple and repeatable way to measure page load

Usually, less than 20% of the time is spent in fetching the HTML

So how do we optimize the other 80%?

YSlow

• Created by Steve Souders• Observes the netpanel data, applies rules and gives you a

score.• Deep understanding of how the browser works• Its the best way to have a repeatable, measurement of your

site's load performance

http://developer.yahoo.com/yslow

YSlow

• Minimize HTTP Requests• Use a Content Delivery Network• Add an Expires or a Cache-Control Header• Gzip Components• Put StyleSheets at the Top• Put Scripts at the Bottom• Avoid CSS Expressions• Make JavaScript and CSS External• Reduce DNS Lookups• Minify JavaScript and CSS• Avoid Redirects• Remove Duplicate Scripts• Configure ETags• Make AJAX Cacheable• Use GET for AJAX Requests• Reduce the Number of DOM Elements• No 404s• Reduce Cookie Size• Use Cookie-Free Domains for Components• Avoid Filters• Do Not Scale Images in HTML• Make favicon.ico Small and Cacheable

Yslow: Created by Steve Souders

http://stevesouders.com/cuzillion/

Google Page speed

• Similar to YSlow• includes paint events• separates out ads, trackers from your content• better recommendations and optimizations• 1 click to see optimized version of your files

http://code.google.com/speed/page-speed

Google Page speed

Google Page speed

Make it repeatable

http://www.showslow.com/ track your site, or your competitors will check every 24hrs for you

• webpagetest.org• gtmetrix.org• zoompf.com• loadimpact.com• gomez.com

Summary

• Page load is the place to start• Read everything in the Yslow and PageSpeed rules• Encourage other devs to play with YSlow, PageSpeed• Show Webpagetest.org, and zoompf.com to the boss

Now that we have a way to measure page performance, we can think about making changes

Measuring Page Load

Fixing page load problems

Analyzing Server Performance

Database issues

Ruby issues Q&A

Starting off on the right foot

Isn't everyone already doing this?

Analysis of the Alexa top 1000 sites found:

• 42% don't gzip• 44% have more than 2 css files• 56% serve css from a cookied domain• 62% don't minify• 31% have more than 100k size css

Top 1000 retail sites 50% aren't doing both keep-alive and compression (the 2 easiest things!!)

Compress with Gzip

• Can save you 50% to 80% of your bandwidth• Simplest performance improvement you can make• Beware differences in browsers• File types: html, js, css, xml, rss, txt, font, json

Probably the simplest and smartest thing you can do for your app!

Combine and minify javascript and css

Rails• javascript_include_tag :all, :cache => "all"• Asset_packager, Jammit

Load common js libraries from googlehttp://github.com/rpheath/google_ajax_libraries_api

Jsmin or YUICompressorClosure compiler

Let's check it out!

Combine and minify javascript and css

Sprites and image optimization• spriteme.org• JqueryUI already gives you sprited images

Image optimization• Smush.it

Expires and Browser Caching

Setting a far future expires tells the browser not to ask for the file

Expires:Mon, 02 Sep 2030 21:23:08 GMT

/images/logo.png?1234567890

Using the query string lets us break the caching by having the browser ask for a new file

There is a problem with using a URL query string this way http://blog.eliotsykes.com/2010/05/06/why-rails-asset-caching-is-broken/

/images/logo-fp-839b180ff39a24f8d6e0ee70e4c40fed.png

Rails Caching

Take your dynamic content, and make it static• Page• Action• Fragment

Full length tutorials• http://railslab.newrelic.com/• http://railscasts.com/episodes?search=caching• Greg Pollack Aloha on Rails http://vimeo.com/10860860

Measuring Page Load

Fixing page load problems

Analyzing Server Performance

Database issues

Ruby issues Q&A

Starting off on the right foot

The Stack

You are here

Request-log-analyzer

• http://github.com/wvanbergen/request-log-analyzer• Request distribution per hour• Most requested• HTTP methods• HTTP statuses returned• Rails action cache hits• Request duration• View rendering time• Database time• Process blockers• Failed requests

Rack::Bug

Rack::Bug

Commercial tools

New Relic• Developer plugin is a good start• Free version doesn’t get you much (Bronze version with heroku)• Works for .NET

• Scoutapp.com• Similar to new Relic• Large selection of community plugins

Testing tools

Apache bench ab -n 10 -c 2 http://www.somewhere.com/

Httperfhttperf --server localhost --port 3000 --uri / --num-conns 10000

Jmeteryes, its ugly, but its powerful

Measuring Page Load

Fixing page load problems

Analyzing Server Performance

Database issues

Q&A

Starting off on the right foot

Database issues

Common database issues• Bad queries• Not utilizing explain• Inadequate indexes• N+1 queries• Selecting more data than is needed• Inconsistent queries for the same data

Query_reviewer

Common database issues• Bad queries• Not utilizing explain and slow query log• http://github.com/dsboulder/query_reviewer • Runs explain on all of your queries, outputs to div in

page

Rails_indexes

http://github.com/eladmeidar/rails_indexes

Foreign key indexes

• Columns that need to be sorted

• Lookup fields

• Columns that are used in a GROUP BY

• Rake tasks to find missing indexes.

New one I haven’t tried yet

http://github.com/samdanavia/ambitious_query_indexer

Bullet plugin

Bullet plugin

• http://github.com/flyerhzm/bullet

Help you reduce the number of queries

with alerts (and growl).

Slim_scrooge

Be more specific in your select• http://github.com/sdsykes/slim_scrooge• Instruments your code• Observes the usage pattern• Suggests/changes your select statement• When invoked within the exact same context

Ruby Issues

Most are due to memory problems• Slow GC• Not release references• Capturing scope

• Profiling will reveal what’s going on• http://guides.rubyonrails.org/performance_testing.html

If you want to learn everything there is to know about profiling • Aman Gupta & Joe Damato• http://timetobleed.com/ • http://memprof.com/ • They’ve already identified and fixed several memory leaks

Ruby issues: Faster Ruby Libraries

C Extension++

• XML parser http://nokogiri.org/

• JSON parser http://github.com/brianmario/yajl-ruby/

• CSV parser http://www.toastyapps.com/excelsior/

• HTTP client http://github.com/pauldix/typhoeus

Date

http://github.com/jeremyevans/home_run

Take home points

Performance• Performance affects the bottom line• The biggest performance win is usually in improving the load time• Continue to monitor and test your apps performance over time• Gzip, combine and minify to get a big boost• Lack of indexes is likely to be one of your biggest backend issues• Try to stay up to date with Libraries and patches• A well tested codebase is easier to tune!• Upgrading to ruby 1.9 will give you a huge performance boost

Links: great info on performance

Performance• Scaling Rails series http://railslab.newrelic.com• RailsCasts.com• Velocity Conf http://en.oreilly.com/velocity2010 • Google io conf http://code.google.com/events/io/2010/• dynatrace site http://ajax.dynatrace.com/pages/• http://www.igvita.com• http://www.mysqlperformanceblog.com

Wait, I made it through that?

Designing for performance• Watch how they really use it• Multiple tabs?• Back and forth between list and detail view

• Great presentation at WindyCityRails2009 by David Eisinger • Optimizing perceived performance• Make it feel fast

Wait, I made it through that?

Architectural musings

•Background long running tasks• Email, image processing, reports, feeds•Leverage Rack, Sinatra or Padrino•Parallel processing, Event Machine•Learn from heroku, scalable from the start•Revaluate your cloudy-ness• Great RailsConf presentation by James Golick

Links

Questions?

@j_mccaffrey

Railsperformance@gmail.com

Beyond ySlow: improving page performance outside the initial load

MySql and Postgres Database tuning

Heroku: Building Scalable apps from the start

Tuning performance for Mobile devices

Performance Improvements coming in Rails 3, and 3.1

Speeding up your tests

Questions for you

Architectural musings

• Moving to or already on Ruby 1.9?• Serving your content from a separate domain• Cookieless?