6 tips for improving ruby performance
-
Upload
engine-yard -
Category
Technology
-
view
5.267 -
download
0
Transcript of 6 tips for improving ruby performance
In Today’s Session You Will Learn how to: • Gain visibility on site performance • Improve scalability and uptime • Find and fix key bottlenecks
• Database • Web Servers • Caching • Background Processing
New Relic + Engine Yard
Web Request Overview
Web Application Overview
DATA BASE
Lazy loading associated data can quickly lead to an N+1 query problem.
ORMs (ActiveRecord, DataMapper, etc.) make it easy to get our data but also make it easy to forget to optimize and refactor.
N+1 problems are hard to spot in development since you are working with limited data sets.
Database Performance
# app/models/customer.rb class Customer < ActiveRecord::Base has_many :addresses
end
# app/models/address.rb class Address < ActiveRecord::Base belongs_to :customer
end
# app/controllers/customers_controller.rb class CustomersController < ApplicationController def index @customers = Customer.all end
end
# app/views/customers/index.html.erb <% @customers.each do |customer| %> <%= content_tag :h1, customer.name %>
<% end %>
N+1 Query Creep
# app/views/customers/index.html.erb
<% @customers.each do |customer| %>
<%= content_tag :h1, customer.name %>
<%= content_tag :h2, customer.addresses.first.city %>
<% end %>
If @customers has 100 records, you'll have 101 queries:
SELECT "customers".* FROM "customers"
SELECT "addresses".* FROM "addresses" WHERE "addresses"."customer_id" = 1 AND "addresses"."primary" = 't' LIMIT 1
SELECT "addresses".* FROM "addresses" WHERE "addresses"."customer_id" = 2 AND "addresses"."primary" = 't' LIMIT 1
SELECT "addresses".* FROM "addresses" WHERE "addresses"."customer_id" = 3 AND "addresses"."primary" = 't' LIMIT 1
...
...
SELECT "addresses".* FROM "addresses" WHERE "addresses"."customer_id" = 100 AND "addresses"."primary" = 't' LIMIT 1
N+1 Query Creep
# app/controllers/customers_controller.rb
class CustomersController < ApplicationController
def index
@customers = Customer.includes(:addresses).all
end
end
If @customers has 100 records, now we only have 2 queries:
SELECT "customers".* FROM "customers"
SELECT "addresses".* FROM "addresses" WHERE "addresses"."customer_id" IN (1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, 100)
Eager Loading with .includes
New Relic > App Server > Web Transactions > Performance Breakdown
Finding N+1 in New Relic
Missing Indexes == Slow Queries
# db/migrate/20120201040247_add_index_for_shop_id_on_orders.rb
class AddIndexForShopIdOnOrders < ActiveRecord::Migration
def change
add_index :orders, :shop_id
end
end
Adding an Index is Simple
Index Protips:
• Searching an index on a table with 1,000 rows is 100x faster than searching a table without an index.
• Put an index on any columns you will likely query against, it's better to have too many than too few indexes.
• Adding an index to a table will lock the table!
RUBY WEB SERVERS
• Simple to operate • Simple configuration • Handles worker management • Great for multi-application environments • Great for low resource environments • Attached to Nginx/Apache HTTPD
Passenger 3
Passenger Request Queue
solo i-c3f2d8a2 ~ # passenger-status ----------- General information ----------- max = 3 count = 3 active = 0 inactive = 3 Waiting on global queue: 0 ----------- Application groups ----------- /data/john_yerhot_org/current: App root: /data/john_yerhot_org/current * PID: 19802 Sessions: 0 Processed: 3 Uptime: 3h 10m 13s /data/scalingrails/current: App root: /data/scalingrails/current * PID: 28726 Sessions: 0 Processed: 3 Uptime: 59m 22s /data/sites/clmeisinger/current: App root: /data/sites/clmeisinger/current * PID: 22147 Sessions: 0 Processed: 70 Uptime: 10h 45m 57s
• Independent of front end web server • More configuration options • Master process will reap children on timeout • Great for single application environments • Allows for zero downtime deploys
Unicorn
Unicorn Request Queue?
Raindrops solo i-5b74313d ~ # gem install raindrops Fetching: raindrops-0.10.0.gem (100%) Building native extensions. This could take a while... Successfully installed raindrops-0.10.0 1 gem installed solo i-5b74313d ~ # ruby -rubygems -e "require 'raindrops'; puts Raindrops::Linux.unix_listener_stats(['/var/run/engineyard/unicorn_appname.sock']).inspect" {"/var/run/engineyard/unicorn_appname.sock"=>#<struct Raindrops::ListenStats active=0, queued=0>}
Request Queuing in New Relic
Request Queuing in New Relic
NOT COOL
Request Queuing in New Relic
• Time between first ActionContoller hit - X-Queue-Start = Time spent in queuing.
Internet => LB inserts X-Queue-Start => Nginx => Ruby Webserver => Rack => Application
Track Rack Middleware as well def call(env) env["HTTP_X_MIDDLEWARE_START"] = "t=#{(Time.now.to_f * 1000000).to_i}" @app.call(env) end
CACHING
Cache Everything
Rails makes it stupid easy to
cache everything. Do it.
Static Files & Nginx
The best cache is a static file served by Nginx.
# create it on #index, #show, etc.. caches_page :index # expire it on #creates, #updates, #destory, etc... expire_page :action => :index
A Note About Static Files:
Use the front end server. upstream upstream_enki { server unix:/var/run/engineyard/unicorn_enki.sock fail_timeout=0; } location ~ ^/(images|assets|javascripts|stylesheets)/ { try_files $uri $uri/index.html /last_assets/$uri /last_assets/$uri.html @app_enki; expires 10y; } location / { if (-f $document_root/system/maintenance.html) { return 503; } try_files $uri $uri/index.html $uri.html @app_enki; }
Memcached: The Standard
# config/initializers/memcached.rb config.cache_store =:mem_cache_store,
"server-1:11211", "server-2:11211", "server-3:11211",
"server-4:11211"
Next Best: ActionCaching
Will still go through Rack/Rails, but the action gets cached. before_filter :make_sure_youre_ok caches_action :all_the_things def all_the_things @all_things = Thing.all_in_a_complex_way end def expire expire_action :action => :all_the_things end
Fragment Caching
<% cache('my_cache_key') do %> <%= render_large_tag_cloud %> <% end %> ... def update_large_tag_cloud TagCloud.update expire_fragment('my_cache_key') end
Baremetal
Rails.cache.write("john", "yerhot") Rails.cache.read("john")# => "yerhot" # execute a block on miss and cache it. Rails.cache.fetch("miss") do "yerhot" end Rails.fetch("miss")# => "yerhot" Rails.cache.exists("john") # => true Rails.cache.delete("john") # => true Rails.cache.exists("john") # => false
Background Processing
• send email • process images • grab feeds and cache them • complex computations/reports • create/expire caches/pages (like Reddit)
Why Background Processing?
Best Practice:
Use a utility server for background jobs and cron.
Resque to the Rescue
Resque in New Relic
Delayed Job Too
Background Processing baked in.
• Allow an application to switch job systems with minimal code change due to common API
• Very basic queuing system built in • Roll your own wrapper class that responds to push & pop
Rails 4
# application.rb config.queue = QueueName Rails.queue.push(Job.new)
• You need to be monitoring your application.
• Performance has to be reviewed on a regular basis.
• Database indexes are cheap, make lots of them.
• Every application can take advantage of some level of caching: page, action or fragment.
• Background any work that you can.
• Don't neglect front-end performance.
Review
New Relic Standard is Free at Engine Yard
1. If you’re an Engine Yard Customer, select your plan in your Engine Yard Account Settings
2. Add newrelic_rpm to your Gemfile
3. Enable monitoring in the Engine Yard Dashboard
Full Installation Details: http://ey.io/install-newrelic
How to Install New Relic
Questions?
Chris Kelly @amateurhuman www.newrelic.com
John Yerhot @yerhot www.engineyard.com
Thanks for Watching!