Rubinius - What Have You Done For Me Lately

Tuesday, November 23, 2010

RubiniusWhat have you done for me today?


Rubinius?


Modern Ruby Implementation

Rubinius


Philosophy

Rubinius


Philosophy

Use Ruby

Rubinius


Philosophy

Allow Ruby to extend the system

Rubinius


module Enumerable def map if block_given? ary = [] each { |o| ary << yield(o) } ary else to_a end endend


Compatibility

1.8.7

Rubinius


Compatibility

Rails 2.3 - 3+

Rubinius


Compatibility

C extensions

Rubinius


Spin Off Projects

Rubinius


Spin Off Projects

RubySpec

Rubinius


Spin Off Projects

FFI

Rubinius


Drop In Compatible

Rubinius


Technology

Rubinius


Technology

Bring techniques to Ruby

Rubinius


Optimized to Machine Code


Bytecode VM



Bytecode VM

Soph"ticated Garba# Collection



Technology

Efficient Memory Layout

Rubinius


obj.ivars = { :@name => “Evan”, :@age => 31}


obj.class.ivars = { :@name => 0, :@age => 1}

obj.ivars = [ “Evan”, 31]


Flags

Class

Extra Ivars

@name

@age


Developers!


Developers!

Developers!


Developers!

Developers!

Developers!


Build tools around APIs

API Driven


Allow users to build more!

API Driven


Example

Bytecode Compiler

API Driven


Example

Bytecode and Compiler

API Driven


Example

Fancy Language

API Driven


Why useRubinius?


Focus on Developers

Why?


I’m a Ruby developer

Why?


Tools born from need

Why?


Improve an Algorithm

Problem:


Step 1

Improving


Step 1

Benchmark

Improving


require 'benchmark'cc = CoworkersCode.new

Benchmark.bm do |x| x.report("initial") { cc.calculate_awesome_score }end


user system total realinitial 0.071788 0.001601 0.073389 ( 29.998823)


29.998823


user: 0.071788system: 0.001601 total: 0.073389 real: 29.998823


Time running code

User


Time inside the kernel

System


Sum of user and system

Total


Elapsed wall clock time

Real


Time measured by a stop watch

Wall Clock


user: 0.071788system: 0.001601 total: 0.073389 real: 29.998823


0.073389

29.998823!=


0.073389

29.998823!=??


Step 2

Profile

Improving


» rbx -Xprofile scratch/profile_sample.rb

% cumulative self self total time seconds seconds calls ms/call ms/call name------------------------------------------------------------ 99.69 30.00 30.00 10 3000.10 3000.10 Rubinius::Channel#receive_timeout 0.13 0.04 0.04 2 19.82 19.82 IO.open_with_mode 0.11 0.03 0.03 1 33.05 33.05 IO::Buffer#fill 0.02 0.01 0.01 18 0.30 0.30 FFI::Platform::POSIX.stat


» rbx -Xprofile -Xprofiler.graph scratch/profile_sample.rb



0.00 30.00 10/60 Kernel#sleep [15][16] 99.9 30.00 0.00 10 Rubinius::Channel#receive_timeout [16]



0.00 30.00 10/60 Kernel#sleep [15][16] 99.9 30.00 0.00 10 Rubinius::Channel#receive_timeout [16]

0.00 30.00 10/20 CoworkersCode#superhero_factor [14][15] 99.9 0.00 30.00 10 Kernel#sleep [15] 30.00 0.00 10/10 Rubinius::Channel#receive_timeout [16]


def superhero_factor sleep 3 rand(10) end


Step 3

Fix and benchmark

Improving


0.000139


0.073389

29.998823!=


Invisible in all but real time

Sleep!



Problem:



Problem:

SOLVED!


Slow / hung process

Problem:


» bin/rbx scratch/qa_sample.rb 1288125364.684678: Server started, pid 39331


Slow / hung process

Use Query Agent to inspect

Problem:


» bin/rbx -Xagent.start scratch/qa_sample.rb 1288125364.684678: Server started, pid 39331


» bin/rbx consoleVM: bin/rbx -Xagent.start scratch/qa_sample.rbConnecting to VM on port 58273Connected to localhost:58273, host type: x86_64-apple-darwin10.4.0console>


» bin/rbx consoleVM: bin/rbx -Xagent.start scratch/qa_sample.rbConnecting to VM on port 58273Connected to localhost:58273, host type: x86_64-apple-darwin10.4.0console> pidPID: 39331


» bin/rbx -Xagent.start scratch/qa_sample.rb 1288125364.684678: Server started, pid 39331


» bin/rbx consoleVM: bin/rbx -Xagent.start scratch/qa_sample.rbConnecting to VM on port 58273Connected to localhost:58273, host type: x86_64-apple-darwin10.4.0console> pidPID: 39331console> backtrace


» bin/rbx consoleVM: bin/rbx -Xagent.start scratch/qa_sample.rbConnecting to VM on port 58273Connected to localhost:58273, host type: x86_64-apple-darwin10.4.0console> pidPID: 39331console> backtraceThread 0:0x7fff5fbfc250: IO::Buffer#fill_from in kernel/common/io.rb:67 (+72)0x7fff5fbfc690: IO#read in kernel/common/io.rb:1197 (+125)0x7fff5fbfcaf0: ImportantServer#get_request in /Users/evan/git/rbx-release/scratch/qa_sample.rb:10 (+7)0x7fff5fbfced0: MAIN.__script__ in /Users/evan/git/rbx-release/scratch/qa_sample.rb:17 (+91)0x7fff5fbfd390: Rubinius::CodeLoader#load_script in kernel/delta/codeloader.rb:67 (+44)0x7fff5fbfd790: Rubinius::CodeLoader.load_script in kernel/delta/codeloader.rb:91 (+40)0x7fff5fbfdbb0: Rubinius::Loader#script in kernel/loader.rb:467 (+125)0x7fff5fbfdfc0: Rubinius::Loader#main in kernel/loader.rb:578 (+64)0x7fff5fbfe3e0: Rubinius::Loader.main in kernel/loader.rb:617 (+33)0x7fff5fbfe800: Object#__script__ in kernel/loader.rb:628 (+60)


Thread 0:0x7fff5fbfc250: IO::Buffer#fill_from in kernel/common/io.rb:67 (+72)0x7fff5fbfc690: IO#read in kernel/common/io.rb:1197 (+125)0x7fff5fbfcaf0: ImportantServer#get_request in /Users/evan/git/rbx-release/scratch/qa_sample.rb:10 (+7)0x7fff5fbfced0: MAIN.__script__ in /Users/evan/git/rbx-release/scratch/qa_sample.rb:17 (+91)0x7fff5fbfd390: Rubinius::CodeLoader#load_script in kernel/delta/codeloader.rb:67 (+44)0x7fff5fbfd790: Rubinius::CodeLoader.load_script in kernel/delta/codeloader.rb:91 (+40)0x7fff5fbfdbb0: Rubinius::Loader#script in kernel/loader.rb:467 (+125)0x7fff5fbfdfc0: Rubinius::Loader#main in kernel/loader.rb:578 (+64)0x7fff5fbfe3e0: Rubinius::Loader.main in kernel/loader.rb:617 (+33)0x7fff5fbfe800: Object#__script__ in kernel/loader.rb:628 (+60)


Slow / hung process

Sample process via CLI

Problem:


» bin/rbx -Xagent.start -Xagent.verbose \ scratch/qa_sample.rb[QA: Bound to port 58407]1288125364.684678: Server started, pid 39331


» bin/rbx console -p 58431 --btThread 0:0x7fff5fbfc200: IO::Buffer#fill_from in kernel/common/io.rb:67 (+72)0x7fff5fbfc640: IO#read in kernel/common/io.rb:1197 (+125)0x7fff5fbfcaa0: ImportantServer#get_request in /Users/evan/git/rbx-release/scratch/qa_sample.rb:10 (+7)0x7fff5fbfce80: MAIN.__script__ in /Users/evan/git/rbx-release/scratch/qa_sample.rb:17 (+91)0x7fff5fbfd340: Rubinius::CodeLoader#load_script in kernel/delta/codeloader.rb:67 (+44)0x7fff5fbfd740: Rubinius::CodeLoader.load_script in kernel/delta/codeloader.rb:91 (+40)0x7fff5fbfdb60: Rubinius::Loader#script in kernel/loader.rb:467 (+125)0x7fff5fbfdf70: Rubinius::Loader#main in kernel/loader.rb:578 (+64)0x7fff5fbfe390: Rubinius::Loader.main in kernel/loader.rb:617 (+33)0x7fff5fbfe7b0: Object#__script__ in kernel/loader.rb:628 (+60)


Slow / hung process

Easy API to build monitoring tools

Problem:


require 'socket'require 'rubinius/agent'port = 58431agent = Rubinius::Agent.connect( "localhost", port)kind, bt = agent.get("system.backtrace")


Socket based VM API

Query Agent


Socket based VM API

Simple get/set variables

Query Agent


Slow / hung process

Problem:


Slow / hung process

Problem:

SOLVED!


Memory Footprint

Problem:


Memory Footprint

Suspect some kind of memory leak

Problem:


Memory Footprint

Use Heap Dump!

Problem:


» bin/rbx -Xagent.start scratch/hd_demo.rb Waiting...


» bin/rbx consoleVM: bin/rbx -Xagent.start scratch/hd_demo.rbConnecting to VM on port 58710Connected to localhost:58710, host type: x86_64-apple-darwin10.4.0console> set system.memory.dump heap.dump


» bin/rbx -Xagent.start scratch/hd_demo.rb Waiting...Heap dumped to heap.dump


» rbx -Ilib histo.rb ../rbx-release/heap.dump 19798 Rubinius::Tuple 3487912 4867 Rubinius::MethodTable::Bucket 272552 4218 String 303696 4193 Rubinius::CompiledMethod 805056 4193 Rubinius::InstructionSequence 134176 4110 Rubinius::ByteArray 100199256 1547 Rubinius::LookupTable::Bucket 74256 1053 Rubinius::LookupTable 50544 1010 Rubinius::MethodTable 48480 1000 BigCrazyClass 32000 966 Class 104592 783 Rubinius::StaticScope 37584 704 Rubinius::AccessVariable 56320


Complete Heap Layout

Heap Dump



Stable format

Heap Dump



Reference Ruby Reader

Heap Dump


Gauge

Heap Dump


Gauge

Rails Application Explorer

Heap Dump


Future Feature

Tracking allocation sites

Heap Dump


Memory Footprint

Problem:


Memory Footprint

Problem:

SOLVED!


Memory Footprint

Problem:

SOLVED!(mostly)


Future


1.9

Future


Windows

Future


Full Concurrency

Future


Why Not?


Thanks!


Rubinius - What Have You Done For Me Lately

Education

Transcript of Rubinius - What Have You Done For Me Lately