Ruby to Scala in 9 weeks
-
Upload
jutley -
Category
Technology
-
view
121 -
download
0
description
Transcript of Ruby to Scala in 9 weeks
Jake Utley ([email protected])
Ruby to Scala in 9 WeeksEvolving antiquated software with modern tools
Sept. 9, 2014
2WhitePages
Areas of interest over SDLC
• Understanding the problem• Exploring solution space• Speed of prototyping
• Performance• Maintainability• Integration
prototyping production
3WhitePages
Scala and Ruby Similarities
• Readable and concise
• Object oriented
• Highly composable (traits and mixins)
• Highly adaptable
4WhitePages
Background: Location Services
• An internal service at Whitepages, written in Ruby
• Rationalizes Foursquare data against Whitepages people
data
• Modified to use data from Facebook, LinkedIn, and Twitter
• Contained many issues that motivated rewrite
– Unclear code
– Sloppy code organization
– Poor performance
– Minimal documentation
5WhitePages
Migration to Scala
• Concurrency
– Ruby EventMachine vs. Scala Futures
• Type system
• Performance
– Throughput
– Latency
– Hardware Utilization
CONCURRENCY
• Ruby: No built-in concurrency
– Many implementations are single-
threaded
– Cooperative multitasking comes from
external libraries
• Scala: Built-in concurrency
– Built on top of the JVM
– Supports standard concurrency models
o Futures, Actors
7WhitePages
Concurrency
• Ruby library for cooperative
multitasking
• Uses the reactor pattern
• Dangerous: blocking
incorrectly can kill entire
app
• Long chains of callbacks
8WhitePages
Ruby Concurrency: EventMachine
Location Services author created DeferrableSequence to manage callbacks
Wikipedia: http://bit.ly/1qHMez5
9WhitePages
val x = Future {
1 + 1
}
val y = x map { case two =>
two * 2
}
y map println
Scala Concurrency: Futures
• Well known concurrency
model
• Built into Scala
• Functional and imperative:
– Composition (via map, zip, etc.)
– Callbacks (onComplete, etc.)
• Blends nicely into other code
• Can be treated like a 1-
element collection
10WhitePages
11WhitePages Confidential
val x = Future {
1 + 1
}
val y = x map { case two =>
two * 2
}
y map println
Scala Concurrency: Futures
• Drawbacks
– Have to be used in a non-
blocking way (without Await)
– Confusing types
o Future[Future[Int]]
o Seq[Future[Int]]
o flatMap and sequence functions
help to avoid these types
– Difficult to debug
12WhitePages
Areas of interest over SDLC
• Understanding the problem• Exploring solution space• Speed of prototyping
• Performance• Maintainability• Integration
prototyping production
TYPE SYSTEMS
14WhitePages
Type Systems
• Ruby: Dynamically typed
• Scala: Statically typed
• Both have advantages depending on the
circumstance
15WhitePages
Type Systems: Ruby
• Dynamic typing
– Types are checked at runtime
– More flexibility
– Easier to express ideas -> faster
prototyping
– More room for errors
– Less self-documentation, potentially
harder for others to understand code
– Defensive coding
16WhitePages
Type Systems: Scala
• Static typing
– Compiler type checks
– Many errors are caught at compile
time
– Argument types are always
documented
– Easier for others to maintain code
– Strict contracts
– IDEs can tell us the types of any
variable
– Drawback: An IDE becomes a crutch
17WhitePages
Areas of interest over SDLC
• Understanding the problem• Exploring solution space• Speed of prototyping
• Performance• Maintainability• Integration
prototyping production
PERFORMANCE
19WhitePages
Performance: Methodology
• Requests from production sent to
services for 5 minute interval
• Request rate increased until majority of
requests time out (10 seconds)
• Each service ran on identical hardware
• 30 seconds warmups
• 3 trials at each request rate
20WhitePages
Performance tools: Onslaught
• Performance testing tool built at
Whitepages
• Reports throughput, p50, p75, p95, p99,
p999, mean, and max latencies
• Plan to make Onslaught open-source
21WhitePages
Performance: Expectations
• Non-blocking I/O
– Higher throughput
– Better CPU usage (no waits)
– Higher memory usage
• JVM optimizations
– Lower latency
22WhitePages
Performance: Throughput
25 50 75 100 125 1500
50
100
150
200
250
300
350
400
450
Throughput (Ruby)
Requests/s
Su
cces
sfu
l re
spo
nse
s/s
100 200 300 400 500 6000
50
100
150
200
250
300
350
400
450
Throughput (Scala)
Requests/s
Su
cces
sfu
l re
spo
nse
s/s
23WhitePages
Performance: Latency
25 50 75 100 125 1500
1000000
2000000
3000000
4000000
5000000
6000000
7000000
8000000
9000000
10000000
Latency (Ruby)
p50
p95
p99
Request rate (Req/s)
Lat
ency
(µs
)
100 200 300 400 500 6000
1000000
2000000
3000000
4000000
5000000
6000000
7000000
8000000
9000000
10000000
Latency (Scala)
p50
p95
p99
Request rate (Req/s)
Lat
ency
(µs
)
24WhitePages
Performance: CPU
25WhitePages
Performance: CPU
26WhitePages
Performance: Memory
27WhitePages
Areas of interest over SDLC
• Understanding the problem• Exploring solution space• Speed of prototyping
• Performance• Maintainability• Integration
prototyping production
28WhitePages
Summary
Scala or Ruby?
• Prototyping:
– Ruby’s dynamic typing allows for fast prototyping
• Production:
– Scala’s static typing catches more errors and can make code clearer
– Scala supports concurrency with standard library, Ruby does not
– Scala performs better than Ruby in throughput and hardware
utilization
Thank you.Questions?