Managing Virtual Sprawl(How to not let this happen to you)
Jeremy Hitchcock, [email protected]
Why care?
What you have
What you want What you got
Managing clouds like managing single systems increases "system" management by 10x-20x
Clouds Promise
• Greater efficiency
• Faster deploys/less management
• Little/no capital costs and no step functions
Sprawl Eats Potential
• Greater efficiency
• Faster deploys/less management
• Little/no capital costs and no step functions
15 years 3 years
Just not good, yet
Don’t just change broken light bulbs
Wait until it gets dark, then change them all
Let’s get started
1. Architectures
2. Pain points
3. Best practices
4. What do we get?
1: Architectures
• Architecture changes
• Decoupling
• Geography/load balancing
• Disaster recovery
2004
2007-2008
Opera dynamic resource pricing model
Decoupling
• Apps and infrastructure mirror each other
• Years of coupled development
• Hard to retrofit, easier to do from start
Decoupling
New:
Old:Web App DB
Dispatcher
Processing
Storage
Decoupling is Hard
• Logging/debugging
• Common scratch
• Images and provisioning
• Configuration data (run/boot)
• Job dispatch (async/sync)
Images and provisioning
Publish New Code
Add __ new front ends
Even better is that is automatic
Configuration Data
• Most config data is on each image
• Instead, auto populate into source control
• Config, image, controller re-architected
Job Dispatch (sync)
Read photo off disc
Request for photo
Log
Resize/reformat
Return photo to user
Job Dispatch (async)
Read photo off disc
Request for photo Log
Resize/reformat
Return photo to user
1
2
3
4
5
Geography/load balancing
• Data centers do not house eyeballs
• Intra/inter-site load balancing
• Names to numbers (users think names)
• Between clouds/interoperability?
Disaster Recovery
• Practice them
• Failovers should be automatic
• DNS (Quick DNS nit: use short TTLs)
• Contingency plans
Case Study: Authorize.net
Case Study: Authorize.net
Case Study: Authorize.net
; QUESTION SECTION:;secure.authorize.net. IN A
;; ANSWER SECTION:secure.authorize.net. 86400 IN A 64.94.118.32secure.authorize.net. 86400 IN A 64.94.118.33
Case Study: Authorize.net
Case Study: Authorize.net
; QUESTION SECTION:;secure.authorize.net. IN A
;; ANSWER SECTION:secure.authorize.net. 86400 IN A 64.94.118.32secure.authorize.net. 86400 IN A 64.94.118.33
GAH!
2: Pain Points
• Inventory
• Delivery speed
• Supply/demand
• Configuration
• Points of failure
“I can ping it but I don’t know where it is!”
Inventory
• Does it matter?
• Not an asset tag but provisioning scripts
• Audit bills (operational costs)
Delivery Speed
• May actually suffer (more pieces, not iron)
• Be analytical about what can be slow
• Limiting factor of what’s virtualized
• Were you looking before?
•Where is the testing from?
•Is this load dependent?
•Do users notice/care?
•Does it matter?
•Cost to make it faster?
•Savings to make it slower?
Graph from Gomez
Delivery Speed
Supply/demand
• Capital investments versus operating costs
• Big architecture changes to constant tuning
• Sampling time
Configuration
• Configuration in source control
• Has to move to a centralized location
• Patches, updates, revision images
• Lot of hard work here (no return)
Points of Failure
• It’s about risk
• All in the name, DNS
• 99.9% is different from 99.99%
• Any page is better than nothing
3: Best Practices
• App rewrite
• Controller (code, monitoring)
• Configuration (chef, puppet, etc)
• Dev/staging/production (Django/Rails)
• Security
• Monitoring and verification
Dev/Staging/Production
• This stuff works, use it
• Clouds make this possible
• ONLY exception is load testing (big exception)
• Nothing going to work out of the box
Security
• No “behind the firewall”
• Not an after thought, core feature
• Something to test
• Two hash encryption (private data)
• Centralized management makes security easier (At least double or nothing)
Monitoring and Verification
What you monitorWhat your user sees
Are they the same? Test transactions
4: What do we get?
• More choice on availability
• Less step functions (capacity, cost)
• Reduce computational marginal cost
Final Remarks
• Sprawl eats away from the promised good
• Never truly decoupled, apps dictate arch
• Management tools still lacking, more homegrown
• Make it all automatic, not easy
Questions?Jeremy Hitchcock, [email protected]
DynDNS.com offers a suite of DNS, email, domain registration and virtual servers for the home and small business user.
The Dynect Platform provides the enterprise with external managed DNS and traffic management services.
Top Related