2
Microsoft project code named
for Scalable and Available Applications
Anil Nori, Distinguished Engineer Muralidhar Krishnaprasad (“MK”), Principal ArchitectMicrosoft
Session Code: DAT202
“Velocity”
3
Outline
Need for Distributed CachesUsage ScenariosProject “Velocity” - Introduction and VisionKey ConceptsAPIsCache Configurations ArchitectureRelease ScheduleDemos
4
• Rapidly dropping memory prices, enabling huge amounts of data to reside in-memory• Availability of cheap commodity hardware that can be easily replaced• Multi-core CPUs, cheap flash memory
Most composite applications designed for serious scale and availability are using mid-tier, distributed caching technologies today.
• Data is sourced from all over the place: local, federated, cloud
• An abundance of applications working together, each with different shapes for the same data
• More workloads require increasing scale, availability, and performance as they become mission critical
• Applications must be cheap to scale
Evolving requirements• Logic and data distributed, with data
living closer to mid-tier logic• Global shared data structures enable
application productivity and acceleration
• Recognition of different types of data with different semantics
• Deployment of high-speed data storage where feature set is not “one size fits all”
Evolving application architecturesbeing solved by
Underlying trends
enabled by
5
Distributed Cache UsageH
oriz
onta
lVe
rtica
ls
Scenario
Web
• User-specific HTTP session and shared state across web farm• In-flight shopping carts for web retail• Enabling online self-service applications• Explicit storage of pre-computed or highly-accessed data
LOB • Enterprise-wide product catalog for POS, analytics• Caching frequently used reference data for a ERP application
Telco • Cellular/VOIP: compute utilization, prepay charges, call routing and session info • SMS: message content / notification / receipt, billing
Travel • Aggregated flight pricing / availability retrieved from airlines
Defense • Sensor network data processing and threat detection
Financial• Per-user portfolio data and delayed quote storage for trading
• Aggregate and process ticker stream for algorithmic trading
6
A version of the authoritative dataAggregated or transformed
Each version is uniqueRefreshed periodically Examples
Web and Enterprise (Product) CatalogsUser, Employee data
Access patternMostly readShared & Concurrent Access
ScaleLarge number of accesses
FunctionalityKey based AccessQuery & FilteringLoading
Caching Reference Data
Web Tier
Clients
Local Cache (in Proc)
Distributed Cache Servers
Data Tier
Usernames, Name-> ID Mapping
Friend Lists Usernames
Scenario: Social Networking
7
Data typically generated as part of the application activityActive during business transactions
Typically logged to a backend data sourceHistorical data
Examples Shopping Cart Session StateEnterprise LOB app (Purchase Order)
Access patternRead and writePrimarily exclusive access
ScaleHigh data (and access) scale
FunctionalityKey based accessTransactions
Caching Activity-oriented Data
Data Tier
Mid Tier
Rich Clients
Distributed Cache
IntegrationHub
Thin Clients
Web Tier
Vendor services,
Pricing
VendorSources
Order, Invoice,
Payment
External Systems…
Aggregated Vendor
Catalogs
Scenario: Enterprise LOB Application
8
Caching Resource-oriented Data
Authoritative dataModified by transactions; spans transactionsExamples
Flight Inventory
Access pattern: Read and writeShared access
FunctionalityKey based accessTransactions
ScaleLarge number of concurrent accessesRelaxed consistency for scale
Scenario: Flight Inventory and Pricing
App Logic
Continental
American
United
Distributed Cache
Flight Routing Itinerary
Flight Segment Flight Price
Airli
nes
Inventory
Booking Service
9
Application Cache Requirements
PerformanceMillisecond access10s of 1000s of accesses
Scale10s – 100s of nodes in enterprise100s – 1000s in web applications
FunctionalityKey based and Query based accessTransactions, Optimistic concurrency controlInvalidations
AvailabilityCertain enterprise and e-commerce data (e.g. shopping cart, inventory)
ConsistencyRelaxed consistency for scale and performance
10
Unified Cache View
What is Velocity?An explicit, distributed, in-memory application cache for all kinds of data (CLR objects, rows, XML, Binary data etc.)
Fuse "memory" across machines into a unified cacheClients can be spread across machines or
processes
Clients Access the Cache as if it
was a large single cache
Cache Layer distributes data
across the various cache
nodes
11
Where does Velocity fit?
App (Mid) TierIIS apps, ASP.NET, .NET Apps, PHP Apps
etc.
Cache TierVelocityData Tier
SQL Server, SSDS, etc.
ClientDesktop, Mobile, etc.
Web ClientAJAX, Silverlight etc.
13
HostPhysical processes hosting Velocity instance.
Named CachesCan span across machinesDefined in the configuration file
RegionsPhysically co-located Container of Cache ItemsMay be implicit or explicitly created
Cache ItemKey, Payload (Object ), Tags, TTL, Timestamps, Version
Velocity Logical HierarchyMachine 2Machine 1
Cache Host ACache Host A Cache Host BCache Host B Cache Host C Cache Host C Machine 3
Cache Host DCache Host D
Named Cache : Product Catalog
Named Cache : Electronics Inventory
Regions Region A
Key Payload Tags Key Payload Tags 121 xxxx “Toy” “Child”
123 yyyy “Toy” “Chair”..
Machine -> Cache Host -> Named Caches -> Regions -> Cache Items -> Objects
14
Velocity – Example Usage// Create instance of cachefactory (reads appconfig)CacheFactory fac = new CacheFactory();
// Get a named cache from the factoryCache catalog = fac.GetCache("catalogcache");
// ------------------------------------------------------// Region based Get/Putcatalog.CreateRegion("toyRegion");
// Both toy and toyparts are put in the same region catalog.Put("toyRegion", "toy-101", new Toy( .,.));Catalog.Put("toyRegion", "toypart-100", new ToyParts(…));
Toy toyObj = (Toy)catalog.Get("toyRegion", "toy-101");
//-------------------------------------------------------// Simple Get/Putcatalog.Put("toy-101", new Toy("thomas", .,.));
// From the same or a different clientToy toyObj = (Toy)catalog.Get("toy-101");
15
Velocity Access APIsSimple Access
Get – Returns object or entire Cache item
Add - Creates new entry else exception if entry exists
Put - Replaces existing entry or creates a new one
Remove - Removes existing entry
Tag Based AccessSearch limited to a specific regionSearch on "all" or "any" equality match of tags
EnumerationEnumerate entire region or result from tag search
17
Types of CachesDistributed Configurations
Partitioned CacheData Partitioned across all nodes in the named cacheUsed for Scale & Availability
Replicated CacheData replicated across all nodes of the named cacheUsed for scaling (read) requests
Local CacheCache in the application process space for frequently accessed itemsPayload kept in object form
18
Velocity Deployment
Velocity Cache Service
Velocity Cache Service
Cluster Service
Application ApplicationVelocity Client
Local CacheVelocity Client
Local Cache
Caching Tier
ApplicationTier
19
Velocity Client2Velocity Client1
Partitioned Cache Put (K2, V2) on Client1
Routing Layer routes to Cache2 (primary)Cache2 performs local operations & returns back to client
Get (K2) on Client2Client2 routes to Cache2 for K2 & gets value from there
Put(K2, v2) Get(K2)
Routing layer Routing layer
Cache2Cache1
Primary Regions
K2, V2
Primary Regions
K1, V1
Cache3
Primary Regions
K3, V3
20
Velocity Client
Local Cache
Velocity ClientLocal Cache
Partitioned Cache fronted by Local Cache
Local Cache can help speed up access on clients
Put(K2, v2) Get(K2)
Routing layer Routing layer
Cache2Cache1
Primary Regions
K2, V2
Primary Regions
K1, V1
Cache3
Primary Regions
K3, V3
K2, V2K2, V2
21
Achieving Scale and PerformanceScale on Data Size
More machines => More memory to cacheScale on Cache Throughput
Cache Throughput: Operations/sec from entire cacheMore machines => keys distributed across more machines => better throughput
PerformanceHigh Performance by scaling out data and processingIncreased memoryIncreased processing
Workload distributed across multiple cache nodes
23
Basic Layout
Load Test Controller
…
Server 1 Server 2 Server 3
Cache Client1 Cache Client 2 Cache Client n
Load Controller Spawns Test
Clients
Client Machines run
test
Cache Service Cluster
Partitioned Cache
24
Demo Output
Single ServerThroughput Increases with Increasing Load
Until Server Saturation
Load
Throughput
Latency
Server 2 AddedThroughput Increases
Latency DecreasesUntil Server Saturation
Server 3 AddedThroughput Increases
Latency Decreases
26
Velocity ClientVelocity Client
Cache2
Secondary Regions
Cache1
Secondary Regions
Cache3
Secondary Regions
Primary Regions
Availability
Put(K2, v2)
K2, V2
K2, V2
Get(K2)
Primary Regions Primary Regions
K3, V3 K1, V1 K1, V1K2, V2K3, V3
K3, V3K1, V1
Routing layer Routing layer
Put (K2, V2) on Cache1Cache2 replicates changes to secondaries Cache1 and Cache3Waits for acks from Cache1 and Cache2; returns back to client
Get (K2) on Cache3Cache3 routes to Cache2 primary for K2 & gets value from there
27
Cache2Cache1 Cache3
Replicated Regions
Replicated CachePut (K2, V2) on Cache1
Cache2 performs local operation & Acks back to Cache1Cache2 Asynchronously replicates changes to all machines
Get on cache3Cache 3 reads from local store and returns the result
Put(K2, v2)
K2, V2
Get(K2)
Replicated Regions Replicated Regions
K3, V3 K1, V1 K1, V1K3, V3 K3, V3K1, V1
K2, V2K2, V2
Routing layer
29
Availability Demo
Cache ClusterVelocity
Embedded in ASP.Net
Application
NorthwindApplication
ASP.net SessionCache
VELOCITYVelocity SessionStore Provider
NorthwindApplication
ASP.net SessionCache
VELOCITYVelocity SessionStore Provider
CacheViewerApplication
VELOCITY
Partitioned Cache
31
Velo
city
Ser
vice
Architecture
…
Server 1 Server 2 Server 3
Cache Client 2
Application/ Web Tier
Cache Tier
Configuration Store (Can be database , File share etc.)Stores Global Cache PoliciesStores Current Partitioning Information
Confi
gura
tion
Man
ager
Velo
city
Ser
vice
Velo
city
Ser
vice
Replication Substrate
Communication Substrate(Cluster Ring Management)
One of the Velocity
Service Hosts the
Configuration Manager
Velocity Client Velocity ClientVelocity Client
Users
Data manage, Object Manager
32
Velocity V1 FeaturesUsage
Key Based Caching with simple Get/Put Access and Tag based lookupOptimistic Version Based and Pessimistic Lock Based UpdatesNotification SupportHighly Available Cache with configurable number of secondariesEnterprise and Web Scale 10s – 100s of machinesIntegration with distributed management and monitoring tools
Consistency ModelsStrong/Synchronous – Synchronous communication with secondariesWeak/Asynchronous – Async communication for Higher performance
TopologiesEmbedded configuration – ASP.NetCache Service
Cache TypesPartitioned – For Activity & Resource DataReplicated – For Reference DataLocal Cache – Client cache for speeding up access
HostingIntegration with ASP.Net for session cachingHostable on IIS, Biztalk etc.
33
ASP.Net Session IntegrationSessionStoreProvider class
Plugs in to the ASP.Net Session store
Stores session state in “Velocity”
ScaleSession information accessible from all ASP.NET nodes
Enables non-sticky routing
Dynamically scale by adding new nodes
High AvailabilitySession data is backed up on to additional machines
Resilient to machine or process failures
34
Administration & Monitoring
Logging Provider model; Default ETW, support for file logs
AdministrationAdmin tool
Command Line Tool
Admin operations
Start and stop cluster or service, configure named caches ..
Integrate with Management Tools (e.g. IIS, Microsoft System Center)
Monitoring toolMonitor Cache statistics
Perfmon integration
35
Velocity Vision
In-memory DM + QPComponentized, embeddableQuery and Search
Consistent CacheTransactionsInvalidationsSynchronization
Distributed CacheDistribute Hash TableScale-outFailoverApplication Cache
CLR objects, rows, etc.ID-based access: Get/Put
Distribute Hash Table
Global Queryable,
Transactional Cache
ScalableApplication Cache
In-MemoryCache
In-Memory Data Manager
Distributed Data Manager
Distributed Data Manager,
Query Processor
Enterprise Application CacheAvailable
Application Cache
Search and LINQ queries across the distributed cache
37
Schedule
June 2008 (CTP1)(TechEd 2008)
Oct 2008 (CTP2)(PDC 2008)
RTM
Distributed, partitioned cache service Regions, Tags
Local Cache Scale, Performance
ASP.Net Integration Monitoring Tools
Availability Replicated cache
Administration Tools Invalidation & Events
Multiple hosting environments
H1 CY 2009
Microsoft.com Velocity sitehttp://www.microsoft.com/sql/velocity.aspx
MSDN Data Platform Development Centerhttp://msdn.microsoft.com/data
MSDN Code Galleryhttp://code.msdn.microsoft.com/velocity
Forums – Access from http://msdn.microsoft.com/data
Track Resources
www.microsoft.com/teched Tech·Talks Tech·Ed BloggersLive Simulcasts Virtual Labs
http://microsoft.com/msdn
Developer’s Kit, Licenses, and MORE!
Resources for Developers
42
© 2008 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED
OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
Top Related