Virtual Hierarchies to Support Server Consolidation Mike Marty Mark Hill University of...
-
Upload
austen-norman -
Category
Documents
-
view
216 -
download
2
Transcript of Virtual Hierarchies to Support Server Consolidation Mike Marty Mark Hill University of...
![Page 1: Virtual Hierarchies to Support Server Consolidation Mike Marty Mark Hill University of Wisconsin-Madison ISCA 2007.](https://reader030.fdocuments.in/reader030/viewer/2022020319/56649f2e5503460f94c4862f/html5/thumbnails/1.jpg)
Virtual Hierarchies to Support Server Consolidation
Mike MartyMark Hill
University of Wisconsin-Madison
ISCA 2007
![Page 2: Virtual Hierarchies to Support Server Consolidation Mike Marty Mark Hill University of Wisconsin-Madison ISCA 2007.](https://reader030.fdocuments.in/reader030/viewer/2022020319/56649f2e5503460f94c4862f/html5/thumbnails/2.jpg)
Mike Marty, University of Wisconsin Virtual Hierarchies to Support Server Consolidation
64-core CMP
Motivation: Server Consolidation
www server
database server #1
database server #2
middleware server #1
middleware server #1
Core
L2 Cache
L1
![Page 3: Virtual Hierarchies to Support Server Consolidation Mike Marty Mark Hill University of Wisconsin-Madison ISCA 2007.](https://reader030.fdocuments.in/reader030/viewer/2022020319/56649f2e5503460f94c4862f/html5/thumbnails/3.jpg)
Mike Marty, University of Wisconsin Virtual Hierarchies to Support Server Consolidation
64-core CMP
Motivation: Server Consolidation
www server
database server #1
database server #2
middleware server #1
middleware server #1
![Page 4: Virtual Hierarchies to Support Server Consolidation Mike Marty Mark Hill University of Wisconsin-Madison ISCA 2007.](https://reader030.fdocuments.in/reader030/viewer/2022020319/56649f2e5503460f94c4862f/html5/thumbnails/4.jpg)
Mike Marty, University of Wisconsin Virtual Hierarchies to Support Server Consolidation
64-core CMP
Motivation: Server Consolidation
www server
database server #1
database server #2
middleware server #1
middleware server #1
data
data
Optimize Performance
![Page 5: Virtual Hierarchies to Support Server Consolidation Mike Marty Mark Hill University of Wisconsin-Madison ISCA 2007.](https://reader030.fdocuments.in/reader030/viewer/2022020319/56649f2e5503460f94c4862f/html5/thumbnails/5.jpg)
Mike Marty, University of Wisconsin Virtual Hierarchies to Support Server Consolidation
64-core CMP
Motivation: Server Consolidation
www server
database server #1
database server #2
middleware server #1
middleware server #1
Isolate Performance
![Page 6: Virtual Hierarchies to Support Server Consolidation Mike Marty Mark Hill University of Wisconsin-Madison ISCA 2007.](https://reader030.fdocuments.in/reader030/viewer/2022020319/56649f2e5503460f94c4862f/html5/thumbnails/6.jpg)
Mike Marty, University of Wisconsin Virtual Hierarchies to Support Server Consolidation
64-core CMP
Motivation: Server Consolidation
www server
database server #1
database server #2
middleware server #1
middleware server #1
Dynamic Partitioning
![Page 7: Virtual Hierarchies to Support Server Consolidation Mike Marty Mark Hill University of Wisconsin-Madison ISCA 2007.](https://reader030.fdocuments.in/reader030/viewer/2022020319/56649f2e5503460f94c4862f/html5/thumbnails/7.jpg)
Mike Marty, University of Wisconsin Virtual Hierarchies to Support Server Consolidation
64-core CMP
Motivation: Server Consolidation
www server
database server #1
database server #2
middleware server #1
middleware server #1
data
Inter-VM Sharing
VMWare’s Content-based Page Sharing Up to 60% reduced memory
![Page 8: Virtual Hierarchies to Support Server Consolidation Mike Marty Mark Hill University of Wisconsin-Madison ISCA 2007.](https://reader030.fdocuments.in/reader030/viewer/2022020319/56649f2e5503460f94c4862f/html5/thumbnails/8.jpg)
Mike Marty, University of Wisconsin Virtual Hierarchies to Support Server Consolidation
Executive Summary
Motivation: Server Consolidation• Many-core CMPs increase opportunities
Goals of Memory System:• Performance• Performance Isolation between VMs• Dynamic Partitioning (VM Reassignment)• Support Inter-VM Sharing• Hypervisor/OS Simplicity
Proposed Solution: Virtual Hierarchy• Overlay 2-level hierarchy on physically-flat CMP• Harmonize with VM Assignment
![Page 9: Virtual Hierarchies to Support Server Consolidation Mike Marty Mark Hill University of Wisconsin-Madison ISCA 2007.](https://reader030.fdocuments.in/reader030/viewer/2022020319/56649f2e5503460f94c4862f/html5/thumbnails/9.jpg)
Mike Marty, University of Wisconsin Virtual Hierarchies to Support Server Consolidation
Outline
Motivation• Server consolidation• Memory system goals• Non-hierarchical approaches
Virtual Hierarchies
Evaluation
Conclusion
![Page 10: Virtual Hierarchies to Support Server Consolidation Mike Marty Mark Hill University of Wisconsin-Madison ISCA 2007.](https://reader030.fdocuments.in/reader030/viewer/2022020319/56649f2e5503460f94c4862f/html5/thumbnails/10.jpg)
Mike Marty, University of Wisconsin Virtual Hierarchies to Support Server Consolidation
TAG-DIRECTORY
duplicate tagdirectory
A Read A
![Page 11: Virtual Hierarchies to Support Server Consolidation Mike Marty Mark Hill University of Wisconsin-Madison ISCA 2007.](https://reader030.fdocuments.in/reader030/viewer/2022020319/56649f2e5503460f94c4862f/html5/thumbnails/11.jpg)
Mike Marty, University of Wisconsin Virtual Hierarchies to Support Server Consolidation
duplicate tagdirectory
TAG-DIRECTORY
A
getM A
1fwd data3
duplicate tagdirectory
2
Read AA
![Page 12: Virtual Hierarchies to Support Server Consolidation Mike Marty Mark Hill University of Wisconsin-Madison ISCA 2007.](https://reader030.fdocuments.in/reader030/viewer/2022020319/56649f2e5503460f94c4862f/html5/thumbnails/12.jpg)
Mike Marty, University of Wisconsin Virtual Hierarchies to Support Server Consolidation
STATIC-BANK-DIRECTORY
getM A
1
2
fwd
data3
A
A Read A
![Page 13: Virtual Hierarchies to Support Server Consolidation Mike Marty Mark Hill University of Wisconsin-Madison ISCA 2007.](https://reader030.fdocuments.in/reader030/viewer/2022020319/56649f2e5503460f94c4862f/html5/thumbnails/13.jpg)
Mike Marty, University of Wisconsin Virtual Hierarchies to Support Server Consolidation
STATIC-BANK-DIRECTORY
getM A
1
2fwd
data3
A
A Read A
with hypervisor-managed cache
![Page 14: Virtual Hierarchies to Support Server Consolidation Mike Marty Mark Hill University of Wisconsin-Madison ISCA 2007.](https://reader030.fdocuments.in/reader030/viewer/2022020319/56649f2e5503460f94c4862f/html5/thumbnails/14.jpg)
Mike Marty, University of Wisconsin Virtual Hierarchies to Support Server Consolidation
Goals
Optimize Performance
Isolate Performance
Allow Dynamic Partitioning
Support Inter-VM Sharing
Hypervisor/OS Simplicity
Yes
Yes
?
Yes
No
No
No
Yes
Yes
Yes
STATIC-BANK-DIRECTORY
TAG-DIRECTORY
STATIC-BANK-DIRECTORY w/ hypervisor-managed cache
No
No
Yes
Yes
Yes
![Page 15: Virtual Hierarchies to Support Server Consolidation Mike Marty Mark Hill University of Wisconsin-Madison ISCA 2007.](https://reader030.fdocuments.in/reader030/viewer/2022020319/56649f2e5503460f94c4862f/html5/thumbnails/15.jpg)
Mike Marty, University of Wisconsin Virtual Hierarchies to Support Server Consolidation
Outline
Motivation
Virtual Hierarchies
Evaluation
Conclusion
![Page 16: Virtual Hierarchies to Support Server Consolidation Mike Marty Mark Hill University of Wisconsin-Madison ISCA 2007.](https://reader030.fdocuments.in/reader030/viewer/2022020319/56649f2e5503460f94c4862f/html5/thumbnails/16.jpg)
Mike Marty, University of Wisconsin Virtual Hierarchies to Support Server Consolidation
Virtual Hierarchies
Key Idea: Overlay 2-level Coherence Hierarchy on CMP
- First level harmonizes with VM/Workload
- Second level allows inter-VM sharing, migration, reconfig
![Page 17: Virtual Hierarchies to Support Server Consolidation Mike Marty Mark Hill University of Wisconsin-Madison ISCA 2007.](https://reader030.fdocuments.in/reader030/viewer/2022020319/56649f2e5503460f94c4862f/html5/thumbnails/17.jpg)
Mike Marty, University of Wisconsin Virtual Hierarchies to Support Server Consolidation
VH: First-Level Protocol
Intra-VM Directory Protocol w/ interleaved directories
Questions: • How to name directories?• How to name sharers?
Dynamic home tile selected by VM Config Table• Hardware VM Config Table at each tile• Set by hypervisor during scheduling
Full bit-vector to track any possible sharer• Intra-VM broadcast also possible
INVgetM
![Page 18: Virtual Hierarchies to Support Server Consolidation Mike Marty Mark Hill University of Wisconsin-Madison ISCA 2007.](https://reader030.fdocuments.in/reader030/viewer/2022020319/56649f2e5503460f94c4862f/html5/thumbnails/18.jpg)
Mike Marty, University of Wisconsin Virtual Hierarchies to Support Server Consolidation
VH: First-Level Protocol
Example:
Hypervisor/OS can freely change VM Config Table • No cache flushes• No atomic updates• No explicit movement of directory state
Address……000101
Home Tile: p14offset
6
VM Config Table
p12p13p14
012
63 p12
p12p13
p14
345
p13
p12
p14
Core
L2 Cache
L1
per-Tile
Dynamic
![Page 19: Virtual Hierarchies to Support Server Consolidation Mike Marty Mark Hill University of Wisconsin-Madison ISCA 2007.](https://reader030.fdocuments.in/reader030/viewer/2022020319/56649f2e5503460f94c4862f/html5/thumbnails/19.jpg)
Mike Marty, University of Wisconsin Virtual Hierarchies to Support Server Consolidation
Virtual Hierarchies
Two Solutions for Global Coherence: VHA and VHB
memory controller(s)
![Page 20: Virtual Hierarchies to Support Server Consolidation Mike Marty Mark Hill University of Wisconsin-Madison ISCA 2007.](https://reader030.fdocuments.in/reader030/viewer/2022020319/56649f2e5503460f94c4862f/html5/thumbnails/20.jpg)
Mike Marty, University of Wisconsin Virtual Hierarchies to Support Server Consolidation
Protocol VHA
Directory as Second-level Protocol• Any tile can act as first-level directory• How to track and name first-level directories?
Full bit-vector of sharers to name any tile• State stored in DRAM• Possibly cache on-chip
+ Maximum flexibility
- DRAM State
- Complexity
![Page 21: Virtual Hierarchies to Support Server Consolidation Mike Marty Mark Hill University of Wisconsin-Madison ISCA 2007.](https://reader030.fdocuments.in/reader030/viewer/2022020319/56649f2e5503460f94c4862f/html5/thumbnails/21.jpg)
Mike Marty, University of Wisconsin Virtual Hierarchies to Support Server Consolidation
VHA Example
dir
ecto
ry/m
emo
ry c
on
tro
ller
getM A
1
2
data6
A
AFwd
data
4
3
getM A
5
Fwd
data
A
![Page 22: Virtual Hierarchies to Support Server Consolidation Mike Marty Mark Hill University of Wisconsin-Madison ISCA 2007.](https://reader030.fdocuments.in/reader030/viewer/2022020319/56649f2e5503460f94c4862f/html5/thumbnails/22.jpg)
Mike Marty, University of Wisconsin Virtual Hierarchies to Support Server Consolidation
Protocol VHB
Broadcast as Second-level Protocol
Attach token count for each block [token coherence]• T tokens for each block. One token to read, all to write• Allows 1-bit at memory per block• Eliminates system-wide ACK
+ Minimal DRAM State
+ Enables easier optimizations
- Global coherence requires more activity
![Page 23: Virtual Hierarchies to Support Server Consolidation Mike Marty Mark Hill University of Wisconsin-Madison ISCA 2007.](https://reader030.fdocuments.in/reader030/viewer/2022020319/56649f2e5503460f94c4862f/html5/thumbnails/23.jpg)
Mike Marty, University of Wisconsin Virtual Hierarchies to Support Server Consolidation
VHB Example
mem
ory
co
ntr
oll
ergetM A
1
2
3global getM A
getM A
Data+tokens
4
A
A
![Page 24: Virtual Hierarchies to Support Server Consolidation Mike Marty Mark Hill University of Wisconsin-Madison ISCA 2007.](https://reader030.fdocuments.in/reader030/viewer/2022020319/56649f2e5503460f94c4862f/html5/thumbnails/24.jpg)
Mike Marty, University of Wisconsin Virtual Hierarchies to Support Server Consolidation
Memory System Goals
Optimize Performance
Isolate Performance
Allow Dynamic Partitioning
Support Inter-VM Sharing
Hypervisor/OS Simplicity
VHA and VHB
Yes
Yes
Yes
Yes
Yes
![Page 25: Virtual Hierarchies to Support Server Consolidation Mike Marty Mark Hill University of Wisconsin-Madison ISCA 2007.](https://reader030.fdocuments.in/reader030/viewer/2022020319/56649f2e5503460f94c4862f/html5/thumbnails/25.jpg)
Mike Marty, University of Wisconsin Virtual Hierarchies to Support Server Consolidation
Outline
Motivation
Virtual Hierarchies
Evaluation
Conclusion
![Page 26: Virtual Hierarchies to Support Server Consolidation Mike Marty Mark Hill University of Wisconsin-Madison ISCA 2007.](https://reader030.fdocuments.in/reader030/viewer/2022020319/56649f2e5503460f94c4862f/html5/thumbnails/26.jpg)
Mike Marty, University of Wisconsin Virtual Hierarchies to Support Server Consolidation
Evaluation: Methods
Wisconsin GEMS• Full-system, execution-driven simulation• Based on Virtutech Simics• http://www.cs.wisc.edu/gems
64-core tiled CMP• In-order SPARC cores• 512 KB, 16-way L2 cache per tile• 2D mesh interconnect, 16-byte links, 5-cycle link latency• Four on-chip memory controllers, 275-cycle DRAM latency
![Page 27: Virtual Hierarchies to Support Server Consolidation Mike Marty Mark Hill University of Wisconsin-Madison ISCA 2007.](https://reader030.fdocuments.in/reader030/viewer/2022020319/56649f2e5503460f94c4862f/html5/thumbnails/27.jpg)
Mike Marty, University of Wisconsin Virtual Hierarchies to Support Server Consolidation
Evaluation: Methods
Workloads: OLTP, SpecJBB, Apache, Zeus• Separate instance of Solaris for each VM
Approximating Virtualization• Multiple Simics checkpoints interleaved onto CMP• Assume workloads map to adjacent cores• Bottomline: No hypervisor simulated
![Page 28: Virtual Hierarchies to Support Server Consolidation Mike Marty Mark Hill University of Wisconsin-Madison ISCA 2007.](https://reader030.fdocuments.in/reader030/viewer/2022020319/56649f2e5503460f94c4862f/html5/thumbnails/28.jpg)
Mike Marty, University of Wisconsin Virtual Hierarchies to Support Server Consolidation
Evaluation: Protocols
TAG-DIRECTORY:• 3-cycle central tag directory (1024 ways!)
STATIC-BANK-DIRECTORY
• Home tiles interleaved by frame address
VHA
• All data allocates in L2 bank of dynamic home tile
VHB
• Unshared data always allocates in local L2
All Protocols: one L2 copy of block on CMP
![Page 29: Virtual Hierarchies to Support Server Consolidation Mike Marty Mark Hill University of Wisconsin-Madison ISCA 2007.](https://reader030.fdocuments.in/reader030/viewer/2022020319/56649f2e5503460f94c4862f/html5/thumbnails/29.jpg)
Mike Marty, University of Wisconsin Virtual Hierarchies to Support Server Consolidation
0
0.2
0.4
0.6
0.8
1
1.2
1.4
No
rma
lize
d R
un
tim
e
Result: Runtime
OLTP Apache
TAG-DIR
STATIC-B
ANK-DIR
VH AVH B
Zeus SpecJBB
TAG-DIR
STATIC-B
ANK-DIR
VH AVH B
TAG-DIR
STATIC-B
ANK-DIR
VH AVH B
TAG-DIR
STATIC-B
ANK-DIR
VH AVH B
Eight VMs x Eight Cores Each = 64 Cores(e.g. eight instances of Apache)
![Page 30: Virtual Hierarchies to Support Server Consolidation Mike Marty Mark Hill University of Wisconsin-Madison ISCA 2007.](https://reader030.fdocuments.in/reader030/viewer/2022020319/56649f2e5503460f94c4862f/html5/thumbnails/30.jpg)
Mike Marty, University of Wisconsin Virtual Hierarchies to Support Server Consolidation
Result: Memory Stall Cycles
0
0.2
0.4
0.6
0.8
1
1.2
1.4
No
rma
lize
d M
em
ory
Sta
ll C
yc
les
Off-chip Local L2 Remote L1 Remote L2
OLTP Apache
TAG-DIR
STATIC-B
ANK-DIR
VH AVH B
Zeus SpecJBB
TAG-DIR
STATIC-B
ANK-DIR
VH AVH B
TAG-DIR
STATIC-B
ANK-DIR
VH AVH B
TAG-DIR
STATIC-B
ANK-DIR
VH AVH B
Eight VMs x Eight Cores Each = 64 cores(e.g. eight instances of Apache)
![Page 31: Virtual Hierarchies to Support Server Consolidation Mike Marty Mark Hill University of Wisconsin-Madison ISCA 2007.](https://reader030.fdocuments.in/reader030/viewer/2022020319/56649f2e5503460f94c4862f/html5/thumbnails/31.jpg)
Mike Marty, University of Wisconsin Virtual Hierarchies to Support Server Consolidation
Executive Summary
Server Consolidation an Emerging Workload
Goals of Memory System:• Performance• Performance Isolation between VMs• Dynamic Partitioning (VM Reassignment)• Support Inter-VM Sharing• Hypervisor/OS Simplicity
Proposed Solution: Virtual Hierarchy• Overlay 2-level hierarchy on physically-flat CMP• Harmonize with Workload Assignment
![Page 32: Virtual Hierarchies to Support Server Consolidation Mike Marty Mark Hill University of Wisconsin-Madison ISCA 2007.](https://reader030.fdocuments.in/reader030/viewer/2022020319/56649f2e5503460f94c4862f/html5/thumbnails/32.jpg)
Mike Marty, University of Wisconsin Virtual Hierarchies to Support Server Consolidation
Backup Slides
![Page 33: Virtual Hierarchies to Support Server Consolidation Mike Marty Mark Hill University of Wisconsin-Madison ISCA 2007.](https://reader030.fdocuments.in/reader030/viewer/2022020319/56649f2e5503460f94c4862f/html5/thumbnails/33.jpg)
Mike Marty, University of Wisconsin Virtual Hierarchies to Support Server Consolidation
Omitting 2nd-Level Coherence: Protocol VH0
Impacts:• Dynamic Partitioning• Inter-VM Sharing (VMWare’s Content-based Page Sharing)• Hypervisor/OS complexity
Example: Steps for VM Migration from Tiles {M} to {N}
1. Stop all threads on {M}
2. Flush {M} caches
3. Update {N} VM Config Tables
4. Start threads on {N}
![Page 34: Virtual Hierarchies to Support Server Consolidation Mike Marty Mark Hill University of Wisconsin-Madison ISCA 2007.](https://reader030.fdocuments.in/reader030/viewer/2022020319/56649f2e5503460f94c4862f/html5/thumbnails/34.jpg)
Mike Marty, University of Wisconsin Virtual Hierarchies to Support Server Consolidation
Omitting 2nd-Level Coherence: Protocol VH0
Example: Inter-VM Content-based Page Sharing• Up to 60% reduced memory demand
Is read-only sharing possible with VH0?
VMWare’s Implementation:• Global hash table to store hashes of pages• Guest pages scanned by VMM, hashes computed• Full comparison of pages on hash match
Potential VH0 Implementation:• How does hypervisor scan guest pages? Are they modified
in cache? • Even read-only pages must initially be written at some point
![Page 35: Virtual Hierarchies to Support Server Consolidation Mike Marty Mark Hill University of Wisconsin-Madison ISCA 2007.](https://reader030.fdocuments.in/reader030/viewer/2022020319/56649f2e5503460f94c4862f/html5/thumbnails/35.jpg)
Mike Marty, University of Wisconsin Virtual Hierarchies to Support Server Consolidation
Physical Hierarchy / Clusters
PL1 $
PL1 $
PL1 $
PL1 $
PL1 $
PL1 $
PL1 $
PL1 $
Shared L2 Shared L2
PL1 $
Shared L2 Shared L2
PL1 $
PL1 $
PL1 $
PL1 $
PL1 $
PL1 $
PL1 $
![Page 36: Virtual Hierarchies to Support Server Consolidation Mike Marty Mark Hill University of Wisconsin-Madison ISCA 2007.](https://reader030.fdocuments.in/reader030/viewer/2022020319/56649f2e5503460f94c4862f/html5/thumbnails/36.jpg)
Mike Marty, University of Wisconsin Virtual Hierarchies to Support Server Consolidation
Physical Hierarchy / Clusters
PL1 $
PL1 $
PL1 $
PL1 $
PL1 $
PL1 $
PL1 $
PL1 $
Shared L2 Shared L2
PL1 $
Shared L2 Shared L2
PL1 $
PL1 $
PL1 $
PL1 $
PL1 $
PL1 $
PL1 $
www server
database server #1
middleware server #1