8/3/2019 04 - Impact of Advanced Virtualization Technologies on Grid Computing Centers
1/17
Virtualization
in GridComputing
Centers
S. Freitag
Virtualization
Impact on
GridComputing
Conclusion
Impact of Advanced VirtualizationTechnologies on Grid Computing Centers
International Symposium on Grid Computing 2009
Stefan Freitag
Robotics Research Institute
Dortmund University of Technology
23. April 2009
http://find/8/3/2019 04 - Impact of Advanced Virtualization Technologies on Grid Computing Centers
2/17
Virtualization
in GridComputing
Centers
S. Freitag
Virtualization
Impact on
GridComputing
Conclusion
Overview
1 Virtualization
2 Impact on Grid Computing
3 Conclusion
http://find/http://goback/8/3/2019 04 - Impact of Advanced Virtualization Technologies on Grid Computing Centers
3/17
Virtualization
in GridComputing
Centers
S. Freitag
Virtualization
Impact on
GridComputing
Conclusion
Introduction
ApplicationResource
Virtualization
OS Level
Network
Platform
Full
Storage Input/Output
Paravirtualized
Figure: Types of Virtualization
http://find/http://goback/8/3/2019 04 - Impact of Advanced Virtualization Technologies on Grid Computing Centers
4/17
Virtualization
in GridComputing
Centers
S. Freitag
Virtualization
Impact on
GridComputing
Conclusion
Network
VPN (Virtual Private Network)
disjunct network partitions
gateway service for tunneling
WAN for interconnect
WAN
LAN 1
LAN 2
LAN 3
FWFW
FW
GWGW
GW
http://find/http://goback/8/3/2019 04 - Impact of Advanced Virtualization Technologies on Grid Computing Centers
5/17
Virtualization
in GridComputing
Centers
S. Freitag
Virtualization
Impact on
GridComputing
Conclusion
Network
VLAN (Virtual Local Area Network)
Layer 2 (data link layer) construct, IEEE 802.1Q standardDevices on different physical LAN segmentsCommunication as if on same physical LAN segment(bcast domain)
Configuration of VLAN through softwareTraffic shaping/ QoS
LAN
. . .
. . .
S1
S4
. . .
S3
.
.
.
S2
VLAN2
VLAN1
http://find/http://goback/8/3/2019 04 - Impact of Advanced Virtualization Technologies on Grid Computing Centers
6/17
Virtualization
in GridComputing
Centers
S. Freitag
Virtualization
Impact on
GridComputing
Conclusion
Storage
decreasing storage costs (GByte/$)
increasing management complexity
full SAN bandwidth for I/O requests
separation of data and metadata into different places
control unit: appliances or SAN switches
Virtualization
23
4 1
Virtualization
2
1
Control
5
36
4
Virtualization
1a
1b
23
4
Control
1
http://find/http://goback/8/3/2019 04 - Impact of Advanced Virtualization Technologies on Grid Computing Centers
7/17
Virtualization
in GridComputing
Centers
S. Freitag
Virtualization
Impact on
GridComputing
Conclusion
Platform
Full virtualized Environmentguest unaware of virtualization, run native OSreduced performance because of hardware emulation
Para virtualized Environmentmanagement module (hypervisor or virtual machine
monitor) operates with modified operating systemguest OS has much closer control of the underlyinghardware (security, influence on other VMs)
VMManagement
VirtualHardware
VirtualHardware
Guest OS
Application
Guest OS
Application
X86 Hardware
Host OS
Virtualization Layer
X86 Hardware
Virtualization Layer
ApplicationApplication
ManagementVM
Guest OS Guest OS(modified) (modified)
Host OS
http://find/http://goback/8/3/2019 04 - Impact of Advanced Virtualization Technologies on Grid Computing Centers
8/17
Virtualization
in GridComputing
Centers
S. Freitag
Virtualization
Impact on
GridComputing
Conclusion
Input/ Output (IOV)
Typical configuration: 4 to 6 I/O cards per serverEthernetInfinibandFiber Channel
add I/O virtualization capability to PCIe
disaggreationconsolidationvirtualization
/WANLAN
SANPCIe
NICs
IOV Switch
HBAs
Server
http://find/http://goback/8/3/2019 04 - Impact of Advanced Virtualization Technologies on Grid Computing Centers
9/17
Virtualization
in GridComputing
Centers
S. Freitag
Virtualization
Impact on
GridComputing
Conclusion
Input/ Output (IOV)
I/O consolidation
100% of server connected to Ethernet, only 20% to FC
FiberChannel over Ethernet (FCoE), pre-standard
converged Network Adapter (C-NIC/ CNA)benefit for compute centers
fewer adapterspower savingscable management
LAN /WAN
SAN
Server
CNIC
FCoE
Switch
Ethernet
Switch
http://find/http://goback/8/3/2019 04 - Impact of Advanced Virtualization Technologies on Grid Computing Centers
10/17
Virtualization
in GridComputing
Centers
S. Freitag
Virtualization
Impact on
GridComputing
Conclusion
Present situation
Platform and storage virtualization in use
server consolidation
improved utilization of existing servers
increased number of services per m2
reducing TCO (maintenance,...) in the long-run
high availability
run legacy applications
managing mass-storage backendsmigration of virtual machines is bound to restrictions
storage network reconfiguration
http://find/http://goback/8/3/2019 04 - Impact of Advanced Virtualization Technologies on Grid Computing Centers
11/17
Virtualization
in GridComputingCenters
S. Freitag
Virtualization
Impact on
GridComputing
Conclusion
Platform Virtualization @ LRMS level
Job = Virtual Machine
share same operations (start, stop, suspend, checkpointing)
Scenarios:
user submits self-prepared virtual machineon-demand creation of virtual machine by LRMS
VM description as part of job specification
LRMS schedules jobs and/ or virtual machines
Not so good:black-box execution
performance loss (MPI) compared to non-virtualizedsolution
http://find/http://goback/8/3/2019 04 - Impact of Advanced Virtualization Technologies on Grid Computing Centers
12/17
Virtualization
in GridComputingCenters
S. Freitag
Virtualization
Impact on
GridComputing
Conclusion
IOV @ LRMS level
vNIC (virtual Network Interface Card)
vHBA (virtual Host Bus Adapter)
FC WWN used to identify to segregate devices for accesscontrolVirtualized environments: adapters WWN represents allDomUs (Xen)DomU migrates to new server, but WWN does not
zoning and LUN masking (access control) ineffectivevHBA gets own WWN which migrates with VM
http://find/http://goback/8/3/2019 04 - Impact of Advanced Virtualization Technologies on Grid Computing Centers
13/17
Virtualization
in GridComputingCenters
S. Freitag
Virtualization
Impact on
GridComputing
Conclusion
Platform Virtualization @ Grid Level
Benefits
provide uniform environment to Grid users
rapid on-demand provisioning of Grid nodes
non-interfering execution of multiple Grid middlewares
decoupling of knowledge (grid operator/ grid user)
job exchange, e.g. inter cluster/ Grid resources
http://find/http://goback/8/3/2019 04 - Impact of Advanced Virtualization Technologies on Grid Computing Centers
14/17
Virtualization
in GridComputingCenters
S. Freitag
Virtualization
Impact on
GridComputing
Conclusion
Platform Virtualization @ Grid Level
Drawback
resources run different virtualization software
Xen, KVM, VMware, . . .
creation of user environment for each resource
http://find/http://goback/8/3/2019 04 - Impact of Advanced Virtualization Technologies on Grid Computing Centers
15/17
Virtualization
in GridComputingCenters
S. Freitag
Virtualization
Impact on
GridComputing
Conclusion
Virtual Appliances
encapsulate user environment in a virtual appliance
virtual appliance = virtual machine + operating system
+ application level software + MD5
Job = virtual machine virtual appliance
virtual appliance is already technology-dependent :-(
better approach
step 1: create
operating system + application level software + MD5
step 2: make it technology-dependent
http://find/http://goback/8/3/2019 04 - Impact of Advanced Virtualization Technologies on Grid Computing Centers
16/17
Virtualization
in GridComputingCenters
S. Freitag
Virtualization
Impact on
GridComputing
Conclusion
KIWI
idea: prepare once, run anywheredistribution independent, currently only SuSE supported
centralized image description based on XML
output formats: xen, vmware, Amazon EC2, iso, . . .
adoption to Scientific Linux ongoing in Dortmund
PackedImage
description
Image
software
packages
additional
ready 2 serve
accessv
iachro
ot
access
vialoop
UnpackedImage
Figure: from: KIWI Cookbook
http://find/http://goback/8/3/2019 04 - Impact of Advanced Virtualization Technologies on Grid Computing Centers
17/17
Virtualization
in GridComputingCenters
S. Freitag
Virtualization
Impact on
GridComputing
Conclusion
Conclusion & Outlook
Conclusion
platform and storage virtualization widely spread
increased flexibility with I/O virtualization/ FCoE
Grids profit at LRMS and Grid layer from virtualization
on-demand provisioning of virtual appliances
Outlook
Grid resource provides bare metal + virtualization layer
provision of resources to more than one Gridinfrastructure/ VO
http://find/http://goback/Top Related