Grid & Virtualization Working Group
description
Transcript of Grid & Virtualization Working Group
© 2007 Open Grid Forum
Grid & Virtualization Working Group
OGF20 gridvirt-wg
May 7, 2007Manchester, UK
Erol Bozak, ChairSAP, Development Architect
Wolfgang Reichert, Co-ChairIBM, Senior Technical Staff Member
2© 2007 Open Grid Forum
• “I acknowledge that participation in this meeting is subject to the OGF Intellectual Property Policy.”• Intellectual Property Notices Note Well: All statements related to the activities of the OGF and addressed to
the OGF are subject to all provisions of Appendix B of GFD-C.1, which grants to the OGF and its participants certain licenses and rights in such statements. Such statements include verbal statements in OGF meetings, as well as written and electronic communications made at any time or place, which are addressed to:
• the OGF plenary session, • any OGF working group or portion thereof, • the OGF Board of Directors, the GFSG, or any member thereof on behalf of the OGF, • the ADCOM, or any member thereof on behalf of the ADCOM, • any OGF mailing list, including any group list, or any other list functioning under OGF auspices, • the OGF Editor or the document authoring and review process
• Statements made outside of a OGF meeting, mailing list or other function, that are clearly not intended to be input to an OGF activity, group or function, are not subject to these provisions.
• Excerpt from Appendix B of GFD-C.1: ”Where the OGF knows of rights, or claimed rights, the OGF secretariat shall attempt to obtain from the claimant of such rights, a written assurance that upon approval by the GFSG of the relevant OGF document(s), any party will be able to obtain the right to implement, use and distribute the technology or works when implementing, using or distributing technology based upon the specific specification(s) under openly specified, reasonable, non-discriminatory terms. The working group or research group proposing the use of the technology with respect to which the proprietary rights are claimed may assist the OGF secretariat in this effort. The results of this procedure shall not affect advancement of document, except that the GFSG may defer approval where a delay may facilitate the obtaining of such assurances. The results will, however, be recorded by the OGF Secretariat, and made available. The GFSG may also direct that a summary of the results be included in any GFD published containing the specification.”
• OGF Intellectual Property Policies are adapted from the IETF Intellectual Property Policies that support the Internet Standards Process.
OGF IPR Policies Apply
3© 2007 Open Grid Forum
• Status of GridVirt Activities
• Management of Virtual Machines in Grid Infrastructures• Speaker: Ruben S. Montero
• Use Cases
• Virtual Workspaces
• Work Streams for the GridVirt-Working Group
Agenda
4© 2007 Open Grid Forum
Status
• Introduction of the working group at OGF19• Virtualization Concepts• Goals and Scope Definition• Milestones Definition
• Milestones• Milestone 1 (OGF 19)
• Introduction, scope definition & milestones definition• Milestone 2 (OGF 20)
• Terminology definition • Collection of use cases• Determine relations to other OGF WGs and SDOs
• Milestone 3 (OGF 21)• Requirements collection• Determine relation to other standards• First draft of a profile
5© 2007 Open Grid Forum
EGA Reference Model
Grid Management Entity
Grid Component
Policies
Monitor
Enterprise
Other GME
Accounting / Billing
Manage
External Events
The role of Grid Management Entity (GME)
6© 2007 Open Grid Forum
GCGC
Logical GME Grid Components & Dependencies
EGA Reference Model
Grid Management Entity
Grid Component
GC …
…
Monitor
GC GCOther GME
Manage
Policies
Enterprise
Accounting / Billing
Policies Accounting / Billing
Manage
Monitor
GCGCGC
7© 2007 Open Grid Forum
Logical GME
Reference Model – Virtualization
VS Scenario /Landscape
Virtualization Platform GME
VS System GME
Application GME Application
Hypervisor
VSVSVS
Manage
Monitor
Monitor
Manage
Policies Accounting / Billing
PoliciesAccounting / Billing
Grid Components & Dependencies
Manage
Monitor
8© 2007 Open Grid Forum
EGA Reference Model
Grid Management Entity
Grid Component
Policies
Monitor
Enterprise
Accounting / Billing
ManageResources (other GCs)
Resources (other GCs) MetricsMetrics
Service Level Management
Assigns / Provisions
Reconciles
ConsumedGenerates
9© 2007 Open Grid Forum
EGA Reference Model
Lifecycle of a Grid Component
Unconfigured
Inactive
Active
Create / Discover Destroy
Configure Unconfigure
Start Stop
Provision Decommission
Manage
10© 2007 Open Grid Forum
Use Cases Structure
System Virtualization
Provision Virtual System(s)
Provision Virtual System(s)
Manage Virtual System(s)
Manage Virtual System(s)
Decommission Virtual System(s)
Decommission Virtual System(s)
Create / Discover Image(s)
Create / Discover Image(s)
Configure Image(s)
Configure Image(s)
Deploy Virtual System(s) from
Image(s)
Deploy Virtual System(s) from
Image(s)
11© 2007 Open Grid Forum
Use Cases Structure
MigrationMigration
Dynamic Resizing
Dynamic Resizing
MonitoringMonitoring
SnapshottingSnapshotting
System Virtualization
Provision Virtual System(s)
Provision Virtual System(s)
Manage Virtual System(s)
Manage Virtual System(s)
Decommission Virtual System(s)
Decommission Virtual System(s)
12© 2007 Open Grid Forum
Virtualization Use Cases
Migration of virtual system during runtime• Power Saving
The resource manager may pool virtual systems to a reduced number of physical systems in order to save power.
• Planned maintenanceThe physical system as well as the hypervisor could require maintenance activities to be performed (e.g. install patch, hw upgrade or driver etc.). The running job could be migrated to other machines without downtime.
• Adaptation to changing capacity requirements & conditions (availability or offering)
Capacity availability may change in the environment because of recently freed resources by the completed jobs or additional physical systems may be introduced.
13© 2007 Open Grid Forum
Virtualization Use Cases
Power Saving• Policy-driven• Monitor event
• Resource utilization below threshold
• Temperature above threshold
• External event • From hierarchically higher GME
Resulting management actions• Communication with System
Virtualization GME(s) for live migration• Resource allocation / deallocation• Notification of grid component (before
and after live migration)• Accounting event
Grid Management Entity
Grid Component
Policies
Monitor
Enterprise
Accounting / Billing
ManageResources (other GCs)
Resources (other GCs) MetricsMetrics
Assigns / Provisions
Reconciles
ConsumedGenerates
14© 2007 Open Grid Forum
Virtualization Use Cases
Planned Maintenance• External event
Resulting management actions• Communication with System
Virtualization GME(s) for live migration• Resource allocation / deallocation• Notification of grid component (before
and after live migration)• Accounting event
GMEGrid
Component
Policies
Monitor
Enterprise
Other GME
Accounting / Billing
Manage
External Events
15© 2007 Open Grid Forum
Virtualization Use Cases
Dynamic Resizing• Dynamically changing capacity requirements
During runtime the job may require additional capacity (e.g. CPU capacity, Memory capacity, I/O bandwidth etc.). If the underlying physical system is able to serve the requirements more capacity for the job / virtual system can be provided locally on the same physical system.
• Dynamically changing capacity offering / availabilityCapacity availability may change in the physical system (e.g. CPU capacity, Memory capacity, I/O bandwidth etc.) because of recently freed resources by the completed jobs. In these situations available capacity can be utilized for the running jobs.
16© 2007 Open Grid Forum
Virtualization Use Cases
Dynamic Resizing• Policy-driven• Monitor event
• SLO is going to be missed
(progress indicator, trend analysis)
• Grid component event• Dynamic resource requirements
• External event • Dynamic resource availabiliy
Resulting management actions• Communication with System
Virtualization GME for system resizing• Virtual resource reallocation• Changing system parameters/settings• Notification of grid component (before
and after resizing)• Accounting event
Grid Management Entity
Grid Component
Policies
Monitor
Enterprise
Accounting / Billing
ManageResources (other GCs)
Resources (other GCs) MetricsMetrics
Assigns / Provisions
Reconciles
ConsumedGenerates
17© 2007 Open Grid Forum
Virtualization Use Cases
Snapshotting• Stateful cloning
The execution of a job may require costly preparation steps, e.g. retrieving data from the backend, which might be common to all jobs of an activity / application. Rather than doing the preparation separately for each job one (or the first) job can be snapshotted after the preparation and the state can be cloned and distributed.
• Reproducing situationsFor purpose of diagnosis (error or performance analysis) the user may repetitively re-run the same job from a certain persisted state.
• Protecting (long running) jobs from software or hardware failures
By providing recovery points that can be re-activated (long running) jobs can be restarted at a certain persisted state potentially on a different physical system.
18© 2007 Open Grid Forum
Virtualization Use Cases
Stateful Cloning• Policy-driven or external request
from top-level GME
• Subsequent provisioning scenario using the snapshot
Resulting management actions• Communication with System
Virtualization GME to take snapshot• Notification when snapshot has been
taken• Accounting event
Top-level GME manages• Cloning of the snapshot (distribution,
postprocessing)• Provisioning scenario
GMEGrid
Component
Policies
Monitor
Enterprise
Other GME
Accounting / Billing
Manage
External Events
19© 2007 Open Grid Forum
Virtualization Use Cases
Reproducing Situations• External event to take snapshot
• External event to restart from persisted state
Resulting management actions• Communication with System
Virtualization GME to take snapshot
• Communication with System Virtualization GME to restart from snapshot
GMEGrid
Component
Policies
Monitor
Enterprise
Other GME
Accounting / Billing
Manage
External Events
20© 2007 Open Grid Forum
Virtualization Use Cases
Isolation• Metering of job resource consumption
For the purpose of accounting and billing.
• Resource consumption controlThrough isolation the amount / degree of resource consumption can be controlled and leveled, i.e. greedy jobs can be controlled.
21© 2007 Open Grid Forum
Virtualization Use Cases
Metering of Job Resource Consumption
• Data collection at deprovisioning event
Resulting management actions• Communication with System
Virtualization GME to get accurate metering data for the lifetime of the virtual system
GMEGrid
Component
Policies
Monitor
Enterprise
Other GME
Accounting / Billing
Manage
External Events
22© 2007 Open Grid Forum
Virtualization Use Cases
Resource Consumption Control• Policy-driven
Resulting management actions• Communication with System
Virtualization GME to set limits
GMEGrid
Component
Policies
Monitor
Enterprise
Other GME
Accounting / Billing
Manage
External Events
23© 2007 Open Grid Forum
Virtualization Use Cases
Provisioning Scenarios• Resource Provisioning
Definition and activation of the desired runtime environment of a job: Rather than searching for and allocating resources for the job resources can be “created” on demand.
Definition and provisioning of the required software stack (runtime environment)
• EmulationEmulating an environment for legacy jobs: Legacy applications / jobs may require certain physical resources or a certain runtime environment (e.g. operating system). In this situation a virtual system may emulate the legacy environment.
• IsolationAvoiding conflicts.
Ensuring security: To protect the job from spyware the job can be executed in its own dedicated and certified virtual system.
24© 2007 Open Grid Forum
Virtual Workspaces
• A Virtual Workspace is an abstraction of an execution environment…• …that can be made dynamically available • …through well defined protocols,• …the software environment contained in the workspace and
the user submitting the workspace are both trustworthy.
• Virtual Workspaces is not a new idea!• Dynamically setting up cluster nodes
• CoD: http://www.cs.duke.edu/nicl/cod/• bcfg: http://trac.mcs.anl.gov/projects/bcfg2/
• • Providing access to existing installation• Dynamic Accounts: http://workspace.globus.org/da/
• Refining site configuration• Pacman: http://www.archlinux.org/pacman/
25© 2007 Open Grid Forum
Virtual Workspaces
Representation of a Virtual Workspace
Virtual Workspace SpecificationVirtual Workspace
Deployment
XMLXML
MetadataVM Image
XMLXML
Deployment Request
26© 2007 Open Grid Forum
Virtual Workspaces
• Specification of a Virtual Workspace• VM Image• Metadata
• XML Document• Includes deployment-independent information:
VMM and kernel requirementsNICs + IP configuratoinVM image location
Don‘t changes between deployments
• Deployment Request• Specifies availability, memory, CPU%, disk Changes during or between deployments
27© 2007 Open Grid Forum
VW Node
Virtual Workspaces
Node Pool
Node
Node
Node
Node
Node
Node
Node
Node
Node
Node
Node
Node
VW Factory Service
Image Node
VW Service
Trusted Computing Base
28© 2007 Open Grid Forum
VW Node
Virtual Workspaces
Node Pool
Node
Node
Node
Node
Node
Node
Node
Node
Node
Node
NodeVW Factory Service
Image Node
VW Service
Trusted Computing Base
Manage- Start / Stop / Suspend- Migrate- Monitor etc.
Create- Metadata Instance- Deployment Req.
Node
Agent Hypervisor
Node
29© 2007 Open Grid Forum
VW Node
Virtual Workspaces
Node Pool
Node
Node
Node
Node
Node
Node
Node
Node
Node
Node
Node
Node
VW Factory Service
Image Node
VW Service
Trusted Computing Base
Create- Metadata Instance- Deployment Req.
30© 2007 Open Grid Forum
VW Node
Virtual Workspaces
Node Pool
Node
Node
Node
Node
Node
Node
Node
Node
Node
Node
Node
Node
VW Factory Service
Image Node
VW Service
Trusted Computing Base
Create- Metadata Instance- Deployment Req.
31© 2007 Open Grid Forum
VW Node
Virtual Workspaces
Node Pool
Node
Node
Node
Node
Node
Node
Node
Node
Node
Node
Node
Node
VW Factory Service
Image Node
VW Service
Trusted Computing Base
Manage- Start / Stop / Suspend- Migrate- Monitor etc.
32© 2007 Open Grid Forum
Workstreams
• Workstream 1: Refine Use Cases & align Grid Reference Architecture in the Context of System Virtualization• Define the requirements to the grid architecture for
integration with system virtualization platforms
• Workstream 2: Refine the Provisioning Use Case• Define Interaction among the components in the architecture
to create / discover, configure and start a Virtual System• Define information model for definition of Virtual Systems Exploit the concept of „Virtual Workspaces“
33© 2007 Open Grid Forum
Appendix
• Project Homepage• https://forge.gridforum.org/sf/projects/gridvirt-wg
• Mailing list• [email protected]• Subscription: http://www.ogf.org/mailman/listinfo/gridvirt-wg