Powerpath Foundations - Srg

37
Copyright © 2009 EMC Corporation. Do not Copy - All Rights Reserved. PowerPath Foundations - 1 © 2009 EMC Corporation. All rights reserved. PowerPath Foundations PowerPath Foundations Welcome to PowerPath Foundations. Copyright © 2009 EMC Corporation. All rights reserved. These materials may not be copied without EMC's written consent. EMC believes the information in this publication is accurate as of its publication date. The information is subject to change without notice. THE INFORMATION IN THIS PUBLICATION IS PROVIDED “AS IS.” EMC CORPORATION MAKES NO REPRESENTATIONS OR WARRANTIES OF ANY KIND WITH RESPECT TO THE INFORMATION IN THIS PUBLICATION, AND SPECIFICALLY DISCLAIMS IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Use, copying, and distribution of any EMC software described in this publication requires an applicable software license. EMC² , EMC, EMC ControlCenter, AdvantEdge, AlphaStor, ApplicationXtender, Avamar, Captiva, Catalog Solution, Celerra, Centera, CentraStar, ClaimPack, ClaimsEditor, ClaimsEditor, Professional, CLARalert, CLARiiON, ClientPak, CodeLink, Connectrix, Co- StandbyServer, Dantz, Direct Matrix Architecture, DiskXtender, DiskXtender 2000, Document Sciences, Documentum, EmailXaminer, EmailXtender, EmailXtract, enVision, eRoom, Event Explorer, FLARE, FormWare, HighRoad, InputAccel,InputAccel Express, Invista, ISIS, Max Retriever, Navisphere, NetWorker, nLayers, OpenScale, PixTools, Powerlink, PowerPath, Rainfinity, RepliStor, ResourcePak, Retrospect, RSA, RSA Secured, RSA Security, SecurID, SecurWorld, Smarts, SnapShotServer, SnapView/IP, SRDF, Symmetrix, TimeFinder, VisualSAN, VSAM-Assist, WebXtender, where information lives, xPression, xPresso, Xtender, Xtender Solutions; and EMC OnCourse, EMC Proven, EMC Snap, EMC Storage Administrator, Acartus, Access Logix, ArchiveXtender, Authentic Problems, Automated Resource Manager, AutoStart, AutoSwap, AVALONidm, C-Clip, Celerra Replicator, CLARevent, Codebook Correlation Technology, Common Information Model, CopyCross, CopyPoint, DatabaseXtender, Digital Mailroom, Direct Matrix, EDM, E-Lab, eInput, Enginuity, FarPoint, FirstPass, Fortress, Global File Virtualization, Graphic Visualization, InfoMover, Infoscape, MediaStor, MirrorView, Mozy, MozyEnterprise, MozyHome, MozyPro, NetWin, OnAlert, PowerSnap, QuickScan, RepliCare, SafeLine, SAN Advisor, SAN Copy, SAN Manager, SDMS, SnapImage, SnapSure, SnapView, StorageScope, SupportMate, SymmAPI, SymmEnabler, Symmetrix DMX, UltraFlex, UltraPoint, UltraScale, Viewlets, VisualSRM are trademarks of EMC Corporation. All other trademarks used herein are the property of their respective owners.

Transcript of Powerpath Foundations - Srg

Page 1: Powerpath Foundations - Srg

Copyright © 2009 EMC Corporation. Do not Copy - All Rights Reserved.

PowerPath Foundations - 1

© 2009 EMC Corporation. All rights reserved.

PowerPath FoundationsPowerPath Foundations

Welcome to PowerPath Foundations.

Copyright © 2009 EMC Corporation. All rights reserved.

These materials may not be copied without EMC's written consent.

EMC believes the information in this publication is accurate as of its publication date. The information is subject to change without notice.

THE INFORMATION IN THIS PUBLICATION IS PROVIDED “AS IS.” EMC CORPORATION MAKES NO REPRESENTATIONS OR WARRANTIES OF ANY KIND WITH RESPECT TO THE INFORMATION IN THIS PUBLICATION, AND SPECIFICALLY DISCLAIMS IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.

Use, copying, and distribution of any EMC software described in this publication requires an applicable software license.

EMC² , EMC, EMC ControlCenter, AdvantEdge, AlphaStor, ApplicationXtender, Avamar, Captiva, Catalog Solution, Celerra, Centera, CentraStar, ClaimPack, ClaimsEditor, ClaimsEditor, Professional, CLARalert, CLARiiON, ClientPak, CodeLink, Connectrix, Co-StandbyServer, Dantz, Direct Matrix Architecture, DiskXtender, DiskXtender 2000, Document Sciences, Documentum, EmailXaminer, EmailXtender, EmailXtract, enVision, eRoom, Event Explorer, FLARE, FormWare, HighRoad, InputAccel,InputAccel Express, Invista, ISIS, Max Retriever, Navisphere, NetWorker, nLayers, OpenScale, PixTools, Powerlink, PowerPath, Rainfinity, RepliStor, ResourcePak, Retrospect, RSA, RSA Secured, RSA Security, SecurID, SecurWorld, Smarts, SnapShotServer, SnapView/IP, SRDF, Symmetrix, TimeFinder, VisualSAN, VSAM-Assist, WebXtender, where information lives, xPression, xPresso, Xtender, Xtender Solutions; and EMC OnCourse, EMC Proven, EMC Snap, EMC Storage Administrator, Acartus, Access Logix, ArchiveXtender, Authentic Problems, Automated Resource Manager, AutoStart, AutoSwap, AVALONidm, C-Clip, Celerra Replicator, CLARevent, Codebook Correlation Technology, Common Information Model, CopyCross, CopyPoint, DatabaseXtender, Digital Mailroom, Direct Matrix, EDM, E-Lab, eInput, Enginuity, FarPoint, FirstPass, Fortress, Global File Virtualization, Graphic Visualization, InfoMover, Infoscape, MediaStor, MirrorView, Mozy, MozyEnterprise, MozyHome, MozyPro, NetWin, OnAlert, PowerSnap, QuickScan, RepliCare, SafeLine, SAN Advisor, SAN Copy, SAN Manager, SDMS, SnapImage, SnapSure, SnapView, StorageScope, SupportMate, SymmAPI, SymmEnabler, Symmetrix DMX, UltraFlex, UltraPoint, UltraScale, Viewlets, VisualSRM are trademarks of EMC Corporation.

All other trademarks used herein are the property of their respective owners.

Page 2: Powerpath Foundations - Srg

Copyright © 2009 EMC Corporation. Do not Copy - All Rights Reserved.

PowerPath Foundations - 2

© 2009 EMC Corporation. All rights reserved. PowerPath Foundations - 2

Course Objectives

Upon completion of this course, you will be able to:

List the members of the PowerPath family of products

Describe PowerPath devices and concepts

Explain how PowerPath performs load balancing and path failover

Discuss the features and benefits of using PowerPath in storage environments

Explain the basics of PowerPath Encryption with RSA, PowerPath Migration Enabler, and PowerPath/VE

The objectives for this course are shown here. Please take a moment to read them.

Page 3: Powerpath Foundations - Srg

Copyright © 2009 EMC Corporation. Do not Copy - All Rights Reserved.

PowerPath Foundations - 3

© 2009 EMC Corporation. All rights reserved. PowerPath Foundations - 3

PowerPath Family of Products

PowerPath Multipathing– Multipathing and load balancing across a SAN

PowerPath Migration Enabler– Allows for migration of data across a SAN

using a Host

PowerPath Encryption with RSA– Encrypts data traversing a SAN from the Host

using RSA Key Manager Software

PowerPath/VE– PowerPath Multipathing for Virtual

Environments

The PowerPath family of products is the enabler for EMC host-based solutions including multipathing, data migration, and host-based encryption. For the purpose of this course, wherever the product is referred to as “PowerPath” we are referring to PowerPath Multipathing.

PowerPath Multipathing is the most common product in this family and the one we will primarily focus on in this course. It allows for maximizing application availability and performance, automation of path management, and reduction of complexity and cost.

Use PowerPath Encryption with RSA to protect sensitive data against unauthorized access if a disk drive or array is removed.

Use PowerPath Migration Enabler to eliminate application downtime during data migrations or virtualization implementations by integrating EMC PowerPath Migration Enabler with other technologies.

PowerPath/VE is used to optimize VMware vSphere and Microsoft Server 2008 Hyper-V virtual environments with PowerPath Multipathing features.

Page 4: Powerpath Foundations - Srg

Copyright © 2009 EMC Corporation. Do not Copy - All Rights Reserved.

PowerPath Foundations - 4

© 2009 EMC Corporation. All rights reserved. PowerPath Foundations - 4

PowerPath Theory of Operations

Upon completion of this module, you will be able to:

Identify a PowerPath configuration and PowerPath’s function in the I/O path

Describe how PowerPath is licensed

The objectives for this module are shown here. Please take a moment to read them.

Page 5: Powerpath Foundations - Srg

Copyright © 2009 EMC Corporation. Do not Copy - All Rights Reserved.

PowerPath Foundations - 5

© 2009 EMC Corporation. All rights reserved. PowerPath Foundations - 5

PowerPath Configuration

All volumes are accessible through all paths

Redundant components (HBAs, switches, SANs, array ports, etc.) eliminate single points of failure

Storage

Host Application(s)

D

HBA

D

HBA

D

HBA

D

HBA Host BusAdapter

HBADriver

SER

VER

STO

RA

GE

InterconnectTopology

Front-end Ports

InterconnectTopology

This slide illustrates a PowerPath configuration. In a PowerPath configuration, there are one or more paths from the host to a device in a storage array. Each path is unique because it has its own discrete set of components, e.g., HBA, SAN, cable, fibre switch, and array port.

Access to the device in the storage array is accomplished by mapping as a shared device across two or more storage interfaces. An interface is an FA (Fibre Adapter) on a Symmetrix system or an SP (Storage Processor) on a CLARiiON system. The result is that all logical devices can be accessed through all paths to the device. When combined with path failover and load balancing software such as PowerPath, the ability to access devices through multiple paths increases application availability.

In order to take advantage of all array interfaces, the devices should be assigned equally among interface cards. This ensures that the full capacity of the array interfaces is being used.

Page 6: Powerpath Foundations - Srg

Copyright © 2009 EMC Corporation. Do not Copy - All Rights Reserved.

PowerPath Foundations - 6

© 2009 EMC Corporation. All rights reserved. PowerPath Foundations - 6

PowerPath Implementation PowerPath sits between the applications and the HBA driver

Applications direct I/O to PowerPath

PowerPath directs I/O to an optimal path based on current workload and path availability

Open Systems HOSTLogical volume manager

File system

PowerPath

ApplicationsManagement

utilitiesDBMS

Symmetrix/CLARiiON/HDS/IBM/HP

Host busadapter

Host busadapter

Host busadapter

Host busadapter

As the illustration shows, PowerPath operates between the application and the HBA driver. This position in the I/O stack gives PowerPath the ability to intercept and manage all I/O between the application and the array storage device.

PowerPath intelligently manages the paths to a device by sending I/O down the optimal path based on the current PowerPath load balancing and failover policy setting for the device. PowerPath also takes into account path usage and availability before deciding which path to send the I/O down. If a path to the device fails, PowerPath automatically reroutes the I/O down an alternate path.

Page 7: Powerpath Foundations - Srg

Copyright © 2009 EMC Corporation. Do not Copy - All Rights Reserved.

PowerPath Foundations - 7

© 2009 EMC Corporation. All rights reserved. PowerPath Foundations - 7

Volume Path Set

PowerPath– Keeps a table of paths– Presents a single

device to the application

– Combines equivalent paths into a single “Path Set”

PowerPath

HBAHBA

SER

VER

STO

RA

GE Front-end

Ports

Storage Area Network

Volume Path Set

Application

HBAHBA

POWERPATH

“Pseudo”Device

During installation, PowerPath determines all the paths to a device and builds a table to use when deciding how to route I/O. PowerPath references this table and drives the I/O to either a native or pseudo device. The supported operating systems for these device types are listed on the slide.

The operating system creates native devices to represent and provide access to logical devices. A native device is path-specific and represents a single path to a logical device. The device is “native” in that it is provided by the operating system for use with applications. In this slide, there is a native device for each path. The storage system is configured with two shared logical devices, each of which can be accessed by four paths. There are eight native devices, four in white (numbered 0, 2, 4, and 6) that represent a unique path set to logical device 0; and four in black (numbered 1, 3, 5, and 7) that represent a unique path set to logical device 1. When PowerPath is installed, it uses the native device to point to all the paths to the device.

Applications do not need to be reconfigured to use native devices. The idea is to use the existing disk devices created by the operating system.

A PowerPath pseudo device represents a single logical device and the set of all paths leading to it. There is one pseudo device per path. In the example, logical devices 0 and 1 are referred to by pseudo device names emcpower1c and emcpower2c. Each pseudo device represents the set of paths connected to its respective logical device: emcpower1c represents the set of paths connected to logical device 0, and emcpower2c represents the set of paths connected to logical device 1. In most cases, the application must be reconfigured to use pseudo devices; otherwise, PowerPath load balancing and path failover functionality are not available.

Page 8: Powerpath Foundations - Srg

Copyright © 2009 EMC Corporation. Do not Copy - All Rights Reserved.

PowerPath Foundations - 8

© 2009 EMC Corporation. All rights reserved. PowerPath Foundations - 8

PowerPath Array Support Concepts

Mapped to allFront-end ports

Host Application(s)

HBA HBA HBAHBASER

VER

STO

RA

GE

Storage Area Network

Active

Active

Active

Active

PowerPath

Owned by SP-A

Host Application(s)

HBA HBA HBAHBASER

VER

STO

RA

GE

Storage Area Network

SP-A

Active

Active

SP-B

Passive

Passive

PowerPath

0 1 0 1

Active-Active Active-Passive

In an active-active storage array, if multiple interfaces exist to a LUN, they all provide equal access to the logical device. Active-active means all interfaces to a device are active simultaneously.

In a configuration that includes an active-active array, PowerPath can spread the work load across all zoned paths. In addition, PowerPath can failover across any zoned path to the LUN. EMC Symmetrix, IBM ESS, Hitachi Lightning, and EMC Invista are examples of active-active arrays.

In the active-passive array, a LUN is assigned to port 0 and port 1 on Storage Processor A (SP A). In this system, SP A is designated as the primary or active route to the device, and therefore all I/O is directed down the paths through SP A to the device. PowerPath load balances I/O across these active paths as shown by the green arrows.

The LUN can also be accessed through Storage Processor B but only after the device has been re-assigned (trespassed) to SP B. This path is referred to as a passive path. PowerPath does not send I/O down passive paths. Passive paths are shown in orange.

With all active paths to the LUN unavailable, the active paths to an interface card logical device must be moved to another storage processor. This reassignment is initiated by the other, functioning interface. When PowerPath is installed on the host, the reassignment is initiated by PowerPath, which instructs the storage system to make the reassignment. On a CLARiiON array, these reassignments are known as trespassing. Trespassing can take several seconds to complete; however, I/Os do not fail during this time. After devices are trespassed, PowerPath detects the changes and seamlessly sends data via the new route. After a trespass, logical devices can be trespassed back to their original paths once the error is corrected.

Page 9: Powerpath Foundations - Srg

Copyright © 2009 EMC Corporation. Do not Copy - All Rights Reserved.

PowerPath Foundations - 9

© 2009 EMC Corporation. All rights reserved. PowerPath Foundations - 9

PowerPath Multipathing LicensingPowerPath– Unlocks the full load balancing and path failover capabilities of PP and

management of all EMC and non-EMC arrays– Configures the following default load balancing policies for these array

classes Symmetrix Optimize (so) optimized for SymmetrixCLARiiON Optimize (co) optimized for CLARiiON Adaptive (ad) optimized for non-EMC arrays

PowerPath SE– Backend (between switch and array) failover only– Single HBA supported only– No load balancing available– Up to two paths only per device

PowerPath/VE– Same as Full PowerPath license, but supports only Microsoft Windows

2008Hyper-V and VMware vSphere

– Only available through Powerlink licensing

PowerPath Multipathing has three licenses available.

PowerPath: this license permits the user to take advantage of the full set of PowerPath load balancing and path failover functionality. This license was formerly referred to as PowerPath Enterprise or PowerPath Enterprise Plus.

PowerPath SE: this license supports back-end failover only. In other words, only paths from the switch to storage arrays are candidates for failover. Back-end storage failover is for single HBAs that have one path to the switch and are zoned to have a maximum of two paths to the storage array. This license was formerly known as PowerPath Base, Unlicensed PowerPath, PowerPath Fabric Failover, and Utility Kit PowerPath. These now all refer to the same product.

PowerPath/VE: this license enables full PowerPath multipathing in a Virtual Environment. Currently supported environments are listed above. Check Powerlink for more up-to-date information regarding supported environments.

Page 10: Powerpath Foundations - Srg

Copyright © 2009 EMC Corporation. Do not Copy - All Rights Reserved.

PowerPath Foundations - 10

© 2009 EMC Corporation. All rights reserved. PowerPath Foundations - 10

Module Summary

Key points covered in this module:

PowerPath configuration

PowerPath’s operational location on the host

Volume path set

PowerPath array support concepts

PowerPath licensing

These are the key points covered in this module. Please take a moment to review them.

Page 11: Powerpath Foundations - Srg

Copyright © 2009 EMC Corporation. Do not Copy - All Rights Reserved.

PowerPath Foundations - 11

© 2009 EMC Corporation. All rights reserved. PowerPath Foundations - 11

PowerPath Business Justification Benefits

Upon completion of this module, you will be able to:

Cite three major challenges confronting business today

Explain how PowerPath can solve business challenges

Identify the value PowerPath adds to a business

The objectives for this module are shown here. Please take a moment to read them.

Page 12: Powerpath Foundations - Srg

Copyright © 2009 EMC Corporation. Do not Copy - All Rights Reserved.

PowerPath Foundations - 12

© 2009 EMC Corporation. All rights reserved. PowerPath Foundations - 12

Current Business Challenges

Application availability – Businesses require 24x7 access to data

Storage manageability– Optimize information access– Maximize server, storage, and data path utilization

Quality of Service (QoS)– Diverse requirements– Competing applications– Tune the infrastructure performance by application – Protect application availability

The slide lists the main business challenges that IT professionals face today. Customers need their businesses up 24x7 in order to protect revenue streams, support new product development efforts, and increase customer satisfaction.

Businesses need a method to manage their storage in order to maximize their technology investments. IT professionals are searching for new and more efficient, policy-based, automated management tools to manage and reduce the complexity of their environment. Tools must be capable of tuning the environment in order to maximize server, storage, and data path utilization.

These management tools must have the ability to quickly respond to various and changing business requirements, provide continuous data protection, and tune I/O of competing applications.

Page 13: Powerpath Foundations - Srg

Copyright © 2009 EMC Corporation. Do not Copy - All Rights Reserved.

PowerPath Foundations - 13

© 2009 EMC Corporation. All rights reserved. PowerPath Foundations - 13

Optimized performance and high availability; no application disruption

Consistent and improved service levels

Improved manageability – saves time, reduces maintenance costs

Optimized data management through user-selectable storage allocation policies

Automated information utilization; optimized data movement

Impact to Business of PowerPath Features

Automatic path failover and recovery

Dynamic load balancing of I/O

Online configuration and management

Policy-based management

Automated server-to-storage I/O management

Now that you understand the business challenges, let’s explain how PowerPath helps solve these challenges. PowerPath helps by automatically tuning the storage area network when it detects imbalances by selecting alternate paths for the data to be routed through. It also combines multiple path I/O capabilities, automatic load balancing and path failover functions, and online path management into one integrated package.

PowerPath’s automatic load balancing functionality guarantees maximum use of multiple paths to the data. Critical server and array resources, such as memory and CPU, are not constrained by congested paths to the data while other paths to the same data are under used. By maximizing resource use, server and array investments are used to their full potential.

PowerPath’s automatic path fail-over capability guarantees application availability. When the host and storage environment are configured with redundant paths through multiple HBAs and fabrics, PowerPath automatically uses alternate paths to the device and continues to balance I/O load across the remaining paths. When a path is restored, PowerPath automatically adds the path back into the available paths list and uses it for load balancing.

All load balancing and failover policies are tunable while the host and application are online. Policies can be changed for an array or on a device by device basis.

Page 14: Powerpath Foundations - Srg

Copyright © 2009 EMC Corporation. Do not Copy - All Rights Reserved.

PowerPath Foundations - 14

© 2009 EMC Corporation. All rights reserved. PowerPath Foundations - 14

The Value of PowerPath

Support for open systems

Support for multiple storage arrays

Improves SAN performance

Maximizes utilization of shared SAN devices and paths

Reduces connectivity administration overhead and environmental downtime

Protects against path failures

PowerPath can add value in many ways. For example, PowerPath is supported on many operating system platforms, including Windows, Linux, Solaris, HP-UX, and AIX. PowerPath supports all EMC arrays and several non-EMC arrays (referred to as third-party arrays).

PowerPath helps maximize SAN performance by using all the I/O processing and bus capacity of all paths to a device. PowerPath load balances I/O on a host-by-host basis and works by equalizing I/O load for all paths to the array from the host. PowerPath intelligently handles each I/O request and checks current load balancing and path failover settings to choose the least-burdened available path.

PowerPath reduces management time and downtime, because administrators no longer need to configure paths statically across logical devices. With PowerPath, no setup time is required, and paths are always configured for optimum performance.

If there is a failure in the data path, PowerPath fails over the I/O to an alternative path. PowerPath determines the best and optimal way to use devices and paths that are shared across the SAN.

Page 15: Powerpath Foundations - Srg

Copyright © 2009 EMC Corporation. Do not Copy - All Rights Reserved.

PowerPath Foundations - 15

© 2009 EMC Corporation. All rights reserved. PowerPath Foundations - 15

Module Summary

Key points covered in this module:

Major challenges confronting business today

How PowerPath helps solve business challenges

How PowerPath adds value to a business

These are the key points covered in this module. Please take a moment to review them.

Page 16: Powerpath Foundations - Srg

Copyright © 2009 EMC Corporation. Do not Copy - All Rights Reserved.

PowerPath Foundations - 16

© 2009 EMC Corporation. All rights reserved. PowerPath Foundations - 16

PowerPath Features and FunctionalityUpon completion of this module, you will be able to:

Explain how PowerPath’s load balancing and path failover functionality can help maintain and enhance application availability and performance

Describe how PowerPath’s automatic path testing and restore is used to maintain I/O paths

Cite how PowerPath’s online management can help tune an application

Explain how PowerPath adds value to clustered environments

Identify the tools used to manage a PowerPath environment

The objectives for this module are shown here. Please take a moment to read them.

Page 17: Powerpath Foundations - Srg

Copyright © 2009 EMC Corporation. Do not Copy - All Rights Reserved.

PowerPath Foundations - 17

© 2009 EMC Corporation. All rights reserved. PowerPath Foundations - 17

Path Failure without PowerPath

Volumes are spread across all available paths

Each volume has a single path based on the device name used

Host adapter and cable connections are single points of failure Storage

Host Application(s)

D

HBA

D

HBA

D

HBA

D

HBA Host BusAdapter

HBADriver

SER

VER

STO

RA

GE

SAN

Front-end Ports

I/O

error

Error: I/O Interrupted!

On this slide, an application running on a server has four paths to the storage. However, only one path is used because the operating system running on the server only allows one path to be defined. This is expressly done when the administrator chooses a device name to use.

Without PowerPath or another path failover software package, the loss of a channel (represented by the “error”) means one or more applications may stop functioning. This can be caused by the loss of an HBA, storage array front-end port, Fibre Channel switch, hub, or a failed cable. In a standard non-PowerPath environment, these are all single points of failure. In this case, all I/O that was heading down that path is now lost, resulting in an application failure and the potential for data loss or corruption.

Page 18: Powerpath Foundations - Srg

Copyright © 2009 EMC Corporation. Do not Copy - All Rights Reserved.

PowerPath Foundations - 18

© 2009 EMC Corporation. All rights reserved. PowerPath Foundations - 18

Path Failure with PowerPathIf a path component fails, the device driver returns a timeout to PowerPath

PowerPath responds by taking the path offline and re-driving I/O through an alternate path

Subsequent I/Os use surviving path(s)

Application is unaware of the failure

App

HBA HBA

D DD

HBA Host BusAdapter

HBADriver

Storage

SER

VER

STO

RA

GE

D

HBA

PowerPath

Front-end Ports

SAN

I/O

Ack

I/O

I/O Sucessful!

error

This illustration shows the general I/O flow on a host with PowerPath connected to an active-active array. The host has four HBAs which are cabled to a port on a switch. There are four LUNs in the array, each configured to be visible through all four ports. The switch is configured so that each HBA is zoned to each port in the array. Therefore, each HBA can access a LUN through any port on the array.

The following sequence of events occurs. 1. The application creates an I/O request designated for one of the LUNs. 2. The application forwards the I/O request to the appropriate HBA driver. 3. PowerPath intercepts the I/O request while enroute to the HBA driver. PowerPath checks the volume path set for the

device and recognizes there are 16 possible paths to this device. 4. PowerPath checks load-balancing policy for the LUN and forwards the I/O to the proper driver. The driver forwards

the I/O to the HBA driver. 5. The HBA driver sends I/O over the SAN to the target port on the array.6. A cable failure occurs while the I/O is being passed down the cable as shown by the explosion. At this point there is an I/O timeout value that comes into play. 7. When the timeout value has expired, the HBA returns an error to the HBA driver. The driver then attempts to return

the error to the application. 8. PowerPath intercepts the error, checks the volume path set for the LUN to determine the load balancing policy, then, 9. Redirects the I/O request to another open path. 10.The I/O is completed and the acknowledgement is returned through the same path the I/O was sent, directly to the

application, PowerPath does not intercept.

Page 19: Powerpath Foundations - Srg

Copyright © 2009 EMC Corporation. Do not Copy - All Rights Reserved.

PowerPath Foundations - 19

© 2009 EMC Corporation. All rights reserved. PowerPath Foundations - 19

PowerPath Fabric FailoverAvailable for CLARiiON and Symmetrix

Provides back-end failover, non-disruptive upgrades, and single HBA configuration support for single-attach hosts

Included in CLARiiON Utility Kit

Non-HA environment

No multi-pathing capabilities

Application

SD

HBA

Request

PowerPath

SPSP--BBSP ASP A

Fibre Channel Switch

CLARiiON

SERVER

As a special case for non-high availability environments, PowerPath Fabric Failover provides single HBA configuration support, back-end failover support, and non-disruptive upgrade support.

Fabric failover has no multi-pathing or load balancing capabilities and therefore should not be used in a high availability environment.

PowerPath Fabric Failover is a version of PowerPath without a license key that provides only basic failover functionality. It protects against CLARiiON SP failures, Symmetrix FA port failures, back-end storage system failures, and supports non-disruptive upgrade (NDU) of storage system software. While a server is running normally, PowerPath Fabric Failover takes no action.

If a failure occurs in an SP or an FA port, PowerPath Fabric Failover attempts to fail over (transfer) the I/Os to a different SP or FA port. PowerPath Fabric Failover does not protect against HBA, switch, or switch port failures. To protect against such failures in storage systems, you must have multiple HBAsconnected to a storage system, and order PowerPath with the full license.

In this example, PowerPath sends an I/O down the path to SP B. A failure occurs on the back end. PowerPath receives the error and resends the I/O down the path to SP A.

PowerPath Fabric Failover is called Utility Kit PowerPath in CLARiiON documentation.

Page 20: Powerpath Foundations - Srg

Copyright © 2009 EMC Corporation. Do not Copy - All Rights Reserved.

PowerPath Foundations - 20

© 2009 EMC Corporation. All rights reserved. PowerPath Foundations - 20

I/O without PowerPath: Queues out of BalanceI/O performance is partially based on queue length

Optimal performance cannot be achieved unless work is balanced

I/O workload across HBAs is seldom balanced

At any point in time, some paths may be idle while other paths have multiple I/O operations queued

SAN

Host Application(s)

Host BusAdapter

HBADriverD DD D

HBA HBAHBA HBA

Request

Request

Request

Request

Request

Request

Request

RequestSER

VER

STO

RA

GE

Request

This example illustrates how I/O throughput is unbalanced without PowerPath installed.

On the slide, several applications running on a server have four paths to the storage. However, only one path is used due to the lack of multipathing software.

The administrator can manually configure the storage system to spread I/O load across the paths based on snapshot measurements, guesstimates of average loading, and predictive loads. The final configuration has some applications sharing devices down one path.

The example depicts a snapshot of the system at a moment in time when the depth of the I/O queues is very unbalanced. Host applications sitting on top of deep queues are not getting the data they need.

In this instance, two of the applications are currently generating high I/O traffic, causing two channels to become overloaded, as depicted by the red disks and the pending request stack. The two other channels are lightly loaded. Eventually, the requests are handled and the system returns to a more balanced load. In the meantime, the applications are “data starved” and the users or applications experience less than optimal performance.

To solve the problem, the System Administrator has to once again manually configure the system to better balance the load, requiring application downtime. In any system, there are times when the load is unbalanced due to an application experiencing heavy I/O requirements.

Page 21: Powerpath Foundations - Srg

Copyright © 2009 EMC Corporation. Do not Copy - All Rights Reserved.

PowerPath Foundations - 21

© 2009 EMC Corporation. All rights reserved. PowerPath Foundations - 21

SAN

Host Application(s)

Host BusAdapter

HBADriverD DD D

HBA HBAHBA HBA

Request

Request

Request

Request Request

Request

Request

Request

PowerPath

SER

VER

STO

RA

GE

PowerPath Dynamic Load BalancingPowerPath dynamically balances workload across all available paths

PowerPath provides greatest performance improvement in environments where the workload is not balanced

– Workloads are seldom balanced

– Workloads dynamically change

Administrator can change load balancing polices on the fly

When PowerPath is installed, applications transparently access PowerPath instead of the HBA driver. PowerPath allocates the requests across all available channels, reducing bottlenecks and improving performance. This diagram shows a similar snapshot with PowerPath using multiple channels to minimize the queue depth on all channels.

Since the Channel Directors or Storage Processors are writing to cache and not to disks, any Channel Director/Storage Processor can handle any request. This allows PowerPath to constantly tune the server to adjust to changing loads from the applications running on the server. This, in turn, improves the performance of the server by enabling it to make better use of the storage. For every I/O, the PowerPath filter driver looks at the volume path set and selects the path based on the load balancing policy and failover setting for the device. The result is better application performance with fewer operational resources spent on the care and feeding of the system, which results in more financial value from your server investment.

PowerPath does not manage the I/O queues; it manages the placement of I/O requests in the queue.

Page 22: Powerpath Foundations - Srg

Copyright © 2009 EMC Corporation. Do not Copy - All Rights Reserved.

PowerPath Foundations - 22

© 2009 EMC Corporation. All rights reserved. PowerPath Foundations - 22

PowerPath Load Balancing PoliciesSymm_opt / CLAR_opt / Adaptive (default)

– I/O requests are balanced across multiple paths based on composition of reads, writes, and user-assigned device / application priorities

– Default policy on systems with a valid PowerPath license

Round_Robin– I/O requests are distributed to each

available path in turn

Least_I/Os– I/O requests are assigned to the path

with the fewest number of requests in the queue

Least_Blocks– I/O requests are assigned to the path

with the fewest total blocks in the queue

Request– Path failover only

No Redirect – Disable path failover and load

balancing – Default for Symmetrix when there is

no license key– Default on CLARiiON with base

license

Basic Failover – PowerPath fabric failover functionality – Default for CLARiiON and Symmetrix

when there is no license key

The slide lists current PowerPath load balancing policies. Please take a moment to review them.

Symm_opt, CLAR_opt, or Adaptive policies choose paths based on path load and logical device priority. Symm_opt (so) is the default on Symmetrix arrays and CLAR_opt (co) is the default for CLARiiON arrays. Adaptive (ad) is the default priority on Invista and non-EMC arrays and performs similarly to Symm_opt and CLAR_opt.

In Round Robin (rr) policy, I/O requests are assigned to each available path in rotation. It can be used for all arrays that PowerPath supports.

Least I/Os (lo) policy load balance is based on the number of pending I/Os. I/O requests are routed to the path with the fewest queued requests, regardless of total block volume. Least Blocks (lb) policy load balancing is based on the number of blocks in pending I/Os. I/O requests are routed to the path with the fewest queued blocks, regardless of the number of requests involved. Both policies can be set on all arrays that PowerPath supports.

Request (re) uses the path that would have been used if PowerPath had not been installed. For pseudo devices, it uses one arbitrary path for all I/O. For all devices, path failover is in effect, but load balancing is not. No Redirect (nr) policy has no load balancing nor is failover in effect. This policy is valid for all arrays that PowerPath supports. Both policies can be used on all arrays.

Basic failover policy sets load balancing not in effect. I/O routing on failure is limited to one host bus and one port on each Storage Processor. This policy is required for a non-disruptive upgrade. It protects against Storage Processor and back-end failures, but not against HBA or host loop failures. Basic failover is valid for Symmetrix, CLARiiON, and HP EVA arrays.

Page 23: Powerpath Foundations - Srg

Copyright © 2009 EMC Corporation. Do not Copy - All Rights Reserved.

PowerPath Foundations - 23

© 2009 EMC Corporation. All rights reserved. PowerPath Foundations - 23

PowerPath Automatic Path Testing and Restore

Auto-probe: tests for dead paths– Periodically probes inactive paths to identify failed paths before

sending user I/O to them

Autorestore: tests for restored paths– Periodically probes failed/closed paths to determine if they have

been repaired

The PowerPath autoprobe function periodically probes inactive paths to identify failed paths before sending user I/O. This process allows PowerPath to proactively close paths before an application experiences a time-out when sending I/O over failed paths.

The autoprobe function uses SCSI Inquiry commands for the probe, so that even a not-ready device returns successfully. It tests one LUN per HBA and port connection roughly every 30 seconds and skips LUNs that have received I/O in the last 30 seconds. If one probe fails down a path, it kicks off error checking routines. For failed paths, these routines check every LUN down that path to proactively close paths.

Autorestore runs every 5 minutes and probes every failed or closed path to determine if it has been repaired. Like autoprobe, it uses SCSI Inquiry commands.

Using a combination of autoprobe and autorestore features means that if a path fails, it is marked failed, but the application continues to run. Autorestore then tests the path’s viability and automatically restores the connection in approximately 5 minutes. Administrators can also restore paths manually using the PowerPath CLI or Administrator GUI.

Page 24: Powerpath Foundations - Srg

Copyright © 2009 EMC Corporation. Do not Copy - All Rights Reserved.

PowerPath Foundations - 24

© 2009 EMC Corporation. All rights reserved. PowerPath Foundations - 24

Online Management: Performance Tuning

Available bandwidth can be segmented per applicationSTO

RA

GE

HBA Driver

Host BusAdapter

ChannelGroup

ChannelGroup

Standby PathsApplication A

D DD D D D

HBA HBAHBA HBA HBA HBA

RequestRequest

Request

Request

Request

Request RequestRequest

Request

Standby PathsApplication B

HBA Driver

Host BusAdapter

PowerPath

SER

VER

Host Application A Host Application B

SAN

This example illustrates how PowerPath channel groups can be used to tune I/O performance.

PowerPath provides the flexibility to define paths to a device as “active” or “standby”. A standby path is only used if all the active paths to a device fail. PowerPath considers active paths as available for load balancing and failover. Paths can be dynamically added and removed by setting them to standby or active mode. For example, you have eight paths partitioned, four for one application and four for another. When one application needs more bandwidth (as during batch), you can add more paths through scripts during the batch run and take them away when you're done. PowerPath calls these dedicated paths to the application’s Channel Groups.

In this slide, there are two applications running on a server, Application A and Application B. The server is configured for six paths to the devices. Both applications are very busy. You decide that you don’t want the activity of one application to impact the performance of the other. You use PowerPath to set two paths to the devices used by Application A into standby mode. These paths are shown in yellow. Application A now has four active paths to the devices. To further isolate the I/O for both applications, you use PowerPath to set four of the paths to the devices used by Application B to standby mode. These paths are represented in green. Now Application A has four paths and Application B has two paths to the devices.

Page 25: Powerpath Foundations - Srg

Copyright © 2009 EMC Corporation. Do not Copy - All Rights Reserved.

PowerPath Foundations - 25

© 2009 EMC Corporation. All rights reserved. PowerPath Foundations - 25

PowerPath and HA ClustersPowerPath enhances High Availability cluster environments Increases application availability by eliminating the need to failover when a path failure occurs Tested with leading HA cluster products

– HACMP for AIX– HP MC/ServiceGuard– Veritas Cluster Server– Veritas Storage Foundation– Sun Cluster– Oracle RAC– Legato AAM– Microsoft MSCS– Red Hat Cluster Suite

STO

RA

GE

SER

VER

SAN

Application

SD

HBA

PowerPath

Application

SD

HBA

SD

HBA

PowerPath

HBA

SER

VER

SD

Open systems clustering technology manages application availability by detecting failures and restarting high availability applications on a surviving cluster node.

The deployment of PowerPath in the cluster eliminates the application downtime due to a channel failure. PowerPath detects the channel failure and uses alternate channels so the cluster software does not have to reconfigure the cluster to keep the applications running.

PowerPath improves the availability of the applications running in the cluster.

Many clusters are deployed to provide performance scalability. PowerPath’s load balancing can help the customer maximize performance and get the greatest value from their cluster investment.

PowerPath must work cooperatively with the cluster software. This does not require special code in the cluster, but does require PowerPath to work with the storage management component of the cluster software. Validating PowerPath with the cluster software systems requires extensive knowledge and testing of the cluster software with PowerPath.

Page 26: Powerpath Foundations - Srg

Copyright © 2009 EMC Corporation. Do not Copy - All Rights Reserved.

PowerPath Foundations - 26

© 2009 EMC Corporation. All rights reserved. PowerPath Foundations - 26

Management Interfaces – Command Line and GUIPowerPath Administrator GUI PowerPath Command Line

powermt check

powermt check_registration

powermt config

powermt display

powermt display options

powermt display paths

powermt load

powermt remove

powermt restore

powermt save

powermt set mode

powermt set periodic_autorestore

powermt set policy

powermt set priority]

powermt set write_throttle

powermt set write_throttle_queue

powermt version

The PowerPath Administrator GUI is used to configure, monitor, and manage PowerPath devices.

This slide shows a sample of the PowerPath Administrator Console. The PowerPath Administrator has two panes. On the left is the Scope pane, where PowerPath objects are displayed in a hierarchical list that can be expanded or collapsed. On the right is the Results pane that provides a view of configuration statistics for PowerPath objects selected in the Scope pane.

The CLI interface is available on all hosts. The commands are used to view or change the PowerPath options for the HBA port and path of the device.

This slide displays a help menu for the different options, which is invoked with the powermt helpcommand.

Page 27: Powerpath Foundations - Srg

Copyright © 2009 EMC Corporation. Do not Copy - All Rights Reserved.

PowerPath Foundations - 27

© 2009 EMC Corporation. All rights reserved. PowerPath Foundations - 27

Module Summary

Key points covered in this module:

Path failover

Fabric failover

Load balancing

Automatic path testing and restore

Performance tuning

PowerPath’s value in clustered environments

PowerPath’s management tools

These are the key points covered in this module. Please take a moment to review them.

Page 28: Powerpath Foundations - Srg

Copyright © 2009 EMC Corporation. Do not Copy - All Rights Reserved.

PowerPath Foundations - 28

© 2009 EMC Corporation. All rights reserved. PowerPath Foundations - 28

Other PowerPath Products

Upon completion of this module, you will be able to:

Identify and explain PowerPath Encryption with RSA

Identify and explain PowerPath Migration Enabler

Identify and explain PowerPath/VE

The objectives for this module are shown here. Please take a moment to read them.

Page 29: Powerpath Foundations - Srg

Copyright © 2009 EMC Corporation. Do not Copy - All Rights Reserved.

PowerPath Foundations - 29

© 2009 EMC Corporation. All rights reserved. PowerPath Foundations - 29

PowerPath Encryption with RSA – BenefitsSensitive data protection– Safeguard data in the event that a disk is removed from an array or

unauthorized data access

Centralized key management– Reduce complexity and cost in encryption deployments by centralizing the

management of encryption keys across the enterprise without managing multiple point solutions

Consistent encryption technology– Standardize across EMC and nonEMC storage platforms when using

PowerPath Encryption with RSA

Flexible encryption– Choose the logical unit numbers (LUNs) or volumes to encrypt

Replication support– Use seamlessly with array replication—both local and remote

PowerPath Encryption with RSA® is a host-based data-at-rest encryption solution utilizing EMC PowerPath and RSA Key Manager (RKM) for the Datacenter. It is flexible—choose the LUNs or volumes that need to be encrypted.

The benefit of PowerPath Encryption with RSA is that it protects information in the event it becomes compromised through unauthorized access or disk removal. Adding this level of protection enables compliance with internal, private, and government standards, including the Payment Card Industry Data Security Standard (PCI DSS) which is one of the most widely applicable compliance standards in the market today.

Page 30: Powerpath Foundations - Srg

Copyright © 2009 EMC Corporation. Do not Copy - All Rights Reserved.

PowerPath Foundations - 30

© 2009 EMC Corporation. All rights reserved. PowerPath Foundations - 30

Hosts with PowerPath Encryption

PowerPath Encryption with RSA

Host-based encryption for data-at-rest on disk

Primary Storage

SANSAN

RSA Key ManagerAppliance

Power-Path

IPNetwork

IPNetwork

PowerPath Encryption with RSA incorporates flexible, easy-to-use, centralized enterprise key management via the RSA Key Manager for the Datacenter and enables consistent encryption methodology in heterogeneous environments.

The RSA Key Manager server manages the cryptographic keys used by the PowerPath Encryption hosts. These keys are used to encrypt and decrypt the data. It is the responsibility of the RSA Key Manager server to ensure that hosts have access to only the intended keys and that these keys are stored for later retrieval.

Encryption occurs at the PowerPath level on the Host. It is encrypted using an RSA key from the RKM server. The key is used to encrypt the data before it is sent into the SAN. The data is then stored on the array encrypted. When the data is retrieved, PowerPath encryption decrypts the data the same way and returns the data to the application unencrypted.

Page 31: Powerpath Foundations - Srg

Copyright © 2009 EMC Corporation. Do not Copy - All Rights Reserved.

PowerPath Foundations - 31

© 2009 EMC Corporation. All rights reserved. PowerPath Foundations - 31

PowerPath Migration Enabler – BenefitsNondisruptive data migrations

– Migrate data or introduce virtualization into your environment without taking application downtime

Source and target synchronization– Keep the source and target arrays in sync until commit decision when you use PowerPath

Migration Enabler

“Tryout” of target– Allow rollback to the source array if the data move is not beneficial to the environment

Host and process crash recovery– Recover to the point the failure occurred without restarting the process

Adoption without copying– Perform EMC Invista encapsulation without copying data or introducing additional storage

into the environment

Common interface– Provide a single management interface for multiple migration methods

TimeFinder/Clone support– Allows nondisruptive migrations between RAID types within a Symmetrix array

EMC PowerPath Migration Enabler is a solution that leverages the same underlying technology as PowerPath, and enables other technologies, like array-based replication and virtualization, to eliminate application downtime during data migrations or virtualization implementations. PowerPath Migration Enabler insulates hosts from changes in the storage infrastructure and allows non-disruptive data migrations. PowerPath Migration Enabler enables solutions such as EMC Open Replicator, EMC TimeFinder®/Clone, and EMC Invista® to eliminate downtime during data migrations and virtualization deployments.

PowerPath Migration Enabler Host Copy eliminates downtime by enabling data migrations through the host in heterogeneous environments without the need for an underlying migration technology. PowerPath Migration Enabler keeps arrays in sync during Open Replicator for bulk data moves between Symmetrix arrays or TimeFinder/Clone for movement across data tiers within a Symmetrix system with minimal impact to host resources.

In addition, PowerPath Migration Enabler enables seamless deployment of Invista virtualized environments by encapsulating (or bringing under the control of) the volumes that will be virtualized. It supports data migrations to or from any network or supported array type by enabling data migrations through the host to support high availability and integration with existing business processes.

Page 32: Powerpath Foundations - Srg

Copyright © 2009 EMC Corporation. Do not Copy - All Rights Reserved.

PowerPath Foundations - 32

© 2009 EMC Corporation. All rights reserved. PowerPath Foundations - 32

PowerPath Migration Enabler

Nondisruptive data migration performance

SANSAN

PowerPathMigration Enabler

PowerPath Migration Enabler is an ideal solution for eliminating planned downtime, conducting high-performance migrations for large data sets, and for frequent upgrades to storage technology, information consolidation, or during upgrades to lease contracts with new equipment. Whether Open Replicator or TimeFinder/Clone for Symmetrix or Invista is deployed with PowerPath Migration Enabler, or if the host is used as the migration point, downtime is eliminated, keeping applications up and running.

Page 33: Powerpath Foundations - Srg

Copyright © 2009 EMC Corporation. Do not Copy - All Rights Reserved.

PowerPath Foundations - 33

© 2009 EMC Corporation. All rights reserved. PowerPath Foundations - 33

PowerPath/VE – BenefitsStandardized path management– Unify management across heterogeneous physical and virtual

environments

Optimized utilization– Leverage all channels to provide optimal, predictable, and consistent

information access

Dynamic load balancing– Constantly adjust I/O path usage and respond to changes in I/O loads from

Virtual Machines

Automatic I/O path failure detection– Keep virtual environment and applications running in the event of failure

Simplified management– Eliminate the need to monitor and rebalance the dynamic environment

PowerPath for Virtual Environments (PowerPath/VE) allows customers to standardize on a single multipathing solution across their entire environment. Customers who are standardizing on VMware®have access to flexible and automatic I/O load balancing to manage the complexity of virtual machines and I/O-intensive applications in hyper-consolidated environments.

Additionally, PowerPath support for VMware ESX Server and Microsoft Hyper-V virtual environments increases the number of always-active initiators, as opposed to idle failover-standby initiators for each path, to each LUN, resulting in newfound I/O “horsepower” from all virtual server hosts in the form of multiple data streams per host.

Page 34: Powerpath Foundations - Srg

Copyright © 2009 EMC Corporation. Do not Copy - All Rights Reserved.

PowerPath Foundations - 34

© 2009 EMC Corporation. All rights reserved. PowerPath Foundations - 34

PowerPath/VE on VMwareVMware NMP

Storage Ports

ESX Server ESX Server

Active ChannelsStand-by Channels

PowerPath/VE

Storage Ports

ESX Server ESX Server

All Channels are active and used to load balance

PowerPath PowerPath

VMware Native Multipathing (NMP) is a limited path management solution. The typical deployment provides for channel failover only, without load balancing across multiple paths. Stand-by paths are designated for failover which cannot maximize the potential performance. Configuration of NMP is on a LUN-by-LUN basis. The need to map the active and stand-by channels for each LUN quickly becomes cumbersome as the number of VMs and VMware datastores on LUNs increases.

With the addition of PowerPath/VE to the environment, we have more than just simple channel failover. In addition to automatic fail-back, PowerPath/VE brings PowerPath’s established load balancing policies to virtual environments. Rather than designate some channels as active and others as stand-by, PowerPath leverages all channels for I/O and can dynamically distribute traffic over them. This gives PowerPath/VE superior and predictable performance over NMP.

Page 35: Powerpath Foundations - Srg

Copyright © 2009 EMC Corporation. Do not Copy - All Rights Reserved.

PowerPath Foundations - 35

© 2009 EMC Corporation. All rights reserved. PowerPath Foundations - 35

PowerPath/VE – Architecture Overview

PowerPath Remote Management Server

ESX Hosts

PowerPath remote tools (rpowermt) installed

Used to remotely manage PowerPath/VE on any number of ESX hosts

PowerPath software is installed

The PowerPath/VE solution consists of two major components, the PowerPath Remote Management Server (RMS) and one or more ESX hosts. A functional TCP/IP connection is required between these components.

The PowerPath Remote Management Server is where the PowerPath CLI, rpowermt, is installed. From this server, you can send PowerPath commands to an ESX host via PowerPath remote tools (rpowermt).

This server is also used to install, upgrade, or uninstall PowerPath/VE code on an ESX server after VMware vSphere CLI (vCLI) is installed.

With this architectural change, no assumptions need be made about the ability to open a console and execute CLI commands directly on the server running PowerPath. This is particularly relevant when deploying embedded servers such as ESXi.

Page 36: Powerpath Foundations - Srg

Copyright © 2009 EMC Corporation. Do not Copy - All Rights Reserved.

PowerPath Foundations - 36

© 2009 EMC Corporation. All rights reserved. PowerPath Foundations - 36

Module Summary

Key points covered in this module:

PowerPath Encryption

PowerPath Migration Enabler

PowerPath/VE

These are the key points covered in this module. Please take a moment to review them.

Page 37: Powerpath Foundations - Srg

Copyright © 2009 EMC Corporation. Do not Copy - All Rights Reserved.

PowerPath Foundations - 37

© 2009 EMC Corporation. All rights reserved. PowerPath Foundations - 37

Course SummaryKey points covered in this course:

PowerPath configuration

PowerPath devices and concepts

Environments that PowerPath supports

PowerPath licensing

Features and benefits of using PowerPath in storage environments

New features included in the current release of PowerPath

User tools to manage PowerPath

These are the key points covered in this training. Please take a moment to review them.

This concludes the training. Please proceed to the Course Completion slide to take the assessment.