Power5 Vio Server

© 2004 IBM Corporation

Virtual IO Server

Americas Advanced Technical Support TeamWestlake - Texas

2

IBM Systems Group

POWER5 VIO Server © 2004 IBM Corporation

This document was developed for IBM offerings in the United States as of the date of publication. IBM may not make these offerings available in other countries, and the information is subject to change without notice. Consult your local IBM business contact for information on the IBM offerings available in your area.Information in this document concerning non-IBM products was obtained from the suppliers of these products or other public sources. Questions on the capabilities of non-IBM products should be addressed to the suppliers of those products.IBM may have patents or pending patent applications covering subject matter in this document. The furnishing of this document does not give you any license to these patents. Send license inquires, in writing, to IBM Director of Licensing, IBM Corporation, New Castle Drive, Armonk, NY 10504-1785 USA. All statements regarding IBM future direction and intent are subject to change or withdrawal without notice, and represent goals and objectives only. The information contained in this document has not been submitted to any formal IBM test and is provided "AS IS" with no warranties or guarantees either expressed or implied.All examples cited or described in this document are presented as illustrations of the manner in which some IBM products can be used and the results that may be achieved. Actual environmental costs and performance characteristics will vary depending on individual client configurations and conditions.IBM Global Financing offerings are provided through IBM Credit Corporation in the United States and other IBM subsidiaries and divisions worldwide to qualified commercial and government clients. Rates are based on a client's credit rating, financing terms, offering type, equipment type and options, and may vary by country. Other restrictions may apply. Rates and offerings are subject to change, extension or withdrawal without notice.IBM is not responsible for printing errors in this document that result in pricing or information inaccuracies.All prices shown are IBM's United States suggested list prices and are subject to change without notice; reseller prices may vary.IBM hardware products are manufactured from new parts, or new and serviceable used parts. Regardless, our warranty terms apply.Many of the pSeries features described in this document are operating system dependent and may not be available on Linux. For more information, please check: http://www.ibm.com/servers/eserver/pseries/linux/whitepapers/linux_pseries.htmlAny performance data contained in this document was determined in a controlled environment. Actual results may vary significantly and are dependent on many factors including system hardware configuration and software design and configuration. Some measurements quoted in this document may have been made on development-level systems. There is no guarantee these measurements will be the same on generally-available systems. Some measurements quoted in this document may have been estimated through extrapolation. Users of this document should verify the applicable data for their specific environment.

Special Notices

Revised February 6, 2004

3

IBM Systems Group


Special Notices (Cont.)The following terms are registered trademarks of International Business Machines Corporation in the United States and/or other countries: AIX, AIX/L, AIX/L(logo), alphaWorks, AS/400, Blue Gene, Blue Lightning, C Set++, CICS, CICS/6000, CT/2, DataHub, DataJoiner, DB2, DEEP BLUE, developerWorks, DFDSM, DirectTalk, DYNIX, DYNIX/ptx, e business(logo), e(logo)business, e(logo)server, Enterprise Storage Server, ESCON, FlashCopy, GDDM, IBM, IBM(logo), ibm.com, IBM TotalStorage Proven, IntelliStation, IQ-Link, LANStreamer, LoadLeveler, Lotus, Lotus Notes, Lotusphere, Magstar, MediaStreamer, Micro Channel, MQSeries, Net.Data, Netfinity, NetView, Network Station, Notes, NUMA-Q, Operating System/2, Operating System/400, OS/2, OS/390, OS/400, Parallel Sysplex, PartnerLink, PartnerWorld, POWERparallel, PowerPC, PowerPC(logo), Predictive Failure Analysis, pSeries, PTX, ptx/ADMIN, RISC System/6000, RS/6000, S/390, Scalable POWERparallel Systems, SecureWay, Sequent, ServerProven, SP1, SP2, SpaceBall, System/390, The Engines of e-business, THINK, ThinkPad, Tivoli, Tivoli(logo), Tivoli Management Environment, Tivoli Ready(logo), TME, TotalStorage, TURBOWAYS, VisualAge, WebSphere, xSeries, z/OS, zSeries.

The following terms are trademarks of International Business Machines Corporation in the United States and/or other countries: Advanced Micro-Partitioning, AIX/L(logo), AIX 5L, AIX PVMe, AS/400e, BladeCenter, Chipkill, Cloudscape, DB2 OLAP Server, DB2 Universal Database, DFDSM, DFSORT, Domino, e-business(logo), e-business on demand, eServer, GigaProcessor, HACMP, HACMP/6000, Hypervisor, i5/OS, IBMLink, IBM Virtualization Engine, IMS, Intelligent Miner, Micro-Partitioning, iSeries, NUMACenter, OpenPower, POWER, Power Architecture, Power Everywhere, PowerPC Architecture, PowerPC 603, PowerPC 603e, PowerPC 604, PowerPC 750, POWER2, POWER2 Architecture, POWER3, POWER4, POWER4+, POWER5, POWER5+, POWER6, Redbooks, Sequent (logo), SequentLINK, Server Advantage, ServeRAID, Service Director, SmoothStart, SP, S/390 Parallel Enterprise Server, ThinkVision, Tivoli Enterprise, TME 10, TotalStorage Proven, Ultramedia, VideoCharger, Visualization Data Explorer, X-Architecture, z/Architecture.

A full list of U.S. trademarks owned by IBM may be found at: http://www.ibm.com/legal/copytrade.shtml.

UNIX is a registered trademark in the United States, other countries or both.

Linux is a registered trademark of Linus Torvalds in the United States, other countries or both.

Microsoft, Windows, Windows NT and the Windows logo are registered trademarks of Microsoft Corporation in the United States and/or other countries.

Intel, Itanium and Pentium are registered trademarks and Intel Xeon and MMX are trademarks of Intel Corporation in the United States and/or other countries

AMD Opteron is a trademark of Advanced Micro Devices, Inc.

Java and all Java-based trademarks and logos are trademarks of Sun Microsystems, Inc. in the United States and/or other countries.

TPC-C and TPC-H are trademarks of the Transaction Performance Processing Council (TPPC).

SPECint, SPECfp, SPECjbb, SPECweb, SPECjAppServer, SPEC OMP, SPECviewperf, SPECapc, SPEChpc, SPECjvm, SPECmail, SPECimap and SPECsfs are trademarks of the Standard Performance Evaluation Corp (SPEC).

NetBench is a registered trademark of Ziff Davis Media in the United States, other countries or both.

Other company, product and service names may be trademarks or service marks of others. Revised August 23, 2004

4

IBM Systems Group


The IBM benchmarks results shown herein were derived using particular, well configured, development-level and generally-available computer systems. Buyers should consult other sources of information to evaluate the performance of systems they are considering buying and should consider conducting application oriented testing. For additional information about the benchmarks, values and systems tested, contact your local IBM office or IBM authorized reseller or access the website of the benchmark consortium or benchmark vendor.

IBM benchmark results can be found in the IBM eServer p5, pSeries and IBM RS/6000 Performance Report at http://www-1.ibm.com/servers/eserver/pseries/hardware/system_perf.html

Unless otherwise indicated for a system, the performance benchmarks were conducted using AIX V4.3 or AIX 5L. IBM C Set++ for AIX and IBM XL FORTRAN for AIX with optimization were the compilers used in the benchmark tests. The preprocessors used in some benchmark tests include KAP 3.2 for FORTRAN and KAP/C 1.4.2 from Kuck & Associates and VAST-2 v4.01X8 from Pacific-Sierra Research. The preprocessors were purchased separately from these vendors. Other software packages like IBM ESSL for AIX and MASS for AIX were also used in some benchmarks.

For a definition and explanation of each benchmark and the full list of detailed results, visit the web site of the benchmark consortium or benchmark vendor.

TPC http://www.tpc.orgSPEC http://www.spec.orgLinpack http://www.netlib.org/benchmark/performance.pdfPro/E http://www.proe.comGPC http://www.spec.org/gpcNotesBench http://www.notesbench.orgVolanoMark http://www.volano.comSTREAM http://www.cs.virginia.edu/stream/SAP http://www.sap.com/benchmark/Oracle Applications http://www.oracle.com/apps_benchmark/PeopleSoft - To get information on PeopleSoft benchmarks, contact PeopleSoft directly Siebel http://www.siebel.com/crm/performance_benchmark/index.shtmBaan http://www.ssaglobal.comMicrosoft Exchange http://www.microsoft.com/exchange/evaluation/performance/default.aspVeritest http://www.veritest.com/clients/reportsFluent http://www.fluent.com/software/fluent/fl5bench/fullres.htmTOP500 Supercomputers http://www.top500.org/Ideas International http://www.microsoft.com/exchange/evaluation/performance/default.aspStorage Performance Council http://www.storageperformance.org/results

Notes on Benchmarks and Values

Revised August 26, 2003

5

IBM Systems Group


rPerf–rPerf (Relative Performance) is an estimate of commercial processing performance relative to other pSeries systems. It is derived from an IBM analytical model which uses characteristics from IBM internal workloads, TPC and SPEC benchmarks. The rPerf model is not intended to represent any specific public benchmark results and should not be reasonably used in that way. The model simulates some of the system operations such as CPU, cache and memory. However, the model does not simulate disk or network I/O operations.

–rPerf estimates are calculated based on systems with the latest levels of AIX 5L and other pertinent software at the time of system announcement. Actual performance will vary based on application and configuration specifics. The IBM ~ pSeries 640 is the baseline reference system and has a value of 1.0. Although rPerf may be used to approximate relative IBM UNIX commercial processing performance, actual system performance may vary and is dependent upon many factors including system hardware configuration and software design and configuration.

–All performance estimates are provided "AS IS" and no warranties or guarantees are expressed or implied by IBM. Buyers should consult other sources of information, including system benchmarks, and application sizing guides to evaluate the performance of a system they are considering buying. For additional information about rPerf, contact your local IBM office or IBM authorized reseller.

Notes on Performance Estimates

Revised June 28, 2004

6

IBM Systems Group


What is Advanced POWER Virtualization - APV

Hardware Feature code for Power5 server, which enables:

Micro-partitioning – use of processor shared pool, or fractional cpu

Partition Load Manager (PLM)

VIO ServerPhysical disk can be shared as virtual disks to client partitionsShared Ethernet Adapter (SEA) – Physical adapter or EtherChannel in VIO server can be shared by client partitions. Clients have virtual Ethernet Adapters

Virtual Ethernet – LPAR to LPAR within a Power5 Server, does not require APV feature code

7

IBM Systems Group


Why Virtual I/O Server

Driven by a large number of partitions being configurable in a micr-partitioning environment

POWER5 systems will support more partitions than i/o slots available Allow partitions to be created without physical slot restrictionsNo more the need to have the typical slot requirement of 1 NIC and 1 storage adapter

There are exceptional servers – 90% active all the time. However,Broad statistical average utilization ? 10-15% in a large UNIX server farmCan some of this hardware resource be shared? Be virtualized?

Optimized utilization of resourcesPartitions can be created without adding any additional hardware resourcesEfficient utilization of physical resources through sharing on the serverFacilitates server consolidation

8

IBM Systems Group


Virtual I/O Server characteristicsVirtual I/O server configured as a VIO Server lpar

AIX 5.3 requiredSupported only on Power5 Server

VIO server lpar is a single function appliance lparHost based – LVMStorage based – ESS, EMC etc

Based on a client/server modelVIO Server LPAR has physical diskClient LPAR sees standard scsi disks, accesses lun’s via virtual scsi adapter

Inter-partition communication provided via PHYP

Virtual disk in client LPAR may be disk or logical volume in VIO server

A physical disk in VIO server can provide virtual disks to several client LPARs

9

IBM Systems Group


Creating Virtual IO Server lpar

10

IBM Systems Group


Virtual I/O Server installation

Packaged and shipped as AIX mksysb image – on VIO CD

VIO Server installation methodsCD installHMC install - Open rshterm and type “installios”, follow promptsNIM Install now supportedhttp://publib.boulder.ibm.com/infocenter/eserver/v1r2s/en_US/index.htm?info/iphb1/iphb1_vios_configuring_installnim.htm

VIO Server can support multiple clientsAIX 5.3SUSE Linux Enterprise Server 9 for POWERRed Hat Enterprise Linux AS for POWER Version 3

11

IBM Systems Group


Virtual I/O Server Administration

Closed box only supporting encapsulated commands – padmin userpadmin id runs a restricted shell onlypadmin initial login, prompted for password changepadmin user runs “license –accept” commandCommand Line Interface for administrative functions

no smitty with padmin userdevice management, lan, install, security, users, maintenance

padmin can obtain AIX root access by oem_setup_env command

12

IBM Systems Group


VIO Server/Client Overview

13

IBM Systems Group


VIO Server Configuration with MPIO

MPIO in client LPAR automatically configures

Client sees one hdisk –with two MPIO pathslspath –l hdisk0

Paths are fail_over only. Noload balancing in client MPIO

hdisk1 in each VIO serverattached to vscsi serveradapter. Did not bring hdisk1into volume group in VIO

Set reserve_policy attributeon hdisk1 to no_reserve ineach VIO server

LUN appears in each VIOserver as hdisk1

Single RAID5 LUN carvedin ESS, made visible to onefibre channel adapter inboth VIO servers

14

IBM Systems Group


VIO Server Configuration with LVM Mirroring

15

IBM Systems Group


Internals AIX

Virtual I/O Server AIX client

LVM

OEMmultipathing

disk DDHBA DD

VSCSI client

disk DD

LVM

MPIOPCM

MPIOPCM

VSCSIserverinternal or

external storage

logical volumes

OEM device paths

physical volumes

16

IBM Systems Group


Supported configurations today

PHYP

LVM

PHYSICALADAPTERDRIVER

AIXClient LPAR

AIX LVM

VSCSIclient

DISKDRIVER

Virtual I/O Server

ESS SAN STORAGE


SDD-PCM for ESS

VSCSIclient

MPIO

LVM


Virtual I/O Server


SDD-PCM for ESS

ESS SAN STORAGEPHYP

LVM


AIXClient LPAR

AIX LVM

VSCSIclient

DISKDRIVER

Virtual I/O Server

ESS SAN STORAGE


SDD or SDD-PCM for ESS

ESS SAN Storage ConfigurationAIX Client uses MPIO to protect against Virtual I/O Server failures Virtual I/O Server uses SDD-PCM to protect against adapter failures

ESS SAN Storage ConfigurationVirtual I/O Server uses SDD, or SDD-PCM to protect against adapter failures

PHYP

LVM


AIXClient LPAR

AIX LVM

VSCSIclient

DISKDRIVER

Virtual I/O Server

FASTT SAN STORAGE


FASTT RDAC Driver

ESS SAN Storage ConfigurationVirtual I/O Server uses FASTT RDAC Driver to protect against adapter failures

Other configurations with EMC and HDS are being tested currentlyOther configurations with EMC and HDS are being tested currently

17

IBM Systems Group


Virtual IO Server resource configuration

Add Physical devices to VIO Server

Create a volume group on one or more disks with mkvgmkvg [-f] [-vg VolumeGroup] PhysicalVolume

mkvg –f –vg rootvg_clients hdisk2rootvg_clients

Create logical volumes on the volume group

mklv [-mirror] [-lv NewLogicalVolume | -prefix Prefix ] VolumeGroup Size [PhysicalVolume … ]mklv –lv aix_sq07 rootvg_clients 7G hdisk2

18

IBM Systems Group


Create Virtual SCSI adapter - server

Same panel whether you are creating VIO server for first time, or DLPAR addingvirtual scsi server adapter to running VIO server later.

19

IBM Systems Group


VIO Server virtual adapters configuration

DLPAR added virtual adapter doesn’t show up in VIO server until cfgdev

lsdev –virtualname status description

ent2 Available Virtual I/O Ethernet Adapter (l-lan)vhost0 Available Virtual SCSI Server Adaptervhost1 Available Virtual SCSI Server Adaptervsa0 Available LPAR Virtual Serial Adapter

cfgdev –dev vio0

lsdev –virtualname status description

ent2 Available Virtual I/O Ethernet Adapter (l-lan)vhost0 Available Virtual SCSI Server Adaptervhost1 Available Virtual SCSI Server Adaptervhost2 Available Virtual SCSI Server Adaptervsa0 Available LPAR Virtual Serial Adapter

20

IBM Systems Group


Virtual IO server resource mapping

Configuring virtual target devicemkvdev –vdev aix_sq07 –vadapter vhost0 –dev vt_aix_sq07mkvdev –vdev hdisk7 –vadapter vhost1 –dev vt_hdisk7

$ lsdev -virtualname status description

ent2 Available Virtual I/O Ethernet Adapter (l-lan)vhost0 Available Virtual SCSI Server Adaptervhost1 Available Virtual SCSI Server Adaptervsa0 Available LPAR Virtual Serial Adaptervt_aix_sq07 Available Virtual Target Device - Logical Volumevt_hdisk7 Available Virtual Target Device - Disk

21

IBM Systems Group


VIO resources map$ lsmap -allSVSA Physloc Client Partition ID--------------- -------------------------------------------- ------------------vhost0 U9111.520.10C1C1C-V3-C2 0x00000001

VTD vtscsi0LUN 0x8100000000000000Backing device aix_sq07Physloc

SVSA Physloc Client Partition ID--------------- -------------------------------------------- ------------------vhost1 U9111.520.10C1C1C-V3-C4 0x00000001

VTD vtscsi1LUN 0x8100000000000000Backing device hdisk7Physloc U787A.001.DNZ00ZE-P1-C1-T1-L4-L0

$

22

IBM Systems Group


More use of lsmap$for v in `ioscli lsdev -virtual | grep vhost | awk '{print $1}'`doioscli lsmap -vadapter $v -fmt : | awk -F: '{ print $1" "$2" "$4"

"$6}'donevhost0 U9117.570.10C0EDC-V16-C7 virt_aix_sq07 aix_sq07vhost1 U9117.570.10C0EDC-V16-C8 virt_suse_sq07 suse_sq07vhost2 U9117.570.10C0EDC-V16-C9 virt_aix_sq08 aix_sq08vhost3 U9117.570.10C0EDC-V16-C10 virt_suse_sq08 suse_sq08vhost4 U9117.570.10C0EDC-V16-C11 virt_aix_sq09 aix_sq09vhost5 U9117.570.10C0EDC-V16-C12 virt_suse_sq09 suse_sq09vhost6 U9117.570.10C0EDC-V16-C13 virt_aix_sq10 aix_sq10vhost7 U9117.570.10C0EDC-V16-C14 virt_suse_sq10 suse_sq10vhost8 U9117.570.10C0EDC-V16-C15 virt_rhel_sq07 rhel_sq07vhost9 U9117.570.10C0EDC-V16-C16 virt_rhel_sq08 rhel_sq08vhost10 U9117.570.10C0EDC-V16-C17 virt_rhel_sq09 rhel_sq09vhost11 U9117.570.10C0EDC-V16-C18 virt_rhel_sq10 rhel_sq10

23

IBM Systems Group


Create Virtual SCSI adapter - client

Similar panel to create client virtual scsi adapter as the panel for server virtualscsi adapter. “This slot connects to which slot in which remote LPAR?” thinkplanning, think spreadsheet

24

IBM Systems Group


Client virtual disk attributes

[email protected] / # lsdev -Cc diskhdisk0 Available Virtual SCSI Disk Drive

[email protected] / # lscfg -vl hdisk0hdisk0 U9117.570.10C0EDC-V7-C5-T1-L810000000000 Virtual SCSI Disk Drive

[email protected] / # lsattr -El hdisk0PCM PCM/friend/vscsi Path Control Module Falsealgorithm fail_over Algorithm Falsemax_transfer 0x20000 Maximum TRANSFER Size Truepvid 00cc0edc916c5bd80000000000000000 Physical volume identifier Falsequeue_depth 3 Queue DEPTH Falsereserve_policy no_reserve Reserve Policy False

[email protected] / # lscfg -vl vscsi1vscsi1 U9117.570.10C0EDC-V7-C6-T1 Virtual SCSI Client Adapter

Device Specific.(YL)........U9117.570.10C0EDC-V7-C6-T1

25

IBM Systems Group


Virtual Ethernet

Virtual EthernetEnable inter-lpar communications without a physical adapterIEEE-compliant ethernet programming modelImplemented through inter-partition in-memory communication

VLAN splits up groups of network users on a physical network onto segments of logical networks

Virtual switch provides support for multiple (up to 4K) VLANsEach partition can connect to multiple networks, through one or more adaptersVIO server can add VLAN ID tag to the ethernet frame as appropriate. Ethernet switch restricts frames to ports that are authorized to receive frames with specific VLAN ID

Virtual network can connect to physical network through "routing" partitionsipforwarding, routing at IP layer – this is not Virtual Ethernet switch

26

IBM Systems Group


Create virtual ethernet adapter

27

IBM Systems Group


Configure shared ethernet adapter - SEA

28

IBM Systems Group


SEA configuration$ lsdev | grep ent[0-9]ent0 Available Virtual I/O Ethernet Adapter (l-lan)ent1 Available 10/100 Mbps Ethernet PCI Adapter II (1410ff01)

$ mkvdev –sea ent1 –vadapter ent0 –default ent0 –defaultid 1ent2 Available

$ lsdev | grep ent[0-9]ent0 Available Virtual I/O Ethernet Adapter (l-lan)ent1 Available 10/100 Mbps Ethernet PCI Adapter II (1410ff01)ent2 Available Shared Ethernet Adapter

$ lsattr -El ent2pvid 1 PVID to use for the SEA device Truepvid_adapter ent0 Default virtual adapter to use for non-VLAN-tagged packets Truereal_adapter ent1 Physical adapter associated with the SEA Truethread 0 Thread mode enabled (1) or disabled (0) Truevirt_adapters ent0 List of virtual adapters associated with the SEA (comma separated) True

29

IBM Systems Group


Shared Ethernet Adapter setup

VIO Server after mksysb InstallPhysicalEthernet

VirtualEthernet

VIO Server after sea config

mkvdev –sea ent1 –vadapter ent0 –default ent0 –defaultid 1ent2 Available

If VIO server requires a localIP address on this adapter con-figuration, the address is placedon shared adapter interfaceen2 (smitty chinet as root). It isnot configured on physical, noron virtual adapter.

ent1 ent0

9.19.126.98Shared Ethernet ent2

ent0ent1

30

IBM Systems Group


Supported configurations today

Virtual I/O Server

AIX Client LPAR

PHYP

Shared Ethernet Adapter

Virtual Ethernet

VirtualEthernet

Interface Failover

Virtual I/O Server

Shared Ethernet Adapter

Client LPAR: Interface Failover protects against Virtual I/O Server failures Restriction: VLAN Tags cannot be used

31

IBM Systems Group


Reference

InfoCenterhttp://publib.boulder.ibm.com/infocenter/eserver/v1r2s/en_US/index.htm

Virtualizing your compute environmenthttp://publib.boulder.ibm.com/infocenter/eserver/v1r2s/en_US/info/iphb1/iphb2.pdf

VIO Server and PLM command line referencehttp://publib.boulder.ibm.com/infocenter/eserver/v1r2s/en_US/info/iphb1/commands/commands.pdf

Redbook Introduction to Advanced POWER Virtualization on IBM p5 Servershttp://www.redbooks.ibm.com search on SG24-7940http://www.redbooks.ibm.com/redpieces/pdfs/sg247940.pdf

IBM eServer P5 Architecture and Performance Considerationshttp://publib-b.boulder.ibm.com/redpieces/abstracts/sg245768.html

Power5 Vio Server

Documents

Transcript of Power5 Vio Server