1

© 2016, 2017 IBM Corporation

Cognitive Systems

Session ID:

Agenda Key:

33CO How to Get Started with VIO Server Planning and Best PracticesGary Konicek (konicek@us.ibm.com)IBM Lab Services

170294

33CO

© 2016, 2017 IBM Corporation

Cognitive Systems

2

Agenda

• VIOS General Planning References

• Installation Planning

• Virtual Storage / Tape Planning

• Virtual Network Planning

• Miscellaneous Tips

– Multipath Driver Considerations

– POWER8 Considerations

– HMC Considerations

2

© 2016, 2017 IBM Corporation

Cognitive Systems

3

Where Do I Start with Planning & Installing VIOS

• IBM PowerVM Getting Started Guide– http://www.redbooks.ibm.com/redpieces/abstracts/redp4815.html

• IBM PowerVM Virtualization Introduction and Configuration– http://www.redbooks.ibm.com/redpieces/abstracts/sg247940.html

• PowerVM Managing and Monitoring– http://www.redbooks.ibm.com/redpieces/abstracts/sg247590.html

• IBM PowerVM Best Practices– http://www.redbooks.ibm.com/abstracts/sg248062.html

• VIOS Manual (POWER8) – Knowledge Center– http://public.dhe.ibm.com/systems/power/docs/hw/p8/p8hb1.pdf

© 2016, 2017 IBM Corporation

Cognitive Systems

4

Where Do I Start with Planning & Installing VIOS

• IBM Power Systems HMC Implementation and Usage Guide– http://www.redbooks.ibm.com/abstracts/sg247491.html?Open

– Version 7 and Classic focus

• IBM Power Systems HMC Implementation and Usage Guide– http://www.redbooks.ibm.com/abstracts/sg248334.html?Open

– Version 8 and Classic and Enhanced+ interface

• IBM i Virtualization and Open Storage white paper (2014)– http://IBM i Virtualization

3

© 2016, 2017 IBM Corporation

Cognitive Systems

5

IBM i Virtualization Support Overview

• Note: Supported functions continue to evolve Review website

https://www.ibm.com/developerworks/community/wikis/home/wiki/IBM%20i%20Technology%20Updates/page/IBM%20i%20Virtualization%20Summary

© 2016, 2017 IBM Corporation

Cognitive Systems

VIOS Installation Planning

6

4

© 2016, 2017 IBM Corporation

Cognitive Systems

7

What are you going to virtualize & how

• Processors, storage, memory, network, all of these or subset

• Which VIO functions to use

– Active memory sharing, shared storage pools, etc

• Storage virtualization – Internal storage, external storage (SAN), tape storage, N_Port ID (NPIV), virtual scsi (VSCSI)

• Number of VIO servers – 1, 2, 3, 4 or more per system

– Pair for production, pair for dev/test

– Pair for Ethernet, pair for SAN storage

• Boot VIO from internal disk or external storage

© 2016, 2017 IBM Corporation

Cognitive Systems

8

VIOS Sizing

• Recommended Minimum– 0.5 to 1.0 cores and 6-8 GB of memory, but depends on several factors

Storage/ethernet throughput, number of virtual adapters, number of cores hosted, etc.

• VIOS Sizing References– PowerVM Best Practices – Chapter 1.3

• http://www.redbooks.ibm.com/abstracts/sg248062.html• Disregard the comment about starting with 1.5 cores for vSCSI with IBM i

– VIOS Manual on Knowledge Center• http://public.dhe.ibm.com/systems/power/docs/hw/p8/p8hb1.pdf

• IBM Workload Estimator can be used to size VIOS– Use Collection Services data to input I/Os and throughput for storage and networking– Allocates 0.1 cores for the redundant VIOS

• Use VIOS Advisor, topas, or nmon to track utilization and make adjustments as needed

5

© 2016, 2017 IBM Corporation

Cognitive Systems

9

VIOS Installation Planning – Install Disks

• When installing VIOS on internal disk– Order at least two disks per VIOS and use mirroring for protection– Must be seen as 512-byte sector disk. If install disk is not found, will need to reformat

using AIX Diagnostic CD• Download at

http://www14.software.ibm.com/webapp/set2/sas/f/diags/home.html• Load this CD and activate partition• Navigate menu options to reformat the disks or create RAID array• Shut down LPAR and then install with VIOS media

• When installing VIOS on external disk– Protect LUNs on the storage server– Use 512-byte sector LUNs for VIOS (AIX, pseries, etc.)– VIOS requires minimum of 30GB but use larger LUNs to store fixes, management

tools, optical media library, etc.• 70 – 100 GB LUNs are a common size for VIOS

© 2016, 2017 IBM Corporation

Cognitive Systems

10

Dual-VIOS Considerations on Internal Disk

• If dual VIOS will be implemented and you want to install both in a single system unit or CEC, order the Power Server with a split backplane

– Each VIOS must own a disk adapter for its OS

– Split backplane provides two disk adapters (one for each VIOS)

– BUT one adapter also owns the DVD

Disk DrivesAdapter Adapter

DVD

6

© 2016, 2017 IBM Corporation

Cognitive Systems

11

Installing Second VIOS

• So if VIOS1 owns the DVD, how do you install VIOS2?

• Several options

– Use the install option on the HMC to install from:

• HMC image repository

• HMC’s DVD

– Requires external IP address for the HMC and the VIOS LPAR

• Use a NIM Server if you have one

– Swap the disk drives after installing VIOS1. Then install VIOS2 using the DVD on the Power System.

© 2016, 2017 IBM Corporation

Cognitive Systems

12

Installing second VIOS - External Storage

• If installing VIOS on SAN Storage, one approach is to build the second VIOS by using storage copy functions to create a copy of the first VIOS

• In addition to changing the host name and IP address, each VIOS needs it’s own unique RMC ID to ensure the HMC can access the second VIOS using RMC

• Follow these steps to generate a new RMC node id and restart the RMC subsystem– $ oem_setup_env

– # /usr/bin/odmdelete -o CuAt -q 'attribute=node_uuid‘

– # /usr/sbin/rsct/bin/mknodeid –f

– # /usr/sbin/rsct/install/bin/recfgct

7

© 2016, 2017 IBM Corporation

Cognitive Systems

13

Network Requirements for VIOS

• Each VIOS needs an IP address on the external network

• The IP address needs to be in a subnet that allows access to the external IP address of the HMC

• Ensure your network team activates the IP addresses on the network switch ports

• Ensure one ethernet port for each VIOS is cabled to the ethernet switch(es)

© 2016, 2017 IBM Corporation

Cognitive Systems

Virtual Storage and Tape Planning for IBM i LPARs

14

8

© 2016, 2017 IBM Corporation

Cognitive Systems

15

Planning for Virtualization of Internal Disk to IBM i Client LPARs

• If you plan to protect the disks by building RAID-5 or RAID-6 arrays in VIOS– Order IBM i drives– Can reuse drives that were previously owned by an IBM i LPAR– If AIX drives were ordered, reformat to 528-byte sectors using SAS RAID Array Manager in AIX

• Reformat could take a long time so plan for it!

• If you plan to protect the disks by mirroring on IBM i– Order AIX drives– If IBM drives were ordered or reusing i drives, reformat to 512 byte sectors using SAS RAID

Array Manager in AIX– Use RAID-0 on VIOS

• In all cases, you will lose up to 1/9th of the storage PLUS protection for RAID or mirroring

© 2016, 2017 IBM Corporation

Cognitive Systems

16

Planning for Virtualization of External Storage to IBM i Client LPARs

• Storage Volumes do not have to be the size of a physical disk on the storage server– But the number of volumes is important for IBM i workloads

• More volumes typically result in better I/O performance

• All volumes in an IBM i LPAR should be the same size

• Recommended minimum number of LUNs per IBM i LPAR = 6

• Consider storage usage for 512 to 520-byte sector conversion – Results in 1/9th of storage used for IBM i tag bytes when virtualizing storage from V7000/V3700, XIV, etc.– Exception: Not an issue with DS8000 LUNs which are 520-byte sector for IBM i

• When using vSCSI, assign physical volumes to IBM i LPARs

• Use Disk Magic tool to size proper number of storage volumes, physical disks, storage controllers, etc.

9

© 2016, 2017 IBM Corporation

Cognitive Systems

17

EMC

IBM i

FCPVIOS

FC HBAs

VIOS

FC HBAs

DS8000

• All storage subsystems* and internal storage

supported

• Storage assigned to VIOS first, then virtualized

to IBM i

• Some storage subsystems and some FC tape libraries

supported

• Storage mapped directly to Virtual FC adapter in IBM

i, which uses N_Port on FC adapter in VIOS

NPIV

IBM i

VIOS

FC HBAs

XIV

generic scsi disk

generic scsi disk

DS3500

VIOS

FC HBAs

SAN

SCSI

VSCSI

* See following charts for list of IBM supported storage devices

V7000

DS8000

SAN

V7000

© 2016, 2017 IBM Corporation

Cognitive Systems

18

Requirements for NPIV with VIOS and IBM i Client

Partitions

• Must use 8 Gb or 16 Gb fiber channel adapters on the Power System and assign to VIOS partitions

• Must use a fiber channel switch to connect Power System and Storage Server

• Fiber Channel switch must be NPIV-capable

• Storage Server must support NPIV as an attachment between VIOS and IBM i

10

© 2016, 2017 IBM Corporation

Cognitive Systems

19

IBM Storage system support

• Type of support may vary (i.e.)– NPIV or VSCSI

– Direct or via SAN switch

• Storage supported varies – see IBM System Storage Interoperability Center http://www-03.ibm.com/systems/support/storage/ssic/interoperability.wss

• IBM i POWER External Storage Support Matrix – Quick reference document

– http://www.ibm.com/support/techdocs/atsmastr.nsf/WebIndex/PRS4563

© 2016, 2017 IBM Corporation

Cognitive Systems

20

IBM Storage System Interoperability Center• Example from website

11

© 2016, 2017 IBM Corporation

Cognitive Systems

21

IBM i POWER External Storage Support Matrix – POWER8 Example

© 2016, 2017 IBM Corporation

Cognitive Systems

22

Redundant VIOS Support–Enables utilization of redundant VIOS partitions with 1 set of storage LUNs

–Improves availability of virtualized environment

–Each VIOS needs their own unique storage adapters

–External SAN Storage required

• Internal disk is controlled by a single host or server

Power Hypervisor

VIOS VIOS

12

© 2016, 2017 IBM Corporation

Cognitive Systems

23

Protection Options

VIOS

VIO Client VIO Client

VIOS VIOS

M

MMirror

VIOS

Multi-path

VIOS

Multi-path

VIO Client

MPIO

Internal Storage Internal Storage SAN-Attached Storage

© 2016, 2017 IBM Corporation

Cognitive Systems

SAN Switch

NPIV Disk Configuration

SAN Switches

IBM i LPARs

fcs0 C2-T1

fcs1C2-T2

VIOS 1

VIOS 2

31

41

52

62

32

42

51

61

fcs2C5-T1

fcs3C5-T2

A

B

SAN Switch

LPAR462

61

SAN Switch

LPAR352

51

SAN Switch

LPAR242

41

SAN Switch

LPAR132

31

Virtual Physical

Virtual Physical

C0xx00

C0xx02

C0xx04

C0xx06

C0xx08

C0xx0a

C0xx0c

C0xx0e

LPAR2 Disk

LPAR1 Disk

LPAR3 Disk

LPAR4 Disk24

13

© 2016, 2017 IBM Corporation

Cognitive Systems

25

Planning for Tape Devices with VIOS and IBM i

https://www.ibm.com/developerworks/community/wikis/home/wiki/IBM Removable Media on IBM i

© 2016, 2017 IBM Corporation

Cognitive Systems

SAN Switch

NPIV Tape Configuration

Tape F

Tape E

Tape D

Tape C

Tape B

Tape A

SAN Switches

IBM i LPARs

fcs4C4-T1

fcs5C4-T2

VIOS 1

VIOS 2

81

83

86

88

82

84

85

87

fcs4C6-T1

fcs5C6-T2

A

B

SAN Switch

LPAR488

87

SAN Switch

LPAR386

85

SAN Switch

LPAR284

83

SAN Switch

LPAR182

81

Control Path 1

Control Path 2

Virtual Physical

Virtual Physical

C0xx20

C0xx22

C0xx24

C0xx26

C0xx28

C0xx2a

C0xx2c

C0xx2e

26

14

© 2016, 2017 IBM Corporation

Cognitive Systems

Virtual Network Planning

27

© 2016, 2017 IBM Corporation

Cognitive Systems

28

Network Virtualization Design Guidelines• Shared ethernet adapters can be built using one physical ethernet port or an

aggregated link comprised of multiple physical ports

VIOS IBM i

Client

Hyp

erv

isor

Switch

PV

ID =

15

PV

ID =

15

CMNx

Virt 1GbEnt 2

Virt

Ent 0

Phy

Ent 1

Phy

Ent3

SEA

VIOS IBM i

Client

Ent4

SEA

Hyp

erv

isor

Switch

PV

ID =

15

PV

ID =

15

Ent 3

Aggr

CMNx

Virt1GbEnt 2

Virt

Ent 0

Phy

Ent 1

Phy

15

© 2016, 2017 IBM Corporation

Cognitive Systems

29

Network Virtualization Design Guidelines (cont.)

• Simple Design for IBM i client LPARs:– One shared ethernet adapter per network subnet per VIOS– i.e., SEA #1 for 123.123.111.xxx and SEA #2 for 123.123.222.xxx

• More Complex Design for IBM i client LPARs– Split a single shared ethernet adapter over multiple network subnets using VLANs – Network switch must be aware of and configured for each VLAN– By default, IBM i doesn’t recognize VLAN tags so PVID in IBM i virtual adapter needs to match a secondary VLAN in

the VIOS virtual adapter

• Recommend using a separate port for management of VIOS itself and other ports for client LPARs

• Duplicate the above requirements for each VIOS

• Use Performance Tools TCP/IP report to identify bandwidth requirements

© 2016, 2017 IBM Corporation

Cognitive Systems

30

Example: Network Bandwidth Planning

VIOS1

ent0

C10-T1

ent1

C10-T2

ent2

C10-T3

ent3

C10-T4

VIOS2

ent0

C11-T1

ent1

C11-T2

ent2

C11-T3

ent3

C11-T4

1Gb Management Port

3Gb Link Virtualized

to IBM i LPARs

Primary

1Gb Management Port

3Gb Link Virtualized

to IBM i LPARs

Failover

VIOS1

ent0

C10-T1

ent1

C10-T2

ent2

C10-T3

ent3

C10-T4

VIOS2

ent0

C11-T1

ent1

C11-T2

ent2

C11-T3

ent3

C11-T4

1Gb Management Port

2Gb Link Virtualized

to IBM i LPARs

Primary

1Gb Management Port

2Gb Link Virtualized

to IBM i LPARs

Failover

Single Subnet

1Gb Link Virtualized

to IBM i LPARs

Failover

1Gb Link Virtualized

to IBM i LPARs

Primary

Two Subnets

16

© 2016, 2017 IBM Corporation

Cognitive Systems

31

Example: Network Bandwidth Planning - Continued

VIOS1

ent0

C10-T1

ent1

C10-T2

ent2

C10-T3

ent3

C10-T4

VIOS2

ent0

C11-T1

ent1

C11-T2

ent2

C11-T3

ent3

C11-T4

1Gb Management Port

6Gb Link Virtualized

to IBM i LPARs

Primary

1Gb Management Port

6Gb Link Virtualized

to IBM i LPARs

Failover

VIOS1

ent0

C10-T1

ent1

C10-T2

ent2

C10-T3

ent3

C10-T4

VIOS2

ent0

C11-T1

ent1

C11-T2

ent2

C11-T3

ent3

C11-T4

1Gb Management Port

4Gb Link Virtualized

to IBM i LPARs

Primary

1Gb Management Port

4Gb Link Virtualized

to IBM i LPARs

Failover

Single Subnet

2Gb Link Virtualized

to IBM i LPARs

Failover

2Gb Link Virtualized

to IBM i LPARs

Primary

Two Subnets

ent0

C6-T1

ent1

C6-T2

ent2

C6-T3

ent3

C6-T4

ent0

C6-T1

ent1

C6-T2

ent2

C6-T3

ent3

C6-T4

ent0

C12-T1

ent1

C12-T2

ent2

C12-T3

ent3

C12-T4

ent0

C12-T1

ent1

C12-T2

ent2

C12-T3

ent3

C12-T4

© 2016, 2017 IBM Corporation

Cognitive Systems

Miscellaneous Tips

32

17

© 2016, 2017 IBM Corporation

Cognitive Systems

33

Multipath Device Driver Considerations

• For IBM SAN storage, Subsystem Device Driver Path Control Module (SDDPCM) multi-pathing software can be installed on each VIOS

– Provides load balancing and redundancy support

– Replaces the built in multipath support

– SDDPCM Support Matrix

• http://www.ibm.com/support/docview.wss?rs=540&context=ST52G7&dc=DA400&uid=ssg1S7001350

– Not needed in VIOS for IBM i support

• When using non-IBM SAN storage, you may need to install the vendor’s multipath device drivers

– Contact vendor for requirements

© 2016, 2017 IBM Corporation

Cognitive Systems

34

SDDPCM Download Info• http://www-01.ibm.com/support/docview.wss?uid=ssg1S4000201• SDDPCM User’s Guide also located on this page• Google ‘SDDPCM Download’ for latest version info

• See Chapter 3 of SDDPCM User’s Guide• Might need to update SDDPCM host attachment code and fibre channel firmware

first• Install SDDPCM driver• If updating earlier versions, might need to delete earlier version first – read user’s

guide

18

© 2016, 2017 IBM Corporation

Cognitive Systems

35

POWER8 Considerations

• New E870C and E880C systems do not have disk in the system units so VIOS install disks need to be in EXP24S I/O drawer

• IBM recommendation for Dual-VIOS on E870C/E880C:

– http://www.redbooks.ibm.com/redpieces/abstracts/redp5137.html

– Section 2.10.2 recommends 4 PCIe SAS Adapters (FC 5901) – two for each VIOS

© 2016, 2017 IBM Corporation

Cognitive Systems

36

HMC Considerations

• HMC provides simple browser interfaces to configure VIOS, virtual storage and virtual networking

• Recommend using at least V7.7.5 to get most of the virtual management support

– Recommend at least V7.7.7 for remote install

– VIOS is shipped with V8 on new HMCs, but you can upload VIOS image to older HMCs at V7.7.7

• V8.8.2 or later provides two interfaces – Classic and Enhanced

– Can change interfaces at log in

– Enhanced version is a new template based interface

19

#COMMONS17www.common.org

Don’t Forget Your Session Surveys

• Sign in to the Online Session Guide (www.common.org/sessions)

• Go to your personal schedule

• Click on the session that you attended

• Click on the Feedback Survey button located above the abstract

Completing session surveys helps us plan future programming and provides feedback used in speaker awards. Thank you for your participation.

© 2016, 2017 IBM Corporation

Cognitive Systems

https://facebook.com/IBMPowerSystems

https://twitter.com/IBMPowerSystems

https://www.linkedin.com/company/ibm-power-systems

IBM Power Systems Official Channels:

http://www.youtube.com/c/ibmpowersystems

https://www.ibm.com/blogs/systems/topics/servers/power-

systems/

Power Systems Social Media

20

© 2016, 2017 IBM Corporation

Cognitive Systems

39

More to Follow:

Blogs to Follow More to Follow Hashtags To Use

• IBM Systems Magazine You and I (Steve Will)

• IBM Systems Magazine i-Can (Dawn May)

• IBM Systems Magazine: iDevelop (Jon Paris and

Susan Gantner)

• IBM Systems Magazine: iTalk with Tuohy

• Aaron Bartell Blog

• Steve Pitcher Blog

• Trevor Perry Blog

• IBM DB2 for i (Mike Cain)

• IBM DB2 Web Query for i (Doug Mack)

• Modern-i-zation (Tim Rowe)

@IBMSystems@COMMONug

@IBMChampions@IBMSystemsISVs

@LinuxIBMMag@OpenPOWERorg

@AIXMag@IBMiMag

@ITJungleNews@SAPonIBMi@SiDforIBMi

@IBMAIXeSupp@IBMAIXdoc

#PowerSystems

#IBMi

#IBMAIX

#POWER8

#LinuxonPower

#OpenPOWER

#HANAonPower

#ITinfrastructure

#OpenSource

#HybridCloud

#BigData