D73819GC10 Sg Solaris11 What's New

8/12/2019 D73819GC10 Sg Solaris11 What's New

1/178

What's New in Oracle Solaris

11

Student Guide

D73819GC10

Edition 1.0

October 2011

D74667


2/178

Copyright 2011, Oracle and/or it affiliates. All rights reserved.

Disclaimer

This document contains proprietary information and is protected by copyright and

other intellectual property laws. You may copy and print this document solely for your

own use in an Oracle training course. The document may not be modified or altered

in any way. Except where your use constitutes "fair use" under copyright law, you

may not use, share, download, upload, copy, print, display, perform, reproduce,

publish, license, post, transmit, or distribute this document in whole or in part without

the express authorization of Oracle.

The information contained in this document is subject to change without notice. If you

find any problems in the document, please report them in writing to: Oracle University,

500 Oracle Parkway, Redwood Shores, California 94065 USA. This document is not

warranted to be error-free.

Restricted Rights Notice

If this documentation is delivered to the United States Government or anyone using

the documentation on behalf of the United States Government, the following notice is

applicable:

U.S. GOVERNMENT RIGHTS

The U.S. Governments rights to use, modify, reproduce, release, perform, display, or

disclose these training materials are restricted by the terms of the applicable Oracle

license agreement and/or the applicable U.S. Government contract.

Trademark Notice

Oracle and Java are registered trademarks of Oracle and/or its affiliates. Other names

may be trademarks of their respective owners.

Authors

Michael Ernest

Gary Riseborough

Marcus Flieri

Bart Smaalders

Dave Miner

Nicolas DrouxDan Price

Cindy Swearingen

Glenn Fadden

Liane Praza

Technical Contributors

and Reviewers

Mike Tracey

Mike Carew

Editor

MalavikaJinka

Publishers

Nita Brozowski

Sumesh Koshy


3/178

iii

Contents

Preface

1 Introduction

Oracle Solaris: The Mission Critical OS 1-2

Raising the Bar Set by Solaris 10 1-3

SPARC Enterprise Servers 1-4

SPARC T3 Servers: Scaling to New Heights 1-5

Oracle Solaris: Platform Choice and Flexibility 1-6

Serious About Oracle Solaris 1-7

Oracle Addresses Range of Customer Needs 1-8

Topic Outline 1-10

Module Structure 1-11

2 Image Packaging System (IPS) and Automated Installer (AI)

IPS Design Goals 2-2

IPS Implementation 2-3

IPS Package 2-4

Package Naming 2-5

IPS Repository 2-6

Starting the packagemanager GUI 2-7

Starting the packagemanager GUI - 2 2-8

pkg Subcommands 2-9

pkg Subcommands 2 2-10

Example: Search, List, and Install 2-11

Installing a Package with Dependencies 2-12

Verifying a Package 2-13

Fixing a Package 2-14

Listing Package Contents 2-15

Removing a Package 2-16

Updating a Package 2-17

Creating a Package 2-18

Group Packages 2-19

Other Commands and Utilities 2-20

AI: Why Replace JumpStart? 2-21

Rosetta Stone for Solaris 10 Users 2-22

AI Components and Features 2-23

AI Terminology 2-24


4/178

iv

Flow of Automated Installation 2-25

Creating an AI Service 2-26

Creating an IPS Repository 2-28

Creating AI Clients 2-29

JumpStart to AI Mapping 2-30

IPS References 2-31

AI References 2-32

3 Network Virtualization 1

Feature: Overview 3-2

Virtual NICs (VNICs) 3-3

Virtual NICs (VNICs) 2 3-4

Virtual Switches 3-5

Physical Wire, Physical Machines 3-6

Virtual Network: Example 3-7Creating VNICs and Etherstubs 3-8

Unified Data Link Properties 3-9

Virtual Bridges 3-10

ipadm 3-11

Managing Interfaces and IP Addresses 3-12

Managing Interface Properties 3-13

Creating Flows 3-14

Data Link Vanity Naming 3-15

Resource Pools 3-16

dlstat(1M) 3-17Other Network Observability Enhancements 3-18

Rethinking Zones 3-19

Other Solaris 11 Enhancements 3-20

4 ZFS Features in Solaris 11

Enhancements 4-2

Boot Environments 4-3

Boot Environments (BE) 4-4

Creating a Boot Environment 4-5

Activating a Boot Environment 4-6

Destroying a Boot Environment 4-7

Mounting and Unmounting a Boot Environment 4-8

Creating New Boot Environments 4-9

Creating New Boot Environments - 2 4-10

BE Upgrade with pkg-update 4-11

Deduplication 4-12


5/178

v

Deduplication Example - 1 4-13

Deduplication Example - 2 4-14

Root Pool Mirroring 4-15

Snapshot Differences 4-16

zfs diff Output 4-17

Send Stream Enhancements 4-18

Send Stream: Override Example 4-19

Send Stream: Enforce Example 4-20

Send Stream: Ignore Example 4-21

Pool Import: Log Device Recovery 4-22

Pool Import Recovery: Example 4-23

Pool Import: Read-Only Mode 4-24

Synchronous Write Behavior Property 4-25

Values for sync Property 4-26

ZFS Synchronous Behavior: Tuning Caveats 4-27RAIDZ/Mirror Performance 4-28

Integrating ZFS into Deployment 4-29

Performance Notes 4-30

Other ZFS Features 4-31

ZFS References 4-32

5 Zones

Changes Since Solaris 10 FCS 5-2

Design and Features 5-7

Storage 5-8Networking: Exclusive IP Zones 5-9

Networking: Shared IP ZonesIPMP 5-11

Zones Observability 5-12

zonestat Command 5-13

zonestat Interval: Example 5-14

zonestat by Resource: Example 5-15

Resource Management 5-16

Zones Security 5-17

Solaris 10 Containers 5-18

Solaris 10 Container: Expected Migration Path 5-19

References 5-20

6 Network Virtualization 2

Advanced Network Features 6-2

ilbadm: L3/L4 Integrated Load Balancing 6-3

Load Balancing Components 6-4


6/178

vi

ilbadm: Example 6-5

IP Filter, Forwarding in a Zone 6-6

Hardware Lanes and Dynamic Polling 6-7

Hardware Lanes 6-8

ipmpstat: Observability for IPMP Groups 6-9

ipmpstat: Example 6-10

Fiber Channel over Ethernet (FCoE) 6-11

Virtual Router Redundancy Protocol (VRRP) 6-12

IP over Infiniband (IPoIB) 6-13

Non-Uniform Memory Architecture (NUMA) I/O 6-14

NUMA I/O Architecture: Overview 6-15

GLDv3 Public Driver APIs 6-16

Network Performance Highlights 6-17

7 Security

Features 7-2

Root Implemented as a Role 7-3

File system encryption: zfs(1M) 7-4

Configuring ZFS Encryption 7-5

File system encryption: lofiadm 7-6

Network Spoofing Protection 7-7

Zones: Delegated Administration 7-8

SMF: Delegated Administration 7-9

SMF: Method Context 7-10

SMF: Firewall Integration 7-11Least Privilege Changes 7-12

In-kernel pfexec 7-13

Basic Privileges: More is Less 7-14

Role-Based Access Control 7-15

Sandboxing Enhancements 7-16

Kerberos Improvements 7-17

Key Management: pkcs11_kms Provider 7-18

Other Enhancements 7-19

Oracle Solaris 11 Trusted Extensions 7-20

Trusted Extensions Changes 7-21

Trusted Platform Modules (TPM) 7-22

8 Services Management Facility (SMF)

SMF Design Goals 8-2

SMF Is the Glue in Solaris 11 8-3

Service Templates 8-4


7/178

vii

Early Manifest Imports 8-5

SMF Enhanced Profiles 8-6

Fault Notification 8-7

IPS Actuators 8-8

FMRI Stored in proc_t Structure 8-9


8/178


9/178

Preface


10/178


11/178

Profile

Before You Begin This Course

You should be able to configure and manage a system running the Oracle Solaris

Operating system.

How This Course Is Organized

An understanding of Oracle Solaris features and working knowledge of the OracleSolaris 10 Operating System is beneficial, but not required

How This Course Is Organized

S What's New in Oracle Solaris 11 is an instructor-led seminar featuring lecture and

demonstrations. Online demonstrations and written practice sessions reinforce the

concepts and skills introduced.


12/178

Related Publications

System release bulletins

Installation and users guides

read.me files

International Oracle Users Group (IOUG) articles

Oracle Magazine


13/178

Copyright 2011, Oracle and/or i ts affiliates. All rights reserved.

Introduction


14/178

What's New in Oracle Solaris 11 1 - 2


Oracle Solaris: The Mission Critical OSIf It Must Work, It Runs on Solaris

The #1 deployment platform for the

#1 mission critical Oracle Database Extreme data integrity: ZFS

Hardened security: Secure by Default, Cryptographic

Framework, Least Privilege model

Predictive Self HealingFMA, SMF

Complete Virtualization with application isolation and resource

management: Containers

Production Safe Observability: DTrace

Scalable to thousands of threads, terabytes of memory


15/178



Raising the Bar Set by Solaris 10

Oracle Solaris 11

The Only Completely Virtualized OS Availability: Greatly improved with new packaging tools, safe

online upgrades, faster reboots

Scalability and Performance: Thousands of threads, terabytes of

RAM, hundreds of Gbps network bandwidth

Efficiency: Virtualized network, storage and server resources;

binary compatibility; advanced power management

Security: On-disk data encryption, secure process execution, HWcertification of the OS at boot time


16/178



2010 2011 2012 2013 2014 2015

T-Series

1-4 Socket

+ 2x

Throughput

M-Series1-64 Socket

+ 20%

M-Series

8-64 Sockets

+6x Throughput

+1.5x Single

Strand

T-Series

1-4 Sockets

+3x Single Strand

M-Series

8-64 Sockets

+2x

ThroughputT-Series

1-8 Sockets

+3x

Throughput

SPARC

1-64 Sockets

+2x Throughput

+1.5x Single

Strand

Solaris 11

Express

Solaris 11

Update

Solaris 11 Solaris 11

Update

Solaris 11

Update

SPARC Enterprise ServersThe Leader in System Scalability

5 Year Trajectory

Cores 4x

Threads 32x

Memory Capacity 16x

Database TPM 40x

Java Ops Per Second 10x


17/178



SPARC T3 Servers: Scaling to New HeightsIntegrated, High Throughput SPARC Systems for Massive Scale

SPARC T3-4

SYS

TEMT

HROUGHPUT

CONSOLIDATION

SPARC T3-1

SPARC T3-1B Bladefor Blade 6000

SPARC T3-2

VIRTUALIZATION

64 cores

512 threads

Best scale

Most security

Enterprise-

ready

32 cores

256 threads

Medium scale

Middleware

consolidation

Enterprise-

ready

16 cores

128 threads

Entry-level

Price/performa

nce

Best RAS

16 cores 128 threads

Best density

HIGH

HIGH

Worlds First 16

Core Processor

HIGH


18/178



Oracle SPARC x86 Oracle x86

Solaris

Zone

Solaris 10

Zone*

Solaris

Zone

Solaris

8 or 9Zone*

Consolidation path for older Solaris

versions Leverages server virtualization

technology

Built-in scalable, platform-

independent virtualization Native, bare metal performance

Binary Compatibility Guaranteed

Oracle Solaris: Platform Choice and Flexibility


19/178



Compute, Storage, Network

Serious About Oracle Solaris

Investments in Oracle Solaris 11

SPARC, x86 support

Exadata and Exalogic

Over 2,700 projects, over 400 inventions

Over 20 million hours of development

Over 60 million hours of testing

Over 56 million tests

Over 11,000 applications

Solaris 11: Coming in 2011


20/178



Oracle Addresses Range of Customer NeedsHigh Performing Application-to-Disk Solutions from a Single Vendor

Server

Storage

VM Solaris/OEL

Database

Fusion Middleware

Applications

Engineered Systems

Efficiency

Manageability and Simplicity

Compute,Storage, Network,

Software

HIGH

HIGH

Oracles OptimizedSolutions


21/178



The preceding is intended to outline our general product

direction. It is intended for information purposes only, and maynot be incorporated into any contract. It is not a commitment to

deliver any material, code, or functionality, and should not be

relied upon in making purchasing decisions.

The development, release, and timing of any features or

functionality described for Oracles products remain at the sole

discretion of Oracle.


22/178



Topic Outline

Morning

Image Packaging System Automated Installer

Networking (Crossbow)

Afternoon

Solaris Containers

ZFS

Security

SMF (Application Deployment)


23/178



Module Structure

Focus on enhancements since Oracle Solaris 10 9/10 release

Command-line examples included with slidesFeature demonstrations at instructor's discretion

Use cases blogged daily

Demo environment is generic

VirtualBox instance

Unless special arrangements are made

Text install, slim_profile added

Demo scripts available to those interested


24/178


25/178

Copyright 2011, Oracle and/or its affiliates. All rights reserved.

Image Packaging System (IPS) and

Automated Installer (AI)


26/178



IPS Design Goals

Use one process for installing, patching, and upgrading

Minimize system downtime Reverse install operations easily


27/178



IPS Implementation

Relies on ZFS for safety

Makes fast, safe copies with snapshots and clones Can apply changes to cloned BEs when desired

Avoids conditions imposed by patches that overwrite files

Single-user mode to prevent untimely access

Deferred activation to prevent uncoordinated access

Problem: A file that has been patched is available immediatelyfor use. A program that depends on it, however, will not workuntil the system is rebooted.

http://blogs.oracle.com/patch/entry/deferred_activation_patching


28/178



IPS Package

New model incorporates all software change types

Includes dependencies automatically Installs only what is required to complete a package

Each package is associated with apublisher

Replaces metacluster model with profiles that can overlap

Supports signed packages

Uses a fatpackage model

All variations in one: SPARC/x86/debug/nondebug

Available from a repository


29/178



Package Naming

Packages use a Fault Management Resource Identifier

(FMRI) pkg://solaris/library/[email protected],5.11-

0.75:20071001T163427Z

Package categories establish a namespace

Similar to SMF service names

Each version has its own tuple [email protected],5.11-0.75:20071001T163427Z

,-:


30/178



IPS Repository

Networked software catalog service

Incremental or monolithic downloads Built-in software release versioning

Avoids media size as a delivery constraint

Publishes catalog of available software

Automates retrieval of new dependencies, updates

Download/unzip/install steps unnecessary

Default publisher

http://pkg.oracle.com/solaris/release/


31/178



or

Starting the packagemanagerGUI


32/178



Starting the packagemanagerGUI - 2


33/178



pkg Subcommands

/usr/bin/pkg

pkg list List packages installed on the system

pkg search

Identify the package that a file (or pattern) belongs to

Install packages and configure repositories

Limit search to local packages with -l option

pkg info

Lists package details


34/178



pkg Subcommands 2

pkg install

pkg uninstall pkg verify

Validate a packages installation

pkg fix

Fix errors reported by pkg verify

pkg contents

Display the objects making up a package


35/178



Example: Search, List, and Install

# pkg search /usr/bin/ncftpINDEX ACTION VALUE PACKAGE

path file usr/bin/ncftp pkg:/network/ftp/[email protected]

# pkg list pkg:/network/ftp/ncftp

pkg list: no packages matching 'pkg:/network/ftp/ncftp' installed

# pkg install ncftp

Packages to install: 1

Create boot environment: No

DOWNLOAD PKGS FILES XFER (MB)Completed 1/1 13/13 0.5/0.5

PHASE ACTIONS

Install Phase 39/39

PHASE ITEMS

Package State Update Phase 1/1

Image State Update Phase 2/2


36/178



Installing a Package with Dependencies

# pkg install gimp

Refreshing catalog 1/1 solarisCaching catalogs ...

Creating PlanPackages to install: 24

Create boot environment: No

Services to restart: 6

DOWNLOAD PKGS FILES XFER (MB)

library/desktop/libgweather 0/24 0/8732 0.0/68.0

...

image/library/gegl 23/24 8714/8732 68.0/68.0

Completed 24/24 8732/8732 68.0/68.0

PHASE ACTIONS

Install Phase 1/10557

...Install Phase 10557/10557

PHASE ITEMS


...


37/178



Verifying a Package

# pkg verify ncftp# ls -l /usr/bin/ncftp

-r-xr-xr-x 1 root bin 276012 Dec 7 20:39 /usr/bin/ncftp

# chmod 775 /usr/bin/ncftp

# pkg verify ncftpVerifying: PACKAGE

STATUSpkg://solaris/network/ftp/ncftp ERROR

file: usr/bin/ncftpMode: 0775 should be 0555


38/178



Fixing a Package

# pkg fix ncftpVerifying: pkg://solaris/network/ftp/ncftp ERROR

file: usr/bin/ncftpMode: 0775 should be 0555

Created ZFS snapshot: 2010-12-07-23:29:09

Repairing: pkg://solaris/network/ftp/ncftp


Completed 1/1 2/2 0.1/0.1

PHASE ACTIONSUpdate Phase 2/2

PHASE ITEMS

Package State Update Phase 1/1Package Cache Update Phase 1/1Image State Update Phase 2/2

# pkg verify ncftp


39/178



Listing Package Contents

# pkg contents ncftpPATHusr

usr/binusr/bin/ncftpusr/bin/ncftpbatchusr/bin/ncftpbookmarksusr/bin/ncftpgetusr/bin/ncftplsusr/bin/ncftpputusr/bin/ncftpspoolerusr/sfwusr/sfw/bin

usr/sfw/bin/ncftp...


40/178



Removing a Package

# pkg uninstall ncftpCreating Plan

Packages to remove: 1

Create boot environment: NoPHASE ACTIONSRemoval Phase 1/33Removal Phase 33/33

PHASE ITEMSPackage State Update Phase 1/1


Package Cache Update Phase 1/1




PHASE ITEMSReading Existing Index 1/8

Reading Existing Index 5/8Reading Existing Index 8/8

Indexing Packages 1/1


41/178



Updating a Package

Updating all installed packages to the latest version

# pkg updatePackages to install: 1

Packages to update: 795

Create boot environment: Yes


Completed 796/796 4754/4754 205.2/205.2

PHASE ACTIONS

Removal Phase 2561/2561

Install Phase 3967/3967

Update Phase 6277/6277...

A clone of solaris-39 exists and has been updated and activated.

On the next boot the Boot Environment solaris-40 will be mounted on '/'.

Reboot when ready to switch to this updated BE.


42/178



$ pkgsend generate ~/fu

file gnome_terminal_fu group=bin mode=0644 owner=rootpath=gnome_terminal_fu pkg.size=326

file netbeans_fu group=bin mode=0644 owner=root path=netbeans_fupkg.size=283

file awk_fu group=bin mode=0644 owner=root path=awk_fu pkg.size=110

$ pkgrepo -s file:/tmp/test-repo create$ pkgrepo -s file:/tmp/test-repo set publisher/prefix=michael.oow.com

$ eval `pkgsend -s file:/tmp/test-repo open [email protected]`

$ pkgsend -s file:/tmp/test-repo import ~/ilb_demo

$ pkgsend -s file:/tmp/test-repo close

pkg://michael.oow.com/[email protected],5.11:20110912T012101Z

PUBLISHED

Creating a Package

Easy to package existing software

Or emit a manifest


43/178



Group Packages

Part of manual or automated install process

Controls other installed packages (or package groups) babel_install installs slim_install

slim_install is LiveCD content

Must uninstall group packages to customize what theycontrol

Remove babel_install to manage slim_install

Remove slim_install to manage individual packages

The automated installer will do this for you


44/178



Other Commands and Utilities

Other pkg(5) utilities

pkg publisher pkg set-publisher

pkgrepo(1)

pkgsend(1)

pkgrecv(1)

pkgdepend(1)

pkg.depotd(1M)

pkgmogrify(1M)


45/178



AI: Why Replace JumpStart?

To make updating/patching:

Faster More reliable

Easily reversible

To leverage current technology

Integrate with ZFS

Leverage the IPS repository

Apply SMF naming scheme

To separate client and server dependencies Make the installer platform-neutral

Let clients select their software repository


46/178



Solaris 10 Solaris 11

SVR4 Packages IPS (SVR4 still supported)

Install media Starter image + IPS repository

Live Upgrade beadm(1M)

Upgrade option pkg update, Update Manager

JumpStart Automated Installer(AI)

JumpStart Profiles AI ManifestsFlash Install replication No equivalent yet

Blueprints for custom DVDs Distribution Constructor

Rosetta Stone for Solaris 10 Users


47/178



AI Components and Features

Three service components

DHCP server (requires mDNS) SMF-based installer

IPS repository

Tools for managing andobserving process

Configure with installadm(1M)

Observe clients using livessh install parameter

Manage image with beadm(1M)

AI is WAN Boot-ready


48/178



AI Terminology

Client (installation target)

Can be physical or virtual (not zones, yet) SMF Services

svc:/network/dhcp-server:default

svc:/system/install/server:default

svc:/application/pkg/server

Manifest SMF-named install configuration

Criteria Properties that match client details to an

appropriate manifest


49/178



Flow of Automated Installation


50/178



Creating an AI Service

Use Oracle Solaris DHCP or ISC DHCP

installadm(1M) will manage DHCP if: svc:/network/physical:default (Not nwam)

svc:/network/dns/multicast:default

/etc/netmasks entry exists

Default route is set

Use AI-specific image

sol-11-exp-201011-ai-{x86|sparc}.iso

Server and client platforms do not have to match Cannot super-size the AI image from Text or LiveCD


51/178



# pkg verify installadm

# installadm create-service -a sparc -n solaris_11 \> -i 192.168.1.10 -c 3 -s ai_sparc_image.iso \> /export/ai/sparc/solaris_11

# installadm list

Creating an AI Service

-n Install service name

-i DHCP start address

-c DHCP range

-s AI source image


52/178



Creating an IPS Repository

Download Repository Image (two files)

http://www.oracle.com/technetwork/server-storage/solaris11/downloads/index.html

Combine the files and:

Burn it to media

Or, mount it by using lofiadm(1M)

Or, copy it to a ZFS file system with rsync(1)

Enable repository service

svc:/application/pkg/server:default

For more details, see How to Copy An Oracle Solaris 11Software Package Repository.


53/178



# installadm create-client -b "console=ttya,livessh=enable" \> -e 0:e0:81:5d:bf:e0 -n s11-x86

# installadm create-client -e 00:14:4f:a7:65:70 -n s11-sparc

Creating AI Clients

The client will get AI service location from DHCP.

The client will get boot image, configuration, and repositorylocation from AI service.

AI service identifies clients by MAC address.

x86 clients can add other boot parameters.

AI service binds clients to a named install service.


54/178



JumpStart AIsetup_install_server installadm create-service

add_install_client installadm create-client

begin script

Client profiles, rules Manifests with client criteria

finish scriptpkg actuators (before reboot)

First-boot SMF services

sysidcfg file SMF profile

Manifests, driver updates, custom image

from Distribution Constructor

JumpStart to AI Mapping


55/178



IPS References

Adding and Updating Oracle Solaris 11 Software Packages

http://www.oracle.com/pls/topic/lookup?ctx=E23824&id=AUOSS


56/178



AI References

Creating a Custom Oracle Solaris Installation Image

http://www.oracle.com/pls/topic/lookup?ctx=E23824&id=CCOSI Transitioning From Oracle Solaris 10 JumpStart to Oracle

Solaris 11 Automated Installerhttp://www.oracle.com/pls/topic/lookup?ctx=E23824&id=MFJAI

Creating and Administering Oracle Solaris 11 BootEnvironmentshttp://www.oracle.com/pls/topic/lookup?ctx=E23824&id=CMBEA

Installing Oracle Solaris 11 Systems

http://www.oracle.com/pls/topic/lookup?ctx=E23824&id=IOSUI


57/178


Network Virtualization 1


58/178



Feature: Overview

Virtualized NICs, switches, and bridges

Dynamic IP address management Quality of Service (QoS)

Control bandwidth by transport, service, protocol, or

connection

Vanity naming for devices

Fencing compute resources

Assign NICs/VNICs to processor sets or pools

Real time usage and history


59/178



Virtual NICs (VNICs)

Same control as a physical NIC

Private TCP/IP stack Managed with ifconfig, dladm, and so on

Dedicated MAC address

May be random, chosen, or device-assigned

Can be bound to hardware and kernel resources


60/178



Virtual NICs (VNICs) 2

Private TCP/IP stack

Data path is separate, does not rely on modules added to aglobal stack

A complete, standards-based virtualization solution

VLAN tags supported

Priority Flow Control (PFC)

With supporting hardware, can be fully encapsulated to the

switch


61/178



Virtual Switches

VNICs sharing a VLAN id on one data link need a switch

MAC layer provides built-in switching semantics Data path among VNICs sits on top of the data link

Connects VNIC to physical network

Isolates broadcast domains

Want an explicit virtual switch? Use an etherstub:

Makes any virtual network topology possible

Can reduce or eliminate trips to physical NIC

Can also manage resource controls


62/178



Client Router

Virtual Wire, Virtual Machines

Host 1 Host 2

Port 620.0.03

1 Gbps 1 Gbps 100 Mbps 1 Gbps

Port 920.0.01

Port 310.0.03

Port 110.0.01

Port 210.0.02

Switch 3 Switch 1

ClientVirtual

Router

VNIC620.0.03

1 Gbps 1 Gbps 1 Gbps 100 Mbps 1 Gbps

VNIC920.0.01

VNIC310.0.03

VNIC110.0.01

VNIC210.0.02

1 Gbps

Etherstub 3 Etherstub 1

Host 1 Host 2

Physical Wire, Physical Machines


63/178



Virtual Network: Example


64/178



# dladm create-vnic -l bge1 vnic1

# dladm create-vnic -l bge1 -m random p maxbw=100M -p cpus=4,5,6 vnic2

# dladm create-etherstub vswitch1

# dladm show-etherstub

LINK

vswitch1

# dladm create-vnic -l vswitch1 -p maxbw=1000M p cpus=4,5,6 vnic3

# dladm show-vnic

LINK OVER MACTYPE MACVALUE BANDWIDTH CPUS

vnic1 bge1 factory 0:1:2:3:4:5 - -

vnic2 bge1 random 2:5:6:7:8:9 max=100M 4,5,6

vnic3 vswitch1 random 4:3:4:7:0:1 max=1000M -# dladm create-vnic -l ixgbe0 -v 1055 -p maxbw=500M -p cpus=1,2 vnic9

Creating VNICs and Etherstubs


65/178



Unified Data Link Properties

dladm [set,reset,show]-linkprop

Alternative to ndd(1M) utility Single, stable interface for network property consumers

Changes can be made temporary or persistent

$ dladm show-linkprop e1000g0

LINK PROPERTY PERM VALUE DEFAULT POSSIBLE

e1000g0 speed r- 1000 1000 --

e1000g0 duplex r- full full half,full

e1000g0 state r- up up up,down

e1000g0 flowctrl rw no bi no,tx,rx,bi

e1000g0 maxbw rw -- -- --e1000g0 priority rw high high low,medium,high

e1000g0 protection rw -- -- mac-nospoof,

restricted,

ip-nospoof,

dhcp-nospoof

e1000g0 rxrings rw -- -- --


66/178



NIC NIC

Bridge

VNIC VNIC VNIC

etherstub

Virtual Bridges

Data Link (Layer 2), 802.1D

Detects MAC addresses Connects NICs, etherstubs,

link aggregations

Lets you move a VNIC

without changing IP address

Supports RBridges

(TRILL Transparent

Interconnect of Lots ofLinks)

Manages with dladm


67/178



ipadm

Consolidates management of

Network interface state IP address assignment

TCP/IP protocol properties

Uses action-object subcommands like dladm

create-if, show-if, disable-addr, and so on

Supercedes various commands and files

ifconfig

/etc/hostname. ndd


68/178



# dladm create-vnic l bge0 play1# ipadm create-addr T static d a 10.2.3.5/24 play1/v4static2

# ipadm show-if

IFNAME STATE CURRENT PERSISTENT

lo0 ok -m-v------46 ---

bge0 ok bm--------46 ---

play1 down bm--------46 -46

# ipadm show-addr

ADDROBJ TYPE STATE ADDR

play1/v4static2 static down 10.2.3.5/24

#

# ipadm up-addr play1/v4static2

# ipadm show-addr play1/v4static2

ADDROBJ TYPE STATE ADDRplay1/v4static2 static ok 10.2.3.5/24

Managing Interfaces and IP Addresses


69/178



# ipadm show-ifprop play1IFNAME PROPERTY PROTO PERM CURRENT PERSISTENT DEFAULT POSSIBLE

play1 arp ipv4 rw on -- on on,off

play1 forwarding ipv4 rw off -- off on,off

play1 metric ipv4 rw 0 -- 0 --

play1 mtu ipv4 rw 1500 -- 1500 68-1500

play1 exchange_routes ipv4 rw on -- on on,off

play1 usesrc ipv4 rw none -- none --

play1 forwarding ipv6 rw off -- off on,off

play1 metric ipv6 rw 0 -- 0 --

play1 mtu ipv6 rw 1500 -- 1500 1280-1500

play1 nud ipv6 rw on -- on on,off

play1 exchange_routes ipv6 rw on -- on on,off

play1 usesrc ipv6 rw none -- none --

Managing Interface Properties


70/178



# flowadm create-flow -l bge0 protocol=tcp,local_port=443 -p maxbw=50M http-1

# flowadm set-flowprop -l bge0 -p maxbw=100M http-1

Creating Flows

Define a flow by:

Service (protocol + port address) Transport type (TCP, UDP, SCTP, iSCSI, and so on)

IP address/subnet

Differentiated Service Code Point (DSCP) label

Flows can assign bandwidth caps (maxbw)

Flows maintain their own kstat counters

Use flowstat(1M)

Use extended accounting for historical reference


71/178



Data Link Vanity Naming

Vanity naming

Set desired name via dladm(1M) List device interfaces in /dev/net

Supports alternative to so-called PPA hack

PPA: Physical Point of Attachment

Name calculated with (VID*1000 + instance)

Example: bge + (487 * 1000 + 1) = bge487001

knickknack@os11e:/dev/net$ ls -l

total 0

crw-rw-rw- 1 root sys 58, 1001 2010-12-19 17:37 beatnic0

crw-rw-rw- 1 root sys 20, 1 2010-12-19 14:22 e1000g0


72/178



Resource Pools

Assigned CPUs process network traffic for a data link

Both kernel threads and network interrupts Configured through pools data link property

# dladm show-linkprop p pool

Alternative to manual setting (cpus property)

Pool configuration determines the CPUs selected

svc:/system/pools:default

Automatically updated if CPUs migrate to other pools

Some zones use dynamic pools svc:/system/pools/dynamic:default

Assigns CPUs on zone bootup, releases on shutdown


73/178



dlstat(1M)

Observability for data link and flow statistics

Measured per hardware/software ring For VirtualBox instance:

# kstat -n mac_rx_ring0

Includes network traffic spread to other CPUs (aka fanout)

Hardware lane counters (if NIC supports them)$ dlstat -i 30

LINK IPKTS RBYTES OPKTS OBYTES

bge0 25.89K 16.90M 18.23K 4.42M

play0 5.64K 1.51M 226 15.61K

play1 5.55K 1.49M 131 7.63K

bge0 81 13.29K 19 7.13K

play0 62 9.37K 0 0

play1 62 9.37K 0 0


74/178



Other Network Observability Enhancements

IP-layer observability

Snoop loopback traffic between zones using shared-IP # snoop -I lo0

Network DTrace providers

udp: send, receive probes

ip: send, receive, drop-in, drop-out probes

tcp: send, receive, state-change,connect-[request|refused|established|, accept-[refused|established]

tcpdump and wireshark are IPS packages

Observe flows with flowstat

Observe IPMP groups with ipmpstat


75/178



Rethinking Zones

Consider using the global zone (GZ) as a system service

processor NGZs isolate processes, software stacks

Resource controls cap NGZ consumption

CPU binding, psets, or pools

Virtual, resident set size (RSS), or paging memory

Shared memory, semaphores

An exclusive TCP/IP stack completes the picture.

L2/L3 boundary: Data links (exclusive-IP property)

Per-NIC in Solaris 10, per-VNIC in Solaris 11

One example: the Immutable Service Container

http://blogs.sun.com/video/entry/immutable_service_containers


76/178



Other Solaris 11 Enhancements

Still more stuff in dladm(1M)

VLAN, WiFi, IP tunnel management

Network Auto-Magic (NWAM) service

svc:/network/physical:nwam

Automagic setup

User can modify security, name services

Manual control (CLI or GUI)

Location-specific configurations


77/178


ZFS Features in Solaris 11


78/178



Enhancements

Key enhancements discussed in this module:

Root pool boot environments (BE) Deduplication

Root pool mirroring

Snapshot diff capability

Synchronous write behavior property

Send stream enhancements

Improved pool recovery


79/178



Boot Environments

Makes updates safe, reliable, and recoverable

Similar to Solaris 10 Live Upgrade ZFS only

Managed by beadm(1M)

Subcommands provide means to:

List

Activate

Create, Destroy, Rename

Mount, Unmount


80/178



Boot Environments (BE)

ZFS is required.

A BE is a special-purpose ZFS snapshot. beadm(1M) replaces lu* commands.

All BEs reside in the root pool.

No need to maintain partitions

Integrated with IPS

New BEs with package actuators

Make new BE with pkg image-update or pkg update


81/178



Creating a Boot Environment

Initial boot environment after installation

# beadm listBE Active Mountpoint Space Policy Created

-- ------ ---------- ----- ------ -------

solaris NR / 2.81G static 2010-12-06 03:48

Create a new boot environment by using beadm create# beadm create S11-BE-1 && beadm list

BE Active Mountpoint Space Policy Created

-- ------ ---------- ----- ------ -------

S11-BE-1 - - 110.0K static 2010-12-09 04:23

solaris NR / 2.81G static 2010-12-06 03:48

Active flags N = Active Now

R = Active next Reboot


82/178



Activating a Boot Environment

Activating a boot environment

# beadm activate S11-BE-1# beadm list


-- ------ ---------- ----- ------ -------

S11-BE-1 R - 2.81G static 2010-12-09 04:23

solaris N / 120.5K static 2010-12-06 03:48

After reboot# beadm list


-- ------ ---------- ----- ------ -------S11-BE-1 NR / 2.82G static 2010-12-09 04:23

solaris - - 7.37M static 2010-12-06 03:48


83/178



Destroying a Boot Environment

Destroying a boot environment

# beadm destroy solaris

Are you sure you want to destroy solaris? This action cannot beundone(y/[n]): y

# beadm list


-- ------ ---------- ----- ------ -------

S11-BE-1 NR / 2.83G static 2010-12-09 04:23


84/178



Mounting and Unmounting a Boot Environment

Mounting and unmounting a boot environment

# beadm create S11-BE-2 && beadm listBE Active Mountpoint Space Policy Created

-- ------ ---------- ----- ------ -------

S11-BE-1 NR / 2.83G static 2010-12-09 04:23

S11-BE-2 - - 45.0K static 2010-12-09 04:53

# beadm mount S11-BE-2 /mnt && beadm list


-- ------ ---------- ----- ------ -------

S11-BE-1 NR / 2.83G static 2010-12-09 04:23

S11-BE-2 - /mnt 11.67M static 2010-12-09 04:53

# beadm unmount S11-BE-2 && beadm listBE Active Mountpoint Space Policy Created

-- ------ ---------- ----- ------ -------

S11-BE-1 NR / 2.83G static 2010-12-09 04:23

S11-BE-2 - - 12.08M static 2010-12-09 04:53


85/178



Creating New Boot Environments

Create a new BE with an IPS package change

# beadm listBE Active Mountpoint Space Policy Created

-- ------ ---------- ----- ------ -------

S11-BE-1 NR / 2.84G static 2010-12-09 04:23

S11-BE-2 - - 12.08M static 2010-12-09 04:53

# pkg install --require-new-be --be-name=S11-BE-3 ncftpPackages to install: 1

Create boot environment: Yes


Completed 1/1 13/13 0.5/0.5

PHASE ACTIONSInstall Phase 39/39

PHASE ITEMS




86/178



Creating New Boot Environments - 2

PHASE ITEMS

Reading Existing Index 8/8

Indexing Packages 1/1

A clone of S11-BE-1 exists and has been updated and activated.

On the next boot the Boot Environment S11-BE-3 will be mountedon '/'.


# beadm list


-- ------ ---------- ----- ------ -------

S11-BE-1 N / 352.0K static 2010-12-09 04:23

S11-BE-2 - - 12.08M static 2010-12-09 04:53

S11-BE-3 R - 2.85G static 2010-12-09 05:19


87/178



BE Upgrade with pkg-update

New BE names are incremented by default

# pkg update A clone of zfsBE exists and has been updated andactivated.

On the next boot the Boot Environment zfsBE-1 will bemounted on '/'.


# init 6

# beadm list


-- ------ ---------- ----- ------ -------

zfsBE - - 9.38M static 2010-10-15 09:18

zfsBE-1 NR / 10.76G static 2010-11-05 09:57


88/178



Deduplication

Drops redundant data blocks

Enabled per-file system: dedup property To determine benefit on the existing ZFS storage:

# zdb -S

http://hub.opensolaris.org/bin/view/Community

+Group+zfs/dedup

Benefit is expressed similarly to compressratio

Observable via zpool status

Dedup operations have pool scope.


89/178



Deduplication Example - 1

bayle@os11e:~$ ls -l /usr/java/src.zip

-rw-r--r-- 1 root bin 19160179 2010-12-06 04:44

/usr/java/src.zip

bayle@os11e:~$ zfs set dedup=on rpool1/home/deirdre

bayle@os11e:~$ cp /usr/java/src.zip /home/deirdre/src1.zip

bayle@os11e:~$ zfs list rpool1/home/deirdre

NAME USED AVAIL REFER MOUNTPOINT

rpool1/home/deirdre 110M 8.10g 110M /home/deirdre


90/178



Deduplication Example - 2

bayle@os11e:~$ zpool list

NAME SIZE ALLOC FREE CAP DEDUP HEALTH ALTROOT

rpool1 15.9G 6.61G 9.27G 41% 6.00x ONLINE -

bayle@os11e:~$ rm /home/deirdre/*zip

bayle@os11e:~$ zpool list

NAME SIZE ALLOC FREE CAP DEDUP HEALTH ALTROOT

rpool1 15.9G 6.61G 9.27G 41% 1.00x ONLINE -

bayle@os11e:~$ zfs list rpool1/home/deirdre

NAME USED AVAIL REFER MOUNTPOINT

rpool1/home/deirdre 31K 8.12G 31K /home/deirdre


91/178



Root Pool Mirroring

Root pools can be mirrored after installation

# zpool attach rpool

Allow resilvering to complete

# zpool status rpool

Boot blocks are installed automatically

Verify bootability


92/178



Snapshot Differences

The zfs diff command lists differences between two

snapshots.$ ls /home/timh

fileA

$ zfs snapshot tank/home/timh@old

$ ls /home/timh

fileA fileB

$ zfs snapshot tank/home/timh@new

$ zfs diff tank/home/timh@old tank/home/timh@newM /tank/home/timh/

+ /tank/home/timh/fileB


93/178



zfs diff Output

Differences listed for files and directories:

M: Modification or link count change -: Object is present in the first snapshot only

+: Object is present in the second snapshot only

R: Object has been renamed


94/178



Send Stream Enhancements

Modify property values in a received dataset

Enforce property value(s) in a sent dataset Disable property settings in a received dataset


95/178



Send Stream: Override Example

File compression is off for the tank/data file system. You

want to enable compression for the bpool/data file system.# zfs get compression tank/data

NAME PROPERTY VALUE SOURCE

tank/data compression off default

# zfs send -p tank/data@snap1 | zfs recv -ocompression=on -d bpool

# zfs get -o all compression bpool/data

NAME PROPERTY VALUE RECEIVED SOURCE

bpool/data compression on off local


96/178



Send Stream: Enforce Example

The -b option declares the file system as a property source.

# zfs send -b bpool/data@snap1 | zfs recv -d restorepool# zfs get -o all compression restorepool/data

NAME PROPERTY VALUE RECEIVED SOURCE

restorepool/data compression off off received


97/178



Send Stream: Ignore Example

The receive -x option ignores property settings.

Applies recursively to contained file systems For example: Ignore quota property setting:

# zfs send -R tank/home@1020 | zfs recv -x quotabpool/home

# zfs get -r quota bpool/home

NAME PROPERTY VALUE SOURCE

bpool/home quota none default

bpool/home@1020 quota - -

bpool/home/cindys quota none localbpool/home/cindys@1020 quota - -

bpool/home/tom quota none local

bpool/home/tom@1020 quota - -


98/178



Pool Import: Log Device Recovery

Importing a pool with a missing log causes an error.

# zpool import dozerThe devices below are missing, use '-m' to import thepool anyway:

c3t3d0 [log]

cannot import 'dozer': one or more devices is currently

unavailable

Now, you can import the pool as-is (-m).

Attach the missing log device.

Use zpool clear to resolve errors. Works for mirrored log devices


99/178



Pool Import Recovery: Example

Example: Import Pool With Missing Log Device

# zpool import -m dozer

# zpool status dozer

pool: dozer

state: DEGRADED

status: One or more devices could not be opened. Sufficient replicasexist for the pool to continue functioning in a degraded state.

action: Attach the missing device and online it using 'zpool online'. see:http://www.sun.com/msg/ZFS-8000-2Q

config:

NAME STATE READ WRITE CKSUM

dozer DEGRADED 0 0 0

mirror-0 ONLINE 0 0 0

c3t1d0 ONLINE 0 0 0c3t2d0 ONLINE 0 0 0

logs

14685044587769991702 UNAVAIL 0 0 0 was c3t3d0


100/178



Pool Import: Read-Only Mode

May help in recovering a damaged pool

All datasets are mounted in the read-only mode. Disables pool transaction processing

No pending synchronous writes in the intent log are played.

Ignored attempts to set a pool property

# zpool import -o readonly=on tank

# zpool scrub tank

cannot scrub tank: pool is read-only

To revert to read-write, export, and import the pool


101/178



Synchronous Write Behavior Property

The sync property defines per-file system write behavior

Replaces the zil_disable tunable parameter The default setting is standard

Write synchronous transactions to the intent log, flush

devices

# zfs set sync=always tank/home/perrin


102/178



Values for sync Property

Possible sync property values include:

standard Synchronous-write transactions: allfsync(3C) calls, open(2) calls flagged with O_DSYNC,

O_SYNC.

always Write and flush all transactions to stable

storage. The system call returns upon completion.

disabled Commit transactions to stable storage with

the next flush, regardless of delay. Fast performance, no

risk of pool corruption. Data corruption is another matter.


103/178



ZFS Synchronous Behavior: Tuning Caveats

A sync property value of disabled on the active BE or

/var may produce undefined behavior. Increases vulnerability to replay attacks

Understand all the risks before using this value

Processes that rely on synchronous behavior can losedata with the disabled value.


104/178



RAIDZ/Mirror Performance

Latest-and-greatest RAIDZ pools automatically mirror

latency-sensitive metadata. Pools created with b148 or later

Pool version 29 or later

Boosts I/O throughput

Applies to all newly-written data

Trades off space for time

Does not improve resilience to failure


105/178



Integrating ZFS into Deployment

Consider a separate file system per significant application.

Monitor with fsstat(1M). Use snapshots for easy rollbacks.

Use zfs diff to monitor changes.

Apply encryption if appropriate.

Use zfs send/receive for replication or backup.


106/178



Performance Notes

On-disk encryption costs ~7% on random I/O and ~3% on

sequential I/O. RAID-Z mirror allocation Some workloads show 2-4x

speedup on directory searches.

Scrub/resilver ops now prefetch their metadata.

System duty cycle (SDC) scheduler balances thread

priorities for CPU time.

Slim ZIL reduces metadata I/O if data blocks are not full.

Explicit ZIL behavior is controlled via sync property.


107/178



Other ZFS Features

Dynamic LUN expansion

autoexpand property Splittable mirrored pools (zpool split)

Triple-parity RAID-Z (raidz3)

Improved ACL compatibility with CIFS

Automatic snapshots/Time Slider

SMF service auto-snapshot

User/group quotas

Via userspace and groupspace subcommands


108/178



ZFS References

Oracle Solaris Administration: ZFS File Systems

http://www.oracle.com/pls/topic/lookup?ctx=E23824&id=ZFSADMIN


109/178


Zones


110/178



Changes Since Solaris 10 FCS

Core

Configurable privileges (limitpriv) Supports DTrace inside a zone

Zone rename and move operations

Zone migration (attach, detach)

Software update on attach

Default update is conservative

Option -U will update all

Boot arguments (bootargs)

Packaging

Parallel patching, turbo SVR4 packaging

Live Upgrade support


111/178




Resource management

Overhauled and simplified (zone.*) CPU Caps added

zone.cpu-cap, zone.cpu-shares

See resource_controls(5)

Enhanced observability

Supported by getvmusage(2)

Integration with ZFS

Assign datasets to zones

Faster provisioning with clones and snapshots


112/178




Networking

ip-type defrouter

Brands

Oracle Solaris 8 Containers

Oracle Solaris 9 Containers

Trusted extensions

Sun Cluster integration

Oracle Enterprise Manager Ops Center 2.5 Integration


113/178




Physical to virtual (p2v) migration

Consolidate legacy instances as zones onto new hardware Available for Oracle Solaris 8, 9, and (other) 10 instances

Process

Create a system image

Transfer to zonepath location

Install the zone

Image automatically updated during installation

User-land/kernel need to be in sync Need to emulate Host ID


114/178



Changes in Oracle

Solaris 11


115/178



Design and Features

lofiadm support

v2v and p2v migration Branded Oracle Solaris 10 containers

Exclusive-IP network stack enhancements

zonestat

IPMP support for ip-type


116/178



Storage

lofiadm(1M), lofi(7D) supported

New resource control to limit lofi devices zone.max-lofi

zonecfg:zone1> add rctl

zonecfg:zone1:rctl> set name=zone.max-lofi

zonecfg:zone1:rctl> add value (priv=privileged, limit=10, action=none)

zonecfg:zone1:rctl> end

zonecfg:zone1>


117/178



Networking: Exclusive IP Zones

Exclusive-IP options

allowed-address property defines usableaddress/range.

defrouter property supports ip-type=exclusive.

# zonecfg -z zone1

zonecfg:zone1> set ip-type=exclusive

zonecfg:zone1> add net

zonecfg:zone1:net> set allowed-address=192.168.1.10/32

zonecfg:zone1:net> set physical=vnic1

zonecfg:zone1:net> set defrouter=192.168.1.1

zonecfg:zone1:net> end


118/178



Networking: Exclusive IP Zones

Administration/tools available inside a zone

dladm, flowadm, ipadm IP Tunnels

IPMP

Zones are ideal for virtual networking

Configurable with multiple vnics

Internal namespace for flows

Layers 2 and 3 network protection

Prohibit mischievous traffic from exclusive-IP zones (Try dladm show-linkprop protection)


119/178



Networking: Shared IP Zones IPMP

Solaris 10 IPMP, interface name changes on failover,

creating issues for some users For example: Using interface ce0:2 one moment, ce1:1

the next

Zone admin has no control

Solaris 11 IPMP

Zone retains same interface

ipmp0:2 remains ipmp0:2 for the zone session

Zone admin can test interface for IPMP flag

If set, the address is highly available.


120/178



Zones Observability

Improved utilization monitoring

CLI and Oracle Enterprise Manager integration Uses extended accounting (see acctadm)

Also svcs extended-accounting

Reports on both shared and dedicated resources

Measures utilization against configured limits

zonestat(1M)


121/178



zonestat Command

zonestatd daemon performs monitoring

Nonroot users and nonglobal zone users can see (some of)the information

zonestat can monitor:

Virtual, physical, and locked memory

Pools, psets, LWPs, and processes

Shared-memory, semaphore, and message resources

Can report specific zones, resource types

Supports sorting by column Machine-parseable output is also available


122/178



zonestat Interval: Example

End-of-run reporting for average, high, and total usage

$ zonestat 5

Collecting data for first interval...

Interval: 1, Duration: 0:00:05

SUMMARY Cpus/Online: 32/32 Physical: 32.0G Virtual: 47.9G

----------CPU---------- ----PHYSICAL----- -----VIRTUAL-----

ZONE USED %PART %CAP %SHRU USED PCT %CAP USED PCT %CAP

[total] 1.57 4.92% - - 5660M 17.2% - 9.9G 20.6% -

[system] 0.09 0.28% - - 5086M 15.5% - 9275M 18.8% -

kodiak-dp 1.00 100% - 100% 46.0M 0.14% 4.49% 36.2M 0.07% 1.17%

global 0.48 1.56% - 1.56% 419M 1.27% - 673M 1.37% -

kodiak-ab 0.00 0.00% - 0.01% 67.0M 0.20% - 115M 0.23% -

kodiak-rie 0.00 0.00% - 0.02% 41.6M 0.12% - 62.4M 0.12% -


123/178



zonestat by Resource: Example

Example: Monitor lwps and processes

$ zonestat -r processes,lwps 5PROCESSES SYSTEM LIMIT

system-limit 292K

ZONE USED PCT CAP %CAP

[total] 191 0.63% - -

[system] 0 0.00% - -

global 167 0.55% - -

foo 24 0.08% 300 8.00%

LWPS SYSTEM LIMIT

system-limit 2047M

ZONE USED PCT CAP %CAP

[total] 713 0.00% - -

[system] 0 0.00% - -

global 618 0.00% - -

foo 95 0.00% 1000 9.50%


124/178



Resource Management

New max-processes resource control

# zonecfg -z zone1zonecfg:zone1> set max-processes=300

prctl now reports resource utilization# prctl -i zone foo

zone: 4: foo

NAME PRIVILEGE VALUE FLAG ACTION

zone.max-lofi

usage 0

system 18.4E max deny

zone.max-swapusage 28.3MB

privileged 3.00GB - deny

system 16.0EB max deny


125/178



Zones Security

Delegated administration

Authorizations can be configured directly in zonecfg login, manage, clonefrom

# zonecfg -z zone1

zonecfg:zone1> add admin

zonecfg:zone1:admin> set user=jack

zonecfg:zone1:admin> set auths=login,manage

zonecfg:zone1:admin> end

zonecfg:zone1> commit

Authorizations are added to user/role entry in/etc/user_attr by zonecfg.


126/178



Solaris 10 Containers

Solaris 10 branded zone

Similar to the existing solaris8and solaris9 brand settings onSolaris 10

Promote adoption and compatibility of Oracle Solaris 11

Leverage existing investment in Solaris 10

Infrastructure, training, support

Allow new technology to support Oracle Solaris 10 context

Virtualized networking among Solaris 10 instances

Application recertification for Solaris 11 unnecessary

Use p2v installation process

Or v2v for moving the existing Solaris 10 zones

Support instances on Solaris 10 10/09 or later


127/178



Solaris 10 Container: Expected Migration Path

Solaris 10

Solaris 10

zone: db27-prod

Solaris 11

zone: db27-prod

Solaris 11

zone: db27-prod

db27-prod

p2v

Solaris10Brand

redeploy


128/178



References

Oracle Solaris Administration: Oracle Solaris Zones, Oracle

Solaris 10 Zones, and Resource Managementhttp://www.oracle.com/pls/topic/lookup?ctx=E23824&id=SYSADRM


129/178


Network Virtualization 2


130/178



Advanced Network Features

ilbadm

IP Filtering, forwarding in a zone Hardware Lanes and dynamic polling

ipmpstat

Fiber Channel over Ethernet (FCoE)

VRPP support

NUMA I/O

Public GLDv3 APIs


131/178



ilbadm: L3/L4 Integrated Load Balancing

Operational modes

Stateless Direct Server Return (DSR) Half or Full NAT

Algorithms supported

Round robin

IP hashing: Source address or source address + port

Health-checking built-ins

TCP, UDP, ICMP probes

Apply as parameters to user-scripted tests Performance comparable to IP forwarding


132/178



Load Balancing Components

pkg://solaris/service/network/load-

balancer/[email protected],5.11-0.148: To configure:

Server group: list of host+port addresses

Virtual IP (aka logical host)

Algorithm, operational type

Healthcheck program and parameters (optional)

The configured elements form a rule.

ilbadm subcommands follow dladm model.


133/178



# ilbadm create-servergroup \

> -s servers=apache-zone1:80,apache-zone2:80 \

> apache_group

#

# ilbadm create-rule \

> e p I vip=10.1.2.3,port=80 \

> -m lbalg=rr,type=HALF-NAT \

> -h hc-name=/var/hc/apache_check \

> -o servergroup=apache_group \

> apacheload_rrobin

ilbadm: Example


134/178



IP Filter, Forwarding in a Zone

Same operational semantics as the GZ

For IP Filter in a zone # pkg install ipfilter; pkg contents ipfilter

Filter/NAT configuration files in the /etc/ipf directory

See /usr/share/ipfilter/examples

# svcadm enable ipfilter

Or just forwarding # svcadm enable ipv4-forwarding


135/178



Hardware Lanes and Dynamic Polling

A Hardware Lane is defined by

NIC-supported partitions (Receive/Transmit Rings, DMA) Kernel queues/threads bound to CPU, pset, or pool

Same CPUs assigned to a VNIC or a flow

Dynamic polling Switches from interrupt handling to polling rate in low traffic

Reduces context switching and lock contention

mpstat output with NIC and legacy driver:intr ithr csw icsw migr smtx srw syscl usr sys wt idl

10818 8607 4558 1547 161 1797 289 19112 17 69 0 12

mpstat with NIC and GLDv3-based driver:

intr ithr csw icsw migr smtx srw syscl usr sys wt idl

2823 1489 875 151 93 261 1 19825 15 57 0 27


136/178



Physical MachinePhysical NIC

Hardware Lane

C

L

A

S

S

I

F

I

E

R

VNICHardware

Rings/DMA

Kernel Threads

and Queues

VNICKernel Threads

and Queues

FlowHardware

Rings/DMA

Kernel Threads

and Queues

Virtual

Machine/Zone

Virtual

Machine/Zone

Application

Switch

VLAN

Separated

Hardware

Rings/DMA

Hardware Lanes

Intended for multicore platforms with multi-10gigE NICs

Hardware Lanes + dedicated resources = linear scaling Integrated with virtualization and QoS controls

Dynamic polling, packet chaining boost efficiency


137/178



ipmpstat: Observability for IPMP Groups

Reads sockets opened by in.mpathd

Five output modes Address (-a)

Group (-g)

Interface (-i)

Probe (-p)

Target (-t)

VNICs are valid IPMP group members.

Useful for testing


138/178



# ifconfig blut0 ipmp

# ifconfig play0 group blut0

# ifconfig play1 group blut0

# ipmpstat -a

ADDRESS STATE GROUP INBOUND OUTBOUND

fe80::897f:b644:ae41:e0b up blut0 -- --

10.2.3.5 up blut0 play1 play1 play0

10.9.8.7 up blut0 play0 play1 play0

# ifconfig play0 group ""

# ipmpstat -a

ADDRESS STATE GROUP INBOUND OUTBOUND

fe80::897f:b644:ae41:e0b up blut0 -- --

10.2.3.5 up blut0 play1 play1

10.9.8.7 up blut0 play1 play1

#

ipmpstat: Example


139/178



MAC Layer APIs To Create VNICs,

Dedicate Resources, Bandwidth

for both Network Stack and FCoE

Virtualized Data Link Layer

10gB Port

VirtualNIC

Rx/Tx Ring

DMAChannel

H/W Flow Classifier

FCoE Port

Rx/Tx Ring

DMAChannel

FCoEGlue

MACClient

MACClient

MAC Layer

NetworkStack

App LeadvilleFiber

ChannelStack

10gEthernet PortPseudo FC instance presented to storage

Fiber Channel over Ethernet (FCoE)


140/178



Virtual Router Redundancy Protocol (VRRP)

HA support for routers and load balancers

Treats active server as a primary Other servers are passive

Solaris framework monitors control messages

Upon primary failure, framework elects a new primary

Moves the Virtual IP address (VIP)

Each VRRP router associates a VNIC with the VRRP id

VNIC attributes are set via dladm(1M).


141/178



IP over Infiniband (IPoIB)

Used in Exalogic systems (BOND0 interface)

Runs on top of IB's verb layer Control over IB partitions in dladm(1M)

*-part subcommands

IB data links show up as Host Channel Adapter (HCA) ports

Create partition data links over IB data links

Plumb them with IP addresses, assign them to zones

All dladm(1M) link properties apply


142/178



Non-Uniform Memory Architecture (NUMA) I/O

On NUMA platforms, I/O performance factors include:

Kernel resource location (memory placement) Hardware topology

Device location (backplane attachment)

NUMA I/O Framework

Defines affinity for all I/O subsystems

I/O subsystems register affinity to needed resources

Framework uses affinity to determine memory placement

Consumer-transparent process


143/178



I/O

Subsystem

DeviceDriver

Core NUMA I/O

Framework

I/O

SubsystemKernelAffinityAPIs

Admin

Interface

PCI/DDI

Framework

I/O topology

constructor

NUMA lgrp

sub-system

CPUS/poolconstraints

Interrupt

handles

Bind

interrupt

NUMAtopology

I/O

topology

NUMA I/O Architecture: Overview


144/178



GLDv3 Public Driver APIs

Dynamic polling

Packet chaining Hardware checksumming offload

Large Send Offload (LSO)

Revamped driver property interface

Simplify driver development

Extensibility for future releases

First supported in Solaris 10 U9 (09/10 release)

See Chapter 19, Document #816-4854


145/178



Network Performance Highlights

Dynamic polling on receive rings boosts efficiency

Aggregation, flow control on transmit rings Binding available to psets or pools

Supports Message Signaled Interrupts (MSI)

Used in PCI Express (PCIe) hardware

Alternative to traditional Pin-Based Interrupt

Hardware Lanes

Improve cache locality, isolates traffic


146/178


147/178


Security


148/178



Features

Root as a role

On-disk file encryption Network spoofing protection

Delegated administration

Zones, SMF services

In-kernel pfexec

Forced Privilege and Stop Profile


149/178



installer@os11e:~$ rolesroot

installer@os11e:~$ profilesConsole User

Suspend To RAM

Suspend To Disk

Brightness

CPU Power Management

Network Autoconf UserNetwork Wifi Info

Desktop Removable Media User

Basic Solaris User

All

Root Implemented as a Role

User defined during installation receives the root role

sudo is enabled with 5-minute grace


150/178



$ zfs create -o encryption=on rpool1/home/fngEnter passphrase for 'rpool1/home/fng':

Enter again:

$ zfs list rpool1/home/fngNAME USED AVAIL REFER MOUNTPOINT

rpool1/home/fng 31K 8.29G 31K /export/home/fngfir@os11e:/$ zfs get all rpool1/home/fng | grep keyrpool1/home/fng keysource passphrase,prompt local

rpool1/home/fng keystatus available -

rpool1/home/fng rekeydate Fri Dec 10 10:35 2010 local

File system encryption: zfs(1M)

Applicable to datasets or volumes

Need a wrapper key to mount file system Passphrase or file-based, delegatable key control

See man page examples 22-27 for zfs(1M)


151/178


Copyright 2011, Oracle and/or i ts affiliates. All rights

D73819GC10 Sg Solaris11 What's New

Documents

Transcript of D73819GC10 Sg Solaris11 What's New