Unix Linux Administration III

Class 7: Solaris zones and containers. Perl and regular expressions.

Agenda Review last weeks lecture Review homework Lab 6c IPS and package mgmt. Solaris zones and containers Perl regular expressions Perl control structures.

Review: pluginsplugins are independent programs that run outside nagios.

They can be written in any language the host supports.

The plugins should be executable, provide a short text response, include a standaridzed return value (0-3).

0 = OK 1 = Warning 2 = Critical 3 = Unknown

plugins exist under <nagroot>/libexec and should be defined in commands.cfg

Review Perl hashHashes are similar to the Arrays except the index is based on scalars.A Hash index keys can be ANY scalar value.Hash is defined by %

HASH reference vs Array reference:$hash{key};

Array reference:$array[#];

Reviewkey/value pairs

$hash{key} = "pandora";or%hash = ('key', 'pandora');

Hash functionskeys -> array of all keys in the hashvalues -> array of all values in the hasheach -> combo of keys and values.delete -> remove key/value pairs.

Review Image packaging system (IPS) introduced in

Solaris 11. primary command line tool "pkg" pkg provides the ability to install, query, search,

among other functions as related to packages. The default repository= http://pkg.oracle.command using beadm and BE instances we can create

duplicate boot environments. Using BE we can safely upgrade Solaris and

provide a fallback path. BE uses zfs snapshots and datasets

Homework review Nagios monitors

smtpsshCustom dns monitor

Perl hash Solaris package mgmt

In class lab 6c

Lab notes for this session can be found here: http://www.ulcert.uw.edu -> Class Content -> InClass labs ->

Q3, Class 7, Unit 1

What we are going to cover: The Solaris Zones and Containers.What you should leave this session with: Knowledge of how zones and containers

work together. Administrative basics for zones and

containers.

Solaris Containers and Zones

Zones and Containers were introduced with Solaris 10. Zones provide a method for multiple virtual environments to run on the same disk like Virtual PC, VMware or XEN.

Containers are used to manage resources on a host. Zones are a subset of container functionality.

Zones vs ContainersContainers are Oracle’s Solaris based

operating system virtualization solution.

Containers allow user to manage resources between applications and services.

Zones are in fact one component of that strategy

Solaris Zones

Zones provide virtual environments within a single physical machine.

Zones are similar to VMware guests.

Zones can provide isolation between applications.

Zones can prevent one process or application from impacting another.

Zones provide:

Security – a user or process in one zone is not able to directly impact another zone*.

Isolation – multiple apps can exist on the same host yet be entirely separate of each other.

Zones provide:

Network Isolation – Normally two apps on the same server communicate at the socket level and as such never utilize the network stack. However, with zones applications in separate zones on the same host DO communicate over the network stack.

Zones provide:

Virtualization – each zone is managed separately

Granularity – resources can be shared between zones or allocated on a per zone basis.

Standard environments – provides the same standard Solaris interface and application environments that on a given physical host

Zones, BSD jails, VMware guests

Zones are often associated with BSD jails and there are some similarities.

Zones are similar also to VMware however with VMware guests we don’t see shared disk typically.

Zones typically use less than 1% of the available CPU.

Using immutable zones, requires as little as disk 150MB of disk space.

Resource ManagementResource management is one component of the

Solaris 11 container technology

Resource management provides the ability to: Allocate resources such as CPU and memory Monitor how resource allocations are being used. Generate detailed accounting information

Resource management provides a new daemon (rcapd) which controls physical memory

Consolidation and resource mgmt.

RSM (resource mgmt) is an Oracle technology provided with Solaris containers that allows administrators to:

Allocate specific resources; CPU time, RAM and

Dynamically manage resource allocations. Provide great accounting detail The ability to cap daemons for zones.

Consolidation

Consolidation has become a fundamental driver in the Corporate world over the last decade or so. Primarily driven as both more and better virtualization technologies entered the market.

Consolidation cont.

Solaris provides functionality (immutable Zones) to allow for shared files and configurations while limiting the resources available (CPU and Memory) to a given process, application or group of applications on the same host using Resource Pools.

Types of Zones: global or not?The Global zone is the default zone. This

zone always exists. The Global zone provides system wide configuration and control.

Non-Global or simply zones are created from the Global zone. There can be up to 8192 zones on a single host.

Applications running in a non-global zone are isolated from applications running in other zones.

The state of your Zone.Zone states relate to the non-global zones. The

global zone must always be available.Zone states:Configured- zone configuration and storage

allocated are complete but final post initial boot configuration is still pending.

Incomplete- typically during install or uninstall process.

Installed- initial configuration complete, zoneadm confirms configuration however, no virtual platform associated to the zone.

Zone state cont.Ready- virtual platform is defined, network

interfaces plumbed and file systems mounted. Zone ID defined; however, no processes associated with this zone.

Running- The zone enters this state when the first user process is created. This is the normal state for a zone.

Shutting down + down – This state should only be visible during shut down.

Global zone features Zone ID = 0 Provides the single bootable instance of Solaris Contains all system packages May contain additional software, packages, files

or data not installed using pkgadm Maintains global zone configuration information. Only zone aware of all zones (files, devices, etc). Only zone from which a non-global zone can be

configured, installed, managed or removed.

Non-global zone features. Assigned zone ID on boot (dynamic value). Shares the Solaris kernel that is booted

from the global zone Contains a subset of installed Solaris

system packages May contain additional software shared

from global zone and some code that is not shared.

Non-global zone features cont. May contain software, files or data not

installed using pkgadm or shared from the global zone

Includes a complete product database including software shared from the global zone and installed directly into the zone.

Unaware of other zones, cannot manage other zones.

Maintains all its own local information such as hostname, network settings, etc.

zone file system modelsA non-global zone contains its own root file system.

The size and configuration of the file system is based upon administration decisions.

No technical limit on how much disk can be consumed by a zone however, there are methods to manage or control this:Disk partitionsSoft partitionsLoopback file systems (managed with lofi and

lofiadm)

Zonecfg file-mac-profile

Solaris 10 had the concept of “sparse root zones”. This has been removed from Solaris 11. However, we can limit the ability for zones to write to the global zone using the “file-mac-profile. There are four preconfigured profiles. The default configuration does not enable this policy and therefore allows a writable root file system.

file-mac-profile options None – default config. Zone has read-write

access to the root file system. Strict – read-only file system. No IPS package

installs. Data can only be logged remotely. fixed-configuration – updates to /var/* allowed.

Data can be logged locally. Flexible-configuration – updates to /var/*

and /etc/* allowed, closest to the sparse root zones found in Solaris 10.

Networking your zone.Zones can communicate with other zones over

the network. Each zone has its own set of binding. Zones

run all of there own network services and daemons.

Meaning: an httpd daemon on one zone will not conflict with another zone also running an httpd daemon even on the same port.

Only the global zone can monitor traffic on all zones

Zone Daemons

The zone management service utilizes the SMF framework:svc:/system/zones:default

The two daemons that support zones are: zoneadmd: 1 per zone zsched: 1 per each Active zone (ready or

running) This is also known as the “zone schedule”. Keeps track of the kernel resources associated to the zone.

Zone daemons The zoneadmd daemon starts when a

zone needs to be managed. zoneadmd provides support for: zone ID allocation, system resource controls, network interface, loopback and conventional file systems.

zsched is started by zoneadmd and exists if the zone state is: running, ready, or shutting down. The zsched monitors the kernel threads within the zone.

Zone configurationZones are configured using the zonecfg utility.zonecfg is also used to verify the zone.zonecfg can be run interactively or it can read

in a configuration file.zonecfg can: create or delete a zone configuration add or remove resources in a configuration set the properties for a resource in a

configuration query and verify a configuration and save or

revert a configuration.

Zone configuration cont.Viewing a zone configuration file can done by

viewing the zone xml file directly: /etc/zones/<zonename>.xml

or exporting the zone configuration:zonecfg –z <zonename> export

The data is printed to stdout*.

Installing a Zone

After you have configured a zone the next step is to verify the install:zoneadm –z <zonename> verify

Once the zone is verified you can begin the installation: During this process the necessary files from the global zone will be copied to the local zone and product database will be populated.zoneadm –z <zonename> install

Boot the zoneBefore the zone can be booted it needs to transition to a ready

state*: zoneadm –z <zonename> ready

This step will plumb the interfaces and mount any required file systems. At this stage there are still no running processes in the zone.

Now you boot the zone zoneadm –z <zone_name> boot

Use list to confirm that the zone has started zoneadm –z <zone_name> list –v

The zone state should be “running” at this point.

*if you simply boot the zone Solaris it will pass the ready operation by default.

Halting and or Rebooting a zone

To shutdown or halt a zone usezoneadm –z <zone_name> halt

The zone state will change from running to halted.

You are able to reboot a zone without impacting any other zones.zoneadm –z <zone_name> reboot

Uninstall and deleting a zone

Zones are registered when installed. The should also be uninstalled before being deleted* to avoid issues.zoneadm –z <zone_name> uninstall

Once the zone is uninstalled it can be deleted zonecfg –z <zone_name> delete

*Using a –F will avoid the confirmation request on uninstall

zone login (zlogin)From the global zone you can use zlogin* to

access non-global zones. However, this can only be done as root or a user with the RBAC role “Zone Management”.

zlogin can be used in Interactive, non-interactive or console mode.

Basic syntax is: zlogin zonename utility [argument]

Your initial login will use –C for console zlogin –C <zonename>

*standard access methods still apply such as ssh.

zlogin options

-C = console connection to zone.

-E = disable the use of extended functions.

-l username = changes the login user to something other than root

-S = recovery mode login or safe mode login.

Final zone configuration

Once the zone is booted there are still some final configurations to complete. They include:

Language Terminal Type (try CDE terminal type) Host name Security Policy

Final zone configuration cont. Name Service Time Zone Root PasswordThese settings are configured interactively*

the first time you login which is similar to a standard install. Once completed the zone reboots and the it should be complete. The initial console login is required otherwise the zone will not be available.

*You can use a sysidcfg file to complete the install

Review zones allow for multiple virtual instances containers manage resources on a host. zones are a subset of container

functionality. zones can provide application isolation,

network isolation, security boundaries. Sparse root zones replaced with

immutable zones. shared files and configuration, reduced

disk foot print.

Review cont. Global zone == default zone (always exists). non-global zone == anything other than the

global zone. zone states: configured, incomplete, installed,

ready, running, shutting down. zone daemons: zoneadm is the primary zone

admin tool. zsched is started by zoneadm and monitors the zone.

zones are configured using zonecfg. zlogin is used to login from the global zone and

can provide console level access.

In class lab 7a

Lab notes for this session can be found here: http://www.ulcert.uw.edu -> Class Content -> InClass labs ->

Q3, Class 7, Unit 2

What we are going to cover: Perl Regular Expressions (regex), file input

and output.What you should leave this session with: The Perl implementation of regex. How to work with Perl file handles.

Regular Expression Concepts

A regex is a pattern to which a string is compared, this either succeeds or fails.

A regex can be used to extract data from a string as well as replacing pieces of the string.

PERL regexps are a superset of regexps found in grep, awk, sed, vi, emacs, etc…

Regular Expressions concepts

Regex are little programs built-in to Perl.

Regex have their own syntax and language within Perl to define functions.

Regex seek to match or find data.

Simple Uses: Literals

By default regexps match against $_

Looking for literal strings:

if (/abc/) { print;}will print out any line that contains the literal string ‘abc’

Simple Uses: Substitutions

s/regexp/string/;

Takes the entire matching string from the regexp portion and replaces it with string

$_ = ‘This is a long line of stuff’;s/This is a long line/short line/;$_ now equals ‘short line of stuff’;

Patterns: Single Characters

a-z, A-Z, 0-9 match themselves

‘.’ matches any character, except newline

Character Classes allow matching from a “class” of possibilities

Classes are enclosed in square brackets []

/[abcde]/ will match a single character of either a, or b, or c, or d, or e

Can use ‘-’ as a range operator [0-9], [a-zA-Z]

Patterns: Negation & Predefined ClassesUse the carat ‘^’ immediately after the left bracket ‘[‘ to signal the negative case

[^a-z] any character except lowercase alphas

\d for [0-9], \D for [^0-9]

\w for [a-zA-Z0-9_] i.e. “word characters”

\W for [^a-zA-Z0-9_] i.e.”non-word chars”

\s for [ \r\t\n\f] i.e. “white space characters”

\S for [^ \r\t\n\f] i.e. “non-white-space”

Grouping Patterns: Sequences

Literal sequences match themselvesabc matches an ‘a’ followed by a ‘b’ followed by a ‘c’

Simplest group

Grouping Patterns: Multipliers

Asterisk ‘*’: zero or more of the preceding character:

/a*/: matches any numbers of ‘a’

/a*/ will match ‘baaaaaaaaaaaaaaaaaaaa’

/ba*c/ will match ‘bc’

Grouping Patterns: Multipliers

Plus “+”: one or more matched of the preceding character

/a+/ requires at least one ‘a’ be present

/ba+c/ will NOT match ‘bc’

Question Mark “?”: zero or one matched/ba?c/ will only match for ‘bc’ or ‘bac’

Grouping Patterns: Using parenthesis to store matches. Adding a pair of () around some portion of the regexp allows later reference to the value in () or until the next successful pattern match.

Inside the regexp use \N where N is the N’th set of parenthesis*

N starts at 1 NOT zero

Outside the regexp the value can be referenced as $N

Grouping Patterns: Using parenthesis to store matches cont.

/(abc)(xyz)\2\1/ matches abcxyzxyzabc

A later reference to $1 will yield the string ‘abc’

Grouping Patterns: Alternatives

Can specify alternative matching using the pipe “|” operator

/bill|bob/ will match either ‘bill’ or ‘bob’

Similar to the single character matching using square brackets

/[abc]/ is the same a /a|b|c/

Anchoring Patterns

Anchors allow to force matches at particular locations in a string

\^ anchors the match to the start of a string/^A/ matches strings whose first character is an “A”

\$ anchors the match to the end of a string/Z$/ matches strings whose last character is a “Z”

Anchors do NOT take up space

Anchoring Patterns\b anchors a string at a word boundary

/fred\b/ matches fred, but not frederick as it wants a space AFTER fred

Word boundary occurs betweencharacters that match \w and \Wcharacters that match \w and the beginning or ending of a string (^ or $)

\B anchors a string at a non-word boundary/fred\B/ matched frederick but NOT fredDoes NOT want a space after fred

Precedence*Ordered from highest to lowest precedence. Parenthesis: ( …), (?:… ), (?<LABEL>…) Quantifiers: a* a+ a? a{n,m} Anchors and Sequences: abc ^a a$ Alternation: a|b|c Atoms: a [abc] \d \1

*Reference page 130 Table 8-1

Matching Operator Use the =~ option when comparing

something other than $_ if ($a =~ /yes|YES|y|Y/ ) {};

Useful for assigning default memory matches

if ($name =~ /(\w+)/) [

my $new_name = $1

}

Ignoring Case

To make matches case insensitive add an ‘i’ after the “/” which ends the regexp if ($a =~ /y|yes/i ) {} will match either ‘y’ or

‘yes’ in any capitalization

Alternative Delimiters It is permissible to use a different

character than the forward slash ‘/’ as the delimiterCommon when you are matching lots of ‘/’s

Proceed any non-alphanumeric or whitespace character by an ‘m’ if (m@/etc/shadow@) if (m#/users/rick/source#)

Using Variable Interpolation You can use variables in regular

expressionsif ($phrase =~ /$word/) {}

Variable is evaluated at the time regexp is evaluated and its value is placed in the regexp

Automatic match variables

Used for finding matches Before and After the value defined.

match space, word comma /\s(\w+),/

if (“Hello there, neighbor” =~ /\s(\w+),/) {

print found “first $`, then $& and finally $’\n”;

} $& matches the part matched $` matches the part BEFORE the match $’ matched the part AFTER the match

Substitutions

Use the ‘s’ operator to a pair of regexpss/foo/bar/; #replaces first ‘foo’ with ‘bar’s/foo/bar/g; # replaces all ‘foo’ with ‘bar’s/foo/bar/i; # ignore case when doing

substitutions Can use =~ to operate on variables other

than $_

Split Function Breaks a scalar up into pieces using

regular expressions@array = split (regexp, string)defaults: regexp: /\s+/, string: $_

Parts the DO NOT match the regexp are returned If using “:” as separators there will be no “:”

in the values of @array

Join Function

Opposite of the split function Takes an array of values, and a “glue”

value and puts them together $bigstring = join($glue, @array)

Slurping in a filemy $file = "data.txt";

open (IN, $file) || die "can't open file\n: $!";

while (my $line = <IN>) {

next if $line =~ /s*#/;

my ($name, $price, $date) = ( split/:/, $line);

print "I found name: $name, price: $price, Date: $date\n";

}

Input from STDIN

$input = <STDIN> One line of input

@input = <STDIN> All input until EOF

Common Usage while (<STDIN>) {

chomp; #Other operations}

Input from the Diamond Operator<> can be used like <STDIN> but it reads data, line by line, from any files specified on the command lineThis can be used to create a simple perl version of “cat”. Here we call it perl_cat.

while (<>) { print;}

Run: perl_cat file1 file2 file3

Input from the Diamond Operator

<> really goes through files in the @ARGV array

Set by default in PERL from any file names on the command line

Can be overwritten in the program@ARGV = (“file1”, “file2”, “file3”);

Using print for Normal Output via STDOUT

print takes a list of functions and sends them to the filehandle STDOUTInserts no spaces or formattingprint is a function which returns either true (1) for a successful completion of false (0) if an error occursParenthesis are optional unless the first item you are printing has them

print (3+2), “hello”; #only prints 5

Review: Regular Expressions (regex) are either successful or fail (0|1). we can use regex to extract or replace data. Perl regular expressions are a superset of those found in other

common UNIX utilities. By default perl regex match against $_ if (/foundit/) { print;} # This will print any line that contains

"foundit". Regular expression substitutions. s/old/new/; This can be a

simple string match or regex values such as: /abc/, /[0-4]/, etc. Regex grouping patterns:

* zero or more of the preceding character + match 1 or more of the preceding character ? zero or one of the preceding character

() stores the pattern match in memory for later use. (similar to shell).

Review: use | (logical or) provides alternative matches like: /yes|

YES|Y|y/ Perl anchor patterns: ^ start, $ end, \b word boundary, \B

=!\b Regex conforms to precedence rules Parenthesis,

Quantifiers, ... matching operator ~, if ($a =~ /yes|YES/y|Y/ {}; i = case

insensitive. matching before or after the match $&, $`, $' split function, breaks scalars into chunks using regex join function, just the opposite of the split function. slurp in a file my $file = "data.txt";

open ( IN, $file ) || die "can't open file\n: $!"; This can be updated to use STDIN also $input =

<STDIN> use <> like STDIN to read data line by line.

In class lab 7b

Lab notes for this session can be found here: http://www.ulcert.uw.edu -> Class Content -> InClass labs ->

Homework

Will be posted later this evening.