Smf deepdive-tran

Sun Proprietary/Use by Permission Only

SOLARIS 10 DEEP DIVE: SMF

SOLARIS 10DEEP DIVE: SMF

Christine Tran

US Solaris Adoption Practice

Sun Microsystems



Topics

• Part I: SMF Fundamentals> Motivation for SMF> Core concepts and terminology> Command overview> New boot process

• Part II: SMF in Action> Quick tour of service administration> Managing inetd services> Tips for troubleshooting> Recovering a repository



Topics

• Part III: Advanced SMF> Inside the manifest> A little about XML> Writing a manifest from scratch> Exploring the repository with svccfg and svcprop

• Part IV: More Advanced SMF> Security aspects> Snaphots> Handling signals> Portable services



Topics

• Part V: SMF Tools and SMF Community> Tools: Webmin> Tools: xmllint, jedit, Morphon> SMF Community on OpenSolaris> Resources> Review> Q & A> Closing



Part I:SMF Fundamentals



Motivation for SMF

• What is a service?• Is running a software program the same as delivering a

service? What exactly is a service?> Answer> Answer> Answer

• Does there currently exist an OS framework for service support and management?> Answer> Answer> Answer



How are services started today?

• init(1M) vi rc scripts in /etc/rc?.d/*> Long-time running or one-time initializations

• inetd(1M) as defined by inetd.conf> Short-lived to provide network functions

• init(1M) as defined by /etc/initab > Restartable or one-time functions

• How are these services grouped? Could they be related? How are the relationships described?



Diagnostic ability

• An application fails to start> Missing configuration file?> Failed to mount a filesystem?> Database is late in starting up?> Missing another component?

• Lack of knowledge of service boundary and service interdependencies limits error handling ability of system.• Lacks common framework of error-handling• Lacks restart capability



Service Management Today

• Thousands of different text files, arbitrarily grouped and managed with multiple administration techniques.• Undeclared, often unknown dependencies; linear startup

is a by-product.• Lacks common interface, each service is started a

different way.• Does not address multiple instances of a service, for

example, web servers.• Does not address services that may span multiple hosts,

for example, a grid.



Motivation for SMF

A Service Oriented Architecture requires a definition of service

and a more robust framework to deliver and manage it.



SMF gives you

• A consistent service model in a common framework: command interface, service description, property specification, status view, etc.• A meaningful system view.• Ability to state dependencies (by-product is parallel boot).• Restart capabilities • All data stored in persistent, transaction-based repository.• Snapshots allow undo and rollback to a working

configuration.



Core concepts and terminology

• A service is:> An abstract description of a long-lived software object.> May describe object on a collection of machines (grid).> May have multiple instances (httpd).> Each instance of a service has a well-defined state and a well-

defined error boundary [process contract].> Each service defines methods: start, stop, refresh, etc.> Each service defines dependencies: what it depends on, and

may define dependents: what depends on it.




• A service has a unique identifier called FMRI – Fault Management Resource ID

svc://localhost/network/login:rlogin

Schemesvc – SMF managed servicelrc – legacy RC script






Locationlocalhost – currenthostname – future release





svc://localhost/network/login:rloginFunctional category

applicationsystemdevicenetworkmilestoneplatformsite



Functional Categories

• Application – traditional daemons• Device – useful for dependencies• Milestone – similar to run levels• Network – converted services from inetd.conf• Platform – platform-specific services• System – platform-independent system services• Site – reserved for local site use






Service Description – related to method or RC script






Instance – “default” is the default instance




• FMRI can be addressed by the shortest unique match, for example:> svc://localhost/network/login:rlogin> svc:/network/login:rlogin> network/login:rlogin> rlogin

• Some common names have changed to a different FMRI, for example: syslog is now system-log



Service states

• online – the service instance is enabled and has successfully started.• offline – the service instance is enabled, but the service is

not yet running or available to run, usually due to a dependency that has not been satisfied, or an error in the start method.• disabled – the service instance is not enabled and is not

running.• maintenance – the service instance has encountered an

error that must be resolved before it could be started again. There are many reasons why a service could be in this state.



Service states

• legacy_run – the legacy service is not managed by SMF, but the service can be observed.> Faults not handled by SMF, no automated restart> Administrative error undetected> Software or hardware error results in process death

• degraded – the service instance is enabled, but is running at a limited capacity.• uninitialized – this state is the initial state for all services

before their configuration has been read.



Service dependency

• A dependency is a formal description of other services that are required to start a service.• A service can be dependent on another service or files.

When a dependency is not met, the service stays offline. Whether a dependency is satisfied depends on its type:> require_all – all services are running or all files are present.> require_any – at least one is running or at least one file is

present.> optional_all – all are running, disabled, in maintenance, or not

present. For files, this type is the same as require_all.> exclude_all – all are disabled, in maintenance, or when files

are not present.



Dependency actions

• Once running (online or degraded states are considered running) if a service dependency is stopped or refreshed, SMF determines whether to restart the service depending on the restart_on attribute specified.

restart_on attributeReason for dependency stop None Error Restart RefreshError no yes yes yesNon-error stop no no yes yesRefresh no no no yes



Service restarter

• /lib/svc/bin/svc.startd is the master process starter and restarter. It> reads the Service Configuration Repository and start services

on boot;> restart services that have failed;> shutdown services whose dependencies are no longer

satisfied;> run legacy RC scripts at run-level transitions;> provides system view of service status.

• There is a delegated restarter, inetd.



Delegated restarter

• Assumes responsibility for executing a set of services.• Manages service faults; may gracefully handle conditions

such as signals or core dumps.• May define/require additional service configuration.• Often provides functionality common to a specific set of

services, for example: inetd



SMF Manifest

• Description of the initial configuration of a service in XML.• Loaded into the SMF repository at boot time.• Sun-delivered services live in /var/svc/manifest. • For ISVs, manifests should be placed in the appropriate

subdirectory /var/svc/manifest.• Manifests for anything that is specific to the customer's

site only can go in /var/svc/manifest/site. This is reserved for local use.



SMF Manifest

• A manifest identifies, at the very least: > the service name> what the service is dependent on> ways to start and stop the service

• Can contain other attributes, such as the delegated restarter, config files, log files and others.



Service Configuration Repository

• Located in /etc/svc.• Distributed between local memory (volatile) and local file

(repository.db).• repository.db holds the persistent service description read

from the manifests.• volatile is a tmpfs allocated from swap containing

transient data (lock files, init state, log files).• The underlying DB engine is sqlite 2.8• Zones have their own repository.• Changes to services should be made against repository.



Profile and Archive

• Profile – a set of service instances and their enabled or disabled state. Useful for copying service states between systems.• Generated by svccfg extract• Archive – a complete set of persistent data for all service

instances. Useful for copying service definitions between systems.• Generated by svccfg archive• Output is an XML file similar to manifests.



SMF Snapshots

• snapshot – a complete collection of properties for a service instance.• It provides a historical view of a service and simplifies

rollback of service configuration changes.• No snapshot-at-will capability yet.• Snapshot types:> initial – taken on the first import of the manifest> last_import – taken during last import> running – properties of the currently running service instance> previous – properties of the service instance just before running> start – taken at the last successful service start



Key files and directories

• /var/svc – manifests, logs and profiles• /lib/svc – binaries, methods, seed, support files• /etc/svc/repository.db – global configuration database• /lib/svc/seed/global.db – minimal configuration database• /lib/libscf.so.1 – repository APIs



New filesystems

• /etc/svc/volatile> Directory where svc.startd stores log files in early stages of

boot, before /var is mounted read-write. Contains transient data.

• /system/contracts> Primary interface into contract subsystem, to allow service

instances to be restarted. Indirectly related to SMF.



Command overview

• General commands> svcs(1) service status listings> svcadm(1M) administrative actions> svccfg(1M) general property manipulation> svcprop(1) property reporting

• inetd management commands> inetadm(1M) administrative actions/property modification> inetconv(1M) conversion of legacy inetd.conf entries



Command overview

• Daemons> svc.startd(1M) dependency engine, master restarter> svc.configd(1M) repository, authorization> inetd(1M) delegated restarter

• Contracts subsystem> ctrun(1M) execute with process contract> ctstat(1M) display active contracts> ctwatch(1M) monitor contract events> libcontract(3LIB) contract APIs



New boot process

• Instead of booting to run level, SMF introduces the concept of milestone.• milestone – a service which specifies a collection of

services which declare a specific state of system-readiness.• You can boot to specific milestone, services not part of

that milestone are temporarily disabled.• The default milestone is the milestone you will transition

to at every boot. It is usually milestone “all”.• init S, boot -s and friends still work.



Milestones and run levels

SVR4 Run Level SMF Milestone - none s, S single-user 2 multi-user 3 multi-user-server - all



Legacy services

• Milestones are services, as such, they have manifests which have defined dependencies.• After all its dependencies have been satisfied, the start

method of the SMF major milestone executes each 'S' script within /etc/rc?.d/ with the argument 'start'. • /var/svc/manifest/milestone/single-user.xml specifies a

series of dependencies, and the start method is /sbin/rcS start.• SMF services first, legacy services last, run once only.



Part II:SMF in Action



Service administration

svcs – reports service status

• svcs -a show all services• svcs -x show services not running, impact• svcs -d show dependencies (what this depends on)• svcs -D show dependents (what depends on this)• svcs -p show member processes• svcs -l/-v show additional details



svcs

# svcs -a

STATE STIME FMRI

legacy_run Oct_12 lrc:/etc/rc2_d/S20sysetup

online Oct_12 svc:/network/ssh:default

disabled Oct_12 svc:/application/print/server:default

offline Oct_12svc:/application/print/rfc1179:default

...



svcs

# svcs -x

svc:/application/print/server:default (LP print server)

State: disabled since Wed Oct 12 16:18:38 2005

Reason: Disabled by an administrator.

See: http://sun.com/msg/SMF-8000-05

See: lpsched(1M)

Impact: 2 dependent services are not running. (Use -v for list.)



svcs

# svcs -d http

STATE STIME FMRI

online Oct_12 svc:/network/loopback:default

online Oct_12 svc:/network/physical:default

# svcs -D loopback

STATE STIME FMRI

disabled Oct_12 svc:/network/dns/client:default

disabled Oct_12 svc:/network/dns/server:default

disabled Oct_12 svc:/network/rarp:default



svcs

# svcs -p sendmail

STATE STIME FMRI

online 16:09:32 svc:/network/smtp:sendmail

16:10:32 23724 sendmail

16:10:32 23725 sendmail



svcs

# svcs -l rarp

fmri svc:/network/rarp:default

name Reverse Address Resolution Protocol (RARP) server

enabled false

state disabled

next_state none

state_time Wed Oct 12 16:18:41 2005

restarter svc:/system/svc/restarter:default

dependency require_all/error file://localhost/etc/nsswitch.conf (online)

dependency require_any/error



Service Administration

svcadm – administer services-t temporary-r recursive-s synchronously waits for completion

• svcadm enable/disable enable/disable services• svcadm enable/disable -t temporarily, until rebooted• svcadm restart restart specified service• svcadm refresh re-read service configuration• svcadm clear/mark clear/mark maintenance state• svcadm milestone transition to a milestone• svcadm -d milestone transition and set as default



svcadm

# svcs sendmail

STATE STIME FMRI

online 16:49:59 svc:/network/smtp:sendmail

# svcadm disable -t sendmail

# svcs sendmail

STATE STIME FMRI

disabled 16:51:27 svc:/network/smtp:sendmail

• Does not show service in temporary state. The presence of this property tells us that this setting is temporary.

# svcprop sendmail | grep general_ovr/enabled

general_ovr/enabled boolean false



svcadm keyserv

# svcs keyserv

STATE STIME FMRI

disabled Oct_12 svc:/network/rpc/keyserv:default

# svcadm enable keyserv

# svcs keyserv

STATE STIME FMRI

maintenance 16:58:57 svc:/network/rpc/keyserv:default

• Keyserv is in maintenance mode, why?



svcadm keyserv

# svcs -x

svc:/network/rpc/keyserv:default (RPC encryption key storage)

State: maintenance since Wed Oct 19 16:58:57 2005

Reason: Start method exited with $SMF_EXIT_ERR_CONFIG.

See: http://sun.com/msg/SMF-8000-KS

See: keyserv(1M)

See: /var/svc/log/network-rpc-keyserv:default.log

Impact: This service is not running.

• Actually, it is because the domain name is not set on this host.



svcadm keyserv

# domainname test.com

# domainname

test.com

• There is a service called domain, which takes care of system identifying info; it has to be restarted.

# svcs -o fmri,desc domain

FMRI DESC

svc:/system/identity:domain system identity(domainname)

# svcs domain

STATE STIME FMRI

online Oct_12 svc:/system/identity:domain

# svcadm restart domain



svcadm keyserv

# svcs keyserv

STATE STIME FMRI

maintenance 16:58:57 svc:/network/rpc/keyserv:default

• Keyserv is still in maintenance, why?# svcadm clear keyserv

# svcs keyserv

STATE STIME FMRI

online 17:02:48 svc:/network/rpc/keyserv:default

• Will this survive a reboot?



svcadm milestone

• What milestone am I in?# svcprop -p options_ovr/milestone system/svc/restarter:default

svcprop: Couldn't find property `options_ovr/milestone' for instance `svc:/system/svc/restarter:default'.

• If you see this, you are in milestone “all”.# svcadm milestone single-user

# Requesting System Maintenance Mode

Console login service(s) cannot run

Oct 19 17:19:00 zone1 syslogd:going down on signal 15

svc.startd: The system is coming down for administration. Please wait.

svc.startd: Killing user processes: done.



svcadm milestone

svc.startd: Killing user processes: done.

Root password for system maintenance (control-d to bypass):

single-user privilege assigned to /dev/console.

Entering System Maintenance Mode

# svcprop -p options_ovr/milestone system/svc/restarter:default

svc:/milestone/single-user:default

# reboot


svcprop: Couldn't find property `options_ovr/milestone' for instance `svc:/system/svc/restarter:default'.

• I'm back in milestone “all”.



svcadm milestone

# svcadm milestone -d single-user

# Requesting System Maintenance Mode

# reboot


svc:/milestone/single-user:default

• Every subsequent reboot will get me back here. To get back to full service, set default milestone to be “all”.

# svcadm milestone -d all



Managing inetd services

• Definition for inetd-started services were in inetd.conf but inetd.conf is now deprecated. Look how short it is!• inetd.conf entries are converted to SMF manifests and

imported into the repository on initial boot.• The old way of adding a service by modifying inetd.conf

and HUPing inetd no longer works.• If you have an application that depends on what's in

inetd.conf, there is a tool called inetconv that will convert the inetd.conf entry to an SMF manifest.



inetd

• inetd is now a delegated restarter for inet services.• It manages inet-specific properties such as> tcp_wrappers> max connection rate

• It manages state for inet services.• It listens for connections, etc.



inetconv

# grep tftp inetd.conf

# TFTPD - tftp server (primarily used for booting)

# tftp dgram udp6 wait root /usr/sbin/in.tftpdin.tftpd -s /tftpboot

• It's commented out. Uncomment the entry. You can HUP inetd but take my word that that won't do anything.



inetconv

# inetconv

inetconv: Notice: Service manifest for 100235/1 already generated as /var/svc/manifest/network/rpc/100235_1-rpc_ticotsord.xml, skipped

tftp -> /var/svc/manifest/network/tftp-udp6.xml

Importing tftp-udp6.xml ...Done

inetconv: Notice: Service manifest for 100083/1 already generated as /var/svc/manifest/network/rpc/100083_1-rpc_tcp.xml, skipped

inetconv: Notice: Service manifest for 100068/2-5 already generated as /var/svc/manifest/network/rpc/100068_2-5-rpc_udp.xml, skipped



inetconv

# svcs tftp

svcs: Pattern 'tftp' doesn't match any instances

# svcs -l udp6

fmri svc:/network/tftp/udp6:default

name tftp

enabled true

state online

next_state none

state_time Wed Oct 19 17:40:16 2005

restarter svc:/network/inetd:default




inetadm – administer inetd-started services

• inetadm list all services managed by inetd• inetadm -p show default inetd service property values• inetadm -l show all properties for a service• inetadm -e/-d enable/disable a service• inetadm -m modify property values• inetadm -M modify inetd default property values



inetadm

# inetadm -l udp6

SCOPE NAME=VALUE

name="tftp"

endpoint_type="dgram"

proto="udp6"

isrpc=FALSE

wait=TRUE

exec="/usr/sbin/in.tftpd -s /tftpboot"

user="root"

...

• Let's say you want to run with some other option, -X instead of -s.



inetadm

# inetadm -m udp6 exec="/usr/sbin/in.tftpd -X /tftpboot"

# inetadm -l udp6

SCOPE NAME=VALUE

name="tftp"

endpoint_type="dgram"

proto="udp6"

isrpc=FALSE

wait=TRUE

exec="/usr/sbin/in.tftpd -X /tftpboot"

user="root"



inetadm

• Can also use inetadm to manage inetd-services as well as svcadm.

# svcs udp6

STATE STIME FMRI

online 17:49:44 svc:/network/tftp/udp6:default

# inetadm -d udp6

# svcs udp6

STATE STIME FMRI

disabled 18:01:53 svc:/network/tftp/udp6:default



inetadm

• Usually if you don't use a service, it's enough to disable it using svcadm. But you want it really gone then delete it from your repository using svccfg delete FMRI.




svcprop – get service configuration properties

• -p property FMRI fetch a property• -s snapshot FMRI fetch property from the snapshot

# svcprop -p inetd_start/exec tftp/udp6

/usr/sbin/in.tftpd\ -s\ /tftpboot

# svcprop -p restarter/state tftp/udp6

online




svccfg – modify service configurations

• select FMRI select a service• listprop show properties for a service• setprop change a property value for a service• delprop delete a property value for a service• delete delete a service• export|import export|import a service description in XML• extract|apply extract|apply a profile• archive dump a full XML service description for

all services



svccfg

# svccfg -s print/server listprop 'lpsched/fd_limit'

lpsched/fd_limit count 0

# svccfg

svc:> select print/server

svc:/application/print/server> listprop lpsched/fd_limit


svc:/application/print/server> setprop lpsched/fd_limit = 8192

svc:/application/print/server> exit

# svcadm refresh print/server

# svccfg -s print/server listprop 'lpsched/fd_limit'




svccfg

• Can be used in batch or interactive mode.• Understands globbing to form regex• We'll work through a more pertinent example later.



Tips for troubleshooting

• If a single service is broken, -x is your friend. svcs -x will display the service logfile, if it exists.• See service start message:> boot -m verbose.

• For a system that hangs during boot:> boot -m verbose,milestone=none> log in at prompt> svcadm milestone all

• Watch system progress with svcs.



Tips for troubleshooting

• truss the service starting up by trussing the restarter• Or modify the start method, wrapping it in truss# svcprop -p start/exec apache2

/lib/svc/method/http-apache2 start

# svccfg -s apache2

svc:/network/http:apache2> setprop start/exec = “/usr/bin/truss -ealfo /tmp/apache2.truss /lib/svc/method/http-apache2 start”

• Don't forget svcadm clear once fault is rectified.



Last resorts

• Have changes been made to repository?> How many services are affected?> What snapshots exist?> Can you revert back to a previous snapshot?

• Read /lib/svc/share/README.• Restore repository from backup.



Revert a snapshot

# svccfg -s svc:/network/dns/server:default

svc:/network/dns/server:default> listsnap

initial

last-import

previous

running

start

svc:/network/dns/server:default> revert start

svc:/network/dns/server:default> exit

# svcadm refresh dns/server

# svcadm restart dns/server



Recovering a repository

• The SMF repository can be recovered in case of corruption.• You may want to save the current repository in

/etc/svc/repository.db for post mortem.• Restore script is /lib/svc/bin/restore_repository.• If you restore from seed, next boot would be the initial

boot, you will see SMF reading and importing manifests.# zlogin -C zone1

[Connected to zone 'zone1' console]

40/104



restore_repository

# /lib/svc/bin/restore_repository

The following backups of /etc/svc/repository.db exist, from oldest to newest:

boot-20051012_125317

manifest_import-20051012_130044

boot-20051012_161835

boot-20051019_172301

boot-20051019_180815

# Please enter one of:

1) boot, for the most recent post-boot backup

2) manifest_import, for the most recent manifest_import backup.

3) a specific backup repository from the above list



restore_repository

4) -seed-, the initial starting repository. (All customizations will be lost.)

5) -quit-, to cancel.

Enter response [boot]: -seed-

After confirmation, the following steps will be taken:

svc.startd(1M) and svc.configd(1M) will be quiesced, if running.

/etc/svc/repository.db -- renamed --> /etc/svc/repository.db_old_20051019_182048

/lib/svc/seed/nonglobal.db -- copied --> /etc/svc/repository.db

and the system will be rebooted with reboot(1M).

Proceed [yes/no]? yes



Part III:Advanced SMF



Manifest revisited

• Manifest is the description of the initial configuration of a service, delivered in XML.• Automatically imported into repository on boot, install,

upgrade, and pkgadd by svc:/system/manifest-import.• Ignored once imported unless modified.• Do not modify Sun-delivered manifests! If you need to

change service properties, do it in the repository. We'll talk more about this.



Order is important

• Service bundle• Service name• Create default instance• Single instance• Dependencies• Exec methods• Property groups• Stability level• Template



A little about XML

• XML markups contain data about data, metadata, on the information they bracket.> <time>1200</time>

• Tells you at 1200 is a time unit, instead of weight or distance. Whatever processes XML data can recognize that data for what it is.• Can contain any kind of data. How do we differentiate?

An XML file of train schedule will have different formatting than an XML file of TV schedule, not just formatting, but different processing and different usage. Yet, <time>1200</time> can appear in both XML files.



Introducing DOCTYPE

• Look at the top of each manifest, they all have this line:> <!DOCTYPE service_bundle SYSTEM

'/usr/share/lib/xml/dtd/service_bundle.dtd.1'>

• DTD – Document Type Definition, contrast with DOCTYPE for HTML.• Tells you whether you (especially if you are an XML-

processing program) are reading a train schedule or a TV Guide listing.• DTD is one example of a type of markup called

declaration. Specifically, the DTD is used to constrain the grammar of an XML file.



Grammar?

• Grammar governs the “correctness” of an XML document. A train schedule must have: source, terminus, track number, departure and arrival time. It won't have channel, show name, show time, duration, starring, rating.• Looking at just data, they look very similar. The DTD

defines and enforces how they must be marked up in XML, one way and one way only.• DTD is a collection of rules, or declarations, describing

markup objects.• An element declaration defines what it can contain and in

which order, also known as a content model.



Define your own language

• Strictly speaking, XML is not a markup language. It has no tag with absolute meaning. <time> can mean absolute time or elapsed time. Contrast this with <b></b>, this always means bold.• XML lays down framework for you to construct your own

markup language.• Adding elements in the DTD adds objects to your

language. An element not declared in the DTD is illegal. An element containing anything not declared in the DTD is illegal.• We'll see why this is important later on.

What's in the DTD?

• A bunch of things! None resembling what you might find in a manifest.>  are comments> <!ELEMENT> or <!ATTLIST> are declarations> <blah /> is same as <blah></blah>

• Ordering in DTD is not same as the required ordering in manifest. For example, declaration for property group is before declaration of service FMRI, and service bundle is declared last.



Why XML?

• Think about what's in /etc/rc?.d today. Anyone can write anything in any format and drop it in /etc/rc?.d. In fact, what's in /etc/rc?.d is closer to methods than manifests.• A unified service management tool depends on having

uniform data to process. The repository is a database which relies on having correct data in the correct “cell”.• With XML you get a parser and a validator for free.

Anything that XML can't validate won't get imported.• The parser is in a library called libxml2. It comes with

xmllint, which is a generic XML validator.



A quick tour inside the manifest

• We said a manifest must have at least> the service name> what the service is dependent on> ways to start and stop the service> in theory you need a restarter too but there is the default

svc.startd.

• House-keeping robot analogy> Lights> Depends on Power> On/Off switch on the wall, flip up or down> Robot is the default restarter



tftp-udp6.xml

service

name='network/tftp/udp6'

type='service'

<exec_method

type='method'

name='inetd_start'

exec='/usr/sbin/in.tftpd -s /tftpboot'

<exec_method

type='method'

name='inetd_offline'

exec=':kill_process'



tftp-udp6.xml

<restarter>

<service_fmri value='svc:/network/inetd:default' />

</restarter>

• Note that the method isn't “start”, but “inetd_start”.• Wait! No dependencies? Well, having inetd as your

delegated restarter implicitly marks you as dependent on inetd. Who starts inetd, then? It's started in /var/svc/profile/generic_open.xml• If you look at /var/svc/manifest/network/nfs/server.xml,

you will see that there is no explicitly defined restarter.



Writing a manifest from scratch

• Not all Sun products are delivered in a manifest, the reason seems to be time and effort. The README file for MySQL still says to link a startup script in /etc/rc3.d to the mysql.server script.• If no manifest is delivered, the service can still run as a

legacy service.• We're going to convert MySQL into an SMF service.• You may want to do this in a zone. Remember that each

zone has their own manifests and SMF repository.



Setting up MySQL database

• First, we need to set up the database. Instructions are in /etc/sfw/mysql/README.solaris.mysql. The startup script is /etc/sfw/mysql/mysql.server.

root@mysql~> /usr/sfw/bin/mysql_install_db

Preparing db table

Preparing host table

Preparing user table

Preparing func table

Preparing tables_priv table

Preparing columns_priv table

Installing all prepared tables

13:38:46 /usr/sfw/sbin/mysqld: Shutdown Complete



MySQL

• Add users and groupsroot@mysql~> groupadd mysql

root@mysql~> useradd -g mysql mysql

root@mysql~> chgrp -R mysql /var/mysql

root@mysql~> chmod -R 770 /var/mysql

• Add configuration filesroot@mysql~> installf SUNWmysqlr /var/mysql d 770 root mysql

root@mysql~> cp /usr/sfw/share/mysql/my-medium.cnf

/var/mysql/my.cnf



MySQL

• Start MySQL manually to see that everything worksroot@mysql~> /usr/sfw/sbin/mysqld_safe --user=mysql &

Starting mysqld daemon with databases from /var/mysql

• Set the passwordsroot@mysql~> /usr/sfw/bin/mysqladmin -u root password 'mysql'

root@mysql~> /usr/sfw/bin/mysqladmin -u root -h mysql password 'mysql'

• Check the status of MySQLroot@mysql~> /usr/sfw/bin/mysqladmin -u root -pmysql status

Uptime: 37 Threads: 1 Questions: 1 Slow queries: 0 Opens: 6 Flush tables: 1 Open tables: 0 Queries per second avg: 0.02



Copy and modify

• Great! Everything works. Now you are ready to write your manifest!• Find a manifest similar to your application and tweak it.

Since MySQL is an application, I'll look in /var/svc/manifest/application. The print/server.xml seems closest to what I want, so I'll start from that. Remember that site-specific manifests go into “site”.• Most everything is already filled out, I just change a few

relevant properties.root@mysql~> cp /var/svc/manifest/application/print/server.xml/var/svc/manifest/site/mysql.xml



Name

• Change the name<service_bundle type='manifest' name='SUNWsqlr:mysql'>

<service

name='application/mysql'

type='service'

version='1'>



Dependency

• I leave the existing dependencies alone. I may want to add a dependency on the config file, so MySQL doesn't start if it's missing.

<dependency

name='database'

grouping='require_all'

restart_on='none'

type='path'>

<service_fmri value='file://localhost/var/mysql/my.cnf'/>

</dependency>



This just in

• File dependencies going may be going away.• CR 6240573> Synopsis: File dependencies are currently overused by SMF

consumers. The problem is that a file dependency means "wait for this file to become available before starting". That's not the usual case. Most users of file dependencies are trying to express the concept that "if this file is missing, there is a problem."

• No file event to signal SMF if a file has changed.• Actions in a file dependency doesn't do anything, yet.



Methods

• Change the start/stop methods<exec_method

type='method'

name='start'

exec='/etc/sfw/mysql/mysql.server start'

timeout_seconds='60' />

<exec_method

type='method'

name='stop'

exec='/etc/sfw/mysql/mysql.server stop'

timeout_seconds='60' />



Description and documentation

• For now, delete the property groups as they don't apply to mysql• Change the description and documentation location<loctext xml:lang='C'>

MySQL server

</loctext>

<documentation>

<manpage title='mysql' section='1M'

manpath='/usr/sfw/man' />

</documentation>



Importing your manifest

• You're done! Now import it.root@mysql~> svccfg import mysql.xml

root@mysql~> svcs -l mysql

fmri svc:/application/mysql:default

name MySQL server

enabled false

state disabled

next_state none

state_time Tue Jun 07 19:20:51 2005




Importing your manifest

dependency require_all/none svc:/system/filesystem/local (online)

dependency require_all/none svc:/system/filesystem/usr (online)

dependency require_all/refresh svc:/system/identity:domain (online)

dependency optional_all/none svc:/system/system-log (online)

dependency require_all/none file://localhost/var/mysql/my.cnf (online

root@mysql~> svcadm enable mysql

root@mysql~> svcs mysql

STATE STIME FMRI

online 19:21:19 svc:/application/mysql:default



Done!

• Congratulations! You have successfully written a service manifest. Now MySQL is an SMF-managed service.



SMF on OpenSolaris

• Collection of manifests and methods are at http://www.opensolaris.org/os/community/smf/manifests/• There is a different manifest for MySQL submitted there.



Exploring the repository

• We said the repository is a database containing all descriptions of all services SMF knows about, and their states.• Make changes to service against its content in the

repository, not in the manifest. The days of looking in a text file to find what's running is over. In fact, what's in a manifest can differ vastly with the running configuration.• We'll fix Bug ID 6270245 as an example.• Bug ID: 6270245> Synopsis: apache2 needs svc:/system/filesystem/local

dependency when Sun Volume Manager metadevices are in use



Two ways to fix

• Two ways to fix:• Delete the service from the repository, fix the manifest

and re-import. This is not the recommended way.• Fix the currently running service configuration. This is the

recommended way, and we'll talk about why after we've finished the exercise.• We're going to do this both ways, in order to better

understand the repository and the manifest.• In this case the service can be addressed as http or

apache2, http is the service, apache2 is the instance.• Disable the service, then delete it from the repository.



Fix 1: deleting http from repository

# svccfg

svc:> list network/http

network/http

svc:> delete network/http

svc:> exit

• Or

# svccfg delete apache2

# svcs apache2

svcs: Pattern 'apache2' doesn't match any instances

STATE STIME FMRI



Modifying manifest

• Once again, look around for something to copy from, I find my filesystem/local dependency in inetd-upgrade.xml and ssh.xml, respectively:inetd-upgrade.xml: <dependency name='filesystem'ssh.xml: <dependency name='fs-local'

• Note the name, filesystem and fs-local. These are just common names, they are not FMRI. In fact, these can be anything descriptive.• Add this to my

/var/svc/manifest/network/http-apache2.xml• Import your fixed manifest and check.



filesystem/local dependency

dependency name='myfs-local'


restart_on='none'

type='service'>

<service_fmri value='svc:/system/filesystem/local'/>

</dependency>



In depth

• name is myfs-local, a descriptive name.• grouping is require_all, self-explanatory, but when is

grouping not require_all? When the dependency is a milestone, for example.• restart_on is none. If you depend on a service, and its

state changes, you may want to re-examine your own service.• Refer back to slide “reason for dependency stop”• type is service; can be service or file.



restart_on

• restart_on none: do not restart, you require this dependency to be satisfied once, on startup, that's all.• restart_on error: restart when this service faults. Why?

Opposite of previous case, you depend on this service all the time. If it faults (restart is implicit) and does not come back up, you need to restart and check your dependency. What state would you be in?• restart_on restart: restart when this service has an error

stop, or a non-error stop.• restart_on refresh: restart on all the above conditions,

and when this service re-read its configuration file.



svccfg import

# svccfg import http-apache2.xml

# svcs -l apache2

fmri svc:/network/http:apache2

name Apache 2 HTTP server

enabled false

state disabled

next_state none

state_time Mon Jun 06 18:26:41 2005


dependency require_all/error svc:/network/loopback:default (online)

dependency optional_all/error svc:/network/physical:default (online)




Fix 2: modify properties in repository

• Fix by modifying current service properties in the repository.• Now that I've written the manifest, I know better what to

do. Basically, take that block of dependency and some how get it into the running service configuration of apache2.• Disable and delete apache2 from the repository.• Re-import the original, unrepaired manifest; it should only

have two dependencies.



Put back original manifest

# svccfg -s apache2 delete

# svccfg import http-apache2.xml

# svccfg -s apache2 listpg

loopback dependency

physical dependency

start method

stop method

refresh method

httpd application

startd framework

general framework

restarter framework NONPERSISTENT

restarter_actions framework NONPERSISTENT



What I'm interested in

# svccfg

svc:> help

General commands: help set repository end

Manifest commands: inventory validate import export archive

Profile commands: apply extract

Entity commands: list select unselect add delete

Snapshot commands: listsnap selectsnap revert

Property group commands: listpg addpg delpg

Property commands: listprop setprop delprop editprop

Property value commands: addpropvalue delpropvalue setenv unsetenv

svc:>



Adding properties

• Add a new dependency to my property groupsvc:/network/http:apache2> addpg myfs-local dependency

• In your manifest, this conforms to <dependency name='myfs-local'

• Add other propertiessvc:/network/http:apache2> setprop myfs-local/grouping = astring: ”require_all”

svc:/network/http:apache2> setprop myfs-local/restart_on = astring: ”none”

svc:/network/http:apache2> setprop myfs-local/type = astring: ”servicr”



Oops! A typo.

svc:/network/http:apache2> listprop

myfs-local dependency

myfs-local/group astring require_all

myfs-local/restart_on astring none

myfs-local/type astring servicr

svc:/network/http:apache2> editprop (invokes vi)

# Property group "myfs-local"

# delprop myfs-local

# addpg myfs-local dependency

# setprop myfs-local/group = astring: (require_all)

# setprop myfs-local/restart_on = astring: (none)

# setprop myfs-local/type = astring: (servicr)



Add the FMRI

• Change this line tosetprop myfs-local/type = astring: “service”

• Upon quitting vi, the temp file is executed. This corresponds to


restart_on='none'

type='service'>

• Now add the FMRI of the dependencysvc:/network/http:apache2> setprop myfs-local/entities = fmri: “svc:/system/filesystem/local”



Check your configuration

svc:/network/http:apache2> listprop

...

filesystem-local dependency

filesystem-local/group astring require_all

filesystem-local/restart_on astring none

filesystem-local/type astring service

filesystem-local/entities fmri svc:/system/filesystem/local

...

svc:/network/http:apache2> exit



Refresh the service

# svcadm refresh apache2

# svcs -l apache2

fmri svc:/network/http:apache2

name Apache 2 HTTP server

...


dependency require_all/error svc:/network/loopback:default (online)

dependency optional_all/error svc:/network/physical:default (online)




New property added

# svccfg -s apache2 listpg

loopback dependency

physical dependency

start method

stop method

refresh method

httpd application

startd framework

general framework

restarter framework NONPERSISTENT

restarter_actions framework NONPERSISTENT

myfs-local dependency



The reason why ...

• You've modified the service properties for apache2 in the repository. Don't be confused the next time you look at http-apache2.xml and wonder what happened. Manifests are read and imported only once (unless they are modified.)• Don't modify manifests in place. Upon boot

svc:/system/manifest-import searches /var/svc/manifest for new XML files and ones which have changed and import them.• This allows us to upgrade service properties (via a patch,

for example) by dropping new manifest in, instead of manipulating the repository using svccfg.



The reason why ...

• When svccfg import is run on a new manifest for services which already exist, it first compares the current values to those in the last-import snapshot.• If values don't differ, it assumes that there's been no

customization, and upgrades the properties to what's in the manifest.• Otherwise it issues warnings to the user, but leaves the

customizations in place.• So any patch which alters a manifest will cause manifest-

import to try to upgrade the corresponding service, no post-install action necessary.



The reason why ...

• If you had modified that manifest in place and re-imported it, or even if you copied it somewhere else, modified it, and re-imported it, svccfg will think that the modified version is the original, official version (via the last-import snapshot comparison), will will override all your customization after the patch.• The moral of the story: leave those manifests alone.

Make changes against the repository with svccfg.• man smf_bootstrap(5) for the gory details.



Part IV:More Advanced SMF



SMF security

• SMF uses RBAC to delegate service management responsibility. We'll talk about managing specific services first.• SMF provides these propval to be put into individual

manifest to authorize specific services:> action_authorization – can execute method> value_authorization – can change value of existing properties> modify_authorization – change, add, delete properties

• These are fixed keywords.• Let's take a look smtp-sendmail.xml and ssh.xml as

examples.

action_authorization

smtp-sendmail.xml:

<property_group name='general' type='framework'>

<propval name='action_authorization' type='astring' value='solaris.smf.manage.sendmail' />

</property_group>

ssh.xml:

<property_group name='general' type='framework'>



<propval name='action_authorization' type='astring' value='solaris.smf.manage.ssh' />

</property_group>



prof_auth

• Where will you find this? /etc/security/prof_authMail Management:::Manage sendmail &

queues:auths=solaris.smf.manage.sendmail;help=RtMailMngmnt.html

Network Security:::Manage network and host security:auths=solaris.smf.manage.ssh,solaris.network.*;help=RtNetSecure.html• These RBAC roles have this privilege, which privilege is

this? The one defined by the keyword action_authorization, which allows you to execute methods.



Delegating to a user

• But you haven't set up roles properly, you want to delegate service administration to a user.

# echo christine::::auths=solaris.smf.manage.ssh >> /etc/user_attr

• You have to re-login for authorization to take effect.# su – christine

$ id

uid=85228(christine) gid=10(staff)



Delegating to a user

$ svcs ssh

STATE STIME FMRI

online Oct_19 svc:/network/ssh:default

$ date

Fri Oct 21 15:47:40 EDT 2005

$ svcadm restart ssh

$ tail /var/svc/log/network-ssh:default.log

[ Oct 21 15:47:42 Stopping because service restarting. ]

[ Oct 21 15:47:42 Executing stop method (:kill) ]

[ Oct 21 15:47:42 Executing start method ("/lib/svc/method/sshd start") ]

[ Oct 21 15:47:42 Method "start" exited with status 0 ]



A little problem

$ svcadm disable ssh

svcadm: svc:/network/ssh:default: Permission denied.

$ svcadm disable -t ssh

$ svcs ssh

STATE STIME FMRI

disabled 15:55:43 svc:/network/ssh:default

$ svcprop ssh |grep general_ovr


• Service can only be temporarily disabled, why?See BugID 6303496.



Needs value_authorization

• Bug ID 6303496> Synopsis cron cannot be smf managed with

solaris.smf.manage.cron

• It turns out that you also need value_authorization, the keyword to change existing property values.• Why? Because you're actually changing the value of

some property. Which property?• And exactly where is that general_ovr property listed?• general and general_ovr appear to be property groups of

type framework. Check the DTD.• We'll talk more about instances later.



Looking for general_ovr

# svccfg -s ssh

svc:/network/ssh> listprop general*

general framework

general/action_authorization astring solaris.smf.manage.ssh

general/entity_stability astring Unstable

general/single_instance boolean true

# svccfg -s ssh:default

svc:/network/ssh:default> listprop general*

general framework

general/enabled boolean true

general_ovr frameworkNONPERSISTENT




value_authorization

# svccfg -s ssh

svc:/network/ssh> listprop


• Do you need to addpg?svc:/network/ssh> setprop general/value_authorization = astring: "fooblat"

svc:/network/ssh> exit

• You still need to make one change for this to work.



value_authorization

# tail /etc/user_attr

christine::::auths=solaris.smf.manage.ssh,fooblat

# svcadm refresh ssh

# su - christine

$ svcadm disable ssh

$ svcs ssh

STATE STIME FMRI

disabled 16:42:47 svc:/network/ssh:default

$ svcprop ssh|grep general_ovr

$

• Yes, it's a permanent change.



Value can be anything

• The value of the auth used has no specific meaning except that it must exist in /etc/user_attr. • In the manifest<propval name='value_authorization' type='astring' value='fooblat' />

• ... is equal to<propval name='value_authorization' type='astring'

value='solaris.smf.manage.ssh' />

• ... but the later is more descriptive. In fact:# auths christine

solaris.smf.manage.ssh,fooblat,solaris.device.cdrw,s



SMF Security

• If you look at /etc/security/prof_attr, you will seeService Management:::Manage

services:auths=solaris.smf.manage, solaris.smf.modify• These are two main authorizations in SMF:> solaris.smf.modify – add, delete, modify services> solaris.smf.manage – restart, refresh, or other state

modification

• They don't appear in any manifest because they apply all services.• There are others: method, dependency, application, and

framework. man smf_security(5) for the gory details.



Blueprint

Restricting Service Administration in the Solaris 10 Operating System by Glenn Brunette (June 2005)This Sun BluePrints Cookbook describes how to use the Solaris 10 Service Management Facility (SMF) to require specific authorizations for certain types of operations. Using this capability, it is possible to delegate access to core service management functions based on the concept of least privilege--if a user or service does not strictly need to have some degree of privilege, then that privilege should not be granted. SMF allows organizations to have much finer grained access control policies than was possible before the Solaris 10 Operating System.

http://www.sun.com/blueprints/0605819-2887.pdf



Snapshots

# svccfg -s ssh

svc:/network/ssh> listsnap

Instance not selected.

svc:/network/ssh> select default

svc:/network/ssh:default> listsnap

last-import

initial

running

• Where are snapshot start and previous?# svcs ssh

STATE STIME FMRI

disabled Oct_21 svc:/network/ssh:default



What's the difference?

# svccfg -s ssh listprop '*general*'

general framework




general/value_authorization astring yadayada

# svccfg -s ssh:default listprop '*general*'

general framework

general/enabled boolean false

general_ovr framework NONPERSISTENT



instance

• Instances are units of administrations.• Services are mechanism for sharing properties.• Instance can have properties, and its parent service can

have properties.• Instances have snapshots, services do not.• If unnamed, that instance is called “default”.• Snapshots are composite view of the instance and

service properties at one point in time.



revert

svc:/network/ssh:default> listsnap

last-import initial

running start

svc:/network/ssh:default> selectsnap initial

[initial]svc:/network/ssh:default> revert

svc:/network/ssh:default> unselect

svc:/network/ssh> listprop general*

general framework




• value_authorization has disappeared.



Snapshots

• Snapshots are not good for rolling back a single change because > No control over when snapshots are taken> Alters the service properties as well

• To undo a property change, it's simpler to use svccfg setprop commands• Can use svcprop or svccfg export to take textual

snapshots. • However, when you use svccfg export, you use the

service name, even though the output file contains the description of the instance(s).

ignore_error property

• From http-apache2.xml:<property_group name='startd' type='framework'>



<propval name='ignore_error' type='astring'value='core,signal' />

</property_group>

• SMF is instructed to ignore signals sent to a service or core dumps by a service; otherwise SMF will try to restart the service.• man svc.startd(1M) for the gory details; description at the

startd/ignore_error paragraph.



Signaling httpd

# ps -ef|grep http

webservd 29855 29853 0 11:46:46 ? 0:00 /usr/apache2/bin/httpd -k start

root 29853 27175 0 11:46:45 ? 0:00 /usr/apache2/bin/httpd -k start





# kill -HUP 29853 <-- started by root



Logs

# tail -f /var/apache2/logs/error_log

(nothing ...)

# tail -f /var/svc/log/network-http:apache2.log

[ Oct 26 11:46:44 Executing start method ("/lib/svc/method/http-apache2 start") ]


(nothing ... )

• What would happen, by the way, if you kill just httpd processes started by webservd?



Keep on killing httpd

# ps -ef|grep http


# kill -9 29854 <-- last one


[Wed Oct 26 12:46:53 2005] [notice] Digest: generating secret for digest authentication ...

[Wed Oct 26 12:46:53 2005] [notice] Digest: done

[Wed Oct 26 12:46:54 2005] [notice] Apache/2.0.52 (Unix) DAV/2 configured -- resuming normal operations



httpd restarted by SMF


Stopping because all processes in service exited. ]

[ Oct 26 11:55:05 Executing stop method ("/lib/svc/method/http-apache2 stop") ]

httpd (pid 29853?) not running

[ Oct 26 11:55:05 Method "stop" exited with status 0 ]



# svcs apache2

STATE STIME FMRI

online 12:46:53 svc:/network/http:apache2



Removing ignore_error property

# svccfg -s apache2

svc:/network/http:apache2> listprop startd*

startd framework

startd/ignore_error astring core,signal

svc:/network/http:apache2> delprop startd/ignore_error

svc:/network/http:apache2> listprop startd*

startd framework

svc:/network/http:apache2> exit

# svcadm refresh apache2



Log of httpd refreshing


[Wed Oct 26 12:55:30 2005] [notice] Graceful restart requested, doing restart

[Wed Oct 26 12:55:30 2005] [warn] (128)Network is unreachable: connect to listener





[ Oct 26 12:55:30 Rereading configuration. ]

[ Oct 26 12:55:30 Executing refresh method ("/lib/svc/method/http-apache2 refresh") ]

[ Oct 26 12:55:30 Method "refresh" exited with status 0 ]



httpd without ignore_error# kill -9 29999 <-- started by root


[ Oct 26 13:13:02 Stopping because process received fatal signal from outside the service. ]


httpd (pid 29999?) not running


[ Oct 26 13:14:02 Method or service exit timed out. Killing contract 720 ]

# svcs apache2

STATE STIME FMRI

maintenance 13:14:02 svc:/network/http:apache2



Clearing and restarting

# svcadm clear apache2


[ Oct 26 13:16:04 Leaving maintenance because clear requested. ]

[ Oct 26 13:16:04 Enabled. ]



• What would happen if you kill any httpd processes started by webservd, now that you no longer have the ignore_error property?



Logs


[Wed Oct 26 13:24:23 2005] [notice] caught SIGTERM, shutting down




• SMF stops and restarts httpd:apache because svc.startd will interpret any externally-generated signal as a failure of the service, and will execute the stop and start method to try and recover.



Logs


[ Oct 26 13:24:23 Stopping because process received fatal signal from outside the service. ]





• What's different though? Why isn't it in maintenance mode?



Portable SMF

• Perhaps you have a large number of servers, or even zones, on which you'd like to imprint an identical set of services.• Well, this is not easy. This is “provisioning”, and SMF is a

service management tool.• You can't just “pack up” the database, or the XML dump

generated by svccfg archive, or all the manifests ... Why not? • What about the methods, the config files, the binaries,

everything else that makes up the apps?



Duplicating service properties

• If, however, you have an application already installed, and you would like to transfer service properties (such as dependencies, the exec method, anything that might be defined by the manifest), you can do that by exporting and then importing a manifest.

# svccfg export svc:/network/http > apache2.export

• Move apache2.export to another system.# svccfg import apache2.export

• Note: the argument to svccfg export is a service, not an instance, although the XML file will contain definition of the instance(s).



Duplicating service states

• You can get a profile, or an XML file that describes the state of all services by using svccfg extract.• This XML file can be used to duplicate the on|off state of

services from system to system. Use svccfg apply.



Profiles

• A profile is an XML description of all service instance states on a system.• Profiles are in /var/svc/profile.• Profiles are applied in order: generic, platform, site• Never modify existing profiles, use site.xml for local

customization.• An example of something that may create a use a profile

is the Solaris Security Toolkit 4.2 (aka JASS).



Part V:SMF Tools and Community



Webmin



About Webmin

• A web-based interface for UNIX system administration• It comes with Solaris 10, or get it at

http://www.webmin.com• Version 1.190 an

later supports SMF,even has a feature tocreate a new manifest.



SMF in Webmin



Create new service

• This button will walk you through creating a new service manifest. • You don't need to know XML, but you do need to

understand what goes into a manifest.• Will ask you a series of questions such as:> Name of service> What are the dependencies,> Which is the delegated restarter> Service properties> Man pages



GUI Tools

• With Webmin you can monitor and manage SMF services through a browser.• A GUI interface to SMF is being developed.• Watch for a SunMC SMF module due near the end of this

year.



More tools

• Use xmllint to check for errors in the manifest file without importing the manifest.• svccfg validate does something similar, without changing

the repository.• Jedit is a free text editor with XML with syntax checker

and validator. Comes with Solaris 10 (/usr/demo/jds/bin/jedit) or get it from http://www.jedit.org.• Morphon Technologies has a good and free XML editor,

but sadly no longer in development, get it from http://www.morphon.com.



SFM community on OpenSolaris

• http://www.opensolaris.org/os/community/smf/• Submit manifests or methods you've written here:

http://www.opensolaris.org/os/community/smf/manifests/• [email protected] migrating to

[email protected]• Subscribe here:

http://www.opensolaris.org/os/community/smf/discussions• Read it here:

http://www.opensolaris.org/jive/forum.jspa?forumID=24• File bugs here:• http://bugs.opensolaris.org/



Resources• Internal> Greenline project page: http://greenline.eng> greenline-interest alias.

• External> BigAdmin: http://www.sun.com/bigadmin> Especially the Service Developer Intro athttp://www.sun.com/bigadmin/content/selfheal/sdev_intro.html> SMF community: http://opensolaris.org/os/community/smf/> SMF mailing list: [email protected]

• Blogs> Liane Praza's blog: http://blogs.sun.com/roller/page/lianep> Stephen Hahn's blog: http://blogs.sun.com/sch



Review

•

•

•



Questions?

•

•

•



And lastly ...

Thank you!



References

• PresentationsL. Praza. smf(5): Service Management Facility.R. Manus, G. Hiregoudar. Solaris Management Framework – Workshop.



Changes

• Last modified: 03/10/2005



SOLARIS 10DEEP DIVE: SMF

[email protected]

US Solaris Adoption Practice

Sun Microsystems

Smf deepdive-tran

Education

Transcript of Smf deepdive-tran