1

<Insert Picture Here>

Oracle Solaris 10 Recommended Patching Strategy

Gerry Haskins, Director, Software Patch ServicesOracle Solaris Systems11th January 2011

3

The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions.The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle.

4

Contents

• Strategy– Recommended Patching Strategy

• When to apply

• What to apply

• Where to get patches and updates

• How to apply

• How to further mitigate risk

• Summary

• Oracle Proactive Services and Tools

– Patching Strategy Considerations• The next generation: Image Packaging System• Further information

5

Applicability

• This presentation describes the generic Recommended Patching Strategy for Solaris 10 systems• An alternative maintenance regime which takes

precedence over this strategy may be prescribed for specific systems

6

Recommended Patching Strategy

• When to apply– Major upgrade maintenance windows will typically be

dictated by your business constraints• Often associated with hardware roll-outs• Every 18 to 24 months is recommended

– Minor patching maintenance windows should be scheduled for every 3 months• Align with the Oracle Critical Patch Update (CPU)

release schedule so you can update the rest of your Oracle stack at the same time

– CPUs are released on the Tuesday closest to the 17th of January, April, July, and October – See http://www.oracle.com/technetwork/topics/security/alerts-086861.html

– Reactive patching may occasionally be necessary to address break-and-fix issues

7

Recommended Patching Strategy

• What to apply– Apply latest Solaris 10 Update release in major

maintenance windows• A Solaris 10 Update is a full release image containing

new features with all available patches pre-applied • Provides functionally rich, intensely tested, high quality,

and high performance software baselines on which to standardize deployments

• Can install or upgrade to a Solaris Update release• Alternatively, use the Solaris Update Patch Bundle to

bring all pre-existing packages up to the same software level as the corresponding Solaris Update

• Recommend customers be on a Solaris 10 Update or Solaris Update Patch Bundle released in the last 2 years

8

Recommended Patching Strategy

• Solaris Update Patch Bundles– Patches pre-existing packages to the same software level

as the corresponding Solaris Update release• For example, all ZFS and Zones functionality available

in patches• /etc/release updated to show both the original release

and the Solaris Update Patch Bundle patch level – Not the same as upgrading to, or fresh install of a Solaris

Update release• Patch Bundles do not include new, deleted, or up'rev'd

packages• Some new functionality may depend on new packages

– Some new hardware may only be supported from a specific Solaris Update release forward

9

Recommended Patching Strategy

• Apply latest Solaris 10 OS Recommended Patch Cluster– Minimum amount of change to get critical Solaris 10 OS

Security, Data Corruption, and System Availability fixes– Archived quarterly as the Oracle Solaris 10 Critical Patch

Update (CPU)– Enterprise Installation Standards (EIS) includes a

superset of the Recommended Patch Cluster, and is available as a monthly patch “baseline” in Oracle Enterprise Manager Ops Center

– Recommend customers be on a Solaris 10 OS Recommended Patch Cluster, CPU, or EIS Patch Baseline released within the last 6 months

10

Recommended Patching Strategy

• Apply latest firmware updates– Firmware updates are increasingly important for

SPARC, especially T-series, as well as x86, to:• Provide functional enhancements, e.g. Oracle VM for

SPARC• Resolve many key issues, often misdiagnosed as

hardware failures• Deliver significant performance gains• Provide better diagnostics

– Storage devices, etc., may need firmware updates too– Oracle Sun QA teams test hardware, software, and

patches against the latest firmware– Recommend customers be on firmware released within

the last 6 months

11

Recommended Patching Strategy

• Apply any additional Solaris 10 OS patches required to fix issues specific to your environment• Apply updates for other software and hardware

– Quarterly released Critical Patch Updates (CPUs) for the rest of the Oracle Stack

– Updates for 3rd party software and hardware• Note that some 3rd party and community based

software shipped with Solaris may deliver bug fixes via upgrading the package versions rather than via applying patches

12

Recommended Patching Strategy

• Where to get patches and updates– Oracle Solaris Update releases• Search for “Oracle Solaris” on http://edelivery.oracle.com/

– My Oracle Support (MOS) is the one stop shop for all your support needs, including patches and knowledge articles• You need an Oracle support contract

– Flash (full functionality): https://support.oracle.com – Html (limited functionality):

https://supporthtml.oracle.com – ‘wget’ downloads: See

https://support.oracle.com/CSP/main/article?cmd=show&type=NOT&id=1199543.1

13

Recommended Patching Strategy

• How to apply– Get your tools in order first• Always install the latest patch and package utility

patches before installing any other patches– This is done automatically when applying the

Solaris OS Recommended Patch Cluster, Solaris CPU, or Solaris Update Patch Bundle

• Install the latest Oracle Solaris 10 Live Upgrade (LU) patches if using Live Upgrade

• Install the latest updates for any patch automation tool used

14

Recommended Patching Strategy

• Apply patches and upgrades to an Inactive Boot Environment to minimize risk and downtime– Avoids the need to follow some of the “Special Install

Instructions” contained in patch READMEs– Provides simple rollback mechanism– Use Oracle Solaris Live Upgrade (LU) for most

environments– In Oracle Solaris Cluster environments, a rolling update

of the cluster nodes may be preferred

15

Recommended Patching Strategy

• Mitigating risk through Integrated Stack Testing– Hardware. Software. Complete.– Oracle Solaris 10 Update releases

and patches are tested as part of Oracle Integrated Stack Testing (OIST)

– Designed to minimize risk, deployment times, and TCO while maximizing performance, availability, and robustness

16

Recommended Patching Strategy

• How to further mitigate risk ?– Oracle Solaris, coupled with 3rd party

products and customer apps, provides virtually infinite configurability

– A customer test environment which closely mimics your production environment is an excellent way to further mitigate risk

– Should include functional, peak load, and stress testing

17

Oracle Solaris 10 Recommended Patching Strategy Summary

Major Maintenance Windows

Minor Maintenance Windows

Reactive Patching

Frequency Every 18 to 24 months

Every 3 months, aligned to CPU schedule

As necessary

Install latest patch utility patches Yes Yes Yes

Use Live Upgrade or rolling Cluster node upgrade Yes Yes Yes

Apply Solaris Update or Solaris Update Patch Bundle Yes

Apply Recommended Patch Cluster, CPU, or EIS patch baseline Yes Yes

Update FirmwareYes Yes

If applicable

Apply any other patches requiredYes Yes Yes

Apply updates for 3rd party s/w & h/wYes Yes

If applicable

Conduct pre-deployment testingYes Yes

As much as possible

18

Recommended Patching Strategy

• Oracle provides proactive services and tools to save you time and money in maintaining systems– Oracle Sun Management and Diagnostic tools – See

https://support.oracle.com/CSP/main/article?cmd=show&type=NOT&id=411786.1

• Oracle Sun System Analysis identifies known issues, including security, data corruption, and availability risk associated with specific systems – See https://support.oracle.com/CSP/main/article?cmd=show&type=NOT&id=1194234.1

• Oracle Auto Service Request (ASR) for Sun Systems – See https://support.oracle.com/CSP/main/article?cmd=show&type=NOT&id=1185493.1

• Oracle Services Tools Bundle (STB) – See https://support.oracle.com/CSP/main/article?cmd=show&type=NOT&id=1153444.1

• Oracle Shared Shell – See https://support.oracle.com/CSP/main/article?cmd=show&type=NOT&id=1194226.1

– Advanced Customer Services (ACS) – See

http://www.oracle.com/us/support/software/advanced-customer-services/index.html or contact acsdirect_us@oracle.com

19

Recommended Patching Strategy

• Oracle Enterprise Manager Ops Center 11g, http://www.oracle.com/technetwork/oem/ops-center/index.html – Automatically downloads all firmware and patches to your site

– Covers T, M, and X-series h/w, disk, & RAID Controller firmware

– Offers Enterprise Class deployment features such as rollback and support for Live Upgrade along with audit and policy control

– Leverages enhanced dependency and Special Instructions metadata

– Integrates telemetry and knowledge from the independent government approved common vulnerability repository at mitre.orgTM

– Offers built in profiles to check OS level patches

– Integrates OS level patch compliance reports with Enterprise Manager Grid Control Oracle Applications Violations for a single Oracle stack compliance report

– Facilitates the usage of single software compliance statements that span multiple Operating Systems

– Facilitates the creation of Service Requests (SRs)

20

Agenda

• Strategy– Recommended Patching Strategy– Patching Strategy Considerations

• Objective

• Advantages of Recommended Patch Strategy

• Why not apply all patches ?

• What about the timing of patch application ?

• What about patch quality ?

• The next generation: Image Packaging System• Further information

21

Patching Strategy Considerations

• Typical objective is to maximize production system availability, security, and performance by optimizing proactive maintenance to prevent issues– Change implies risk– Minimizing risk is not as simple as minimizing change– Need to consider the best tested and best quality

baselines upon which to standardize deployments– Prevention is better than cure - scheduled proactive

maintenance windows are usually significantly less costly than reactive break-and-fix maintenance

– A homogeneous environment helps reduce complexity, and hence TCO

22

Patching Strategy Considerations

• Each Solaris Update includes all bug fixes available at the time it was built• Solaris Updates are intensely tested by many teams

across Oracle and so provide a good quality baseline upon which to standardize deployments• The Solaris OS Recommended Patch Cluster

provides critical Solaris OS Security, Data Corruption, and System Availability fixes– Provides critical fixes in minimum amount of change– Includes fixes released since latest Solaris Update

contents were finalized– Tested as a unit as well as individual patches– Sophisticated install script

23

Patching Strategy Considerations

• Advantages of Recommended Patching Strategy– Risk minimization “sweet spot”

– Safety in numbers

• Issues likely to be caught and resolved quickly

• Contrast to “dim sum” patching where you pick and choose patch combinations– Likely to result in unique software combinations

• Rigorous Oracle Sun patch processes ensure issues are very rare

• Issues may be unique, making them more difficult to diagnose and reproduce, leading to delays in resolution

24

Patching Strategy Considerations

• Why not apply all patches ?– Applying all patches is a perfectly reasonable strategy

• Code changes in patches go through an intensive review, verification, and test process

• All patches included in each Solaris Update release and Solaris Update Patch Bundle

• Most bug fixes are for corner case issues which only occur in highly specific configurations

• Debatable whether applying corner case fixes for all configurations in between Solaris Update releases is the optimal system maintenance strategy to minimize risk and maximize system availability

25

Patching Strategy Considerations

• What about timing of patch application ?– Patches are intensely tested, but issues specific to

certain configurations can still occur occasionally

– Some customers like to wait until a patch has been released for a period of time before applying it unless it fixes an urgent security issue

• Analysis of the time between patch release and the withdrawal of problematic patches shows no correlation to any “sweet spot”, although pervasive issues are usually found within 10 days of release

26

Patching Strategy Considerations

• What about patch quality ?– Oracle Sun releases over 4,000 patches every year

• A patch is withdrawn if it does more harm than good for the majority of customers. Just 17 have been withdrawn after release in the last year.

• Configuration specific issues are documented in the Special Install Instructions section of patch READMEs

• Security issues are announced in Critical Patch Updates and http://www.oracle.com/technetwork/topics/security/alerts-086861.html or via the security blog, http://blogs.sun.com/security , for 3rd party components

• An Alert will be issued for Data Corruption or System Availability issues

– See “Alerts” under the MOS “Knowledge” tab

–

27

Agenda

• Strategy– Recommended Patching Strategy– Patching Strategy Considerations

• The next generation: Image Packaging System• Further information

28

Image Packaging System (IPS)

• Next generation packaging architecture used in– Solaris 11 Express

– Exadata

– Exalogic

• All updates delivered as packages– Single tier package architecture

– No more patches

– No error prone scripts

29

Image Packaging System (IPS)

• Packages are downloaded from Repositories– Choice of change control streams

• Latest code for evaluation, developers, ISVs

• Stable features for deployment

• Support Repository Updates (SRUs) for bug fixes

• Leverages technical advances– ZFS Root, Snapshots

– Boot Environments, beadm, like an improved, built-in Live Upgrade

30

Agenda

• Strategy– Recommended Patching Strategy– Patching Strategy Considerations

• The next generation: Image Packaging System• Further information

31

Further Information

• Patch Corner Blog, http://blogs.sun.com/patch • The Oracle Technology Patching Center,

http://www.oracle.com/technetwork/systems/patches/overview/index.html

• Changes in Security Policies for the Sun product lines, http://www.oracle.com/technetwork/topics/security/changesforsunsecuritypolicies-162219.html

• Critical Patch Updates and Security Alerts, http://www.oracle.com/technetwork/topics/security/alerts-086861.html

• Security Blog, http://blogs.sun.com/security • For information on other key issues, see “Alerts” under

the MOS “Knowledge” tab on https://support.oracle.com

• Oracle Solaris Installation, Booting, and Patching Forum,

https://communities.oracle.com/portal/server.pt/community/oracle_solaris_installation,_booting_and_patching/397

• Feedback to Gerry.Haskins@oracle.com

32

33