Oracle Databases on EMC VMAX.pdf

download Oracle Databases on EMC VMAX.pdf

of 512

Transcript of Oracle Databases on EMC VMAX.pdf

  • 5/17/2018 Oracle Databases on EMC VMAX.pdf

    1/511

    Oracle Databases onEMC Symmetrix Storage Systems

    Version 1.3

    Generating Restartable Oracle Copies Using SymmetrixStorage

    Oracle Remote Replication and Disaster Restart UsingSymmetrix Storage

    Oracle Data Layout and Performance Using SymmetrixStorage

    Yaron Dar

  • 5/17/2018 Oracle Databases on EMC VMAX.pdf

    2/511

    Oracle Databases on EMC Symmetrix Storage Systems2

    Copyright 2008, 2009, 2010, 2011 EMC Corporation. All rights reserved.

    EMC believes the information in this publication is accurate as of its publication date. The information issubject to change without notice.

    THE INFORMATION IN THIS PUBLICATION IS PROVIDED AS IS. EMC CORPORATION MAKES NOREPRESENTATIONS OR WARRANTIES OF ANY KIND WITH RESPECT TO THE INFORMATION IN THISPUBLICATION, AND SPECIFICALLY DISCLAIMS IMPLIED WARRANTIES OF MERCHANTABILITY ORFITNESS FOR A PARTICULAR PURPOSE.

    Use, copying, and distribution of any EMC software described in this publication requires an applicablesoftware license.

    For the most up-to-date regulatory document for your product line, go to the Technical Documentation andAdvisories section on EMC Powerlink.

    For the most up-to-date listing of EMC product names, see EMC Corporation Trademarks on EMC.com.

    All other trademarks used herein are the property of their respective owners.

    H2603.3

  • 5/17/2018 Oracle Databases on EMC VMAX.pdf

    3/511

    Oracle Databases on EMC Symmetrix Storage Systems 3

    Preface

    Chapter 1 Oracle on Open Systems

    Introduction ....................................................................................... 26Oracle overview ................................................................................ 27Oracle system elements .............................................................27Oracle data elements ..................................................................29

    Storage management........................................................................ 33Cloning Oracle objects or environments ....................................... 34Backup and recovery........................................................................ 35Oracle Real Application Clusters ................................................... 36

    Optimizing Oracle layouts on EMC Symmetrix .......................... 38EMC and Oracle integration ........................................................... 39Install base ...................................................................................39

    Joint engineering.........................................................................39Joint Services Center...................................................................40

    Chapter 2 EMC Foundation Products

    Introduction ....................................................................................... 42Symmetrix hardware and EMC Enginuity features .................... 45Symmetrix VMAX platform......................................................46EMC Enginuity operating environment..................................47

    EMC Solutions Enabler base management ................................... 49EMC Change Tracker ....................................................................... 52EMC Symmetrix Remote Data Facility.......................................... 53

    SRDF benefits ..............................................................................54

    SRDF modes of operation..........................................................54SRDF device groups and composite groups...........................55

    Contents

  • 5/17/2018 Oracle Databases on EMC VMAX.pdf

    4/511

    Oracle Databases on EMC Symmetrix Storage Systems4

    Contents

    SRDF consistency groups .......................................................... 55

    SRDF terminology ...................................................................... 59SRDF control operations............................................................ 61Failover and failback operations .............................................. 65EMC SRDF/Cluster Enabler solutions....................................67

    EMC TimeFinder............................................................................... 68TimeFinder/Mirror establish operations................................ 69TimeFinder split operations...................................................... 70TimeFinder restore operations ................................................. 71

    TimeFinder consistent split ....................................................... 72Enginuity Consistency Assist ................................................... 72TimeFinder/Mirror reverse split ............................................. 75TimeFinder/Clone operations.................................................. 75TimeFinder/Snap operations ................................................... 78

    EMC Storage Resource Management ............................................ 81EMC Storage Viewer ........................................................................ 86EMC PowerPath................................................................................ 88

    PowerPath/VE............................................................................90EMC Replication Manager .............................................................. 97EMC Open Replicator ...................................................................... 99EMC Virtual Provisioning............................................................. 100

    Thin device ................................................................................100Data device ................................................................................100New Symmetrix VMAX Virtual Provisioning features ...... 101

    EMC Virtual LUN migration ........................................................ 103

    EMC Fully Automated Storage Tiering (FAST).......................... 106

    Chapter 3 Creating Oracle Database Clones

    Overview.......................................................................................... 109Comparing recoverable and restartable copies of databases ... 110

    Recoverable disk copies...........................................................110Restartable disk copies............................................................. 110

    Copying the database with Oracle shutdown............................. 111Creating Oracle copies using TimeFinder/Mirror .............. 111Creating Oracle copies using TimeFinder/Clone ............... 113Creating Oracle copies using TimeFinder/Snap ................. 115

    Copying a running database using EMC consistencytechnology........................................................................................ 118

    Creating Oracle copies using TimeFinder/Mirror .............. 118Creating Oracle copies using TimeFinder/Clone ............... 120

    Creating Oracle copies using TimeFinder/Snap ................. 122Copying the database with Oracle in hot backup mode........... 125

  • 5/17/2018 Oracle Databases on EMC VMAX.pdf

    5/511

    5Oracle Databases on EMC Symmetrix Storage Systems

    Contents

    Putting the tablespaces or database into hot backup

    mode ...........................................................................................125Taking the tablespaces or database out of hot backupmode ...........................................................................................126Creating Oracle copies using TimeFinder/Mirror...............126Creating Oracle copies using TimeFinder/Clone................128Creating Oracle copies using TimeFinder/Snap..................130

    Replicating Oracle using Replication Manager .......................... 133Transitioning disk copies to Oracle database clones.................. 135

    Host considerations ..................................................................135Enabling a cold database copy................................................140Enabling a restartable database copy.....................................141Enabling a hot backup database copy....................................142

    Oracle transportable tablespaces .................................................. 143Benefits and uses of transportable tablespaces.....................143Implementation of transportable tablespaces with EMCTimeFinder and SRDF..............................................................144

    Transportable tablespace example .........................................144Cross-platform transportable tablespaces ................................... 150

    Overview....................................................................................150Implementing cross-platform transportable tablespaces....151

    Choosing a database cloning methodology ................................ 154

    Chapter 4 Backing Up Oracle Environments

    Introduction ..................................................................................... 156Comparing recoverable and restartable copies of databases.... 157

    Recoverable disk copies ...........................................................157Restartable disk copies.............................................................157

    Database organization to facilitate recovery ............................... 159Oracle backup overview ................................................................ 161

    Online (hot) versus offline (cold) backups............................163Point-in-time and roll-forward recovery backups ...............164

    Comparing partial and entire database backups .................165Comparing incremental and full database backups ............165

    Using EMC replication in the Oracle backup process ............... 166Copying the database with Oracle shutdown ............................ 168

    Creating cold Oracle backup copies usingTimeFinder/Mirror ..................................................................168Creating cold Oracle backup copies using TimeFinder/Clone ...........................................................................................170

    Creating cold Oracle backup copies using TimeFinder/Snap.............................................................................................172

  • 5/17/2018 Oracle Databases on EMC VMAX.pdf

    6/511

    Oracle Databases on EMC Symmetrix Storage Systems6

    Contents

    Copying a running database using EMC consistencytechnology........................................................................................ 175

    Creating restartable Oracle backup copies usingTimeFinder/Mirror.................................................................. 176Creating restartable Oracle backup copies usingTimeFinder/Clone ................................................................... 177Creating restartable Oracle backup copies usingTimeFinder/Snap.....................................................................179

    Copying the database with Oracle in hot backup mode........... 182

    Putting the tablespaces or database into hot backupmode...........................................................................................182Taking the tablespaces or database out of hot backupmode...........................................................................................183Creating hot Oracle backup copies using TimeFinder/Mirror .........................................................................................183Creating hot Oracle backup copies using TimeFinder/Clone........................................................................................... 185

    Creating hot Oracle backup copies using TimeFinder/Snap ............................................................................................ 187Backing up the database copy ...................................................... 190Backups using EMC Replication Manager for Oracle

    backups ............................................................................................ 191Backups using Oracle Recovery Manager (RMAN).................. 193Backups using TimeFinder and Oracle RMAN.......................... 195

    Chapter 5 Restoring and Recovering Oracle Databases

    Introduction..................................................................................... 198Oracle recovery types..................................................................... 199

    Crash recovery .......................................................................... 199Media recovery ......................................................................... 200Complete recovery ................................................................... 201Incomplete recovery................................................................. 201

    Restartable database recovery ................................................ 202Oracle recovery overview.............................................................. 203Restoring a backup image using TimeFinder............................. 205

    Restore using TimeFinder/Mirror.........................................205Restore using TimeFinder/Clone .......................................... 208Restore using TimeFinder/Snap............................................211

    Restoring a backup image using Replication Manager ............ 215Oracle database recovery procedures.......................................... 217

    Oracle restartable database recovery procedures................ 217Oracle complete recovery........................................................218

  • 5/17/2018 Oracle Databases on EMC VMAX.pdf

    7/511

    7Oracle Databases on EMC Symmetrix Storage Systems

    Contents

    Oracle incomplete recovery.....................................................220

    Database recovery using Oracle RMAN...................................... 223Oracle Flashback ............................................................................. 224Flashback configuration...........................................................224Flashback Query........................................................................225Flashback Version Query.........................................................226Flashback Transaction Query..................................................226Flashback Table .........................................................................226Flashback Drop..........................................................................226

    Flashback Database...................................................................227

    Chapter 6 Understanding Oracle Disaster Restart & DisasterRecovery

    Introduction ..................................................................................... 230Definitions ........................................................................................ 231

    Dependent-write consistency..................................................231

    Database restart .........................................................................231Database recovery.....................................................................232Roll-forward recovery..............................................................232

    Design considerations for disaster restart and disasterrecovery ............................................................................................ 233

    Recovery Point Objective.........................................................233Recovery Time Objective .........................................................234Operational complexity............................................................234

    Source server activity ...............................................................235Production impact ....................................................................235Target server activity................................................................235Number of copies of data.........................................................236Distance for solution.................................................................236Bandwidth requirements .........................................................236Federated consistency ..............................................................237Testing the solution ..................................................................237

    Cost .............................................................................................238Tape-based solutions....................................................................... 239

    Tape-based disaster recovery..................................................239Tape-based disaster restart......................................................239

    Remote replication challenges....................................................... 241Propagation delay.....................................................................241Bandwidth requirements .........................................................242Network infrastructure ............................................................242

    Method of instantiation............................................................243Method of reinstantiation ........................................................243

  • 5/17/2018 Oracle Databases on EMC VMAX.pdf

    8/511

    Oracle Databases on EMC Symmetrix Storage Systems8

    Contents

    Change rate at the source site ................................................. 243Locality of reference.................................................................244Expected data loss .................................................................... 244Failback operations ..................................................................245

    Array-based remote replication.................................................... 246Planning for array-based replication ........................................... 247SRDF/S single Symmetrix array to single Symmetrix array ... 250

    How to restart in the event of a disaster ............................... 252SRDF/S and consistency groups.................................................. 253

    Rolling disaster ......................................................................... 253Protection against a rolling disaster ...................................... 255SRDF/S with multiple source Symmetrix arrays................ 257

    SRDF/A............................................................................................ 260SRDF/A using a single source Symmetrix array................. 261SRDF/A multiple source Symmetrix arrays ........................262How to restart in the event of a disaster ............................... 264

    SRDF/AR single hop ..................................................................... 266

    SRDF/AR multihop ....................................................................... 269How to restart in the event of a disaster ............................... 271Database log-shipping solutions .................................................. 272

    Overview of log shipping........................................................ 272Log-shipping considerations .................................................. 272Log shipping and remote standby database ........................275Log shipping and standby database with SRDF.................. 276Oracle Data Guard ................................................................... 277

    Running database solutions .......................................................... 286Overview ...................................................................................286Advanced Replication..............................................................286Oracle Streams .......................................................................... 287

    Chapter 7 Oracle Database Layouts on EMC Symmetrix DMX

    Introduction..................................................................................... 290

    The performance stack................................................................... 291Importance of I/O avoidance ................................................. 292Storage-system layer considerations ..................................... 293

    Traditional Oracle layout recommendations .............................. 294Oracle's optimal flexible architecture .................................... 294Oracle layouts and replication considerations.....................295Automated Storage Management .......................................... 296

    Symmetrix DMX performance guidelines .................................. 297

    Front-end connectivity............................................................. 297Symmetrix cache.......................................................................299

  • 5/17/2018 Oracle Databases on EMC VMAX.pdf

    9/511

    9Oracle Databases on EMC Symmetrix Storage Systems

    Contents

    Back-end considerations ..........................................................308Additional layout considerations...........................................309Configuration recommendations............................................310

    RAID considerations....................................................................... 311Types of RAID...........................................................................311RAID recommendations ..........................................................315Symmetrix metavolumes .........................................................316

    Host- versus array-based striping ................................................ 318Host-based striping ..................................................................318

    Symmetrix-based striping (metavolumes)............................319Striping recommendations ......................................................320Data placement considerations ..................................................... 322

    Disk performance considerations...........................................322Hypervolume contention.........................................................324Maximizing data spread across the back end.......................325Minimizing disk head movement ..........................................327

    Other layout considerations .......................................................... 328

    Database layout considerations with SRDF/S .....................328Database cloning, TimeFinder, and sharing spindles .........328Database clones using TimeFinder/Snap .............................329

    Oracle database-specific configuration settings ......................... 331The database layout process.......................................................... 333

    Database layout process...........................................................333

    Chapter 8 Data Protection

    EMC Double Checksum overview ............................................... 340Traditional methods of preventing data corruption............340Data corruption between host and conventional storage ...341Benefits of checking within Symmetrix arrays.....................341

    Implementing EMC Double Checksum for Oracle .................... 342Other checksum operations.....................................................342Enabling checksum options.....................................................343

    Verifying checksum is enabled ...............................................344Validating for checksum operations ......................................344Disabling checksum..................................................................345

    Implementing Generic SafeWrite for generic applications ....... 346Torn pages: Using Generic SafeWrite to protectapplications................................................................................346Why generic? .............................................................................347Where to enable Generic SafeWrite........................................347

    Configuring Generic SafeWrite...............................................348How to disable Generic SafeWrite .........................................350

  • 5/17/2018 Oracle Databases on EMC VMAX.pdf

    10/511

    Oracle Databases on EMC Symmetrix Storage Systems10

    Contents

    Listing Generic SafeWrite devices ......................................... 351Performance considerations....................................................351

    Syntax and examples...................................................................... 353

    Chapter 9 Storage TieringVirtual LUN and FAST

    Overview.......................................................................................... 356Evolution of storage tiering........................................................... 359

    Manual storage tiering.............................................................359Fully Automated Storage Tiering (FAST)............................. 359Fully Automated Storage Tiering for Virtual Pools(FAST VP) ..................................................................................359Example of storage tiering evolution .................................... 359

    Symmetrix Virtual Provisioning................................................... 361Introduction............................................................................... 361Virtual Provisioning and Oracle databases..........................363Planning thin devices for Oracle databases.......................... 368

    Enhanced Virual LUN migrations for Oracle databases........... 372Manual tiering mechanics ....................................................... 372Symmetrix Enhanced Virtual LUN technology................... 372LUN-based migrations and ASM...........................................373Configuration for Virtual LUN migration............................376Symmetrix Virtual LUN VP mobility technology ............... 380

    Fully Automated Storage Tiering for Virtual Pools................... 381FAST VP and Virtual Provisioning........................................ 381

    FAST VP Elements ................................................................... 382FAST VP time window considerations .................................383FAST VP move time window considerations ......................384FAST VP architecture...............................................................384FAST VP and Oracle databases ..............................................386Examples of FAST VP for Oracle databases .........................390Test Case 1: FAST VP optimization of a single Oracledatabase OLTP workload........................................................ 391

    Test Case 2: Oracle databases sharing the ASM diskgroup and FAST policy............................................................ 396Test Case 3: Oracle databases on separate ASM diskgroups and FAST policies ....................................................... 399

    Fully Automated Storage Tiering................................................. 404Introduction............................................................................... 404FAST configuration ..................................................................405FAST device movement...........................................................406

    FAST and ASM ......................................................................... 407

  • 5/17/2018 Oracle Databases on EMC VMAX.pdf

    11/511

    11Oracle Databases on EMC Symmetrix Storage Systems

    Contents

    Example of FAST for Oracle databases..................................407Conclusion........................................................................................ 419

    Appendix A Symmetrix VMAX with Enginuity

    Introduction to Symmetrix VMAX series with Enginuity ........ 422New Symmetrix VMAX ease of use, scalability andvirtualization features ............................................................. 422Oracle mission-critical applications require protectionstrategy ...................................................................................... 423Enterprise protection and compliance using SRDF............ 423Oracle database clones and snapshots with TimeFinder ... 424Oracle database recovery using storage consistentreplications................................................................................ 424Best practices for local and remote Oracle databasereplications................................................................................ 424Symmetrix VMAX Auto-provisioning Groups ................... 425Symmetrix VMAX Enhanced Virtual LUN migrationtechnology................................................................................. 427Symmetrix VMAX TimeFinder product family................... 431Symmetrix VMAX SRDF product family............................. 434ASM rebalancing and consistency technology .................... 442

    Leveraging TimeFinder and SRDF for business continuitysolutions............................................................................................ 444

    Use Case 1: Offloading database backups fromproduction................................................................................. 447Use Case 2: Parallel database recovery................................. 450Use Case 3: Local restartable replicas of production .......... 452Use Case 4: Remote mirroring for disaster protection(synchronous and asynchronous).......................................... 453Use Case 5: Remote restartable database replicas forrepurposing............................................................................... 454Use Case 6: Remote database valid backup replicas .......... 456

    Use Case 7: Parallel database recovery from remotebackup replicas......................................................................... 457Use Case 8: Fast database recovery from a restartablereplicas....................................................................................... 459

    Conclusion........................................................................................ 462Test storage and database configuration ..................................... 463

    General test environment........................................................ 463

  • 5/17/2018 Oracle Databases on EMC VMAX.pdf

    12/511

    Oracle Databases on EMC Symmetrix Storage Systems12

    Contents

    Appendix B Sample SYMCLI Group Creation Commands

    Sample SYMCLI group creation commands .............................. 468

    Appendix C Related Host Operation

    Overview.......................................................................................... 474BIN file configuration ............................................................. 474SAN considerations................................................................. 475Final configuration considerations for enabling LUN

    presentation to hosts ............................................................... 476Presenting database copies to a different host ........................... 477AIX considerations .................................................................. 477HP-UX considerations ............................................................ 480Linux considerations............................................................... 483Solaris considerations ............................................................. 485Windows considerations ........................................................ 487Windows Dynamic Disks ....................................................... 490

    Presenting database copies to the same host.............................. 491AIX considerations .................................................................. 491HP-UX considerations ............................................................ 492Linux considerations............................................................... 495Solaris considerations ............................................................. 496Windows considerations ........................................................ 497

    Appendix D Sample Database Cloning Scripts

    Sample script to replicate a database........................................... 500

    Appendix E Solutions Enabler Command Line Interface (CLI) forFAST VP Operations and Monitoring

    Overview.......................................................................................... 510Enabling FAST.......................................................................... 510

    Gathering detailed information about a Symmetrix thinpool ............................................................................................ 510Checking distribution of thin device tracks across FASTVP tiers ...................................................................................... 511Checking the storage tiers allocation.................................... 512

  • 5/17/2018 Oracle Databases on EMC VMAX.pdf

    13/511

    Oracle Databases on 13

    Title Page

    1 Oracle Systems Architecture......................................................................... 272 Physical data elements in an Oracle configuration................................... 303 Relationship between data blocks, extents, and segments....................... 324 Oracle two-node RAC configuration........................................................... 375 Symmetrix VMAX logical diagram ............................................................. 476 Basic synchronous SRDF configuration...................................................... 547 SRDF consistency group ............................................................................... 578 SRDF establish and restore control operations .......................................... 639 SRDF failover and failback control operations .......................................... 6510 Geographically distributed four-node EMC SRDF/CE clusters............. 6711 EMC Symmetrix configured with standard volumes and BCVs............ 6912 ECA consistent split across multiple database-associated hosts............. 7313 ECA consistent split on a local Symmetrix system ................................... 7414 Creating a copy session using the symclone command ........................... 7715 TimeFinder/Snap copy of a standard device to a VDEV......................... 8016 SRM commands.............................................................................................. 8217 EMC Storage Viewer...................................................................................... 8718 PowerPath/VE vStorage API for multipathing plug-in........................... 9119 Output of rpowermt display command on a Symmetrix VMAX

    device .................................................................................................................94

    20 Device ownership in vCenter Server........................................................... 9521 Virtual Provisioning components.............................................................. 10122 Virtual LUN eligibility tables ..................................................................... 10323 Copying a cold (shutdown) Oracle database with TimeFinder/

    Mirror...............................................................................................................11224 Copying a cold Oracle database with TimeFinder/Clone ..................... 11425 Copying a cold Oracle database with TimeFinder/Snap....................... 11626 Copying a running Oracle database with TimeFinder/Mirror............. 119

    27 Copying a running Oracle database with TimeFinder/Clone .............. 12128 Copying a running Oracle database with TimeFinder/Snap................ 123

    Figures

  • 5/17/2018 Oracle Databases on EMC VMAX.pdf

    14/511

    Oracle Databases on EMC Symmetrix Storage Systems14

    Figures

    29 Copying an Oracle database in hot backup mode with TimeFinder/Mirror...............................................................................................................127

    30 Copying an Oracle database in hot backup mode with TimeFinder/Clone ................................................................................................................ 129

    31 Copying an Oracle database in hot backup mode with TimeFinder/Snap................................................................................................................. 131

    32 Using Replication Manager to make a TimeFinder copy of Oracle...... 13333 Database organization to facilitate recovery............................................ 15934 Copying a cold Oracle database with TimeFinder/Mirror ................... 16935 Copying a cold Oracle database with TimeFinder/Clone..................... 17136 Copying a cold Oracle database with TimeFinder/Snap ...................... 17337 Copying a running Oracle database with TimeFinder/Mirror............. 17638 Copying a running Oracle database using TimeFinder/Clone ............ 17839 Copying a running Oracle database with TimeFinder/Snap................ 18040 Copying an Oracle database in hot backup mode with TimeFinder/

    Mirror...............................................................................................................18441 Copying an Oracle database in hot backup mode with TimeFinder/

    Clone ................................................................................................................ 18642 Copying an Oracle database in hot backup mode with TimeFinder/Snap..................................................................................................................188

    43 Using RM to make a TimeFinder copy of Oracle .................................... 19144 Restoring a TimeFinder copy, all components ........................................ 20645 Restoring a TimeFinder copy, data components only............................ 20646 Restoring a TimeFinder/Clone copy, all components ........................... 20947 Restoring a TimeFinder/Clone copy, data components only............... 209

    48 Restoring a TimeFinder/Snap copy, all components............................. 21249 Restoring a TimeFinder/Snap copy, data components only................. 21250 Restoring Oracle using EMC Replication Manager ................................ 21551 Database components for Oracle............................................................... 24852 Synchronous replication internals............................................................. 25053 Rolling disaster with multiple production Symmetrix arrays .............. 25454 Rolling disaster with SRDF consistency group protection .................... 25655 SRDF/S with multiple source Symmetrix arrays and ConGroup

    protection ........................................................................................................ 25856 SRDF/A replication internals .................................................................... 26057 SRDF/AR single-hop replication internals .............................................. 26658 SRDF/AR multihop replication Internals ................................................ 27059 Log shipping and remote standby database ............................................ 27560 Sample Oracle10g Data Guard configuration.......................................... 28061 "No data loss" standby database................................................................ 28462 The performance stack ................................................................................ 292

    63 Relationship between host block size and IOPS/throughput............... 298

  • 5/17/2018 Oracle Databases on EMC VMAX.pdf

    15/511

    15Oracle Databases on EMC Symmetrix Storage Systems

    Figures

    64 Performance Manager graph of write-pending limit for a singlehypervolume ...................................................................................................305

    65 Performance Manager graph of write-pending limit for a four-member metavolume .....................................................................................306

    66 Write workload for a single hyper and a striped metavolume ............. 30767 3+1 RAID 5 layout detail ............................................................................. 31268 Anatomy of a RAID 5 random write ......................................................... 31369 Optimizing performance with RAID 5 sequential writes....................... 31470 Disk performance factors ............................................................................ 32471 Synchronous replication internals ............................................................. 35172 Storage tiering evolution ............................................................................. 36073 Thin devices and thin pools containing data devices ............................. 36374 Thin device configuration ........................................................................... 36575 Migration of ASM members from FC to EFDs using Enhanced

    Virtual LUN technology ............................................................................... 37676 Virtual LUN migration to configured space............................................. 37777 Virtual LUN migration to unconfigured space........................................ 379

    78 FAST managed objects................................................................................. 38279 FAST policy association............................................................................... 38380 FAST VP components .................................................................................. 38581 Heat map of ASM member devices showing sub-LUN skewing...... 38782 Gold FAST VP policy storage group association..................................... 39383 Storage tier allocation changes during the FAST VP test for FINDB.... 39484 Ddatabase transaction changes with FAST VP........................................ 39685 Storage tier changes during FAST VP enabled run on two

    databases......................................................................................................... 39886 FAST VP enabled test with different FAST policies................................ 40387 Initial FAST policies for DB3....................................................................... 41088 Initial FAST policy for DB3......................................................................... 41289 Initial performance analysis on FAST ....................................................... 41290 FAST configuration wizard: Setting FAST parameters........................... 41391 FAST configuration wizard: Creating performance and move time

    window ........................................................................................................... 413

    92 FAST configuration wizard: Creating FAST policy................................. 41493 FAST configuration wizard: Creating a FAST storage group................ 41594 DB3 FAST policy........................................................................................... 41695 FAST swap/move detail ............................................................................. 41796 Disk utilization map after migration ......................................................... 41897 Oracle RAC and Auto-provisioning Groups............................................ 42698 Migration example using Virtual LUN technology ................................ 42999 SRDF/Synchronous replication ................................................................. 435

    100 SRDF/Asynchronous replication............................................................... 437101 SRDF Adaptive Copy mode........................................................................ 438

  • 5/17/2018 Oracle Databases on EMC VMAX.pdf

    16/511

    Oracle Databases on EMC Symmetrix Storage Systems16

    Figures

    102 Concurrent SRDF ......................................................................................... 439103 Cascaded SRDF ............................................................................................ 440104 SRDF/Extended Distance Protection........................................................ 440105 SRDF/Star..................................................................................................... 441106 Test configuration ........................................................................................ 464107 Windows Disk Management console........................................................ 488

  • 5/17/2018 Oracle Databases on EMC VMAX.pdf

    17/511

    Oracle Databases on EMC Symmetrix Storage Systems 17

    Title Page

    1 Oracle background processes ........................................................................ 282 SYMCLI base commands ............................................................................... 493 TimeFinder device type summary................................................................ 794 Data object SRM commands.......................................................................... 835 Data object mapping commands .................................................................. 836 File system SRM commands to examine file system mapping ................ 847 File system SRM command to examine logical volume mapping ........... 858 SRM statistics command ................................................................................ 859 Comparison of database cloning technologies ......................................... 15410 Database cloning requirements and solutions.......................................... 15411 Background processes for managing a Data Guard environment......... 28012 Initialization parameters .............................................................................. 33113 Background processes for managing a Data Guard environment......... 35314 FAST VP Oracle test environment.............................................................. 39015 Initial tier allocation for test cases with shared ASM disk group .......... 39116 FINDB initial tier allocation......................................................................... 39317 Initial AWR report for FINDB..................................................................... 39318 Oracle database tier allocations-initial and FAST VP enabled ............... 39519 FAST VP enabled database response time from the AWR report ......... 39520 FINDB and HRDB initial storage tier allocation....................................... 397

    21 Initial AWR report for FINDB..................................................................... 39722 FAST VP enabled database transaction rate changes .............................. 39923 Initial tier allocation for a test case with independent ASM disk

    groups ..............................................................................................................39924 Initial AWR report for CRMDB and SUPCHDB....................................... 40125 AST VP enabled AWR report for CRMDB and SUPCHDB .................... 40226 Storage tier allocation changes during the FAST VP-enabled run ........ 40327 Test configuration ......................................................................................... 408

    28 Storage and ASM configuration for each test database........................... 40929 Database storage placement (initial) and workload profile.................... 409

    Tables

  • 5/17/2018 Oracle Databases on EMC VMAX.pdf

    18/511

    Oracle Databases on EMC Symmetrix Storage Systems18

    Tables

    30 Initial Oracle AWR report inspection (db file sequential read).............. 41031 Initial FAST performance analysis results................................................. 416

    32 Results after FAST migration of DB3 to Flash .......................................... 41733 ASM diskgroups, and Symmetrix device and composite groups ........ 44434 Test hardware................................................................................................ 464

  • 5/17/2018 Oracle Databases on EMC VMAX.pdf

    19/511

    Oracle Databases on EMC Symmetrix Storage Systems 19

    Preface

    As part of an effort to improve and enhance the performance and capabilitiesof its product lines, EMC periodically releases revisions of its hardware andsoftware. Therefore, some functions described in this document may not be

    supported by all versions of the software or hardware currently in use. Forthe most up-to-date information on product features, refer to your productrelease notes.

    This document describes how the EMC Symmetrix array manages Oracledatabases on UNIX and Windows. Additionally, this document provides a

    general description of the Oracle RDBMS and EMC products and utilitiesthat can be used for Oracle administration. EMC Symmetrix storage arrays

    and EMC software products and utilities are used to clone Oracleenvironments and to enhance database and storage management backup andrecovery procedures.

    Other topics include:

    Database and storage management administration

    CPU resource consumption

    The time required to clone or recover Oracle systems

    Audience This TechBook is intended for systems administrators, Oracledatabase administrators, and storage management personnelresponsible for managing Oracle databases on opensystemsplatforms. The information in this document is based on Oracle10g.In this document, open-systems platforms are UNIX operatingsystems (including AIX, HPUX, Linux, and Solaris), as well as

    Microsoft Windows platforms.

  • 5/17/2018 Oracle Databases on EMC VMAX.pdf

    20/511

    20 Oracle Databases on EMC Symmetrix Storage Systems

    Preface

    Readers of this document are expected to be familiar with thefollowing topics:

    Symmetrix operation Oracle concepts and operation

    Relateddocumentation

    The following is a list of related documents that provide moredetailed information on topics described in this TechBook.Many of these documents are on the EMC Powerlink site(http://powerlink.EMC.com). For Oracle information, consult the

    Oracle websites including the main site (http://www.oracle.com),the Oracle Technology Network (OTN), and Oracle Metalink.

    EMC-related documents include:

    Solutions Enabler Release Notes(by release)

    Solutions Enabler Support Matrix(by release)

    Solutions Enabler Symmetrix Device Masking CLI Product Guide (byrelease)

    Solutions Enabler Symmetrix Base Management CLI Product Guide(by release)

    Solutions Enabler Symmetrix CLI Command Reference (by release)

    Solutions Enabler Symmetrix Configuration Change CLI ProductGuide (by release)

    Solutions Enabler Symmetrix SRM CLI ProductGuide (by release)

    Solutions Enabler Installation Guide (by release)

    Solutions Enabler Symmetrix TimeFinder Family CLI Product Guide(by release)

    Solutions Enabler Symmetrix SRDF Family CLI Product Guide(by release)

    Symmetrix Remote Data Facility (SRDF) Product Guide

    EnginuityThe EMC Symmetrix Storage Operating Environment - ADetailed Review (white paper)

    Replication Manager Product Guide

    Replication Manager Support Matrix

    Oracle-related documents include:

    Oracle Data Guard Concepts and Administration

    Oracle Database Administrator's Guide Oracle Database Backup and Recovery Basics

  • 5/17/2018 Oracle Databases on EMC VMAX.pdf

    21/511

    Oracle Databases on EMC Symmetrix Storage Systems 21

    Preface

    Oracle Database Backup and Recovery Advanced Users Guide

    Oracle Database Performance Tuning Guide Oracle Database Reference

    Organization This TechBook contains the following chapters and severalappendices:

    Chapter 1, Oracle on Open Systems,provides a high-leveloverview of Oracle.

    Chapter 2, EMC Foundation Products,describes EMC productsused to support the management of Oracle environments.

    Chapter 3, Creating Oracle Database Clones,describes proceduresto clone Oracle instances. It also discusses procedures to clone Oracleobjects within and across Oracle instances using Oracle TransportableTablespaces and EMC TimeFinder.

    Chapter 4, Backing Up Oracle Environments,describes how to

    back up Oracle environments and objects with Oracle RecoveryManager and EMC products including TimeFinder and SRDF.

    Chapter 5, Restoring and Recovering Oracle Databases,describeshow to recover Oracle environments and objects, based upon the typeof backups that were previously performed.

    Chapter 6, Understanding Oracle Disaster Restart & DisasterRecovery,describes the difference between using traditional

    recovery techniques versus EMC restart solutions.

    Chapter 7, Oracle Database Layouts on EMC Symmetrix DMX,describes Oracle RDBMS on EMC Symmetrix DMX data layoutrecommendations and best practices.

    Chapter 8, Data Protection,describes data protection methodsusing EMC Double Checksum to minimize the impact of I/O errorson database consistency during I/O transfers between hosts and

    Symmetrix storage devices.

    Chapter 9, Storage TieringVirtual LUN and FAST,describesstorage tiers available on Symmetrix and methodologies fornondisruptive migration of Oracle data using Symmetrixtechnologies across available storage tiers.

    The appendixes provide sample code, which supplement proceduresdescribed in the document, and additional detail on the Symmetrix

    VMAX Series with Enginuity with Oracle.

  • 5/17/2018 Oracle Databases on EMC VMAX.pdf

    22/511

    22 Oracle Databases on EMC Symmetrix Storage Systems

    Preface

    The references section lists documents that contain more informationon these topics. Examples provided in this document cover methods

    for performing various Oracle functions using Symmetrix arrays withEMC software. These examples were developed for laboratory testingand may need tailoring to suit other operational environments. Anyprocedures outlined in this document should be thoroughly testedprior to production implementation.

    Conventions used inthis document

    EMC uses the following conventions for special notices.

    Note: A note presents information that is important, but not hazard-related.

    IMPORTANT

    An important notice contains information essential to operation ofthe software or hardware.

    Typographical conventionsEMC uses the following type style conventions in this document:

    Normal Used in running (nonprocedural) text for: Names of interface elements (such as names of windows,

    dialog boxes, buttons, fields, and menus)

    Names of resources, attributes, pools, Boolean expressions,buttons, DQL statements, keywords, clauses, environmentvariables, functions, utilities

    URLs, pathnames, filenames, directory names, computernames, filenames, links, groups, service keys, file systems,notifications

    Bold Used in running (nonprocedural) text for: Names of commands, daemons, options, programs, processes,

    services, applications, utilities, kernels, notifications, systemcalls, man pages

    Used in procedures for:

    Names of interface elements (such as names of windows,dialog boxes, buttons, fields, and menus)

    What user specifically selects, clicks, presses, or types

    Italic Used in all text (including procedures) for: Full titles of publications referenced in text

    Emphasis (for example a new term)

    Variables

  • 5/17/2018 Oracle Databases on EMC VMAX.pdf

    23/511

    Oracle Databases on EMC Symmetrix Storage Systems 23

    Preface

    The authors of this Techbook

    This TechBook was written by Yaron Dar, an employee of EMC basedat Hopkinton, Masschusetts. Yaron has over ten years of service withEMC and more than thirteen years of experience in Oracle databases.

    Other primary contributors to this TechBook are David Waddill,

    Udgith Mankad, and the EMC Database and Application Team, alsobased in Hopkinton.

    We'd like to hear from you!

    Your feedback on our TechBooks is important to us! We want ourbooks to be as helpful and relevant as possible, so please feel free tosend us your comments, opinions and thoughts on this or any otherTechBook:

    [email protected]

    Courier Used for:

    System output, such as an error message or script

    URLs, complete paths, filenames, prompts, and syntax whenshown outside of running text

    Courier bold Used for:

    Specific user input (such as commands)

    Courier italic Used in procedures for:

    Variables on command line

    User input variables

    < > Angle brackets enclose parameter or variable values supplied bythe user

    [ ] Square brackets enclose optional values

    | Vertical bar indicates alternate selections - the bar means or

    { } Braces indicate content that you must specify (that is, x or y or z)

    ... Ellipses indicate nonessential information omitted from theexample

    http://localhost/var/www/apps/conversion/tmp/scratch_4/[email protected]://localhost/var/www/apps/conversion/tmp/scratch_4/[email protected]
  • 5/17/2018 Oracle Databases on EMC VMAX.pdf

    24/511

    24 Oracle Databases on EMC Symmetrix Storage Systems

    Preface

  • 5/17/2018 Oracle Databases on EMC VMAX.pdf

    25/511

    Oracle on Open Systems 25

    1

    This chapter presents these topics:

    Introduction ........................................................................................ 26 Oracle overview ................................................................................. 27 Storage management ......................................................................... 33 Cloning Oracle objects or environments ........................................ 34 Backup and recovery ......................................................................... 35 Oracle Real Application Clusters..................................................... 36 Optimizing Oracle layouts on EMC Symmetrix............................ 38 EMC and Oracle integration............................................................. 39

    Oracle on OpenSystems

  • 5/17/2018 Oracle Databases on EMC VMAX.pdf

    26/511

    26 Oracle Databases on EMC Symmetrix Storage Systems

    Oracle on Open Systems

    Introduction

    The Oracle RDBMS on open systems first became available in 1979and has steadily grown to become the marketshare leader inenterprise database solutions. With a wide variety of features andfunctionality, Oracle provides a stable platform for handlingconcurrent, read-consistent access to a customer's application data.

    Oracle database 10g and 11g, the latest releases of the Oracle RDBMS,have introduced a variety of new and enhanced features overprevious versions of the database. Among these are:

    Increased self-management through features such as AutomaticUndo Management, Oracle managed files, and mean time torecovery enhancements.

    Improved toolsets and utilities such as Recovery Manager(RMAN), Oracle Data Guard, and Oracle Enterprise Manager(OEM).

    Introduction of Automatic Storage Management (ASM).

    Enhancements to Oracle Real Application Clusters.

    Introduction of Database Resource Manager.

    Enhancements to Oracle Flashback capabilities.

    Introduction of Oracle VM server virtualization.

    Oracle's architectural robustness, scalability, and availabilityfunctions have positioned it as a cornerstone in many customers'enterprise system infrastructures. A large number of EMCcustomers use Oracle in open-systems environments to support large,mission-critical business applications.

    Oracle on Open S stems

  • 5/17/2018 Oracle Databases on EMC VMAX.pdf

    27/511

    Oracle overview 27

    Oracle on Open Systems

    Oracle overview

    The Oracle RDBMS can be configured in multiple ways. Therequirement for 24x7 operations, replication and disaster recovery,and the capacity of the host(s) that will contain the Oracle instance(s)will, in part, determine how the Oracle environment must bearchitected.

    Oracle system elementsAn Oracle database consists of three basic components: memorystructures, processes, and files. An Oracle instance is defined as theSystem Global Area and the associated background processes.Figure 1shows a simplified example of the Oracle components.

    Figure 1 Oracle Systems Architecture

    The System Global Area (SGA) contains the basic memory structuresthat an Oracle database instance requires to function. The SGAcontains memory structures such as the Buffer Cache (shared area forusers to read or write Oracle data blocks), Redo Log Buffer (circular

    buffer for the Oracle logs), Shared Pool (including user SQL and

    PL/SQL code, data dictionary, and more), Large Pool, and others.

    PMON

    ICO-IMG-

    System Global Area(SGA)

    Shared

    Pool

    Redo Log

    Buffers

    PGA

    Data

    DictionaryDB Block Buffers

    DBWn

    Datafiles Datafiles Datafiles Datafiles

    Snnn

    LGWR

    CKPT

    Active

    Redo

    Log

    Redo

    Log

    Redo

    Log ARCn

    Archive

    Logs

    SMON

    Oracle on Open Systems

  • 5/17/2018 Oracle Databases on EMC VMAX.pdf

    28/511

    28 Oracle Databases on EMC Symmetrix Storage Systems

    Oracle on Open Systems

    In addition to the SGA, the Oracle instance has another memorystructure that is called Program Global Area, or PGA. A PGA is

    allocated for each server process accessing the database as well as forbackground processes. The PGA contains the session information,cursors, bind variable values, and an area for memory intensiveoperations such as sorts, joins, and others. This is particularlyimportant for data warehouses where parallel query executioncommonly requires a lot of PGA space rather than SGA.

    The background processes are started when the instance is initiated;

    they enable Oracle to perform tasks such as reading and writingbetween the data files and the SGA, managing I/O to the redo logs,performing archiving between the redo and archive logs, andconnecting users to the database. Table 1describes some of theOracle background processes shown in Figure 1 on page 27.

    Table 1 Oracle background processes (page 1 of 2)

    Process Description

    DBWn(DatabaseWriter)

    Writes data from buffer cache to the datafiles on disk. Up to 20 databasewriter processes can be started per Oracle instance. The number of writerscan be controlled manually by using the DB_WRITER_PROCESSESinit.ora parameter. If not specified, Oracle will determine automatically thenumber of writers.

    LGWR (LogWriter)

    Manages the redo log buffer and transmitting data from the buffer to the redologs on disk. Log writer writes to the logs whenever one of these fourscenarios occurs:

    A user committed transaction

    Every three seconds

    When the redo buffer is third full

    If DB writer needs to write dirty blocks, but their redo log is still in theredo buffer

    ARCn(Database

    Archiver)

    Copies the redo logs to one or more log directories when a log switchoccurs. The ARCn process is only turned on if the database is in

    ARCHIVELOG mode and automatic archiving is enabled. Up to 10 archiveprocesses can be started per Oracle instance, controlled by the init.oraparameter LOG_ARCHIVE_MAX_PROCESSES.

    CKPT(Checkpoint)

    When the Oracle system performs a checkpoint, DBWn needs to destagedata to disk. The CKPT process updates the data file header accordingly.

    Oracle on Open Systems

  • 5/17/2018 Oracle Databases on EMC VMAX.pdf

    29/511

    Oracle overview 29

    Oracle on Open Systems

    Additional database processes may be started depending on thesystem configuration. Some of these processes include RECO,QMNn, Jnnn, and MMON.

    Finally, the database files are the physical structures that store data ondisk. These files can be created within a file system, as raw partitionsor in Oracle Automatic Storage Management (ASM). Oracle usesdatabase files to maintain the logical structures within the databaseand store data. These logical structures include tablespaces,segments, extents, and data blocks. Database files commonly includedata, control, temp, redo log and archive log files.

    Oracle data elements

    Oracle maintains a set of database elements critical to the operation ofthe Oracle subsystem. These database elements consist of bothphysical and logical data elements.

    Physical data elements

    The required physical data elements include datafile(s) for the Oracle

    SYSTEM tablespace, control files, redo logs, and other miscellaneousdatabase files (the parameter file, alert and trace logs, backup files,and so on). Other physical elements such as the archive logs and

    SMON(SystemMonitor)

    Performs recovery at instance startup. It coalesces free extents in thedatafiles and cleans up temporary segments of failed user processes.

    PMON(ProcessMonitor)

    Cleans up after a user process fails. The process frees up resourcesincluding database locks and the blocks in the buffer cache of the failedprocess.

    Snnn (Serverprocesses)

    Connects user processes to the database instance. Server processes caneither be dedicated or shared, depending on user requirements and theamount of host memory available.

    Table 1 Oracle background processes (page 2 of 2)

    Process Description

    Oracle on Open Systems

  • 5/17/2018 Oracle Databases on EMC VMAX.pdf

    30/511

    30 Oracle Databases on EMC Symmetrix Storage Systems

    p y

    additional tablespaces for data are also typically configured. Aminimal configuration is shown in Figure 2, followed by a

    description of each data structure.

    Figure 2 Physical data elements in an Oracle configuration

    The Oracle SYSTEM and SYSAUX tablespaces consist of the datadictionary, PL/SQL program units, and other database objects suchas users, tablespaces, tables, indexes, performance information, andso on. These tablespaces are the only ones required, although inpractice, other tablespaces containing user data are typically created.

    Every database has one or more physical data files. A data file isassociated with just one tablespace. Data in a data file is read duringnormal database operations and stored in the database buffer cache.Modified or new data is not written to the data file immediately.Instead, The DB Writer background process periodically refreshes thedata files from the buffer cache.

    The Oracle control files consist of one or more configuration files (thecontrol file is typically multiplexed onto separate physical spindles)that contain the name of the database, the name and location of alldatabase datafiles and redo logs, redo and archive log historyinformation, checkpoint information, and other information neededat system startup and while the database is running.

    REDO2

    REDO1

    ARCH 14

    ARCH 15

    ARCH 16

    SYSTEM

    CNTL 1

    CNTL 2

    CNTL 2

    Binaries

    ICO-IMG-000502

    Oracle on Open Systems

  • 5/17/2018 Oracle Databases on EMC VMAX.pdf

    31/511

    Oracle overview 31

    Oracle redo logs contain data and undo changes. All changes to thedatabase are written to the redo logs, unless logging of allowed

    database objects, such as user tables, is explicitly disabled. Two ormore redo logs are configured, and normally the logs are multiplexedto prevent data loss in the event that database recovery is required.

    Archive logs are offloaded copies of the redo logs and are normallyrequired for recovering an Oracle database. Archive logs can bemultiplexed, both locally and remotely.

    Oracle binaries are the executables and libraries used to initiate the

    Oracle instance. Along with the binaries, Oracle uses many otherfiles to manage and monitor the database. These files include theinitialization parameter file (init.ora), server parameter file(SPFILE), alert log, and trace files.

    Logical data elements

    Datafiles are the primary physical data element. Oracle tablespacesare the logical element configured on top of the datafiles. Oracle

    tablespaces are used as containers to hold the customer's information.Each tablespace is built on one or more of the datafiles.

    Tablespaces are the containers for the underlying Oracle logical dataelements. These logical elements include data blocks, extents, andsegments. Data blocks are the smallest logical elements configurableat the database level. Data blocks are grouped into extents that arethen allocated to segments. Types of segments include data, index,

    temporary and undo.

    Oracle on Open Systems

  • 5/17/2018 Oracle Databases on EMC VMAX.pdf

    32/511

    32 Oracle Databases on EMC Symmetrix Storage Systems

    Figure 3shows the relationship between the data blocks, extents, andsegments.

    Figure 3 Relationship between data blocks, extents, and segments

    DataBlocks(16 KB)

    Extent(960 KB)

    Extent(960 KB)

    Segment

    (1920 KB)

    ICO-IMG-000-503

    Oracle on Open Systems

  • 5/17/2018 Oracle Databases on EMC VMAX.pdf

    33/511

    Storage management 33

    Storage management

    Standard Oracle backup/restore, disaster recovery, and cloningmethods can be difficult to manage and time-consuming. EMC

    Symmetrixprovides many alternatives or solutions that make theseoperations easy to manage, fast, and very scalable. In addition, EMCdeveloped many best practices that increase Oracle performance andhigh availability when using Symmetrix storage arrays.

    Oracle on Open Systems

  • 5/17/2018 Oracle Databases on EMC VMAX.pdf

    34/511

    34 Oracle Databases on EMC Symmetrix Storage Systems

    Cloning Oracle objects or environments

    EMC technology enables creation of an instant point-in-time copy ofan Oracle database system. The cloned copy is an identicalenvironment to its source, and can be used for other processingpurposes such as backup, recovery, offline reporting, and testing.

    Transportable tablespaces are an alternative to cloning an entireOracle database. Through Oracle's transportable tablespaces, it ispossible to clone an individual tablespace or all user tablespaces and

    present them to a different Oracle database environment. Clonecreation is facilitated through the use of EMC products such asTimeFinder, SRDF, Open Replicator, and others.

    In addition, Oracle also may clone or replicate individual databaseobjects, such as tables, in a variety of ways. Methods includetrigger-based mechanisms such as snapshots, Oracle's AdvancedReplication, Streams, message queues and Oracle Data Guard.

    Oracle on Open Systems

  • 5/17/2018 Oracle Databases on EMC VMAX.pdf

    35/511

    Backup and recovery 35

    Backup and recovery

    Backup and recovery operations using Oracle utilities typicallyrequire intervention by experienced personnel and can be both labor-and resource-intensive in large Oracle environments. Recovery oflarge Oracle instances, such as in SAP or PeopleSoft environments,are especially complex because the entire system is basically a largereferential set. All data in the set, including the database andassociated application files, needs to be recovered together and to the

    same recovery point.Dynamic manipulation of objects and application-maintainedreferential integrity further complicates recovery efforts. TraditionalOracle recovery techniques require multiple passes of the data, whichcan greatly impact recovery times. Such techniques are generallyunworkable in large Oracle environments due to the time required torecover all objects. EMC hardware and software are used to make theprocess faster and more effective.

    In addition to traditional backup and recovery operations, Oracleprovides the Recovery Manager (RMAN) utility. RMAN provides awide range of backup and recovery procedures through either acommand line interface on a client host or a GUI interface inEnterprise Manager. RMAN performs backup or recovery operations

    by integrating with sessions running on the target database host.Remote procedure calls (RPCs) to specialized packages stored in the

    target database are then made that execute in the backup or recoveryof the database. RMAN also may be configured as a repository forhistorical backup information that supplements records written bythe utility into the database control file. EMC has worked with Oracleengineering to closely integrate RMAN with products such asTimeFinder to offload backup operations from production andreduce recovery time.

    Oracle on Open Systems

  • 5/17/2018 Oracle Databases on EMC VMAX.pdf

    36/511

    36 Oracle Databases on EMC Symmetrix Storage Systems

    Oracle Real Application Clusters

    Typically, Oracle is configured with a single instance that attaches to asingle database. However, Oracle can be configured with multiplehost instances connecting to a single database. This configuration,which originally was called Oracle Parallel Server (OPS) in Oracleversions prior to release Oracle9i, is now known as Oracle RealApplication Clusters (RAC). Implementations of Oracle RAC areconfigured to enhance performance, scalability, and availability over

    a stand-alone Oracle database.An Oracle RAC environment consists of multiple Oracle instancesthat share access to a single Oracle database. Each instance containsits own memory structures and processes. In addition, each instancecontains its own set of redo logs and undo segments. Each instanceshares access to the datafiles making up the database. Since all hostsmust have access to all database files, concurrent access to the datafiles through the use of cluster manager is required. This also permitsone host to assume control of all datafiles in the event of an instancefailure requiring recovery.

    Performance and scalability are enhanced in an Oracle environmentbecause host-based resource limitations such as CPU and memoryconstraints are overcome by permitting two or more host instances toattach to the same database. For example, in a homogeneous hostenvironment, near-linear scaling of host resources is achieved by

    employing Oracle RAC. Additionally, because multiple hosts areconfigured with access to the database, availability is increased. Inthe event of a failure to one host or database instance, userconnections are failed over to the surviving cluster membersensuring continuous operations.

    Figure 4 on page 37shows a typical Oracle RAC configuration withtwo member nodes. Each member of the group has its own SGA,redo logs, and undo space. Though not shown here, each memberalso has its own set of initialization and parameter files. Concurrentaccess to each data file is managed through the cluster managementsoftware. Locking and inter-instance management arecommunicated through a network interconnect between the RACnodes.

    Oracle on Open Systems

  • 5/17/2018 Oracle Databases on EMC VMAX.pdf

    37/511

    Oracle Real Application Clusters 37

    Figure 4 Oracle two-node RAC configuration

    High bandwidth, low latency interconnect

    RAC Node 1

    SGA

    Binaries

    RAC Node 1

    SGA

    Binaries

    Shared storage

    SYSTEM DATA INDEX

    REDO

    1

    UNDO

    1

    Node 1 files

    REDO

    2

    UNDO

    2

    Node 2 files

    ICO-IMG-000504

    Oracle on Open Systems

  • 5/17/2018 Oracle Databases on EMC VMAX.pdf

    38/511

    38 Oracle Databases on EMC Symmetrix Storage Systems

    Optimizing Oracle layouts on EMC Symmetrix

    A primary concern for DBAs and system administrators whenconfiguring an Oracle databases on an EMC Symmetrix VMAXandDMXis the appropriate data layout on the storage. Maximizingperformance, availability, and recoverability of the database requiresa thorough understanding of the I/O characteristics, uptimerequirements, backup, and cloning needs. Careful consideration andplanning of the back-end configuration, including RAID, physical

    spindles, number of front-end directors and HBAs, as well as layoutof the database on the back-end of the Symmetrix array is necessary.These considerations ensure the database implementationsuccessfully meets all business requirements.

    Oracle on Open Systems

  • 5/17/2018 Oracle Databases on EMC VMAX.pdf

    39/511

    EMC and Oracle integration 39

    EMC and Oracle integration

    The EMC/Oracle partnership was established in 1995 and continuesto the present. Through joint engineering efforts, certification testing,collaborative solution offerings, and the Joint Services Center, EMCand Oracle maintain strong ties to ensure successful productintegration for customers' mission-critical database systems.

    Install baseWith more than 55,000 mutual customers, EMC and Oracle arerecognized as the leaders in automated networked storage andenterprise software, respectively. The EMC Symmetrix VMAX andDMX offer the highest levels of performance, scalability andavailability along with industry-leading software for successfullymanaging and maintaining complex Oracle database environments.In addition, EMC IT has one of the largest deployments of Oracle

    Applications in the world, with over 35,000 named users and over3,500 concurrent users at peak periods. Also Oracle IT uses bothCLARiiONand Symmetrix extensively.

    Joint engineering

    Engineers for EMC and Oracle continue to work together to develop

    integrated solutions, document best practices, and ensureinteroperability for customers deploying Oracle databases in EMCSymmetrix VMAX and DMX storage environments. Key EMCtechnologies such as TimeFinder and SRDF have been certifiedthrough Oracle's Storage Certification Program (OSCP). As Oraclephased out OSCP based on the maturity of the technology,Engineering efforts continue between the two companies to ensuresuccessful integration between each company's products. With each

    major technology or new product line EMC briefs Oracle Engineeringabout the technology changes and together they review bestpractices. EMC publishes many of the technology and deployment

    best practices as joint logo papers with the presence of the Oracle logoshowing the strong communication and relationship between thecompanies.

    Oracle on Open Systems

  • 5/17/2018 Oracle Databases on EMC VMAX.pdf

    40/511

    40 Oracle Databases on EMC Symmetrix Storage Systems

    Joint Services Center

    EMC and Oracle maintain a Joint Services Center to handle specificcustomer questions and issues relating to the database in EMCSymmetrix VMAX and DMX environments. When level 1 techsupport from either company requires assistance with jointEMC/Oracle-related issues, calls are automatically escalated to thisservice center. Based in Hopkinton, Mass., this Service Centerprovides answers to EMC- and Oracle-related questions from leadingsupport specialists trained in both database and storage platforms.

    2

  • 5/17/2018 Oracle Databases on EMC VMAX.pdf

    41/511

    EMC Foundation Products 41

    This chapter introduces the EMC foundation products discussed inthis document that work in combined Symmetrix and Oracleenvironments:

    Introduction ........................................................................................ 42 Symmetrix hardware and EMC Enginuity features...................... 45 EMC Solutions Enabler base management .................................... 49 EMC Change Tracker......................................................................... 52 EMC Symmetrix Remote Data Facility ........................................... 53 EMC TimeFinder................................................................................ 68 EMC Storage Resource Management.............................................. 81 EMC Storage Viewer.......................................................................... 86 EMC PowerPath................................................................................. 88 EMC Replication Manager................................................................ 97 EMC Open Replicator ....................................................................... 99 EMC Virtual Provisioning............................................................... 100 EMC Virtual LUN migration.......................................................... 103 EMC Fully Automated Storage Tiering (FAST)........................... 106

    EMC FoundationProducts

    EMC Foundation Products

  • 5/17/2018 Oracle Databases on EMC VMAX.pdf

    42/511

    42 Oracle Databases on EMC Symmetrix Storage Systems

    Introduction

    EMC provides many hardware and software products that supportOracle environments on Symmetrix systems. This chapter provides atechnical overview of the EMC products referenced in this document.The following products, which are highlighted and discussed, wereused and/or tested with VMware Infrastructure deployed on EMCSymmetrix.

    EMC offers an extensive product line of high-end storage solutions

    targeted to meet the requirements of mission-critical databases andapplications. The Symmetrix product line includes the DMX DirectMatrix Architectureseries and the VMAX Virtual Matrixseries.EMC Symmetrix is a fully redundant, high-availability storageprocessor, providing nondisruptive component replacements andcode upgrades. The Symmetrix system features high levels ofperformance, data integrity, reliability, and availability.

    EMC Enginuity

    Operating Environment Enginuity enablesinteroperation between the latest Symmetrix platforms and previousgenerations of Symmetrix systems and enables them to connect to alarge number of server types, operating systems and storage softwareproducts, and a broad selection of network connectivity elements andother devices, ranging from HBAs and drivers to switches and tapesystems.

    EMC Solutions Enabler Solutions Enabler is a package that

    contains the SYMAPI runtime libraries and the SYMCLI commandline interface. SYMAPI provides the interface to the EMC Enginuityoperating environment. SYMCLI is a set of commands that can beinvoked from the command line or within scripts. These commandscan be used to monitor device configuration and status, and toperform control operations on devices and data objects within astorage complex.

    EMC Symmetrix Remote Data Facility (SRDF) SRDF is abusiness continuity software solution that replicates and maintains amirror image of data at the storage block level in a remote Symmetrixsystem. The SRDF component extends the basic SYMCLI commandset of Solutions Enabler to include commands that specificallymanage SRDF.

    EMC Foundation Products

  • 5/17/2018 Oracle Databases on EMC VMAX.pdf

    43/511

    Introduction 43

    EMC SRDF consistency groups An SRDF consistency group is acollection of related Symmetrix devices that are configured to act in

    unison to maintain data integrity. The devices in consistency groupscan be spread across multiple Symmetrix systems.

    EMC TimeFinder TimeFinder is a family of products that enableLUN-based replication within a single Symmetrix system. Data iscopied from Symmetrix devices using array-based resources withoutusing host CPU or I/O. The source Symmetrix devices remain onlinefor regular I/O operations while the copies are created. TheTimeFinder family has three separate and distinct software products,TimeFinder/Mirror, TimeFinder/Clone, and TimeFinder/Snap:

    TimeFinder/Mirror enables users to configure special devices,called business continuance volumes (BCVs), to create amirror image of Symmetrix standard devices. Using BCVs,TimeFinder creates a point-in-time copy of data that can berepurposed. The TimeFinder/Mirror component extends the

    basic SYMCLI command set of Solutions Enabler to include

    commands that specifically manage Symmetrix BCVs andstandard devices.

    TimeFinder/Cloneenables users to make copies of datasimultaneously on multiple target devices from a single sourcedevice. The data is available to a targets host immediatelyupon activation, even if the copy process has not completed.Data may be copied from a single source device to as many as16 target devices. A source device can be either a Symmetrixstandard device or a TimeFinder BCV device.

    TimeFinder/Snapenables users to configure special devices inthe Symmetrix array called virtual devices (VDEVs) and savearea devices (SAVDEVs). These devices can be used to makepointer-based, space-saving copies of data simultaneously onmultiple target devices from a single source device. The data isavailable to a targets host immediately upon activation. Data

    may be copied from a single source device to as many as 128VDEVs. A source device can be either a Symmetrix standarddevice or a TimeFinder BCV device. A target device is a VDEV.A SAVDEV is a special device without a host address that isused to hold the changing contents of the source or targetdevice.

    EMC Foundation Products

  • 5/17/2018 Oracle Databases on EMC VMAX.pdf

    44/511

    44 Oracle Databases on EMC Symmetrix Storage Systems

    EMC Change Tracker EMC Symmetrix Change Tracker softwaremeasures changes to data on a Symmetrix volume or group of

    volumes. Change Tracker software is often used as a planning tool inthe analysis and design of configurations that use the EMCTimeFinder or SRDF components to store data at remote sites.

    Solutions Enabler Storage Resource Management (SRM)component The SRM component extends the basic SYMCLIcommand set of Solutions Enabler to include commands that allowusers to systematically find and examine attributes of various objectson the host, within a specified relational database, or in the EMCenterprise storage. The SRM commands provide mapping supportfor relational databases, file systems, logical volumes and volumegroups, as well as performance statistics.

    EMC PowerPath PowerPath is host-based software that providesI/O path management. PowerPath operates with several storagesystems, on several enterprise operating systems and providesfailover and load balancing transparent to the host application and

    database.

    EMC Foundation Products

  • 5/17/2018 Oracle Databases on EMC VMAX.pdf

    45/511

    Symmetrix hardware and EMC Enginuity features 45

    Symmetrix hardware and EMC Enginuity features

    Symmetrix hardware architecture and the EMC Enginuity operatingenvironment are the foundation for the Symmetrix storage platform.This environment consists of the following components:

    Symmetrix hardware

    Enginuity-based operating functions

    Solutions Enabler

    Symmetrix application program interface (API) for mainframe

    Symmetrix-based applications

    Host-based Symmetrix applications

    Independent software vendor (ISV) applications

    All Symmetrix systems provide advanced data replicationcapabilities, full mainframe and open systems support, and flexible

    connectivity options, including Fibre Channel, FICON, ESCON,Gigabit Ethernet, and iSCSI.

    Interoperability between Symmetrix storage systems enablescustomers to migrate storage solutions from one generation to thenext, protecting their investment even as their storage demandsexpand.

    Symmetrix enhanced cache director technology allows configurations

    of up to 512 GB of cache. The cache can be logically divided into 32independent regions providing up to 32 concurrent 500 MB/stransaction throughput.

    The Symmetrix on-board data integrity features include:

    Continuous cache and on-disk data integrity checking and errordetection/correction

    Fault isolation Nondisruptive hardware and software upgrades

    Automatic diagnostics and phone-home capabilities

    At the software level, advanced integrity features ensure informationis always protected and available. By choosing a mix of RAID 1(mirroring), RAID 1/0, high performance RAID 5 (3+1 and 7+1)protection and RAID 6, users have the flexibility to choose the

    EMC Foundation Products

  • 5/17/2018 Oracle Databases on EMC VMAX.pdf

    46/511

    46 Oracle Databases on EMC Symmetrix Storage Systems

    protection level most appropriate to the value and performancerequirements