RAC Presentation Oracle10gR2 (1)

7/27/2019 RAC Presentation Oracle10gR2 (1)

1/103

Real ApplicationReal Application

Cluster (RAC)Cluster (RAC)

Kishore AKishore A


2/103

Oracle10g - RACOracle10g - RAC What is all the hype about grid computing?What is all the hype about grid computing?

Grid computing is intended to allow businessesGrid computing is intended to allow businesses

to move away from the idea of many individualto move away from the idea of many individualservers, each of which is dedicated to a smallservers, each of which is dedicated to a smallnumber of applications. When configured in thisnumber of applications. When configured in thismanner, applications often either do not fullymanner, applications often either do not fully

utilize the servers available hardware resourceutilize the servers available hardware resourcesuch as memory, CPU and disk or short of thesesuch as memory, CPU and disk or short of theseresources during peak usage.resources during peak usage.

Grid computing addresses these problems byGrid computing addresses these problems by

providing an adaptive software infrastructureproviding an adaptive software infrastructurethat makes efficient use of low-cost servers andthat makes efficient use of low-cost servers andmodular storage, which balances work- loadsmodular storage, which balances work- loadsmore effectively and provides capacity onmore effectively and provides capacity on

demanddemand B scalin out with small servers in smallB scalin out with small servers in small


3/103

WHAT IS ENTERPRISE GRID COMPUTING?WHAT IS ENTERPRISE GRID COMPUTING?

Implement One from Many.Implement One from Many.

Grid computing coordinates the use of clustersGrid computing coordinates the use of clusters

of machines to create a single logical entity,of machines to create a single logical entity,such as a database or ansuch as a database or an

application server.application server.

By distributing work across many servers, gridBy distributing work across many servers, grid

computing exhibits benefits of availability,computing exhibits benefits of availability,scalability, and performance using low-costscalability, and performance using low-costcomponents.components.

Because a single logical entity is implementedBecause a single logical entity is implemented

across many machines, companies can add oracross many machines, companies can add orremove capacity in small increments, online.remove capacity in small increments, online.

With the capability to add capacity on demandWith the capability to add capacity on demandto a particular function, companies get moreto a particular function, companies get moreflexibility for adapting to peak loads, thusflexibility for adapting to peak loads, thus


4/103

Benefits of Enterprise Grid ComputingBenefits of Enterprise Grid Computing

The primary benefit of grid computing toThe primary benefit of grid computing tobusinesses is achieving high quality of servicebusinesses is achieving high quality of serviceand flexibility at lower cost.and flexibility at lower cost.

Enterprise grid computing lowers costs by:Enterprise grid computing lowers costs by:

Increasing hardware utilization and resourceIncreasing hardware utilization and resourcesharingsharing

Enabling companies to scale out incrementallyEnabling companies to scale out incrementallywith low-with low-

cost componentscost components

Reducing management and administrationReducing management and administrationrequirementsrequirements


5/103

New Trends in HardwareNew Trends in Hardware

Much of what makes grid computing possible today areMuch of what makes grid computing possible today arethe innovations in hardware. For example,the innovations in hardware. For example,

ProcessorProcessors. New low-cost, high volume Intel Itanium 2,s. New low-cost, high volume Intel Itanium 2,Sun SPARC, and IBM PowerPC 64-bit processors nowSun SPARC, and IBM PowerPC 64-bit processors nowdeliver performance equal to or better than exoticdeliver performance equal to or better than exoticprocessors used in high-end SMP servers.processors used in high-end SMP servers.

Blade servers.Blade servers. Blade server technology reduces theBlade server technology reduces the

cost of hardwarecost of hardwareand increases the density of servers, which furtherand increases the density of servers, which further

reduces expensivereduces expensive

data center real estate requirements.data center real estate requirements.

Networked storagNetworked storage. Disk storage costs continue toe. Disk storage costs continue toplummet evenplummet even

faster than processor costs. Network storagefaster than processor costs. Network storagetechnologies such astechnologies such as

Network Attached Storage (NAS) and Storage AreaNetwork Attached Storage (NAS) and Storage Area

Networks (SANs) further reduce these costs by enablingNetworks (SANs) further reduce these costs by enabling


6/103

Oracle Database 10Oracle Database 10gg

Oracle Database 10Oracle Database 10gg builds on thebuilds on the

success of Oracle9success of Oracle9ii

Database, and addsDatabase, and addsmany new grid-specific capabilities.many new grid-specific capabilities.

Oracle Database 10Oracle Database 10gg is based on Realis based on RealApplication Clusters, introduced inApplication Clusters, introduced inOracle9Oracle9ii..

There are more than 500 productionThere are more than 500 productioncustomers running Oracles clusteringcustomers running Oracles clusteringtechnology, helping to prove the validitytechnology, helping to prove the validityof Oracles grid infrastructure.of Oracles grid infrastructure.


7/103

Real Application ClustersReal Application Clusters

Oracle Real Application Clusters enables aOracle Real Application Clusters enables asingle database to run across multiple clusteredsingle database to run across multiple clustered

nodes in a grid, pooling the processingnodes in a grid, pooling the processingresources of several standard machines.resources of several standard machines.

In Oracle 10In Oracle 10gg, the database can immediately, the database can immediatelybegin balancing workload across a new nodebegin balancing workload across a new node

with new processing capacity as it gets re-with new processing capacity as it gets re-provisioned from one database to another, andprovisioned from one database to another, andcan relinquish a machine when it is no longercan relinquish a machine when it is no longerneeded--needed--

this is capacity on demand. Other databasesthis is capacity on demand. Other databasescannot grow and shrink while running and,cannot grow and shrink while running and,therefore, cannot utilize hardware as efficiently.therefore, cannot utilize hardware as efficiently.

Servers can be easily added and dropped to anServers can be easily added and dropped to an

Oracle cluster with no downtime.Oracle cluster with no downtime.


8/103

Service

RAC 10g ArchitectureRAC 10g Architecturepublic network

Node1

Operating System

Oracle Clusterware

instance 1

ASM

VIP1

ListenerNode 2

Operating System

Oracle Clusterware

instance 2

ASM

VIP2

Listener

Service Node n

Operating System

Oracle Clusterware

instance n

ASM

VIPn

Listener

Service

//

Redo / Archive logs all instances

shared storage

Database / Control files

OCR and Voting Disks

Managed by ASM

RAW Devices


9/103

Under the CoversUnder the Covers

Redo Log Files

Node nNode 2

Data Files and Control Files

Redo Log Files Redo Log Files

DictionaryCache

Log buffer

LCK0 LGWR DBW0

SMON PMON

LibraryCache

Global Resource Directory

LMS0

Instance 2

SGA

Instance n

Cluster Private High Speed Network

Buffer Cache

LMON LMD0 DIAG

DictionaryCache

Log buffer

LCK0 LGWR DBW0

SMON PMON

LibraryCache


LMS0

Buffer Cache

LMON LMD0 DIAG

DictionaryCache

Log buffer

LCK0 LGWR DBW0

SMON PMON

LibraryCache


LMS0

Buffer Cache

LMON LMD0 DIAG

Instance 1

Node 1

SGA SGA


10/103

Global ResourceGlobal Resource

DirectoryDirectory RAC Database System has two important services. They are GlobalRAC Database System has two important services. They are Global

Cache Service (GCS) and Global Enqueue Service (GES). These areCache Service (GCS) and Global Enqueue Service (GES). These arebasically collections of background processes. These twobasically collections of background processes. These twoprocesses together cover and manage the total Cache Fusionprocesses together cover and manage the total Cache Fusionprocess, resource transfers, and resource escalations among theprocess, resource transfers, and resource escalations among theinstances.instances.

Global Resource DirectoryGlobal Resource Directory GES and GCS together maintain a Global Resource DirectoryGES and GCS together maintain a Global Resource Directory

(GRD) to record the information about the resources and the(GRD) to record the information about the resources and theenqueues. GRD remains in the memory and is stored on all theenqueues. GRD remains in the memory and is stored on all theinstances. Each instance manages a portion of the directory. Thisinstances. Each instance manages a portion of the directory. Thisdistributed nature is a key point for fault tolerance of the RAC.distributed nature is a key point for fault tolerance of the RAC.

Global Resource Directory (GRD) is the internal database thatGlobal Resource Directory (GRD) is the internal database thatrecords and stores the current status of the data blocks.records and stores the current status of the data blocks.Whenever a block is transferred out of a local cache to anotherWhenever a block is transferred out of a local cache to another

instances cache the GRD is updated. The following resourcesinstances cache the GRD is updated. The following resourcesinformation is available in GRD.information is available in GRD.

* Data Block Identifiers (DBA)* Data Block Identifiers (DBA)* Location of most current version* Location of most current version* Modes of the data blocks: (N)Null, (S)Shared, (X)Exclusive* Modes of the data blocks: (N)Null, (S)Shared, (X)Exclusive

* The Roles of the data blocks (local or global) held by each* The Roles of the data blocks (local or global) held by each

instanceinstance* Buffer caches on multiple nodes in the cluster* Buffer caches on multiple nodes in the cluster


11/103

Background Processes in aBackground Processes in a

RAC instanceRAC instance

Select name,description fromSelect name,description fromv$bgprocess where paddr 00v$bgprocess where paddr 00

The one specific to a RAC instanceThe one specific to a RAC instanceare the DIAG, LCK, LMON, LMNDnare the DIAG, LCK, LMON, LMNDnand LMSn process.and LMSn process.


12/103

DIAG : DiagnosabilityDIAG : Diagnosability

DaemonDaemon The diagnosability daemon is responsibleThe diagnosability daemon is responsible

for capturing information on processfor capturing information on processfailures in a RAC environment, and writingfailures in a RAC environment, and writing

out trace information for failure analysis.out trace information for failure analysis. The information produced by DIAG is mostThe information produced by DIAG is most

useful when working in conjunction withuseful when working in conjunction withOracle Support to troubleshoot causes forOracle Support to troubleshoot causes for

a failure.a failure. Only a single DIAG process is needed forOnly a single DIAG process is needed for

each instanceeach instance


13/103

LCK: Lock ProcessLCK: Lock Process

The lock process (LCK) managesThe lock process (LCK) managesrequests that are not cache-fusionrequests that are not cache-fusionrequests, such as row cache requestsrequests, such as row cache requests

and library cache requestsand library cache requests Only a single LCK process is allowedOnly a single LCK process is allowed

for each instance.for each instance.

LCK maintains a list of lock elementsLCK maintains a list of lock elementsand uses this list to validate locksand uses this list to validate locksduring instance recoveryduring instance recovery


14/103

LMD:Lock Manager DaemonLMD:Lock Manager Daemon

ProcessProcess

The global enqueue service daemonThe global enqueue service daemon(LMD) is a lock agent process that(LMD) is a lock agent process thatcoordinates enqueue managercoordinates enqueue manager

service requests. The requests areservice requests. The requests arefor global cache service enqueuesfor global cache service enqueuesthat control access to globalthat control access to global

enqueues and resources.enqueues and resources.The LMD process also handlesThe LMD process also handles

deadlock detection and remotedeadlock detection and remote

enqueue requests.enqueue requests.


15/103

LMON: Lock MonitorLMON: Lock Monitor

ProcessProcess LMON is the global enqueue serviceLMON is the global enqueue service

monitor. It is responsible for themonitor. It is responsible for thereconfiguration of lock resources when anreconfiguration of lock resources when aninstance joins the cluster or leaves theinstance joins the cluster or leaves thecluster, and also is responsible for thecluster, and also is responsible for thedynamic lock remasteringdynamic lock remastering

LMON will generate a trace file whenever aLMON will generate a trace file whenever areconfiguration occurs (as opposed toreconfiguration occurs (as opposed toremastering of a subset of locks).remastering of a subset of locks).

It is the responsibility of LMON to check forIt is the responsibility of LMON to check forthe death of instances clusterwide, and tothe death of instances clusterwide, and to

initiate reconfiguration as quickly asinitiate reconfiguration as quickly aspossiblepossible


16/103

LMS: Lock Manager ServerLMS: Lock Manager Server

ProcessProcess The LMS process (or global cache serviceThe LMS process (or global cache service

process) is in charge of shipping the blocksprocess) is in charge of shipping the blocksbetween instances for cache-fusionbetween instances for cache-fusionrequests. In the event of a consistent-readrequests. In the event of a consistent-read

request, the LMS process will first roll therequest, the LMS process will first roll theblock back, creating the consistent readblock back, creating the consistent read(CR) image of the block, and will then ship(CR) image of the block, and will then shipthat version of the block across thethat version of the block across theinterconnect to the foreground processinterconnect to the foreground process

making the request at the remotemaking the request at the remoteinstance.instance.

In addition, LMS must interact with theIn addition, LMS must interact with theLMD process to retrieve lock requestsLMD process to retrieve lock requests

placed by LMD. An instance mayplaced by LMD. An instance mayd namicall enerate u to LMSdynamically generate up to LMS


17/103

Server Control UtilityServer Control Utility

To manage the RAC database and its instances, Oracle hasTo manage the RAC database and its instances, Oracle hasprovided a new utility called the Server Control Utilityprovided a new utility called the Server Control Utility(SRVCTL). This replaces the earlier utility (SRVCTL). This replaces the earlier utility opsctlopsctlwhich waswhich wasused in the parallel server.used in the parallel server.

The Server Control Utility is a single point of controlThe Server Control Utility is a single point of controlbetween the Oracle Intelligent agent and each node in thebetween the Oracle Intelligent agent and each node in theRAC system. The SRVCTL communicates with the globalRAC system. The SRVCTL communicates with the globaldaemon service (GSD) and resides on each of the nodes.daemon service (GSD) and resides on each of the nodes.

The SRVCTL gathers information from the database andThe SRVCTL gathers information from the database andinstances and acts as an intermediary between nodes andinstances and acts as an intermediary between nodes andthe Oracle Intelligent agent.the Oracle Intelligent agent.

When you use the SRVCTL to perform configurationWhen you use the SRVCTL to perform configurationoperations on your cluster, the SRVCTL stores configurationoperations on your cluster, the SRVCTL stores configurationdata in the Server Management (SRVM) configurationdata in the Server Management (SRVM) configurationrepository. The SRVM includes all the components ofrepository. The SRVM includes all the components ofEnterprise Manager such as the Intelligent Agent, theEnterprise Manager such as the Intelligent Agent, the

Server Control Utility (SRVCTL), and the Global ServicesServer Control Utility (SRVCTL), and the Global ServicesDaemon. Thus, the SRVCTL is one of the SRVM InstanceDaemon. Thus, the SRVCTL is one of the SRVM Instance


18/103

Server Control UtilityServer Control Utility

For the SRVCTL to function, the Global Services Daemon (GSD) should beFor the SRVCTL to function, the Global Services Daemon (GSD) should berunning on the node. The SRVCTL performs mainly two types ofrunning on the node. The SRVCTL performs mainly two types ofadministrative tasks: Cluster Database Tasks and Cluster Databaseadministrative tasks: Cluster Database Tasks and Cluster DatabaseConfiguration Tasks.Configuration Tasks.

SRVCTL Cluster Database tasks include:SRVCTL Cluster Database tasks include:

Starts and stops cluster databases. Starts and stops cluster databases.

Starts and stops cluster database instances. Starts and stops cluster database instances. Starts and stops listeners associated with a cluster database Starts and stops listeners associated with a cluster database

instance.instance. Obtains the status of a cluster database instance. Obtains the status of a cluster database instance. Obtains the status of listeners associated with a cluster database. Obtains the status of listeners associated with a cluster database.

SRVCTL Cluster Database Configuration tasks include:SRVCTL Cluster Database Configuration tasks include: Adds and deletes cluster database configuration information. Adds and deletes cluster database configuration information. Adds an instance to, or deletes an instance from a cluster database. Adds an instance to, or deletes an instance from a cluster database. Renames an instance name within a cluster database configuration. Renames an instance name within a cluster database configuration. Moves instances in a cluster database configuration. Moves instances in a cluster database configuration. Sets and unsets the environment variable for an instance in a Sets and unsets the environment variable for an instance in a

cluster databasecluster database

configuration.configuration. Sets and unsets the environment variable for an entire cluster in a Sets and unsets the environment variable for an entire cluster in a


19/103

RAW Partitions, Cluster FileRAW Partitions, Cluster File

System and Automatic StorageSystem and Automatic Storage

Management (ASM)Management (ASM) Raw Partitions are a set of unformatted devicesRaw Partitions are a set of unformatted devices

on a shared disk sub-system.A raw partition is aon a shared disk sub-system.A raw partition is adisk drive device that does not have a file systemdisk drive device that does not have a file systemset up. The raw partition is portion of the physicalset up. The raw partition is portion of the physical

disk that is accessed at the lowest possible level.disk that is accessed at the lowest possible level.The actual application that uses a raw device isThe actual application that uses a raw device isresponsible for managing its own I/O to the rawresponsible for managing its own I/O to the rawdevice with no operating system buffering.device with no operating system buffering.

Traditionally, they were required for OracleTraditionally, they were required for Oracle

Parallel Server (OPS) and they provided highParallel Server (OPS) and they provided highperformance by bypassing the file systemperformance by bypassing the file systemoverhead. Raw partitions were used in setting upoverhead. Raw partitions were used in setting updatabases for performance gains and for thedatabases for performance gains and for thepurpose of concurrent access by multiple nodespurpose of concurrent access by multiple nodes

in the cluster without system-level buffering.in the cluster without system-level buffering.


20/103

RAW Partitions, Cluster FileRAW Partitions, Cluster File

System and Automatic StorageSystem and Automatic Storage

Management (ASM)Management (ASM) Oracle 9i RAC and 10g now supportsOracle 9i RAC and 10g now supports

both the cluster file system and theboth the cluster file system and theraw devices to store the shared data.raw devices to store the shared data.

In addition, 10g RAC supports sharedIn addition, 10g RAC supports sharedstorage resources from ASMstorage resources from ASMinstance. You will be able to createinstance. You will be able to create

the data files out of the diskthe data files out of the diskresources located in the ASMresources located in the ASMinstance. The ASM resources areinstance. The ASM resources are

sharable and accessed by all thesharable and accessed by all the


21/103

RAW DevicesRAW Devices

Raw Devices have been in use for very long time.Raw Devices have been in use for very long time.They were the primary storage structures for dataThey were the primary storage structures for datafiles of the Oracle Parallel Server. They remain infiles of the Oracle Parallel Server. They remain inuse even in the RAC versions 9i and 10g. Rawuse even in the RAC versions 9i and 10g. RawDevices are difficult to manage and administer,Devices are difficult to manage and administer,but provide high performing shared storagebut provide high performing shared storage

structures. When you use the raw devices forstructures. When you use the raw devices fordata files, redo log files and control files, you maydata files, redo log files and control files, you mayhave to use the local file systems or some sort ofhave to use the local file systems or some sort ofnetwork attached file system for writing thenetwork attached file system for writing thearchive log files, handling the utl_file_dir files andarchive log files, handling the utl_file_dir files and

files supporting the external tables.files supporting the external tables. On Raw Devices On Local File SystemOn Raw Devices On Local File System

Data files Archive log filesData files Archive log files

Redo files Oracle Home filesRedo files Oracle Home filesControl files CRS Home filesControl files CRS Home files

Voting Disk Alert log, Trace filesVoting Disk Alert log, Trace files


22/103


AdvantagesAdvantages Raw partitions have several advantages:Raw partitions have several advantages: They are not subject to any operating system locking.They are not subject to any operating system locking.

The operating system buffer or cache is bypassed, givingThe operating system buffer or cache is bypassed, giving

performance gains and reduced memory consumption.performance gains and reduced memory consumption. Multiple systems can be easily shared.Multiple systems can be easily shared.

The application or database system has full control toThe application or database system has full control to

manipulate the internals of access.manipulate the internals of access.

Historically, the support for asynchronous I/O on UNIXHistorically, the support for asynchronous I/O on UNIXsystems was generally limited to raw partitionssystems was generally limited to raw partitions


23/103


Issues and DifficultiesIssues and Difficulties There are many administrative inconveniences andThere are many administrative inconveniences and

drawbacks such as:drawbacks such as: The unit of allocation to the database is the entire rawThe unit of allocation to the database is the entire raw

partition. We cannot use a raw partition for multiplepartition. We cannot use a raw partition for multiple

tablespaces. A raw partition is not the same as a file systemtablespaces. A raw partition is not the same as a file systemwhere we can create many files.where we can create many files.

Administrators have to create them with specific sizes.Administrators have to create them with specific sizes.When the databases grow in size, raw partitions cannot beWhen the databases grow in size, raw partitions cannot beextended. We need to add extra partitions to support theextended. We need to add extra partitions to support thegrowing tablespace. Sometimes we may have limitations ongrowing tablespace. Sometimes we may have limitations on

the total number of raw partitions we can use in thethe total number of raw partitions we can use in thesystem. Furthermore, there are no database operationssystem. Furthermore, there are no database operationsthat can occur on an individual data file. There is, therefore,that can occur on an individual data file. There is, therefore,no logical benefit from having a tablespace consisting ofno logical benefit from having a tablespace consisting ofmany data files except for those tablespaces that are largermany data files except for those tablespaces that are largerthan the maximum Oracle can support in a single file.than the maximum Oracle can support in a single file.

We cannot use the standard file manipulation commands onWe cannot use the standard file manipulation commands onthe raw partitions, and thus, on the data files. We cannotthe raw partitions, and thus, on the data files. We cannot


24/103


Raw partitions cannot be used for writing the archive logs.Raw partitions cannot be used for writing the archive logs.

Administrators need to keep track of the raw volumes withAdministrators need to keep track of the raw volumes withtheir cryptic naming conventions. However, by using thetheir cryptic naming conventions. However, by using thesymbolic links, we can reduce the hassles associated withsymbolic links, we can reduce the hassles associated withnames.names.

For example, a cryptic name like /dev/rdsk/c8t4d5s4 or aFor example, a cryptic name like /dev/rdsk/c8t4d5s4 or aname like /dev/sd/sd001 is an administrative challenge. Toname like /dev/sd/sd001 is an administrative challenge. Toalleviate this, administrators often rely on symbolic links toalleviate this, administrators often rely on symbolic links toprovide logical names that make sense. This, however,provide logical names that make sense. This, however,substitutes one complexity for another.substitutes one complexity for another.

In a clustered environment like Linux clusters, it is notIn a clustered environment like Linux clusters, it is not

guaranteed that the physical devices will have the sameguaranteed that the physical devices will have the samedevice names on different nodes or across reboots of adevice names on different nodes or across reboots of asingle node. To solve this problem, manual intervention issingle node. To solve this problem, manual intervention isneeded, which will increase administration overhead.needed, which will increase administration overhead.


25/103

Cluster File SystemCluster File System

CFS offers a very good shared storage facility forCFS offers a very good shared storage facility forbuilding the RAC database. CFS provides a sharedbuilding the RAC database. CFS provides a sharedfile system, which is mounted on all the clusterfile system, which is mounted on all the clusternodes simultaneously. When you implement thenodes simultaneously. When you implement the

RAC database with the commercial CFS productsRAC database with the commercial CFS productssuch as the Veritas CFS or PolyServe Matrixsuch as the Veritas CFS or PolyServe MatrixServer, you will able to store all kinds of databaseServer, you will able to store all kinds of databasefiles including the shared Oracle Home and CRSfiles including the shared Oracle Home and CRSHome.Home.

However, the capabilities of the CFS products areHowever, the capabilities of the CFS products arenot the same. For example, Oracle CFS (OCFS),not the same. For example, Oracle CFS (OCFS),used in case of Linux RAC implementations, hasused in case of Linux RAC implementations, haslimitations. It is not a general purpose file system.limitations. It is not a general purpose file system.It cannot be used for shared Oracle Home.It cannot be used for shared Oracle Home.


26/103

Cluster File SystemCluster File System On Cluster File SystemOn Cluster File System

Data files Archive Log filesData files Archive Log filesRedo files Oracle Home FilesRedo files Oracle Home FilesControl files Alert log,Trace filesControl files Alert log,Trace filesVoting Disk Files for External TablesVoting Disk Files for External Tables

OCR File utl_file_dir locationOCR File utl_file_dir location A cluster file system (CFS) is a file system that may be accessedA cluster file system (CFS) is a file system that may be accessed

(read and write) by all the members in the cluster at the same(read and write) by all the members in the cluster at the sametime. This implies that all the members of the cluster have thetime. This implies that all the members of the cluster have thesame view. Some of the popular and widely used cluster filesame view. Some of the popular and widely used cluster filesystem products for Oracle RAC include HP Tru64 CFS, Veritassystem products for Oracle RAC include HP Tru64 CFS, VeritasCFS, IBM GPFS, Polyserve Matrix Server, and Oracle Cluster FileCFS, IBM GPFS, Polyserve Matrix Server, and Oracle Cluster Filesystem. The cluster file system offers:system. The cluster file system offers:

Simple management.Simple management.

The use of Oracle Managed Files with RAC.The use of Oracle Managed Files with RAC.

A Single Oracle Software Installation.A Single Oracle Software Installation.

Auto-extend Enabled on Oracle Data Files.Auto-extend Enabled on Oracle Data Files.

Uniform accessibility of Archive Logs.Uniform accessibility of Archive Logs.


27/103

ASM Automatic StorageASM Automatic Storage

ManagementManagement ASM is the new star on the block. ASM provides a verticalASM is the new star on the block. ASM provides a vertical

integration of the file system and volume manager forintegration of the file system and volume manager forOracle database files. ASM has the capability to spreadOracle database files. ASM has the capability to spreaddatabase files across all available storage for optimaldatabase files across all available storage for optimalperformance and resource utilization. It enables simple andperformance and resource utilization. It enables simple andnon-intrusive resource allocation and provides automaticnon-intrusive resource allocation and provides automatic

rebalancingrebalancing

When you are using the ASM for building shared files, youWhen you are using the ASM for building shared files, youwould get almost the same performance as that of rawwould get almost the same performance as that of rawpartitions. The ASM controlled disk devices will be part ofpartitions. The ASM controlled disk devices will be part ofASM instance, which can be shared by the RAC databaseASM instance, which can be shared by the RAC database

instance. It is similar to the situation where raw devicesinstance. It is similar to the situation where raw devicessupporting the RAC database had to be shared by multiplesupporting the RAC database had to be shared by multiplenodes. The shared devices need to be presented to multiplenodes. The shared devices need to be presented to multiplenodes on the cluster and those devices will be input to thenodes on the cluster and those devices will be input to theASM instance. There will be an ASM instance supportingASM instance. There will be an ASM instance supportingeach RAC instance on the respective nodeeach RAC instance on the respective node


28/103

ASM Automatic StorageASM Automatic Storage

ManagementManagement From the ASM instance On local or CFSFrom the ASM instance On local or CFS

Data files Oracle Home FilesData files Oracle Home Files

Redo files CRS Home FilesRedo files CRS Home Files

Control files Alert log, trace filesControl files Alert log, trace files

Archive log files Files for externalArchive log files Files for externaltablestables

--------------------------- util_file_dir location--------------------------- util_file_dir location

Voting Disk and OCR fileVoting Disk and OCR file

Are located on raw partitionsAre located on raw partitions

ASM is for more Oracle specific data, redo log filesASM is for more Oracle specific data, redo log filesand archived log files.and archived log files.

Automatic Storage ManagementAutomatic Storage Management


29/103

Automatic Storage ManagementAutomatic Storage Management

Automatic Storage Management simplifies storageAutomatic Storage Management simplifies storagemanagement for Oracle Databases.management for Oracle Databases.

Instead of managing many database files, Oracle DBAsInstead of managing many database files, Oracle DBAs

manage only a small number of disk groups. A diskmanage only a small number of disk groups. A diskgroup is a set of disk devices that Oracle manages as agroup is a set of disk devices that Oracle manages as asingle, logical unit. An administrator can define asingle, logical unit. An administrator can define aparticular disk group as the default disk group for aparticular disk group as the default disk group for adatabase, and Oracle automatically allocates storagedatabase, and Oracle automatically allocates storagefor and creates or deletes the files associated with thefor and creates or deletes the files associated with thedatabase object.database object.

Automatic Storage Management also offers the benefitsAutomatic Storage Management also offers the benefitsof storage technologies such as RAID or Logical Volumeof storage technologies such as RAID or Logical Volume

Managers (LVMs). Oracle can balance I/O from multipleManagers (LVMs). Oracle can balance I/O from multipledatabases across all of the devices in a disk group, anddatabases across all of the devices in a disk group, andit implements striping and mirroring to improve I/Oit implements striping and mirroring to improve I/Operformance and data reliability. Because Automaticperformance and data reliability. Because AutomaticStorage Management is written to work exclusively withStorage Management is written to work exclusively with

Oracle, it achieves better performance than generalizedOracle, it achieves better performance than generalizedstora e virtualization solutions.

Shared Disk StorageShared Disk Storage


30/103

Shared Disk StorageShared Disk Storage

Oracle RAC relies on a shared disk architecture. TheOracle RAC relies on a shared disk architecture. Thedatabase files, online redo logs, and control files for thedatabase files, online redo logs, and control files for thedatabase must be accessible to each node in thedatabase must be accessible to each node in thecluster. The shared disks also store the Oracle Clustercluster. The shared disks also store the Oracle ClusterRegistry and Voting Disk. There are a variety of ways toRegistry and Voting Disk. There are a variety of ways toconfigure shared storage including direct attachedconfigure shared storage including direct attacheddisks (typically SCSI over copper or fiber), Storage Areadisks (typically SCSI over copper or fiber), Storage Area

Networks (SAN), and Network Attached Storage (NAS).Networks (SAN), and Network Attached Storage (NAS). Private NetworkPrivate Network

Each cluster node is connected to all other nodes via aEach cluster node is connected to all other nodes via aprivate high-speed network, also known as the clusterprivate high-speed network, also known as the cluster

interconnect or high-speed interconnect (HSI). Thisinterconnect or high-speed interconnect (HSI). Thisnetwork is used by Oracle's Cache Fusion technology tonetwork is used by Oracle's Cache Fusion technology toeffectively combine the physical memory (RAM) in eacheffectively combine the physical memory (RAM) in eachhost into a single cache. Oracle Cache Fusion allowshost into a single cache. Oracle Cache Fusion allowsdata stored in the cache of one Oracle instance to bedata stored in the cache of one Oracle instance to beaccessed by any other instance by transferring it acrossaccessed by any other instance by transferring it across

The private network is typically built withThe private network is typically built with


31/103

The private network is typically built withThe private network is typically built withGigabit Ethernet, but for high-volumeGigabit Ethernet, but for high-volumeenvironments, many vendors offer proprietaryenvironments, many vendors offer proprietary

low-latency, high-bandwidth solutionslow-latency, high-bandwidth solutionsspecifically designed for Oracle RAC. Linux alsospecifically designed for Oracle RAC. Linux alsooffers a means of bonding multiple physicaloffers a means of bonding multiple physicalNICs into a single virtual NIC to provideNICs into a single virtual NIC to provide

increased bandwidth and availability.increased bandwidth and availability. Public NetworkPublic Network

To maintain high availability, each cluster nodeTo maintain high availability, each cluster nodeis assigned a virtual IP address (VIP). In theis assigned a virtual IP address (VIP). In theevent of host failure, the failed node's IPevent of host failure, the failed node's IPaddress can be reassigned to a surviving nodeaddress can be reassigned to a surviving nodeto allow applications to continue accessing theto allow applications to continue accessing the

database through the same IP address.database through the same IP address.

It's all about availability of the application When aIt's all about availability of the application When a


32/103

It s all about availability of the application. When aIt s all about availability of the application. When anode fails, the VIP associated with it is supposed to benode fails, the VIP associated with it is supposed to beautomatically failed over to some other node. When thisautomatically failed over to some other node. When thisoccurs, two things happen.occurs, two things happen.

The new node re-arps the world indicating a new MACThe new node re-arps the world indicating a new MACaddress for the address. For directly connected clients,address for the address. For directly connected clients,this usually causes them to see errors on theirthis usually causes them to see errors on theirconnections to the old address.connections to the old address.

Subsequent packets sent to the VIP go to the newSubsequent packets sent to the VIP go to the newnode, which will send error RST packets back to thenode, which will send error RST packets back to theclients. This results in the clients getting errorsclients. This results in the clients getting errorsimmediately.immediately.

This means that when the client issues SQL to the nodeThis means that when the client issues SQL to the nodethat is now down, or traverses the address list whilethat is now down, or traverses the address list whileconnecting, rather than waiting on a very long TCP/IPconnecting, rather than waiting on a very long TCP/IPtime-out (~10 minutes), the client receives a TCP reset.time-out (~10 minutes), the client receives a TCP reset.

In the case of SQL, this is ORA-3113. In the case ofIn the case of SQL, this is ORA-3113. In the case ofconnect, the next address in tnsnames is used.connect, the next address in tnsnames is used.

TheThe Oracle CRSOracle CRS contains all the cluster andcontains all the cluster and


33/103

TheThe Oracle CRSOracle CRS contains all the cluster andcontains all the cluster anddatabase configuration metadata along withdatabase configuration metadata along withseveral system management features for RAC.several system management features for RAC.

It allows the DBA to register and invite anIt allows the DBA to register and invite anOracle instance (or instances) to the cluster.Oracle instance (or instances) to the cluster.During normal operation, CRS will sendDuring normal operation, CRS will sendmessages (via a special ping operation) to allmessages (via a special ping operation) to all

nodes configured in the clusteroften callednodes configured in the clusteroften calledthe "heartbeat." If the heartbeat fails for any ofthe "heartbeat." If the heartbeat fails for any ofthe nodes, it checks with the CRS configurationthe nodes, it checks with the CRS configurationfiles (on the shared disk) to distinguish betweenfiles (on the shared disk) to distinguish between

a real node failure and a network failure.a real node failure and a network failure.

CRS maintains two files: the Oracle ClusterCRS maintains two files: the Oracle ClusterRegistry (OCR) and the Voting Disk. The OCRRegistry (OCR) and the Voting Disk. The OCR

and the Voting Disk must reside on shared disksand the Voting Disk must reside on shared disks

TheThe Voting DiskVoting Disk is used by the Oracle cluster manageris used by the Oracle cluster manager


34/103

TheThe Voting DiskVoting Diskis used by the Oracle cluster manageris used by the Oracle cluster managerin various layers. The Cluster Manager and Nodein various layers. The Cluster Manager and NodeMonitor accepts registration of Oracle instances to theMonitor accepts registration of Oracle instances to thecluster and it sends ping messages to Cluster Managerscluster and it sends ping messages to Cluster Managers(Node Monitor) on other RAC nodes. If this heartbeat(Node Monitor) on other RAC nodes. If this heartbeat

fails, oracm uses a quorum file or a quorum partition onfails, oracm uses a quorum file or a quorum partition onthe shared disk to distinguish between a node failurethe shared disk to distinguish between a node failureand a network failure. So if a node stops sending pingand a network failure. So if a node stops sending pingmessages, but continues writing to the quorum file ormessages, but continues writing to the quorum file orpartition, then the other Cluster Managers canpartition, then the other Cluster Managers can

recognize it as a network failure. Hence the availabilityrecognize it as a network failure. Hence the availabilityfrom the Voting Disk is critical for the operation of thefrom the Voting Disk is critical for the operation of theOracle Cluster Manager.Oracle Cluster Manager.The shared volumes created for the OCR and the votingThe shared volumes created for the OCR and the votingdisk should be configured using RAID to protect againstdisk should be configured using RAID to protect againstmedia failure. This requires the use of an externalmedia failure. This requires the use of an externalcluster volume manager, cluster file system, or storagecluster volume manager, cluster file system, or storagehardware that provides RAID protection. .hardware that provides RAID protection. .

Oracle Cluster Registry (OCR)Oracle Cluster Registry (OCR) is used to store theis used to store thecluster configuration information among other things.cluster configuration information among other things.OCR needs to be accessible from all nodes in theOCR needs to be accessible from all nodes in the

Cache FusionCache Fusion


35/103

Cache FusionCache Fusion One of the bigger differences between Oracle RAC andOne of the bigger differences between Oracle RAC and

OPS is the presence of Cache Fusion technology. InOPS is the presence of Cache Fusion technology. InOPS, a request for data between nodes required theOPS, a request for data between nodes required thedata to be written to disk first, and then the requestingdata to be written to disk first, and then the requestingnode could read that data. In RAC, data is passed alongnode could read that data. In RAC, data is passed alongwith locks.with locks.

Every time an instance wants to update a block, it hasEvery time an instance wants to update a block, it hasto obtain a lock on it to make sure no other instance into obtain a lock on it to make sure no other instance in

the cluster is updating the samethe cluster is updating the sameblock. To resolve this problem, Oracle does a datablock. To resolve this problem, Oracle does a datablock ping mechanism that allows it to get the status ofblock ping mechanism that allows it to get the status ofthe specific block before reading it from the disk. Cachethe specific block before reading it from the disk. CacheFusion resolves data block read/read, read/write andFusion resolves data block read/read, read/write and

write/write conflicts among ORACLE database nodeswrite/write conflicts among ORACLE database nodesthrough high performance interconnect networks,through high performance interconnect networks,bypassing much slower physical disk operations used inbypassing much slower physical disk operations used inprevious releases. Using Oracle 9i RAC cache fusionprevious releases. Using Oracle 9i RAC cache fusionfeature, close to linear scalability of databasefeature, close to linear scalability of databaseperformance can be achieved when adding nodes toperformance can be achieved when adding nodes to

the cluster. ORACLE enables better Database capacitythe cluster. ORACLE enables better Database capacitylannin and conserves ca ital investments.


36/103

Build Your Own Oracle RAC 10Build Your Own Oracle RAC 10ggRelease 2 Cluster onRelease 2 Cluster onLinux and FireWireLinux and FireWire

ContentsContents IntroductionIntroduction Oracle RAC 10Oracle RAC 10ggOverviewOverview Shared-Storage OverviewShared-Storage Overview

FireWire TechnologyFireWire Technology Hardware & CostsHardware & Costs Install the Linux Operating SystemInstall the Linux Operating System Network ConfigurationNetwork Configuration Obtain & Install FireWire ModulesObtain & Install FireWire Modules Create "oracle" User and DirectoriesCreate "oracle" User and Directories Create Partitions on the Shared FireWire Storage DeviceCreate Partitions on the Shared FireWire Storage Device
http://www.oracle.com/technology/pub/articles/hunter_rac10gr2.html#Real%20Application%20Cluster%20(RAC)%20%20%20Kishore%20A%20http://www.oracle.com/technology/pub/articles/hunter_rac10gr2.html#Real%20Application%20Cluster%20(RAC)%20%20%20Kishore%20A%20http://www.oracle.com/technology/pub/articles/hunter_rac10gr2.html#Oracle10g%20-%20RAChttp://www.oracle.com/technology/pub/articles/hunter_rac10gr2.html#Oracle10g%20-%20RAChttp://www.oracle.com/technology/pub/articles/hunter_rac10gr2.html#Oracle10g%20-%20RAChttp://www.oracle.com/technology/pub/articles/hunter_rac10gr2.html#Oracle10g%20-%20RAChttp://www.oracle.com/technology/pub/articles/hunter_rac10gr2.html#Oracle10g%20-%20RAChttp://www.oracle.com/technology/pub/articles/hunter_rac10gr2.html#Oracle10g%20-%20RAChttp://www.oracle.com/technology/pub/articles/hunter_rac10gr2.html#Slide%203http://www.oracle.com/technology/pub/articles/hunter_rac10gr2.html#Slide%203http://www.oracle.com/technology/pub/articles/hunter_rac10gr2.html#Slide%204http://www.oracle.com/technology/pub/articles/hunter_rac10gr2.html#Slide%204http://www.oracle.com/technology/pub/articles/hunter_rac10gr2.html#Slide%205http://www.oracle.com/technology/pub/articles/hunter_rac10gr2.html#Slide%205http://www.oracle.com/technology/pub/articles/hunter_rac10gr2.html#Slide%206http://www.oracle.com/technology/pub/articles/hunter_rac10gr2.html#Slide%206http://www.oracle.com/technology/pub/articles/hunter_rac10gr2.html#Slide%207http://www.oracle.com/technology/pub/articles/hunter_rac10gr2.html#Slide%207http://www.oracle.com/technology/pub/articles/hunter_rac10gr2.html#RAC%2010g%20Architecture%20http://www.oracle.com/technology/pub/articles/hunter_rac10gr2.html#RAC%2010g%20Architecture%20http://www.oracle.com/technology/pub/articles/hunter_rac10gr2.html#Under%20the%20Covershttp://www.oracle.com/technology/pub/articles/hunter_rac10gr2.html#Under%20the%20Covershttp://www.oracle.com/technology/pub/articles/hunter_rac10gr2.html#Global%20Resource%20Directoryhttp://www.oracle.com/technology/pub/articles/hunter_rac10gr2.html#Global%20Resource%20Directoryhttp://www.oracle.com/technology/pub/articles/hunter_rac10gr2.html#Global%20Resource%20Directoryhttp://www.oracle.com/technology/pub/articles/hunter_rac10gr2.html#Under%20the%20Covershttp://www.oracle.com/technology/pub/articles/hunter_rac10gr2.html#RAC%2010g%20Architecture%20http://www.oracle.com/technology/pub/articles/hunter_rac10gr2.html#Slide%207http://www.oracle.com/technology/pub/articles/hunter_rac10gr2.html#Slide%206http://www.oracle.com/technology/pub/articles/hunter_rac10gr2.html#Slide%205http://www.oracle.com/technology/pub/articles/hunter_rac10gr2.html#Slide%204http://www.oracle.com/technology/pub/articles/hunter_rac10gr2.html#Slide%203http://www.oracle.com/technology/pub/articles/hunter_rac10gr2.html#Oracle10g%20-%20RAChttp://www.oracle.com/technology/pub/articles/hunter_rac10gr2.html#Oracle10g%20-%20RAChttp://www.oracle.com/technology/pub/articles/hunter_rac10gr2.html#Oracle10g%20-%20RAChttp://www.oracle.com/technology/pub/articles/hunter_rac10gr2.html#Real%20Application%20Cluster%20(RAC)%20%20%20Kishore%20A%20


37/103

Build Your Own Oracle RAC 10Build Your Own Oracle RAC 10ggRelease 2 Cluster onRelease 2 Cluster on

Linux and FireWireLinux and FireWire

Configure the Linux Servers for OracleConfigure the Linux Servers for Oracle Configure theConfigure thehangcheckhangcheck-timer Kernel Module-timer Kernel Module Configure RAC Nodes for Remote AccessConfigure RAC Nodes for Remote Access All Startup Commands for Each RAC NodeAll Startup Commands for Each RAC Node Check RPM Packages for Oracle 10Check RPM Packages for Oracle 10ggRelease 2Release 2

Install & Configure Oracle Cluster File System (OCFS2)Install & Configure Oracle Cluster File System (OCFS2) Install & Configure Automatic Storage Management (Install & Configure Automatic Storage Management (ASMLibASMLib

2.0)2.0) Download Oracle 10Download Oracle 10ggRAC SoftwareRAC Software Install Oracle 10Install Oracle 10ggClusterwareClusterwareSoftwareSoftware Install Oracle 10Install Oracle 10ggDatabase SoftwareDatabase Software Install Oracle10Install Oracle10ggCompanion CD SoftwareCompanion CD Software Create TNS Listener ProcessCreate TNS Listener Process
http://www.oracle.com/technology/pub/articles/hunter_rac10gr2_2.html#Background%20Processes%20in%20a%20RAC%20instancehttp://www.oracle.com/technology/pub/articles/hunter_rac10gr2_2.html#Background%20Processes%20in%20a%20RAC%20instancehttp://www.oracle.com/technology/pub/articles/hunter_rac10gr2_2.html#DIAG%20:%20Diagnosability%20Daemonhttp://www.oracle.com/technology/pub/articles/hunter_rac10gr2_2.html#DIAG%20:%20Diagnosability%20Daemonhttp://www.oracle.com/technology/pub/articles/hunter_rac10gr2_2.html#DIAG%20:%20Diagnosability%20Daemonhttp://www.oracle.com/technology/pub/articles/hunter_rac10gr2_2.html#DIAG%20:%20Diagnosability%20Daemonhttp://www.oracle.com/technology/pub/articles/hunter_rac10gr2_2.html#DIAG%20:%20Diagnosability%20Daemonhttp://www.oracle.com/technology/pub/articles/hunter_rac10gr2_2.html#DIAG%20:%20Diagnosability%20Daemonhttp://www.oracle.com/technology/pub/articles/hunter_rac10gr2_2.html#LCK:%20Lock%20Processhttp://www.oracle.com/technology/pub/articles/hunter_rac10gr2_2.html#LCK:%20Lock%20Processhttp://www.oracle.com/technology/pub/articles/hunter_rac10gr2_2.html#LMD:Lock%20Manager%20Daemon%20Processhttp://www.oracle.com/technology/pub/articles/hunter_rac10gr2_2.html#LMD:Lock%20Manager%20Daemon%20Processhttp://www.oracle.com/technology/pub/articles/hunter_rac10gr2_2.html#LMON:%20Lock%20Monitor%20Processhttp://www.oracle.com/technology/pub/articles/hunter_rac10gr2_2.html#LMON:%20Lock%20Monitor%20Processhttp://www.oracle.com/technology/pub/articles/hunter_rac10gr2_2.html#LMON:%20Lock%20Monitor%20Processhttp://www.oracle.com/technology/pub/articles/hunter_rac10gr2_2.html#LMON:%20Lock%20Monitor%20Processhttp://www.oracle.com/technology/pub/articles/hunter_rac10gr2_2.html#LMON:%20Lock%20Monitor%20Processhttp://www.oracle.com/technology/pub/articles/hunter_rac10gr2_2.html#LMON:%20Lock%20Monitor%20Processhttp://www.oracle.com/technology/pub/articles/hunter_rac10gr2_2.html#LMS:%20Lock%20Manager%20Server%20Processhttp://www.oracle.com/technology/pub/articles/hunter_rac10gr2_2.html#LMS:%20Lock%20Manager%20Server%20Processhttp://www.oracle.com/technology/pub/articles/hunter_rac10gr2_2.html#Server%20Control%20Utilityhttp://www.oracle.com/technology/pub/articles/hunter_rac10gr2_2.html#Server%20Control%20Utilityhttp://www.oracle.com/technology/pub/articles/hunter_rac10gr2_2.html#Server%20Control%20Utilityhttp://www.oracle.com/technology/pub/articles/hunter_rac10gr2_2.html#Server%20Control%20Utilityhttp://www.oracle.com/technology/pub/articles/hunter_rac10gr2_2.html#Server%20Control%20Utilityhttp://www.oracle.com/technology/pub/articles/hunter_rac10gr2_2.html#Server%20Control%20Utilityhttp://www.oracle.com/technology/pub/articles/hunter_rac10gr2_2.html#Slide%2018http://www.oracle.com/technology/pub/articles/hunter_rac10gr2_2.html#Slide%2018http://www.oracle.com/technology/pub/articles/hunter_rac10gr2_2.html#Slide%2018http://www.oracle.com/technology/pub/articles/hunter_rac10gr2_2.html#Slide%2018http://www.oracle.com/technology/pub/articles/hunter_rac10gr2_2.html#Slide%2018http://www.oracle.com/technology/pub/articles/hunter_rac10gr2_2.html#Slide%2018http://www.oracle.com/technology/pub/articles/hunter_rac10gr2_3.html#RAW%20Partitions,%20Cluster%20File%20System%20and%20Automatic%20Storage%20Management%20(ASM)http://www.oracle.com/technology/pub/articles/hunter_rac10gr2_3.html#RAW%20Partitions,%20Cluster%20File%20System%20and%20Automatic%20Storage%20Management%20(ASM)http://www.oracle.com/technology/pub/articles/hunter_rac10gr2_3.html#RAW%20Partitions,%20Cluster%20File%20System%20and%20Automatic%20Storage%20Management%20(ASM)http://www.oracle.com/technology/pub/articles/hunter_rac10gr2_3.html#RAW%20Partitions,%20Cluster%20File%20System%20and%20Automatic%20Storage%20Management%20(ASM)http://www.oracle.com/technology/pub/articles/hunter_rac10gr2_3.html#RAW%20Partitions,%20Cluster%20File%20System%20and%20Automatic%20Storage%20Management%20(ASM)http://www.oracle.com/technology/pub/articles/hunter_rac10gr2_3.html#RAW%20Partitions,%20Cluster%20File%20System%20and%20Automatic%20Storage%20Management%20(ASM)http://www.oracle.com/technology/pub/articles/hunter_rac10gr2_3.html#RAW%20Partitions,%20Cluster%20File%20System%20and%20Automatic%20Storage%20Management%20(ASM)http://www.oracle.com/technology/pub/articles/hunter_rac10gr2_3.html#RAW%20Partitions,%20Cluster%20File%20System%20and%20Automatic%20Storage%20Management%20(ASM)http://www.oracle.com/technology/pub/articles/hunter_rac10gr2_3.html#Slide%2020http://www.oracle.com/technology/pub/articles/hunter_rac10gr2_3.html#Slide%2020http://www.oracle.com/technology/pub/articles/hunter_rac10gr2_3.html#Slide%2020http://www.oracle.com/technology/pub/articles/hunter_rac10gr2_3.html#Slide%2020http://www.oracle.com/technology/pub/articles/hunter_rac10gr2_3.html#Slide%2020http://www.oracle.com/technology/pub/articles/hunter_rac10gr2_3.html#Slide%2020http://www.oracle.com/technology/pub/articles/hunter_rac10gr2_3.html#RAW%20Deviceshttp://www.oracle.com/technology/pub/articles/hunter_rac10gr2_3.html#RAW%20Deviceshttp://www.oracle.com/technology/pub/articles/hunter_rac10gr2_3.html#RAW%20Deviceshttp://www.oracle.com/technology/pub/articles/hunter_rac10gr2_3.html#RAW%20Deviceshttp://www.oracle.com/technology/pub/articles/hunter_rac10gr2_3.html#RAW%20Deviceshttp://www.oracle.com/technology/pub/articles/hunter_rac10gr2_3.html#RAW%20Deviceshttp://www.oracle.com/technology/pub/articles/hunter_rac10gr2_3.html#Slide%2022http://www.oracle.com/technology/pub/articles/hunter_rac10gr2_3.html#Slide%2022http://www.oracle.com/technology/pub/articles/hunter_rac10gr2_3.html#Slide%2022http://www.oracle.com/technology/pub/articles/hunter_rac10gr2_3.html#RAW%20Deviceshttp://www.oracle.com/technology/pub/articles/hunter_rac10gr2_3.html#RAW%20Deviceshttp://www.oracle.com/technology/pub/articles/hunter_rac10gr2_3.html#RAW%20Deviceshttp://www.oracle.com/technology/pub/articles/hunter_rac10gr2_3.html#Slide%2020http://www.oracle.com/technology/pub/articles/hunter_rac10gr2_3.html#Slide%2020http://www.oracle.com/technology/pub/articles/hunter_rac10gr2_3.html#Slide%2020http://www.oracle.com/technology/pub/articles/hunter_rac10gr2_3.html#RAW%20Partitions,%20Cluster%20File%20System%20and%20Automatic%20Storage%20Management%20(ASM)http://www.oracle.com/technology/pub/articles/hunter_rac10gr2_3.html#RAW%20Partitions,%20Cluster%20File%20System%20and%20Automatic%20Storage%20Management%20(ASM)http://www.oracle.com/technology/pub/articles/hunter_rac10gr2_3.html#RAW%20Partitions,%20Cluster%20File%20System%20and%20Automatic%20Storage%20Management%20(ASM)http://www.oracle.com/technology/pub/articles/hunter_rac10gr2_3.html#RAW%20Partitions,%20Cluster%20File%20System%20and%20Automatic%20Storage%20Management%20(ASM)http://www.oracle.com/technology/pub/articles/hunter_rac10gr2_2.html#Slide%2018http://www.oracle.com/technology/pub/articles/hunter_rac10gr2_2.html#Slide%2018http://www.oracle.com/technology/pub/articles/hunter_rac10gr2_2.html#Slide%2018http://www.oracle.com/technology/pub/articles/hunter_rac10gr2_2.html#Server%20Control%20Utilityhttp://www.oracle.com/technology/pub/articles/hunter_rac10gr2_2.html#Server%20Control%20Utilityhttp://www.oracle.com/technology/pub/articles/hunter_rac10gr2_2.html#Server%20Control%20Utilityhttp://www.oracle.com/technology/pub/articles/hunter_rac10gr2_2.html#LMS:%20Lock%20Manager%20Server%20Processhttp://www.oracle.com/technology/pub/articles/hunter_rac10gr2_2.html#LMON:%20Lock%20Monitor%20Processhttp://www.oracle.com/technology/pub/articles/hunter_rac10gr2_2.html#LMON:%20Lock%20Monitor%20Processhttp://www.oracle.com/technology/pub/articles/hunter_rac10gr2_2.html#LMON:%20Lock%20Monitor%20Processhttp://www.oracle.com/technology/pub/articles/hunter_rac10gr2_2.html#LMD:Lock%20Manager%20Daemon%20Processhttp://www.oracle.com/technology/pub/articles/hunter_rac10gr2_2.html#LCK:%20Lock%20Processhttp://www.oracle.com/technology/pub/articles/hunter_rac10gr2_2.html#DIAG%20:%20Diagnosability%20Daemonhttp://www.oracle.com/technology/pub/articles/hunter_rac10gr2_2.html#DIAG%20:%20Diagnosability%20Daemonhttp://www.oracle.com/technology/pub/articles/hunter_rac10gr2_2.html#DIAG%20:%20Diagnosability%20Daemonhttp://www.oracle.com/technology/pub/articles/hunter_rac10gr2_2.html#Background%20Processes%20in%20a%20RAC%20instance


38/103


Create the Oracle Cluster DatabaseCreate the Oracle Cluster Database Verify TNS Networking FilesVerify TNS Networking Files Create / AlterCreate / AlterTablespacesTablespaces Verify the RAC Cluster & Database ConfigurationVerify the RAC Cluster & Database Configuration

Starting / Stopping the ClusterStarting / Stopping the Cluster Transparent Application Failover - (TAF)Transparent Application Failover - (TAF) ConclusionConclusion AcknowledgementsAcknowledgements
http://www.oracle.com/technology/pub/articles/hunter_rac10gr2_3.html#Slide%2023http://www.oracle.com/technology/pub/articles/hunter_rac10gr2_3.html#Slide%2023http://www.oracle.com/technology/pub/articles/hunter_rac10gr2_3.html#Slide%2024http://www.oracle.com/technology/pub/articles/hunter_rac10gr2_3.html#Slide%2024http://www.oracle.com/technology/pub/articles/hunter_rac10gr2_3.html#Cluster%20File%20Systemhttp://www.oracle.com/technology/pub/articles/hunter_rac10gr2_3.html#Cluster%20File%20Systemhttp://www.oracle.com/technology/pub/articles/hunter_rac10gr2_3.html#Cluster%20File%20Systemhttp://www.oracle.com/technology/pub/articles/hunter_rac10gr2_3.html#Cluster%20File%20Systemhttp://www.oracle.com/technology/pub/articles/hunter_rac10gr2_3.html#Slide%2026http://www.oracle.com/technology/pub/articles/hunter_rac10gr2_3.html#Slide%2026http://www.oracle.com/technology/pub/articles/hunter_rac10gr2_3.html#ASM%20%E2%80%93%20Automatic%20Storage%20Managementhttp://www.oracle.com/technology/pub/articles/hunter_rac10gr2_3.html#ASM%20%E2%80%93%20Automatic%20Storage%20Managementhttp://www.oracle.com/technology/pub/articles/hunter_rac10gr2_3.html#Slide%2028http://www.oracle.com/technology/pub/articles/hunter_rac10gr2_3.html#Slide%2028http://www.oracle.com/technology/pub/articles/hunter_rac10gr2_3.html#Slide%2029http://www.oracle.com/technology/pub/articles/hunter_rac10gr2_3.html#Slide%2029http://www.oracle.com/technology/pub/articles/hunter_rac10gr2_3.html#Slide%2030http://www.oracle.com/technology/pub/articles/hunter_rac10gr2_3.html#Slide%2030http://www.oracle.com/technology/pub/articles/hunter_rac10gr2_3.html#Slide%2030http://www.oracle.com/technology/pub/articles/hunter_rac10gr2_3.html#Slide%2029http://www.oracle.com/technology/pub/articles/hunter_rac10gr2_3.html#Slide%2028http://www.oracle.com/technology/pub/articles/hunter_rac10gr2_3.html#ASM%20%E2%80%93%20Automatic%20Storage%20Managementhttp://www.oracle.com/technology/pub/articles/hunter_rac10gr2_3.html#Slide%2026http://www.oracle.com/technology/pub/articles/hunter_rac10gr2_3.html#Cluster%20File%20Systemhttp://www.oracle.com/technology/pub/articles/hunter_rac10gr2_3.html#Cluster%20File%20Systemhttp://www.oracle.com/technology/pub/articles/hunter_rac10gr2_3.html#Slide%2024http://www.oracle.com/technology/pub/articles/hunter_rac10gr2_3.html#Slide%2023


39/103



40/103



41/103


DownloadDownload- Red Hat Enterprise Linux 4- Red Hat Enterprise Linux 4

- Oracle Cluster File System Release 2 - (1.2.3-1) - Single- Oracle Cluster File System Release 2 - (1.2.3-1) - SingleProcessor / SMP /Processor / SMP /

HugememHugemem- Oracle Cluster File System Releaase 2 Tools - (1.2.1-1) - Tools /- Oracle Cluster File System Releaase 2 Tools - (1.2.1-1) - Tools /

ConsoleConsole- Oracle Database 10- Oracle Database 10gg Release 2 EE, Clusterware, Companion CD -Release 2 EE, Clusterware, Companion CD -

(10.2.0.1.0)(10.2.0.1.0)- Precompiled RHEL4 FireWire Modules - (2.6.9-22.EL)- Precompiled RHEL4 FireWire Modules - (2.6.9-22.EL)

- ASMLib 2.0 Driver - (2.6.9-22.EL / 2.0.3-1) - Single Processor / SMP- ASMLib 2.0 Driver - (2.6.9-22.EL / 2.0.3-1) - Single Processor / SMP/ Hugemem/ Hugemem

- ASMLib 2.0 Library and Tools - (2.0.3-1) - Driver Support Files /- ASMLib 2.0 Library and Tools - (2.0.3-1) - Driver Support Files /Userspace LibraryUserspace Library


42/103

Build Your Own Oracle RAC 10Build Your Own Oracle RAC 10ggRelease 2 Cluster onRelease 2 Cluster on

Linux and FireWireLinux and FireWire

IntroductionIntroduction

One of the most efficient ways to become familiar with Oracle Real ApplicationOne of the most efficient ways to become familiar with Oracle Real ApplicationClusters (RAC) 10Clusters (RAC) 10ggtechnology is to have access to an actual Oracle RAC 10technology is to have access to an actual Oracle RAC 10ggcluster. There's no better way to understand its benefitsincluding fault tolerance,cluster. There's no better way to understand its benefitsincluding fault tolerance,security, load balancing, and scalabilitythan to experience them directly.security, load balancing, and scalabilitythan to experience them directly.

The Oracle Clusterware software will be installed to /u01/app/oracle/product/crs onThe Oracle Clusterware software will be installed to /u01/app/oracle/product/crs oneach of the nodes that make up the RAC cluster. However, the Clusterwareeach of the nodes that make up the RAC cluster. However, the Clusterwaresoftware requires that two of its filesthe Oracle Cluster Registry (OCR) file andsoftware requires that two of its filesthe Oracle Cluster Registry (OCR) file andthe Voting Disk filebe shared with all nodes in the cluster. These two files will bethe Voting Disk filebe shared with all nodes in the cluster. These two files will beinstalled on shared storage using OCFS2. It is possible (but not recommended byinstalled on shared storage using OCFS2. It is possible (but not recommended byOracle) to use RAW devices for these files; however, it is not possible to use ASMOracle) to use RAW devices for these files; however, it is not possible to use ASMfor these two Clusterware files.for these two Clusterware files.

The Oracle Database 10g Release 2 software will be installed into a separateThe Oracle Database 10g Release 2 software will be installed into a separateOracle Home, namely /u01/app/oracle/product/10.2.0/db_1, on each of the nodesOracle Home, namely /u01/app/oracle/product/10.2.0/db_1, on each of the nodesthat make up the RAC cluster. All the Oracle physical database files (data, onlinethat make up the RAC cluster. All the Oracle physical database files (data, onlineredo logs, control files, archived redo logs), will be installed to different partitions ofredo logs, control files, archived redo logs), will be installed to different partitions ofthe shared drive being managed by ASM. (The Oracle database files can just asthe shared drive being managed by ASM. (The Oracle database files can just aseasily be stored on OCFS2. Using ASM, however, makes the article that mucheasily be stored on OCFS2. Using ASM, however, makes the article that muchmore interesting!)more interesting!)


43/103



44/103

2. Oracle RAC 102. Oracle RAC 10ggOverviewOverview

Oracle RAC, introduced with Oracle9Oracle RAC, introduced with Oracle9ii, is the successor to Oracle Parallel, is the successor to Oracle Parallel

Server (OPS). RAC allows multiple instances to access the sameServer (OPS). RAC allows multiple instances to access the samedatabase (storage) simultaneously. It provides fault tolerance, loaddatabase (storage) simultaneously. It provides fault tolerance, loadbalancing, and performance benefits by allowing the system to scale out,balancing, and performance benefits by allowing the system to scale out,and at the same timebecause all nodes access the same databasetheand at the same timebecause all nodes access the same databasethefailure of one instance will not cause the loss of access to the database.failure of one instance will not cause the loss of access to the database.

At the heart of Oracle RAC is a shared disk subsystem. All nodes in theAt the heart of Oracle RAC is a shared disk subsystem. All nodes in thecluster must be able to access all of the data, redo log files, control filescluster must be able to access all of the data, redo log files, control filesand parameter files for all nodes in the cluster. The data disks must beand parameter files for all nodes in the cluster. The data disks must beglobally available to allow all nodes to access the database. Each nodeglobally available to allow all nodes to access the database. Each node

has its own redo log and control files but the other nodes must be able tohas its own redo log and control files but the other nodes must be able to

access them in order to recover that node in the event of a system failure.access them in order to recover that node in the event of a system failure. One of the bigger differences between Oracle RAC and OPS is theOne of the bigger differences between Oracle RAC and OPS is the

presence of Cache Fusion technology. In OPS, a request for datapresence of Cache Fusion technology. In OPS, a request for databetween nodes required the data to be written to disk first, and then thebetween nodes required the data to be written to disk first, and then therequesting node could read that data. In RAC, data is passed along withrequesting node could read that data. In RAC, data is passed along with

locks.locks.

S S O


45/103

3. Shared-Storage Overview3. Shared-Storage Overview

Fibre Channel is one of the most popular solutions for shared storage. As IFibre Channel is one of the most popular solutions for shared storage. As I

mentioned previously, Fibre Channel is a high-speed serial-transfermentioned previously, Fibre Channel is a high-speed serial-transferinterface used to connect systems and storage devices in either point-to-interface used to connect systems and storage devices in either point-to-point or switched topologies. Protocols supported by Fibre Channelpoint or switched topologies. Protocols supported by Fibre Channelinclude SCSI and IP.include SCSI and IP.

Fibre Channel configurations can support as many as 127 nodes andFibre Channel configurations can support as many as 127 nodes andhave a throughput of up to 2.12 gigabits per second. Fibre Channel,have a throughput of up to 2.12 gigabits per second. Fibre Channel,

however, is very expensive; the switch alone can cost as much ashowever, is very expensive; the switch alone can cost as much asUS$1,000 and high-end drives can reach prices of US$300. Overall, aUS$1,000 and high-end drives can reach prices of US$300. Overall, atypical Fibre Channel setup (including cards for the servers) costs roughlytypical Fibre Channel setup (including cards for the servers) costs roughlyUS$5,000.US$5,000.

A less expensive alternative to Fibre Channel is SCSI. SCSI technologyA less expensive alternative to Fibre Channel is SCSI. SCSI technologyprovides acceptable performance for shared storage, but forprovides acceptable performance for shared storage, but foradministrators and developers who are used to GPL-based Linux prices,administrators and developers who are used to GPL-based Linux prices,even SCSI can come in over budget at around US$1,000 to US$2,000 foreven SCSI can come in over budget at around US$1,000 to US$2,000 fora two-node cluster.a two-node cluster.

Another popular solution is the Sun NFS (Network File System) found on aAnother popular solution is the Sun NFS (Network File System) found on aNAS. It can be used for shared storage but only if you are using a networkNAS. It can be used for shared storage but only if you are using a network

appliance or something similar. Specifically, you need servers thatappliance or something similar. Specifically, you need servers thatuarantee direct I/O over NFS TCP as the trans ort rotocol anduarantee direct I/O over NFS, TCP as the trans ort rotocol, and

4 Fi Wi T h l


46/103

4. FireWire Technology4. FireWire Technology

Developed by Apple Computer and Texas Instruments,Developed by Apple Computer and Texas Instruments,

FireWire is a cross-platform implementation of a high-speedFireWire is a cross-platform implementation of a high-speedserial data bus. With its high bandwidth, long distances (up toserial data bus. With its high bandwidth, long distances (up to100 meters in length) and high-powered bus, FireWire is100 meters in length) and high-powered bus, FireWire is

being used in applications such as digital video (DV),being used in applications such as digital video (DV),

professional audio, hard drives, high-end digital still camerasprofessional audio, hard drives, high-end digital still camerasand home entertainment devices. Today, FireWire operates atand home entertainment devices. Today, FireWire operates attransfer rates of up to 800 megabits per second while nexttransfer rates of up to 800 megabits per second while nextgeneration FireWire calls for speeds to a theoretical bit rate togeneration FireWire calls for speeds to a theoretical bit rate to

1,600 Mbps and then up to a staggering 3,200 Mbps. That's1,600 Mbps and then up to a staggering 3,200 Mbps. That's3.2 gigabits per second. This speed will make FireWire3.2 gigabits per second. This speed will make FireWire

indispensable for transferring massive data files and for evenindispensable for transferring massive data files and for eventhe most demanding video applications, such as working withthe most demanding video applications, such as working withuncompressed high-definition (HD) video or multiple standard-uncompressed high-definition (HD) video or multiple standard-

definition (SD) video streams.definition (SD) video streams.


47/103

Disk Interface Speed

Serial 115 kb/s - (.115 Mb/s)

Parallel (standard) 115 KB/s - (.115 MB/s)

USB 1.1 12 Mb/s - (1.5 MB/s)

Parallel (ECP/EPP) 3.0 MB/s

IDE 3.3 - 16.7 MB/s

ATA 3.3 - 66.6 MB/sec

SCSI-1 5 MB/s

SCSI-2 (Fast SCSI/Fast Narrow SCSI) 10 MB/s

Fast Wide SCSI (Wide SCSI) 20 MB/s

Ultra SCSI (SCSI-3/Fast-20/Ultra

Narrow)

20 MB/s

Ultra IDE 33 MB/s

Wide Ultra SCSI (Fast Wide 20) 40 MB/s

Ultra2 SCSI 40 MB/s

IEEE1394(b) 100 - 400Mb/s - (12.5 - 50 MB/s)

USB 2.x 480 Mb/s - (60 MB/s)

Wide Ultra2 SCSI 80 MB/s

Ultra3 SCSI 80 MB/s

Wide Ultra3 SCSI 160 MB/s

FC-AL Fiber Channel 100 - 400 MB/s

1. Oracle Clusterware - /u01/app/oracle/product/crs/u01/app/oracle/product/crs2 O l 10 S ft (With t d t b ) / 01/ / l / d t/10 1 0/d t 1 (10 2 0


48/103

2. Oracle 10g Software (Without database) /u01/app/oracle/product/10.1.0/data_1 - (10.2.0

1.Oracle Cluster Registry (OCR) File - /u02/oradata/orcl/OCRFile (OCFS2 )2.CRS Voting Disk - /u02/oradata/orcl/CSSFile (OCFS2 )

3.Oracle Database files ASM

5 S ft R i t5 S ft R i t


49/103

5. Software Requirements5. Software Requirements

SoftwareSoftware

At the software level, each node in a RAC clusterAt the software level, each node in a RAC clusterneeds:needs:

1.1. AnAn operatingoperating systemsystem

2.2. Oracle Clusterware SoftwareOracle Clusterware Software3.3. Oracle RAC software, and optionallyOracle RAC software, and optionally

An Oracle Automated Storage Management instance.An Oracle Automated Storage Management instance.

O l A t t d St M t (ASM)O l A t t d St M t (ASM)


50/103

Oracle Automated Storage Management (ASM)Oracle Automated Storage Management (ASM)

ASM is a new feature in Oracle Database 10ASM is a new feature in Oracle Database 10ggthat provides the servicesthat provides the services

of a filesystem, logical volume manager, and software RAID in a platform-of a filesystem, logical volume manager, and software RAID in a platform-independent manner. Oracle ASM can stripe and mirror your disks, allowindependent manner. Oracle ASM can stripe and mirror your disks, allowdisks to be added or removed while the database is under load, anddisks to be added or removed while the database is under load, andautomatically balance I/O to remove "hot spots." It also supports directautomatically balance I/O to remove "hot spots." It also supports directand asynchronous I/O and implements the Oracle Data Manager APIand asynchronous I/O and implements the Oracle Data Manager API(simplified I/O system call interface) introduced in Oracle9i.(simplified I/O system call interface) introduced in Oracle9i.

Oracle ASM is not a general-purpose filesystem and can be used only forOracle ASM is not a general-purpose filesystem and can be used only forOracle data files, redo logs, control files, and the RMAN Flash RecoveryOracle data files, redo logs, control files, and the RMAN Flash Recovery

Area. Files in ASM can be created and named automatically by theArea. Files in ASM can be created and named automatically by thedatabase (by use of the Oracle Managed Files feature) or manually by thedatabase (by use of the Oracle Managed Files feature) or manually by theDBA. Because the files stored in ASM are not accessible to the operatingDBA. Because the files stored in ASM are not accessible to the operatingsystem, the only way to perform backup and recovery operations onsystem, the only way to perform backup and recovery operations on

databases that use ASM files is through Recovery Manager (RMAN).databases that use ASM files is through Recovery Manager (RMAN). ASM is implemented as a separate Oracle instance that must be up ifASM is implemented as a separate Oracle instance that must be up if

other databases are to be able to access it. Memory requirements forother databases are to be able to access it. Memory requirements forASM are light: only 64MB for most systems. In Oracle RAC environments,ASM are light: only 64MB for most systems. In Oracle RAC environments,

an ASM instance must be running on each cluster node.an ASM instance must be running on each cluster node.

6 I t ll th Li O ti S t6 I t ll th Li O ti S t


51/103

6. Install the Linux Operating System6. Install the Linux Operating System

This article was designed to work with the Red Hat Enterprise Linux 4 (AS/ES)This article was designed to work with the Red Hat Enterprise Linux 4 (AS/ES)

operating environment.operating environment. .. You will need three IP addresses for each server: one forYou will need three IP addresses for each server: one forthe private network, one for the public network, and one for the virtual IP address.the private network, one for the public network, and one for the virtual IP address.

Use the operating system's network configuration tools to assign the private andUse the operating system's network configuration tools to assign the private and

public network addresses. Do not assign the virtual IP address using the operatingpublic network addresses. Do not assign the virtual IP address using the operating

system's network configuration tools; this will be done by the Oracle Virtual IPsystem's network configuration tools; this will be done by the Oracle Virtual IP

Configuration Assistant (VIPCA) during Oracle RAC software installation.Configuration Assistant (VIPCA) during Oracle RAC software installation.

Linux1Linux1 eth0:eth0:

- Check off the option to [Configure using DHCP]- Check off the option to [Configure using DHCP]- Leave the [Activate on boot] checked- Leave the [Activate on boot] checked- IP Address: 192.168.1.100- IP Address: 192.168.1.100

- Netmask: 255.255.255.0- Netmask: 255.255.255.0 eth1:eth1:

- Check off the option to [Configure using DHCP]- Check off the option to [Configure using DHCP]- Leave the [Activate on boot] checked- Leave the [Activate on boot] checked- IP Address: 192.168.2.100- IP Address: 192.168.2.100

- Netmask: 255.255.255.0- Netmask: 255.255.255.0

6 I t ll th Li O ti S t6 Install the Lin Operating S stem


52/103

6. Install the Linux Operating System6. Install the Linux Operating System

Linux2Linux2

eth0:eth0:

- Check off the option to [Configure using DHCP]- Check off the option to [Configure using DHCP]- Leave the [Activate on boot] checked- Leave the [Activate on boot] checked

- IP Address: 192.168.1.101- IP Address: 192.168.1.101

- Netmask: 255.255.255.0- Netmask: 255.255.255.0 eth1:eth1:

- Check off the option to [Configure using DHCP]- Check off the option to [Configure using DHCP]- Leave the [Activate on boot] checked- Leave the [Activate on boot] checked

- IP Address: 192.168.2.101- IP Address: 192.168.2.101- Netmask: 255.255.255.0- Netmask: 255.255.255.0

7. Configure Network Settings7. Configure Network Settings


53/103

g gg g

Server 1 (linux1)

Device IP Address Subnet Purpose

eth0 192.168.1.100 255.255.255.0 Connects linux1 to the public network

eth1 192.168.2.100 255.255.255.0 Connects linux1 (interconnect) to linux2 (int-linux2)

/etc/hosts

127.0.0.1 localhost loopback

# Public Network - (eth0) 192.168.1.100 linux1 192.168.1.101 linux2

# Private Interconnect - (eth1) 192.168.2.100 linux1-priv 192.168.2.101 linux2-priv

# Public Virtual IP (VIP) addresses for - (eth0) 192.168.1.200 linux1-vip 192.168.1.201 linux2-vip



54/103

g gg g

Server 2 (linux2)

Device IP Address Subnet Purpose

eth0 192.168.1.101 255.255.255.0 Connects linux2 to the public network

eth1 192.168.2.101 255.255.255.0 Connects linux2 (interconnect) to linux1 (int-linux1)

/etc/hosts

127.0.0.1 localhost loopback

# Public Network - (eth0) 192.168.1.100 linux1 192.168.1.101 linux2

# Private Interconnect - (eth1) 192.168.2.100 int-linux1 192.168.2.101 int-linux2

# Public Virtual IP (VIP) addresses for - (eth0) 192.168.1.200 vip-linux1 192.168.1.201 vip-linux2



55/103

g gg g

Note that the virtual IP addresses only need to beNote that the virtual IP addresses only need to be

defined in the /etc/hosts file for both nodes. Thedefined in the /etc/hosts file for both nodes. Thepublic virtual IP addresses will be configuredpublic virtual IP addresses will be configured

automatically by Oracle when you run the Oracleautomatically by Oracle when you run the Oracle

Universal Installer, which starts Oracle's VirtualUniversal Installer, which starts Oracle's VirtualInternet Protocol Configuration Assistant (VIPCA). AllInternet Protocol Configuration Assistant (VIPCA). All

virtual IP addresses will be activated when the srvctlvirtual IP addresses will be activated when the srvctlstart nodeapps -n command is run.start nodeapps -n command is run.

This is the Host Name/IP Address that will beThis is the Host Name/IP Address that will beconfigured in the client(s) tnsnames.ora file (moreconfigured in the client(s) tnsnames.ora file (more

details later).details later).



56/103

g gg g

Adjusting Network SettingsAdjusting Network Settings

Oracle now uses UDP as the default protocol on Linux for interprocessOracle now uses UDP as the default protocol on Linux for interprocesscommunication, such as cache fusion buffer transfers between thecommunication, such as cache fusion buffer transfers between theinstances.instances.It is strongly suggested to adjust the default and maximum send bufferIt is strongly suggested to adjust the default and maximum send buffersize (SO_SNDBUF socket option) to 256 KB, and the default andsize (SO_SNDBUF socket option) to 256 KB, and the default andmaximum receive buffer size (SO_RCVBUF socket option) to 256 KB. Themaximum receive buffer size (SO_RCVBUF socket option) to 256 KB. The

receive buffers are used by TCP and UDP to hold received data until is isreceive buffers are used by TCP and UDP to hold received data until is isread by the application. The receive buffer cannot overflow because theread by the application. The receive buffer cannot overflow because thepeer is not allowed to send data beyond the buffer size window. Thispeer is not allowed to send data beyond the buffer size window. Thismeans that datagrams will be discarded if they don't fit in the socketmeans that datagrams will be discarded if they don't fit in the socket

receive buffer. This could cause the sender to overwhelm the receiver .receive buffer. This could cause the sender to overwhelm the receiver .

To make the change permanent, add the following lines to theTo make the change permanent, add the following lines to the/etc/sysctl.conf file, which is used during the boot process:/etc/sysctl.conf file, which is used during the boot process:net.core.rmem_default=262144net.core.rmem_default=262144net.core.wmem_default=262144net.core.wmem_default=262144

net.core.rmem_max=262144net.core.rmem_max=262144

net.core.wmem_max=262144net.core.wmem_max=262144

8. Obtain and Install a Proper Linux Kernel8. Obtain and Install a Proper Linux Kernel


57/103

p

http://oss.oracle.com/projects/firewire/dihttp://oss.oracle.com/projects/firewire/dist/files/RedHat/RHEL4/i386/oracle-st/files/RedHat/RHEL4/i386/oracle-firewire-modules-2.6.9-22.EL-1286-firewire-modules-2.6.9-22.EL-1286-1.i686.rpm1.i

RAC Presentation Oracle10gR2 (1)

Documents

Transcript of RAC Presentation Oracle10gR2 (1)