Oracle RAC 11g Installation on ZFS...Sep 01, 2019 · ZFS ZFS Storage Appliance 700_1 “Sun ZFS...

Oracle RAC 11g Installation on ZFS Oracle Database 11g [11.2.0.4] RAC Installation on OEL 5.8

using ZFS Shared Storage

ORACLE 11g RAC on ZFS INSTALLATION

[Oracle Enterprise Linux 5.8]

Required Software

Table-1

Type Software Image & Install

OS Oracle Linux 5.8 OracleLinux-R5-U8-Server-x86_64-dvd.iso

DB Oracle Database 11.2.0.4 p13390677_112040_Linux-x86-64_1of7 p13390677_112040_Linux-x86-64_2of7 p13390677_112040_Linux-x86-64_3of7

VM VM VirtualBox VirtualBox-4.3.2-90405-Win.exe

ZFS ZFS Storage Appliance 700_1 “Sun ZFS Storage Appliance” image

Network Architecture

Table-2

Interface Network IP Full Name Name Entry

eth0 public 192.168.10.101 rac01.taysi.com rac01 HOSTS File

eth0 public 192.168.10.102 rac02.taysi.com rac02 HOSTS File

eth0 virtual 192.168.10.111 rac01-vip.taysi.com rac01-vip HOSTS File

eth0 virtual 192.168.10.112 rac02-vip.taysi.com rac02-vip HOSTS File

eth0 scan 192.168.10.121 rac-scan.taysi.com rac-scan DNS



eth1 private 192.168.20.101 rac01-priv.taysi.com rac01-priv HOSTS File

eth1 private 192.168.20.102 rac02-priv.taysi.com rac02-priv HOSTS File

eth0 Dns 192.168.10.100 racdns.taysi.com racdns HOSTS File

RAC Node 1 Creation

INSTALLATION

Start the VirtualBox and create a new virtual machine by clicking the “New” button. Enter the OS details in

the popup window and then click the “Next” button.

Image-1

After adjusting the memory size, click “Next” button.

Image-2

Choose the “Create a virtual hard drive now” option and click “Create” button.

Image-3

Choose VDI option and click the “Next” button.

Image-4

Choose “Dynamically allocated” and click the “Next” button.

Image-5

Select the location of the image file, adjust the disk size (30GB) and click the “Create” button.

Image-6

Start the virtual machine.

Image-7

Select the OS image file and click the “Start” button.

Image-8

Hit the “Enter” button and continue.

Image-9

Skip the testing and continue.

Image-10

Click the “Next” button.

Image-11

Select the language and click the “Next” button.

Image-12

Select the keyboard and then click the “Next” button.

Image-13

Check the “Review and modify partitioning layout” option and click the “Next” button.

Image-14

Click the “Yes” button and continue.

Image-15

Select the /boot partition and click the “Edit” button.

Image-16

Set the /boot partition size as 150MB click the “OK” button.

Image-17

Leave the “swap” and “/root” partitions as they are and click the “Next” button.

SWAP SPACE RECOMMENDATION:

If RAM between 2 GB and 8 GB then 2 times the size fo RAM.

If RAM between 8 GB to 32 GB then 1.5 times the size of RAM.

If RAM greater than 32 GB then 32 GB.

Image-18

Accept the loader settings and click the “Next” button.

Image-19

Leave the network page as it is, we‟re going to configure it later. Click the “Next” button.

Image-20

Select the region/city and click the “Next” button.

Image-21

Enter a root password and click the “Next” button.

Image-22

Select “Customize now” option to decide the packages to be installed and then click the “Next” button.

Image-23

Select the required groups from the below screen.

Required Packages for Oracle RAC installation:

GNOME Desktop Environment

Editors

Graphical Internet

Text-based Internet

Development Libraries

Development Tools

Server Configuration Tools

Administration Tools

Base

System Tools

X Window System

Image-24

After selecting the required packages click the “Next” button and continue.

Image-25

Click the “Next” button to begin installation.

Image-26

Installing…

Image-27

Reboot.

Image-28

Press the “Forward” button to continue installation after the reboot.

Image-29

Agree the licence agreement and click the “Forward” button.

Image-30

Disable the firewall and click the “Forward” button. Click “Yes” for the security warning.

Image-31

Disable Secure Linux and click the “Forward” button. Click “Yes” for the security warning.

Image-32

Do NOT enable the Kdump, click the “Forward” button to continue.

Image-33

Adjust the Date & Time and then click the “Forward” button to continue.

Image-34

Do NOT create any user, click the “Forward” button to continue.Click the “Continue” button for the warning

popup.

Image-35

Click the “Forward” button to continue.

Image-36

Click the “Finish” button to finish installation. And reboot the machine.

Image-37

NETWORK CONFIGURATION - VIRTUAL BOX

Add new network interface on VirtualBox

Image-38

Click the Network link on the left then the “Add host-only network” button on the right. You should have 2

network interfaces for RAC installation. Then click the “OK” button.

Image-39

Change the settings of the newly installed RAC01 Node, right click the RAC01 tab then choose the

“Settings…” from the popup menu.

Image-40

Configure the adapter settings of the newly installed RAC01 Node

Image-41

Add a second adapter for RAC01 as seen below, and then click the “OK” button.

Image-42

Change the IP settings of VirtualBox adapters from Control Center > Network and Sharing Center. Click

VirtualBox Host-Only Network.

Image-43

Change the properties as below:

Image-44

Do the same thing for VirtualBox Host-Only Network#2 with the following values below:

Image-45

NETWORK CONFIGURATION - RAC01 SERVER NODE

Edit /etc/sysconfig/network [ROOT]

NETWORKING=yes NETWORKING_IPV6=yes HOSTNAME=rac01.taysi.com

Edit /etc/resolv.conf [ROOT]

nameserver 192.168.10.100 search taysi.com

Edit /etc/hosts [ROOT]

127.0.0.1 localhost.taysi.com localhost # Public 192.168.10.101 rac01.taysi.com rac01 192.168.10.102 rac02.taysi.com rac02 # Private 192.168.20.101 rac01-priv.taysi.com rac01-priv 192.168.20.102 rac02-priv.taysi.com rac02-priv # Virtual 192.168.10.111 rac01-vip.taysi.com rac01-vip 192.168.10.112 rac02-vip.taysi.com rac02-vip # SCAN -> TO DNS (written here just for info) #192.168.10.121 rac-scan.taysi.com rac-scan #192.168.10.122 rac-scan.taysi.com rac-scan #192.168.10.123 rac-scan.taysi.com rac-scan

Edit /etc/sysconfig/network-scripts/ifcfg-eth0 [ROOT]

# Intel Corporation 82540EM Gigabit Ethernet Controller DEVICE=eth0 BOOTPROTO=none HWADDR=08:00:27:1F:03:1F ONBOOT=yes NETMASK=255.255.255.0 IPADDR=192.168.10.101 GATEWAY=192.168.10.1 TYPE=Ethernet USERCTL=no IPV6INIT=no PEERDNS=yes


# Intel Corporation 82540EM Gigabit Ethernet Controller DEVICE=eth1 BOOTPROTO=none ONBOOT=yes HWADDR=08:00:27:d6:01:6f TYPE=Ethernet NETMASK=255.255.255.0 IPADDR=192.168.20.101 GATEWAY=192.168.20.1 USERCTL=no IPV6INIT=no PEERDNS=yes

INSTALL RPMS

Mount the Linux OS iso file. Machine > Settings > Storage

If some other image is already mounted here, remove the mounted image and reboot. When the Linux is

up again, mount the OracleLinux Image as seen below.

Image-46

Note: Do not forget to remove the image after you‟re done, else it will try to boot from image next time

you reboot the server.

Install RPMs -> cd /media/OL5.8\ x86_64\ dvd\ 20120229/Server/ (should be mounted somewhere under

/media)

Just copy the whole command and paste it in the terminal and enter. [ROOT]

rpm -Uvh binutils-2.* rpm -Uvh compat-libstdc++-33* rpm -Uvh elfutils-libelf-0.* rpm -Uvh elfutils-libelf-devel-* rpm -Uvh gcc-4.* rpm -Uvh gcc-c++-4.* rpm -Uvh glibc-2.* rpm -Uvh glibc-common-2.* rpm -Uvh glibc-devel-2.* rpm -Uvh glibc-headers-2.* rpm -Uvh ksh-2* rpm -Uvh libaio-0.* rpm -Uvh libaio-devel-0.* rpm -Uvh libgcc-4.* rpm -Uvh libstdc++-4.* rpm -Uvh libstdc++-devel-4.* rpm -Uvh make-3.* rpm -Uvh sysstat-7.* rpm -Uvh unixODBC-2.* rpm -Uvh unixODBC-devel-2.*

Note: If dependecies fail, just resubmit the whole command, sometimes some libraries must be installed

before others.

KERNEL SETTINGS

Edit /etc/sysctl.conf [ROOT]

fs.aio-max-nr = 1048576 fs.file-max = 6815744 kernel.shmall = 2097152 kernel.shmmax = 1054504960 kernel.shmmni = 4096 # semaphores: semmsl, semmns, semopm, semmni kernel.sem = 250 32000 100 128 net.ipv4.ip_local_port_range = 9000 65500 net.core.rmem_default=262144 net.core.rmem_max=4194304 net.core.wmem_default=262144 net.core.wmem_max=1048586

Note: Issue the command “/sbin/sysctl -p” after changing kernel values.

SECURITY LIMITS

Edit /etc/security/limits.conf for oracle and grid users. [ROOT]

oracle hard memlock 50000000 oracle soft memlock 50000000 oracle soft core unlimited oracle hard core unlimited oracle soft nproc 2047 oracle hard nproc 16384 oracle soft nofile 4096 oracle hard nofile 65536 oracle soft stack 10240 grid hard memlock 50000000 grid soft memlock 50000000 grid soft core unlimited grid hard core unlimited grid soft nproc 2047 grid hard nproc 16384 grid soft nofile 4096 grid hard nofile 65536 grid soft stack 10240

Add the following line to /etc/pam.d/login [ROOT]

session required pam_limits.so

CREATE GROUPS and USERS

USER PRIMARY GROUPS SECONDARY GROUPS grid oinstall asmadmin,asmdba,dba oracle oinstall dba,asmdba

Note: The group and user IDs between the two nodes must be exactly the same.

Create groups and users [ROOT]

groupadd oinstall groupadd asmadmin groupadd asmdba groupadd dba useradd -g oinstall -G asmadmin,asmdba,dba grid useradd -g oinstall -G dba,asmdba oracle

Note: Give passwords to the users after creation.

CREATE DIRECTORIES

Table-3

PATH ENV OWNER MOD /u01/app/grid ORACLE_BASE

[grid’s] grid:oinstall 775

/u01/app/11.2.0/grid ORACLE_HOME [grid’s]

grid:oinstall 775

/u02/app/oracle ORACLE_BASE [oracle’s]

oracle:oinstall 775

/u02/app/oracle/product/11.2.0/db_home ORACLE_HOME [oracle’s]

oracle:oinstall 775

Note: For grid user, ORACLE_HOME should NOT be a sub-dir of ORACLE_BASE. It should be a seperate

maybe same level directory for its ownership will be changed with root.sh script.

Create directories [ROOT]

mkdir -p /u01/app/grid mkdir -p /u01/app/11.2.0/grid mkdir -p /u02/app/oracle mkdir -p /u02/app/oracle/product/11.2.0/db_home chown -R grid:oinstall /u01 chown -R oracle:oinstall /u02 chmod -R 775 /u01 chmod -R 775 /u02

SETUP FILES

Create a directory for setup files and copy the the installation files in here using an FTP tool. [ROOT]

mkdir /setup # Copy the following files into this path: # p13390677_112040_Linux-x86-64_1of7.zip # p13390677_112040_Linux-x86-64_2of7.zip # p13390677_112040_Linux-x86-64_3of7.zip # Unzip the files unzip p13390677_112040_Linux-x86-64_1of7.zip unzip p13390677_112040_Linux-x86-64_2of7.zip unzip p13390677_112040_Linux-x86-64_3of7.zip

CLUSTER VERIFICATION RPM

Go to the path /setup/grid/rpm and install the single rpm package residing in this path. [ROOT]

rpm -Uvh cvuqdisk-1.0.9-1.rpm

DECONFIGURE NTP

Deconfigure [ROOT]

service ntpd stop chkconfig ntpd off mv /etc/ntp.conf /etc/ntp.conf.orig rm /var/run/ntpd

Note: This is a virtual env. test installation. In a real env. you may want to configure NTP instead of

disabling it.

CONFIGURE BASH PROFILES

Switch to oracle user and edit the bash_profile [ORACLE]

# Oracle Settings TMP=/tmp;export TMP TMPDIR=$TMP;export TMPDIR ORACLE_HOSTNAME=rac01.taysi.com;export ORACLE_HOSTNAME ORACLE_UNQNAME=RAC;export ORACLE_UNQNAME ORACLE_BASE=/u02/app/oracle;export ORACLE_BASE ORACLE_HOME=/u02/app/oracle/product/11.2.0/db_home;export ORACLE_HOME ORACLE_SID=RAC1;export ORACLE_SID ORACLE_TERM=xterm;export ORACLE_TERM BASE_PATH=/usr/sbin:$PATH;export BASE_PATH PATH=$ORACLE_HOME/bin:$BASE_PATH;export PATH LD_LIBRARY_PATH=$ORACLE_HOME/lib:/lib:/usr/lib; export LD_LIBRARY_PATH CLASSPATH=$ORACLE_HOME/JRE:$ORACLE_HOME/jlib:$ORACLE_HOME/rdbms/jlib;export CLASSPATH if [ $USER = "oracle" ]; then if [ $SHELL = "/bin/ksh" ]; then ulimit -p 16384 ulimit -n 65536 else ulimit -u 16384 -n 65536 fi fi

Switch to grid user and edit the bash_profile [GRID]

# Grid Settings TMP=/tmp;export TMP TMPDIR=$TMP;export TMPDIR ORACLE_HOSTNAME=rac01.taysi.com;export ORACLE_HOSTNAME ORACLE_UNQNAME=RAC;export ORACLE_UNQNAME ORACLE_BASE=/u01/app/grid;export ORACLE_BASE ORACLE_HOME=/u01/app/11.2.0/grid;export ORACLE_HOME ORACLE_SID=RAC1;export ORACLE_SID ORACLE_TERM=xterm;export ORACLE_TERM BASE_PATH=/usr/sbin:$PATH;export BASE_PATH PATH=$ORACLE_HOME/bin:$BASE_PATH;export PATH LD_LIBRARY_PATH=$ORACLE_HOME/lib:/lib:/usr/lib; export LD_LIBRARY_PATH CLASSPATH=$ORACLE_HOME/JRE:$ORACLE_HOME/jlib:$ORACLE_HOME/rdbms/jlib;export CLASSPATH if [ $USER = "oracle" ]; then if [ $SHELL = "/bin/ksh" ]; then ulimit -p 16384 ulimit -n 65536 else ulimit -u 16384 -n 65536 fi fi

RAC Node 2 Creation

Change the default machine folder to the path where your RAC01 resides.

Image-47

Shutdown the RAC01 Server and clone the virtual machine.

Image-48

Enter “RAC02” for the clone name and check “Reinitialize the MAC address of all network cards”

checkbox. Click the “Next” button to continue.

Image-49

Confirm that the “Full clone” option is selected. Click the “Clone” button to begin the cloning process.

Image-50

START THE RAC02 SERVER AND CHANGE THE NETWORK CONFIGURATIONS OF RAC02 NODE


# Intel Corporation 82540EM Gigabit Ethernet Controller DEVICE=eth0 BOOTPROTO=none ONBOOT=yes HWADDR=08:00:27:d5:92:19 NETMASK=255.255.255.0 IPADDR=192.168.10.102 GATEWAY=192.168.10.1 TYPE=Ethernet USERCTL=no IPV6INIT=no PEERDNS=yes


# Intel Corporation 82540EM Gigabit Ethernet Controller DEVICE=eth1 BOOTPROTO=none ONBOOT=yes HWADDR=08:00:27:40:4d:a9 NETMASK=255.255.255.0 IPADDR=192.168.20.102 GATEWAY=192.168.20.1 TYPE=Ethernet USERCTL=no IPV6INIT=no PEERDNS=yes

Delete files with .bak extension and reboot the machine. [ROOT]

rm /etc/sysconfig/network-scripts/ifcfg-eth0.bak rm /etc/sysconfig/network-scripts/ifcfg-eth1.bak

Note: You can also use Linux GUI to change network settings by choosing System > Administration >

Network.

Change the hostname -> edit /etc/sysconfig/network [ROOT]

NETWORKING=yes NETWORKING_IPV6=yes HOSTNAME=rac02.taysi.com

CHANGE THE BASH PROFILES OF RAC02 NODE

Switch to oracle user and edit the bash_profile [ORACLE]

# Oracle Settings TMP=/tmp;export TMP TMPDIR=$TMP;export TMPDIR ORACLE_HOSTNAME=rac02.taysi.com;export ORACLE_HOSTNAME ORACLE_UNQNAME=RAC;export ORACLE_UNQNAME ORACLE_BASE=/u02/app/oracle;export ORACLE_BASE ORACLE_HOME=/u02/app/oracle/product/11.2.0/db_home;export ORACLE_HOME ORACLE_SID=RAC2;export ORACLE_SID ORACLE_TERM=xterm;export ORACLE_TERM BASE_PATH=/usr/sbin:$PATH;export BASE_PATH PATH=$ORACLE_HOME/bin:$BASE_PATH;export PATH LD_LIBRARY_PATH=$ORACLE_HOME/lib:/lib:/usr/lib; export LD_LIBRARY_PATH CLASSPATH=$ORACLE_HOME/JRE:$ORACLE_HOME/jlib:$ORACLE_HOME/rdbms/jlib; export CLASSPATH if [ $USER = "oracle" ]; then if [ $SHELL = "/bin/ksh" ]; then ulimit -p 16384 ulimit -n 65536 else ulimit -u 16384 -n 65536 fi fi

Switch to grid user and edit the bash_profile [GRID]

# Grid Settings TMP=/tmp;export TMP TMPDIR=$TMP;export TMPDIR ORACLE_HOSTNAME=rac02.taysi.com;export ORACLE_HOSTNAME ORACLE_UNQNAME=RAC;export ORACLE_UNQNAME ORACLE_BASE=/u01/app/grid;export ORACLE_BASE ORACLE_HOME=/u01/app/11.2.0/grid;export ORACLE_HOME ORACLE_SID=RAC2;export ORACLE_SID ORACLE_TERM=xterm;export ORACLE_TERM BASE_PATH=/usr/sbin:$PATH;export BASE_PATH PATH=$ORACLE_HOME/bin:$BASE_PATH;export PATH LD_LIBRARY_PATH=$ORACLE_HOME/lib:/lib:/usr/lib; export LD_LIBRARY_PATH CLASSPATH=$ORACLE_HOME/JRE:$ORACLE_HOME/jlib:$ORACLE_HOME/rdbms/jlib;export CLASSPATH if [ $USER = "oracle" ]; then if [ $SHELL = "/bin/ksh" ]; then ulimit -p 16384 ulimit -n 65536 else ulimit -u 16384 -n 65536 fi fi

# At this point we‟ve completed the configuration of RAC Nodes.

DNS Server Creation

From Oracle 11.2.0.2 onward, we need a DNS server to define SCAN addresses.

Just create a new Oracle Linux VM Server as we did in step “RAC Node 1 Creation”, but this time allocate

only 512MB of memory and a limited disk space. Install all the RPMs that we did earlier. Name the server

“RACDNS”

Edit /etc/hosts [ROOT]

127.0.0.1 racdns.racdomain.com racdns localhost.localdomain localhost ::1 localhost6.localdomain6 localhost6 192.168.10.100 racdns.taysi.com racdns

Edit /etc/named.conf [ROOT]

zone "taysi.com" IN { type master; allow-update { none; }; file "taysi.com.zone"; }; zone "10.168.192.in-addr.arpa." IN { type master; file "10.168.192.zone"; allow-update { none; }; };

Edit -> vi /var/named/taysi.com.zone [ROOT]

$TTL 86400 @ IN SOA localhost root.localhost ( 42 ; serial (d. adams) 3H ; refresh 15M ; retry 1W ; expiry 1D ) ; minimum IN NS localhost localhost IN A 127.0.0.1 rac01 IN A 192.168.10.101 rac02 IN A 192.168.10.102 rac01-priv IN A 192.168.20.101 rac02-priv IN A 192.168.20.102 rac01-vip IN A 192.168.10.111 rac02-vip IN A 192.168.10.112 rac-scan IN A 192.168.10.121 rac-scan IN A 192.168.10.122 rac-scan IN A 192.168.10.123

Edit -> vi /var/named/10.168.192.zone [ROOT]

$ORIGIN 10.168.192.in-addr.arpa. $TTL 1H @ IN SOA dnserver.taysi.com. root.dnsserver.taysi.com. ( 2 3H 1H 1W 1H ) 10.168.192.in-addr.arpa. IN NS dnsserver.taysi.com. 101 IN PTR rac01.taysi.com. 102 IN PTR rac02.taysi.com. 111 IN PTR rac01-vip.taysi.com. 112 IN PTR rac02-vip.taysi.com. 121 IN PTR rac-scan.taysi.com. 122 IN PTR rac-scan.taysi.com. 123 IN PTR rac-scan.taysi.com.

Edit -> vi /etc/resolv.conf [ROOT]

search taysi.com nameserver 192.168.10.100

Copy zone files [ROOT]

cd /var/named cp /usr/share/system-config-bind/profiles/default/named/* .

Edit -> vi /etc/sysconfig/network [ROOT]

NETWORKING=yes NETWORKING_IPV6=no HOSTNAME=racdns.racdomain.com

Edit on both RAC01 and RAC02 nodes -> vi /etc/resolv.conf [ROOT]

nameserver 192.168.10.100 search taysi.com

On DNS server test the DNS resolution of both nodes [ROOT]

[root@racdns ~]# nslookup rac01 Server: 192.168.10.100 Address: 192.168.10.100#53 Name: rac01.taysi.com Address: 192.168.10.101 [root@racdns ~]# nslookup rac02 Server: 192.168.10.100 Address: 192.168.10.100#53 Name: rac02.taysi.com Address: 192.168.10.102

Installation and Configuration of ZFS

Download Oracle ZFS simulator from Oracle web site.

Then run the VirtualBox and import by File > Import Appliance.

Image-51

Open appliance

Image-52

Choose the ovf file and click the “Open” button.

Image-53

Click the “Next” button to continue.

Image-54

Click the “Import” button to begin import process.

Image-55

It may take 3-5 minutes to import…

Image-56

Start the virtual machine.

Image-57

Press the „Enter‟ key to select the Storag of interest. (Here we have only one…)

Image-58

Press any key to begin configuration

Image-59

Config as seen below and press “ESC” then “1” to save config.

Image-60

Login to web console using a compatible browser by using the link -> https://192.168.10.200:215/

Enter username and password you prvieded in the configuration.

Press the “Commit” button without changing anything.

Image-61


Image-62


Image-63


Image-64

PRESS the “+” button left to “Available Pool” to add a pool.

Image-65

Allocate all devices (10 disks, since we‟ll not create any other pool) and then “Commit”

Image-66

Choose storage profile (in our case striped is sufficient)

Image-67

Commit

Image-68

After skipping the register part, the Installation is done.

ADD FILESYSTEMS

Table-4

File System Record Size Syn. Write Cache Dev. Usage

SHAREDCONFIG01 128K Throughput All data & metadata



ORCL_DATA_FS 8K Throughput All data & metadata

ORCL_RECO_FS 128K Throughput All data & metadata

ORCL_REDO_FS 128K Throughput All data & metadata

File System Architecture

Table-5

NAME PATH MOUNT POINT

Clusterware Base /u01/app/grid Local File System

Clusterware Home /u01/app/11.2.0/grid Local File System

OCR File 1 /sharedconfig01/ocr1 ZFS: SHARED_CONFIG to /sharedconfig01



VDSK File 1 /sharedconfig01/vdsk1 ZFS: SHARED_CONFIG to /sharedconfig01



Standart Database Base /uxx/app/<DB_OWNER> Local File System

Standart Database Home /uxx/app/<DB_OWNER>/product/11.2.0/db_home Local File System

Standart Database Datafiles /uxx/app/<DB_OWNER>/<DB_NAME>/data/data

ZFS: <DB_NAME>_DATA_FS to /uxx/app/<DB_OWNER>/<DB_NAME>/data

Standart Database Index /uxx/app/<DB_OWNER>/<DB_NAME>/data/index

Standart Database Undo TS /uxx/app/<DB_OWNER>/<DB_NAME>/data/undo

Standart Database Temp TS /uxx/app/<DB_OWNER>/<DB_NAME>/data/temp

Standart Database Redo Log Group 1 /uxx/app/<DB_OWNER>/<DB_NAME>/redo/redo01 ZFS: <DB_NAME>_REDO_FS to /uxx/app/<DB_OWNER>/<DB_NAME>/redo

Standart Database Redo Log Group 2 /uxx/app/<DB_OWNER>/<DB_NAME>/redo/redo02

Standart Database Redo Log Group 3 /uxx/app/<DB_OWNER>/<DB_NAME>/redo/redo03

Standart Database Backup Dir /uxx/app/<DB_OWNER>/<DB_NAME>/reco/backup ZFS: <DB_NAME>_RECO_FS to /uxx/app/<DB_OWNER>/<DB_NAME>/reco

Standart Database Arc. Log Dir 1 /uxx/app/<DB_OWNER>/<DB_NAME>/reco/reco01

Standart Database Arc. Log Dir 2 /uxx/app/<DB_OWNER>/<DB_NAME>/reco/reco02

EBS Database Base /uxx/app/<DB_OWNER> Local File System

EBS Database Home /uxx/<DB_NAME>/tech_st/11.2.0 Local File System

EBS Database Datafiles /uxx/<DB_NAME>/apps_st/data/data

ZFS: <DB_NAME>_DATA_FS to /uxx/<DB_NAME>/data

EBS Database Index /uxx/<DB_NAME>/apps_st/data/index

EBS Database Undo TS /uxx/<DB_NAME>/apps_st/data/undo

EBS Database Temp TS /uxx/<DB_NAME>/apps_st/data/temp

EBS Database Redo Log Group 1 /uxx/<DB_NAME>/apps_st/redo/redo01 ZFS: <DB_NAME>_REDO_FS to /uxx/<DB_NAME>/redo

EBS Database Redo Log Group 2 /uxx/<DB_NAME>/apps_st/redo/redo02

EBS Database Redo Log Group 3 /uxx/<DB_NAME>/apps_st/redo/redo03

EBS Database Backup Dir /uxx/<DB_NAME>/apps_st/reco/backup ZFS: <DB_NAME>_RECO_FS to /uxx/<DB_NAME>/reco

EBS Database Arc. Log Dir 1 /uxx/<DB_NAME>/apps_st/reco/reco01

EBS Database Arc. Log Dir 2 /uxx/<DB_NAME>/apps_st/reco/reco02

Add filesystems from Shares Tab by clicking the “+” button left to the string Filesystems

Image-69

Create the filesystem as follows

Image-70

Edit the filesystem by clicking the pencil icon of the file system.

Image-71

SHARES > GENERAL

Uncheck the “Inherit from project” checkbox and then change the Database record size according to the

table-4 values.

Image-72

SHARES > PROTOCOLS

Uncheck the “Inherit from project” checkbox and then add NFS exceptions as seen blow. Those IP adresses

are the RAC01 and RAC02 public IPs. /24 means IPv4 protocol. Check the ROOT ACCESS checkboxes.

Image-73

Repeat the steps between Image-69 and Image-73 to define te filesystems in the listed in table-4.

DEFINE MOUNT POINTS

Create paths [ROOT]

mkdir -p /sharedconfig01 mkdir -p /sharedconfig02 mkdir -p /sharedconfig03 mkdir -p /u02/app/oracle/ORCL/redo mkdir -p /u02/app/oracle/ORCL/data mkdir -p /u02/app/oracle/ORCL/reco chown -R grid:oinstall /sharedconfig01 chown -R grid:oinstall /sharedconfig02 chown -R grid:oinstall /sharedconfig03 chown -R oracle:oinstall /u02/app/oracle/ORCL

Edit fstab on both RAC01 and RAC02 nodes. Reboot the servers after editing. [ROOT]

192.168.10.200:/export/SHAREDCONFIG01 /sharedconfig01 nfs rw,bg,hard,nointr,rsize=32768,wsize=32768,tcp,noacl,actimeo=0,vers=3,timeo=600 0 0 192.168.10.200:/export/SHAREDCONFIG02 /sharedconfig02 nfs rw,bg,hard,nointr,rsize=32768,wsize=32768,tcp,noacl,actimeo=0,vers=3,timeo=600 0 0 192.168.10.200:/export/SHAREDCONFIG03 /sharedconfig03 nfs rw,bg,hard,nointr,rsize=32768,wsize=32768,tcp,noacl,actimeo=0,vers=3,timeo=600 0 0 192.168.10.200:/export/ORCL_DATA_FS /u02/app/oracle/ORCL/data nfs rw,bg,hard,nointr,rsize=32768,wsize=32768,tcp,noacl,actimeo=0,vers=3,timeo=600 0 0 192.168.10.200:/export/ORCL_RECO_FS /u02/app/oracle/ORCL/reco nfs rw,bg,hard,nointr,rsize=32768,wsize=32768,tcp,noacl,actimeo=0,vers=3,timeo=600 0 0 192.168.10.200:/export/ORCL_REDO_FS /u02/app/oracle/ORCL/redo nfs rw,bg,hard,nointr,rsize=32768,wsize=32768,tcp,noacl,actimeo=0,vers=3,timeo=600 0 0

Test on both nodes RAC01 and RAC02 that the mounts are OK.

[root@rac01 grid]# df -h Filesystem Size Used Avail Use% Mounted on /dev/mapper/VolGroup00-LogVol00 26G 6.9G 17G 29% / /dev/sda1 145M 24M 114M 17% /boot tmpfs 1002M 0 1002M 0% /dev/shm 192.168.10.200:/export/SHAREDCONFIG 49G 32K 49G 1% /u01/sharedconfig 192.168.10.200:/export/ORCL_DATA_FS 49G 32K 49G 1% /u02/app/oracle/ORCL/data 192.168.10.200:/export/ORCL_RECO_FS 49G 32K 49G 1% /u02/app/oracle/ORCL/reco 192.168.10.200:/export/ORCL_REDO_FS 49G 32K 49G 1% /u02/app/oracle/ORCL/redo

CRS Installation

Provide SSH connectivity [ROOT]

Go to the grid installation files path …grid/sshsetup and run the ssh setup script (On RAC01 or RAC02)

cd /setup/grid/sshsetup ./sshUserSetup.sh -user oracle -hosts "rac01 rac02" -advanced -exverify -confirm ./sshUserSetup.sh -user grid -hosts "rac01 rac02" -advanced -exverify -confirm

Verify the cluster structure before the installation

cd /setup/grid ./runcluvfy.sh stage -pre crsinst -n rac01,rac02 -verbose > result.txt

Examine the output file “result.txt”, at the end of the file we should see the phrase:

Pre-check for cluster services setup was successful.

BEGIN CRS INSTALLATION

su - grid cd /setup/grid/ ./runInstaller

Image-74

Image-75

Image-76

Image-77

Image-78

Image-79

Image-80

Image-81

Image-82

Image-83

Image-84

Image-85

Image-86

Image-87

Image-88

Image-89

Image-90

Image-91

Database Software Installation

su - oracle cd /setup/database ./runInstaller

Image-92

Image-93

Image-94

Image-95

Image-96

Image-97

Image-98

Image-99

Image-100

After executing the root.sh scripts, the software installation is done.

Database Installation

On both nodes RAC01 and RAC02 [with oracle user]

su - oracle mkdir -p /u02/app/oracle/ORCL/data/data mkdir -p /u02/app/oracle/ORCL/data/index mkdir -p /u02/app/oracle/ORCL/data/undo mkdir -p /u02/app/oracle/ORCL/data/temp mkdir -p /u02/app/oracle/ORCL/redo/redo01 mkdir -p /u02/app/oracle/ORCL/redo/redo02 mkdir -p /u02/app/oracle/ORCL/redo/redo03 mkdir -p /u02/app/oracle/ORCL/reco/backup mkdir -p /u02/app/oracle/ORCL/reco/reco01 mkdir -p /u02/app/oracle/ORCL/reco/reco02 chown -R oracle:oinstall /u02/app/oracle/ORCL chmod -R 775 /u02/app/oracle/ORCL

Run dbca command with oracle user

dbca

Image-101

Image-102

Image-103

Image-104

Image-105

Image-106

Image-107

Image-108

Image-109

Image-110 (Also select the charset of the database here)

Image-111

Image-112

Image-113

Installation is DONE...

Oracle RAC 11g Installation on ZFS...Sep 01, 2019 · ZFS ZFS Storage Appliance 700_1 “Sun ZFS...

Documents

Transcript of Oracle RAC 11g Installation on ZFS...Sep 01, 2019 · ZFS ZFS Storage Appliance 700_1 “Sun ZFS...