Leveraging Oracle ASM Cluster File System for Private ... · PDF file6/6/2016 ·...

33
Leveraging Oracle ASM Cluster File System for Private Cloud Storage Kai Yu Oracle Solutions Engineering Dell Inc

Transcript of Leveraging Oracle ASM Cluster File System for Private ... · PDF file6/6/2016 ·...

Leveraging Oracle ASM Cluster File System for Private Cloud Storage

Kai YuOracle Solutions EngineeringDell Inc

About AuthorKai Yu, Senior Principal Architect, Dell Database Engineering

21 years live in Austin and work on Oracle Technology

Specializing in Oracle RAC, Virtualization and Database

Infrastructure

Oracle ACE Director, authored 120 + presentations and 30 +

whitepapers

2011 OAUG Innovator of Year

2012 Oracle Excellence Award: Technologist of the Year: Cloud

Architect by Oracle Magazine

IOUG Spotlight for March 2016

Co-founder and VP of IOUG Cloud Computing SIG

My Oracle Blog: http://kyuoracleblog.wordpress.com/

Co-author Apress Book “Expert Oracle RAC 12c”

Tiered Storage Solution for Oracle 12c RAC database

Existing SAN

Flash based Storage

Oracle RAC Database

Rapidly deliver

Agenda

Oracle ACFS for Cloud Applications

Configuration of Oracle ACFS

Leverage ACFS Snapshot Feature

Oracle ACFS or ASM for Database

Oracle ACFS for Cloud Applications

Oracle ASM Cluster File System (ACFS):

• Initially introduced in Oracle 11gR2 to store non-Oracle Database files.

• ACFS mounted in OS and accessible through OS commands

• A cluster file system shared and accessible by single/cluster nodes with unified namespace.

• ACFS files stored in the Oracle ASM diskgroup with benefits from ASM

• IO load balancing through data stripping among the ASM disks

• Data reliability through mirroring within the ASM diskgroup

Oracle ACFS for Cloud Applications

What can be stored in Oracle ASM Cluster File System (ACFS)

• ACFS supports general purpose files, including applications files, data files, log files, etc ,across multiple OS and planforms,

• Starting with Oracle 12cR1, it support Oracle Database files except for data files and redologs for Oracle Restart

• All databases stored on Oracle ACFS can leverage Advanced Database services: snapshots, tagging, security, encrypting, and auditing

• ACFS doesn’t support Oracle Grid Infrastructure software , which should be stored in local disks

Oracle ACFS for Cloud Applications

Oracle Automatic Storage Management Stack• Oracle ASM provide the basic functionality of Automatic Storage

Management

• Oracle ASM files for Oracle Database supports diskgroups for database files

• Oracle ASM Dynamic Volume Manager (ADVM) : provides the logical volume management and standard device interface to files systems such as ACFS, OCFS2 and regular ext4 file systems

• Oracle RAC database can be stored in Oracle ASM files as well as in Oracle ACFS.

Oracle ACFS for Cloud Applications

Oracle ACFS Integration with Oracle ASM• Oracle ACFS is always configured with Oracle ASM storage.

• Traditional Storage Devices presented to Oracle ASM

• ASM diskgroup is built with the storage devices, presented to Oracle ADVM

• Oracle ADVM volumes are created with the ADVM in the ASM diskgroup and presented to OS as an OS logical block device.

• Oracle ACFS file system is bound to the Oracle ADVM volumes through the file system creation

ASM

Volume

ASM

Volume

ASM Diskgroup FG

ASM Disks

ADVM

ASM Cluster File System(ACFS)

/acfs1 /acfs2 ASM

Volume

ASM

Volume

ASM

Volume

/acfs3

Oracle ASM

ASM Diskgroup

DG

ASM Disks

RAC

DB1

RAC

DB2

Cluser

ware

Oracle ACFS for Cloud Applications

Oracle ACFS and Oracle Clusterware

• Oracle ACFS is based on Oracle ASM, and is tightly coupled with Oracle Clusterware , directly participate Clusterware cluster membership state transaction and resource based high availability (HA) management

• Oracle ACFS is a cluster enabled file system that can shared by multiple cluster nodes

• Oracle 12c RAC database files can stored in Oracle ASM diskgroups as well as Oracle ACFS.

Oracle ACFS for Cloud Applications

Oracle ACFS Resource-based Management

• Oracle clusterware resource support ACFS for enabling and disabling

volumes, loading drivers and mounting and unmounting file systems:

• When Oracle ASM instance starts, It starts the Oracle ACFS driver,

Oracle ADVM driver, Oracle Kernel Service drivers (OKS)

• Oracle ACFS file system can be automatically mounted or

dismounted based on Oracle clusterware resource action.

Oracle ACFS for Cloud Applications

ACFS on Standard ASM

• Each node has an ASM instance which costs CPU/memory

• Local ASM instance failure will cause DB instance failure

• ACFS on Standard ASM

.

Oracle ACFS for Cloud Applications

Oracle ACFS with Flex ASM for higher flexibility, scalability and availability

• Flex ASM is an option on Oracle 12c : enabled or disabled

• A small # of ASM instance. (default 3, specified by admin)

• DB instances connects to any ASM instance (local/remote)

Oracle ACFS for Cloud Applications

Two kinds of Oracle ASM configurations:• Local ASM clients connect to local ASM instance

• Flex ASM clients connect to a remote ASM instance

ASM network• Added for Flex ASM for communication between ASM clients and ASM

• On Oracle 12cR1, share the network with cluster private interconnect

Oracle ACFS for Cloud Applications

• Oracle ACFS with Flex ASM for higher flexibility, scalability and availability Database instance can connect to a remote ASM instance once the local ASM instance fails

• ADVM and Oracle ACFS file system leverage the Flex ASM for a better HA

• ADVM and ACFS utilize an Oracle ASM proxy to connect a remote

ASM instance

Configuring Oracle ACFS Prerequisites of creating Oracle ACFS

• Oracle Grid Infrastructure is installed/configured with Oracle ASM instance running:

[grid@inmem1 bin]$ /u01/app/12.1.0/grid/bin/crsctl check crsCRS-4638: Oracle High Availability Services is onlineCRS-4537: Cluster Ready Services is online

CRS-4529: Cluster Synchronization Services is onlineCRS-4533: Event Manager is online[grid@inmem1 bin]$ ps -ef | grep pmon | grep ASM

grid 31863 1 0 2015 ? 00:10:36 asm_pmon_+ASM1

• Required modules are loaded: These modules should be automatically loaded in Oracle 12c RAC.

[root@inmem1 ~]# lsmod | grep oracle

oracleacfs 3307969 3

oracleadvm 506254 8oracleoks 505749 2 oracleacfs,oracleadvm

Configuring Oracle ACFS Outline of ACFS creation:

• ACFS Creation Process:

• Create ASM diskgroup

• Create ASM volumes

• Create ACFS file system

• Tools for ACFS creation:

• ASMCA GUI tool

• Oracle Enterprise Manager

• ASMCMD command line tool

• SQLPLUS

Configuring Oracle ACFS

Example of creating a ACFS with ASMCMD command line tool:

• Create ASM volume “acfs_vm1” in ASM diskgroup “ACFSDG1”

ASMCMD> volcreate -G ACFSDG1 -s 1g acf_vm

• Create ACFS file system on ASM volume

mkfs -t acfs /dev/asm/acf_vm1-105

mkfs.acfs: version = 12.1.0.1.0

mkfs.acfs: on-disk version = 39.0

mkfs.acfs: volume = /dev/asm/acf_vm1-105

mkfs.acfs: volume size = 1073741824

mkfs.acfs: Format complete.

• Mount the acfs file to an OS mount /acfs_01#mkdir /acfs_vol1

#/sbin/acfsutil registry -a /dev/asm/acf_vm1-105 /acfs_vol1

acfsutil registry: mount point /acfs_vol1 successfully added to Oracle Registry

Now you can see the new ACFS file system mounted to /acfs_vol1:

# df -k | grep 'Filesystem \|asm'

Filesystem 1K-blocks Used Available Use% Mounted on

/dev/asm/acf_vm1-105 1048576 119112 929464 12% /u01/acfs/asm_vol1

Configuring Oracle ACFS

Example of creating a ACFS with asmca GUI tool

• Create ASM volume “SSD_ACFS_VM” in ASM diskgroup “SSD_ACFS”

Configuring Oracle ACFS

Example of creating a ACFS with asmca GUI tool

• Create a ACFS file system on the ASM volume

[

Configuring Oracle ACFS

root@inmem1 ~]# df -k | grep 'Filesystem \|asm'Filesystem 1K-blocks Used Available Use% Mounted on/dev/asm/ssd_acfs_vm-329 5077204992 3978716232 1098488760 79% /SSD_ACFS

# /u01/app/grid/cfgtoollogs/asmca/scripts/acfs_script.shACFS file system /SSD_ACFS is mounted on nodes inmem1,inmem2

Configuring Oracle ACFS Creating ACFS for Oracle Database

• Use asmca GUI tool to create ACFS for Oracle Database as well as Oracle RAC Home

Configuring Oracle ACFS

Creating a ACFS for Oracle Database

• Create Oracle RAC database on ACFS with dbca: • ACFS supports Oracle 12c as well as pre-12c DBs

Configuring Oracle ACFS

Creating a ACFS for Oracle Database• After dbca, you can see the 12c RAC database configuration:

Leverage ACFS Snapshot feature

Oracle ACFS snapshot • An online, read-only or read-write, point in time copy of the Oracle

ACFS file system.• Immediately available, always online, can be resized.• Snapshot is stored in the file system . ACFS/snaps directory , can be

resized.• ACFS director not visible

Snapshot is in /SAN_ACFS/.ACFS/snaps/ro_dbsnap

Leverage ACFS Snapshot Feature

Creating an Oracle ACFS Snapshot

• Read-write snapshot can be read and written without impacting of ACFS system as a copy of the production system for testing without impacting the original production data.

• Read-write snapshot : the diskgroup compatiability attribute for ADVM set to 11.2.0.3.0 or higher.

• Take a snapshot for another snapshot, or convert a snapshot between read-only and read-write

• set diskgroup compatiability attribute for ADVM set to 12.1 or higher)

• The commands to create a snapshot :

#acfsutil snap create snapshot_name ACFS_name

• Check the snapshot status:

#acfsutil snap info ACFS_name

Leverage ACFS Snapshot Feature

Perform database backup with Oracle ACFS snapshot • Oracle ACFS in Oracle GI 12.1 supports all Oracle database files

(11.2.0.4 or higher) • Use AFCS snapshot to do a quick backup or clone the database.

Verify the database files stored in the ACFS file system:oracle@inmem1 ACFSDB]$ pwd/SAN_ACFS/db_acfs/ACFSDB[oracle@inmem1 ACFSDB]$ lsarchivelog controlfile datafile onlinelog

Leverage ACFS Snapshot Feature

Perform database backup with Oracle ACFS snapshot • Create an ACFS snapshot:

#acfsutil snap create ro_dbsnap /SAN_ACFS

#acfsutil snap create: Snapshot operation is complete.

• Check the snapshot status

# acfsutil snap info /SAN_ACFSsnapshot name: ro_dbsnapsnapshot location: /SAN_ACFS/.ACFS/snaps/ro_dbsnapRO snapshot or RW snapshot: ROparent name: /SAN_ACFSsnapshot creation time: Wed Jan 30 14:48:43 2016

Leverage ACFS Snapshot Feature

Perform database backup with Oracle ACFS snapshot • What the snapshot looks like:the database files stored in the ACFS file system:

database files in the Snapshot:

• Restore the database from the snapshot

The snapshot can be used to restore the database in case we lose the original database files in the ACFS system.

. Copy the files from the snapshot to the original directories with the cp command

cp /SAN_ACFS/.ACFS/snaps/ro_dbsnap/db_acfs/ACFSDB/onlinelog/*

/SAN_ACFS/db_acfs/ACFSDB/onlinelog

cp /SAN_ACFS/.ACFS/snaps/ro_dbsnap/db_acfs/ACFSDB/datafile/*

/SAN_ACFS/db_acfs/ACFSDB/datafile

Leverage ACFS Snapshot Feature

• Startup the database after restore.

. Create a read-write snapshot of the read snapshot as the database files (create OS softlinks from the original DB file location to the read-write snapshot

$acfsutil snap create -w -p ro_dbsnap rw_dbsnap /SAN_ACFS

# acfsutil snap info /SAN_ACFSsnapshot name: ro_dbsnapsnapshot location: /SAN_ACFS/.ACFS/snaps/ro_dbsnapRO snapshot or RW snapshot: ROparent name: /SAN_ACFSsnapshot creation time: Wed Jan 30 14:48:43 2016snapshot name: rw_dbsnap

snapshot location: /SAN_ACFS/.ACFS/snaps/rw_dbsnap

RO snapshot or RW snapshot: RW

parent name: /SAN_ACFS

snapshot creation time: Wed Jan 30 17:01:20 2016

……

Oracle ACFS Other Usage

High Availability NFS (HANFS) • Provide un-interrupted service of NFS exports

• Exports based on Highly Availability Virtual IPs(HAVIP)

• Oracle Clusterware agents ensure HAVIP and NFS

NFS export always online

• HAFS is configured using ACFS cluster

• If a cluster node fails, the HAVIP and NFS services are

are automatically taken over by a surviving

Oracle Database Appliance (ODA) • Use ACFS to store database files.

• Database Appliance Virtualized Platforms

use ACFS for shared storage access.

Oracle E-Business Suite uses OCFS2 instead of ACFS as the shared storage in the multi-node configuration.

Oracle ACFS or ASM for Database

Standalone database • Use a regular OS file system such as Linux and Windows file system• If use ASM, need to install Oracle Grid infrastructure for single node:

Oracle Restart and ASM • Oracle ACFS doesn’t support database files and redo files in Oracle

Restart configuration Oracle Database on Oracle Database Appliance

• Use ACFS to store the database files.

Oracle RAC Database • Need Oracle GI 12.1 to use Oracle ACFS for database• Pre-11.2.0.4 databases are not supported by ACFS, must use ASM• Pre-12.1.0.2 Oracle databases on Windows are not supported by

ACFS• Use ACFS for advanced features such as snapshot – for test and

development• ASM and ACFS offer the similar performance• ACFS is built on ASM and has all the benefits of ASM management• Oracle Databases Appliance uses ACFS for all databases

References

1. Oracle Automatic Storage Management Administrator's Guide, 12cR1,

Part III Oracle Automatic Storage Management Cluster File System

2. Oracle Whitepaper : Oracle ACFS, July 2014

3. Integrating ACFS Snapshots With RMAN, by Anju Garg, All things Oracle from

Redgate

4. Benefits of Oracle ACFS , Oracle Whitepaper 2Janurary 2015

5. Export Oracle RAC 12c, Apress , by Riyaj Shamsudeen, Syed Jaffar Hussain,

Tariq Farooq, Kai Yu , Chapter 5, Storage and ASM Practices

6. OCFS2 Certified with EBS 12.2 Shared File System Configurations

By Steven Chan - EBS-Oracle on Jun 06, 2016

Thank You and QAContact me at [email protected] or visit my Oracle Blog at http://kyuoracleblog.wordpress.com/

33