Leveraging Oracle ASM Cluster File System for …...Oracle ACFS for Cloud Applications Oracle ASM...

Leveraging Oracle ASM Cluster File System for Private Cloud Storage

Kai YuOracle Solutions EngineeringDell Inc

About AuthorKai Yu, Senior Principal Architect, Dell Database Engineering

21 years Oracle DBA/Apps DBAS and Solutions Engineering

Specializing in Oracle RAC, Virtualization and Database

Infrastructure

Oracle ACE Director, authored 120 + presentations and 30 +

whitepapers

2011 OAUG Innovator of Year

2012 Oracle Excellence Award: Technologist of the Year: Cloud

Architect by Oracle Magazine

IOUG Spotlight for March 2016

Co-founder and VP of IOUG Cloud Computing SIG

My Oracle Blog: http://kyuoracleblog.wordpress.com/

Co-author Apress Book “Expert Oracle RAC 12c”

https://apex.oracle.com/pls/apex/f?p=19297:4:100228132442593::NO:4:P4_ID:1960

https://apex.oracle.com/pls/apex/f?p=19297:4:100228132442593::NO:4:P4_ID:1960

Agenda

Oracle ACFS for Cloud Applications

Configuration of Oracle ACFS

Leverage ACFS Snapshot Feature

Oracle ACFS or ASM for Database


Oracle ASM Cluster File System (ACFS):

• Initially introduced in Oracle 11gR2 to store non-Oracle Database files.

• ACFS mounted in OS and accessible through OS commands

• A cluster file system shared and accessible by single/cluster nodes with unified namespace.

• ACFS files stored in the Oracle ASM diskgroup with benefits from ASM

• IO load balancing through data stripping among the ASM disks

• Data reliability through mirroring within the ASM diskgroup


Oracle ASM Cluster File System (ACFS):

• ACFS doesn’t support Oracle Grid Infrastructure, which should be stored in local disks

• All databases stored on Oracle ACFS can leverage Advanced Database services: snapshots, tagging, security, encrypting, and auditing

• Starting with Oracle 12cR1, it supports general purpose files as well as database files across multiple OS and planforms, except for data files and redologs for Oracle Restart

Oracle ACFS for cloud applications

Oracle Automatic Storage Management Stack• Oracle ASM provide the basic functionality of Automatic Storage Management

• Oracle ASM files for Oracle Database supports diskgroups for database files

• Oracle ASM Dynamic Volume Manager (ADVM) : provides the logical volume management and standard device interface to files systems such as ACFS, OCFS2 and regular ext4 file systems

• Oracle RAC database can be stored in Oracle ASM files as well as in Oracle ACFS.

Oracle ACFS for Cloud Applications Oracle ACFS Integration with Oracle ASM

• Oracle ACFS is always configured with Oracle ASM storage.

• Traditional Storage Devices presented to Oracle ASM

• ASM diskgroup is built with the storage devices, presented to Oracle ADVM

• Oracle ADVM volumes are created with the ADVM in the ASM diskgroup and presented to OS as an OS logical block device.

• Oracle ACFS file system is bound to the Oracle ADVM volumes through the file system creation

ASM

Volume

ASM

Volume

ASM Diskgroup FG

ASM Disks

ADVM

ASM Cluster File System(ACFS)

/acfs1 /acfs2 ASM

Volume

ASM

Volume

ASM

Volume

/acfs3

Oracle ASM

ASM Diskgroup

DG

ASM Disks

RAC

DB1

RAC

DB2

Cluser

ware


Oracle ACFS and Oracle Clusterware

• Oracle ACFS is based on Oracle ASM, and is tightly coupled with Oracle Clusterware , directly participate Clusterware cluster membership state

transaction and resource based high availability (HA) management

• Oracle ACFS is a cluster enabled file system that can shared by multiple

cluster nodes.

• Oracle 12c RAC database files can stored in Oracle ASM diskgroups as well

as Oracle ACFS.


Oracle ACFS Resource-based Management

• Oracle clusterware resource support ACFS for enabling and disabling

volumes, loading drivers and mounting and unmounting file systems:

• When Oracle ASM instance starts, It starts the Oracle ACFS driver,

Oracle ADVM driver, Oracle Kernel Service drivers (OKS)

• Oracle ACFS file system can be automatically mounted or

dismounted based on Oracle clusterware resource action.


Limitations of the standard ASM

• Each node has an ASM instance which costs CPU/memory

• Local ASM instance failure will cause DB instance failure

.


Oracle ACFS with Flex ASM for higher flexibility, scalability and availability

• Flex ASM is an option on Oracle 12c : enabled or disabled

• A small # of ASM instance. (default 3, specified by admin)

• DB instances connects to any ASM instance (local/remote)


Two kinds of Oracle ASM configurations:• Local ASM clients connect to local ASM instance

• Flex ASM clients connect to a remote ASM instance

ASM network• Added for Flex ASM for communication between ASM clients and ASM

• On Oracle 12cR1, share the network with cluster private interconnect


• Oracle ACFS with Flex ASM for higher flexibility, scalability and availability Database instance can connect to remote ASM instance if local ASM instance fails

• ADVM and Oracle ACFS file system leverage the Flex ASM for a better HA

• ADVM and ACFS utilize an Oracle ASM proxy to connect remote

ASM instance

Configuring Oracle ACFS Prerequisites of creating Oracle ACFS

• Oracle Grid Infrastructure is installed/configured with Oracle ASM instance running:

[grid@inmem1 bin]$ /u01/app/12.1.0/grid/bin/crsctl check crsCRS-4638: Oracle High Availability Services is onlineCRS-4537: Cluster Ready Services is online

CRS-4529: Cluster Synchronization Services is onlineCRS-4533: Event Manager is online[grid@inmem1 bin]$ ps -ef | grep pmon | grep ASM

grid 31863 1 0 2015 ? 00:10:36 asm_pmon_+ASM1

• Required modules are loaded: These modules should be automatically loaded in Oracle 12c RAC.

[root@inmem1 ~]# lsmod | grep oracle

oracleacfs 3307969 3

oracleadvm 506254 8oracleoks 505749 2 oracleacfs,oracleadvm

Configuring Oracle ACFS Outline of the ACFS creation:

• ACFS Creation Process:

• Create ASM diskgroup

• Create ASM volumes

• Create ACFS file system

• Tools for ACFS creation:

• ASMCA GUI tool

• Oracle Enterprise Manager

• ASMCMD command line tool

• SQLPLUS

Configuring Oracle ACFS

Example of creating a ACFS with ASMCMD command line tool:

• Create ASM volume “acfs_vm1” in ASM diskgroup “ACFSDG1”

ASMCMD> volcreate -G ACFSDG1 -s 1g acf_vm

• Create ACFS file system on ASM volume

mkfs -t acfs /dev/asm/acf_vm1-105

mkfs.acfs: version = 12.1.0.1.0

mkfs.acfs: on-disk version = 39.0

mkfs.acfs: volume = /dev/asm/acf_vm1-105

mkfs.acfs: volume size = 1073741824

mkfs.acfs: Format complete.

• Mount the acfs file to an OS mount /acfs_01#mkdir /acfs_vol1

#/sbin/acfsutil registry -a /dev/asm/acf_vm1-105 /acfs_vol1

acfsutil registry: mount point /acfs_vol1 successfully added to Oracle Registry

Now you can see the new ACFS file system mounted to /acfs_vol1:

# df -k | grep 'Filesystem \|asm'

Filesystem 1K-blocks Used Available Use% Mounted on

/dev/asm/acf_vm1-105 1048576 119112 929464 12% /u01/acfs/asm_vol1


Example of creating a ACFS with asmca GUI tool

• Create ASM volume “SSD_ACFS_VM” in ASM diskgroup “SSD_ACFS”


Example of creating a ACFS with asmca GUI tool

• Create a ACFS file system on the ASM volume

[


root@inmem1 ~]# df -k | grep 'Filesystem \|asm'Filesystem 1K-blocks Used Available Use% Mounted on/dev/asm/ssd_acfs_vm-329 5077204992 3978716232 1098488760 79% /SSD_ACFS

# /u01/app/grid/cfgtoollogs/asmca/scripts/acfs_script.shACFS file system /SSD_ACFS is mounted on nodes inmem1,inmem2

Configuring Oracle ACFS Creating ACFS for Oracle Database

• Use asmca GUI tool to create ACFS for Oracle Database as well as Oracle RAC Home


Creating a ACFS for Oracle Database

• Create Oracle RAC database on ACFS with dbca: • ACFS supports Oracle 2c as well as pre-12c DBs


Creating a ACFS for Oracle Database• After dbca, you can see the 12c RAC database configuration:

Leverage ACFS Snapshot feature Oracle ACFS snapshot

• An online, read-only or read-write , point in time copy of the Oracle ACFS file system.

• Immediately available, always online, can be resized.• Snapshot is stored in the file system . ACFS/snaps directory , can be

resized.• ACFS director not visible

Snapshot is in /SAN_ACFS/.ACFS/snaps/ro_dbsnap


Oracle ACFS Snapshot

• Read-write snapshot can be read and written without impacting of ACFS system as a copy of the production system for testing without impacting the original production data.

• Read-write snapshot : the diskgroup compatiability attribute for ADVM set to 11.2.0.3.0 or higher.

• Take a snapshot for another snapshot, or convert a snapshot between read-only and read-write

• set diskgroup compatiability attribute for ADVM set to 12.1 or higher)

• The commands to create a snapshot :

#acfsutil snap create snapshot_name ACFS_name

• Check the snapshot status:

#acfsutil snap info ACFS_name


Perform database backup with Oracle ACFS snapshot • Oracle ACFS in Oracle GI 12.1 supports all Oracle database files

(11.2.0.4 or higher) • Use AFCS snapshot to do a quick backup or clone the database.

Verify the database files stored in the ACFS file system:oracle@inmem1 ACFSDB]$ pwd/SAN_ACFS/db_acfs/ACFSDB[oracle@inmem1 ACFSDB]$ lsarchivelog controlfile datafile onlinelog


Perform database backup with Oracle ACFS snapshot • Create an ACFS snapshot:

#acfsutil snap create ro_dbsnap /SAN_ACFS

#acfsutil snap create: Snapshot operation is complete.

• Check the snapshot status

# acfsutil snap info /SAN_ACFSsnapshot name: ro_dbsnapsnapshot location: /SAN_ACFS/.ACFS/snaps/ro_dbsnapRO snapshot or RW snapshot: ROparent name: /SAN_ACFSsnapshot creation time: Wed Nan 30 14:48:43 2016


Perform database backup with Oracle ACFS snapshot • What the snapshot looks like:the database files stored in the ACFS file system:

database files in the Snapshot:

• Restore the database from the snapshot

The snapshot can be used to restore the database in case we lose the original database files in the ACFS system. You just need to copy the files from the snapshot to the original directories with the cp command

Leverage ACFS Snapshot feature

• Restore the database from Oracle ACFS snapshot cp -r /SAN_ACFS/.ACFS/snaps/ro_dbsnap/db_acfs/ACFSDB/archivelog/*

/SAN_ACFS/db_acfs/ACFSDB/archivelog

cp -r /SAN_ACFS/.ACFS/snaps/ro_dbsnap/db_acfs/ACFSDB/archivelog/*

/SAN_ACFS/db_acfs/ACFSDB/archivelog

cp /SAN_ACFS/.ACFS/snaps/ro_dbsnap/db_acfs/ACFSDB/onlinelog/*

/SAN_ACFS/db_acfs/ACFSDB/onlinelog

cp /SAN_ACFS/.ACFS/snaps/ro_dbsnap/db_acfs/ACFSDB/datafile/*

/SAN_ACFS/db_acfs/ACFSDB/datafile

• Startup the database after restore.

Oracle ACFS Other Usage

High Availability NFS (HANFS) • Provide un-interrupted service of NFS exports

• Exports based on Highly Availability Virtual IPs(HAVIP)

• Oracle Clusterware agents ensure HAVIP and NFS

NFS export always online

• HAFS is configured using ACFS cluster

• If a cluster node fails, the HAVIP and NFS services are

are automatically taken over by a surviving

Oracle Database Appliance (ODA) • Use ACFS to store database files.

• Database Appliance Virtualized Platforms

use ACFS for shared storage access.

Oracle E-Business Suite uses OCFS2 instead of ACFS as the shared storage in the multi-node configuration.

Oracle ACFS or ASM for Database

Standalone database • Use a regular OS file system such as Linux and Windows file system• If use ASM, need to install Oracle Grid infrastructure for single node:

Oracle Restart and ASM • Oracle ACFS doesn’t support database files and redo files in Oracle

Restart configuration Oracle Database on Oracle Database Appliance

• Use ACFS to store the database files.

Oracle RAC Database • Need Oracle GI 12.1 to use Oracle ACFS for database• Pre-11.2.0.4 databases are not supported by ACFS, must use ASM• Pre-12.1.0.2 Oracle databases on Windows are not supported by

ACFS• Use ACFS for advanced features such as snapshot – for test and

development• ASM and ACFS offer the similar performance• ACFS is built on ASM and has all the benefits of ASM management• Oracle Databases Appliance uses ACFS for all databases

References

1. Oracle Automatic Storage Management Administrator's Guide, 12cR1,

Part III Oracle Automatic Storage Management Cluster File System

2. Oracle Whitepaper : Oracle ACFS, July 2014

3. Integrating ACFS Snapshots With RMAN, by Anju Garg, All things Oracle from

Redgate

4. Benefits of Oracle ACFS , Oracle Whitepaper 2Janurary 2015

5. Export Oracle RAC 12c, Apress , by Riyaj Shamsudeen, Syed Jaffar Hussain,

Tariq Farooq, Kai Yu , Chapter 5, Storage and ASM Practices

6. OCFS2 Certified with EBS 12.2 Shared File System Configurations

By Steven Chan - EBS-Oracle on Jun 06, 2016

7. Design and Architecture of Dell Acceleration Appliances for Database (DAAD):

A Practical Approach with High Availability Guaranteed, Kai Y, Yuxiang G,

Peng Z., Meikang Q (2015) In proceedings of 2015 IEEE 17th IHPCC-ICESS-CSS,

New York, USA

8. Implementing Oracle Database12c's Heat Map and Automatic Data Optimization for

Database Storage Performance and Cost, Kai.Yu, Yuxiang Gao, IOUG Collaborate 15

Thank You and QAContact me at [email protected] or visit my Oracle Blog at http://kyuoracleblog.wordpress.com/

32

Leveraging Oracle ASM Cluster File System for …...Oracle ACFS for Cloud Applications Oracle ASM...

Documents

Transcript of Leveraging Oracle ASM Cluster File System for …...Oracle ACFS for Cloud Applications Oracle ASM...