Oracle ASM Cluster File System ACFS

Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |

Mike DietrichMaster Product ManagerDatabase Upgrade and MigrationsOracle Corporation

Oracle ASM Cluster File SystemACFS


Safe Harbor Statement

The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle.

2

Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | 3

@ricardoACFSpm

https://twitter.com/ricardoACFSpm


Germany

https://MikeDietrichDE.com

MikeDietrichDE

Mike DietrichMaster Product ManagerOracle Database Upgrades and Migrations

https://mikedietrichde.com/

https://twitter.com/MikeDietrichDE


Program Agenda

Oracle ACFS – A Complete Storage Solution

Features and how to use them

Test & Dev Provisioning with Oracle ACFS

Use Cases and Customer References

1

2

3

4

5


ASMClusterFileSystem

6


Program Agenda





1

2

3

4

7


• Fast file access due to underlying extent-based architecture

• Throughput and response time go on par with ASM excellent performance numbers

• Industry standard POSIX & Windows Compliant Cluster File System

• Easy management through command-line interfaces and graphical tools

• Oracle Clusterwareintegration allows automatic startup and mounting

• ACFS architecture allows quick recovery from outages

• Oracle Clusterware integration allows ACFS to leverage all cluster membership and high availability services

8

AvailabilityManageabilityEnterprise Edition (EE)Performance

Oracle ACFS - A Complete Oracle Storage Solution


Encryption

Snapshot

Security

Replication

Auditing

Tagging

Compression

Plug-ins

HA-NFS&

HA-SMB

Oracle ACFS - ASM Cluster File System

Oracle ADVM - ASM Dynamic Volume Manager

Ora

cle

AS

M

Ora

cle

Clu

ste

rwa

re

ASMDiskGroup

All database files on ACFS

General purpose files on ACFS

Database RAC Application, Middleware & General Files

Application & Middleware on ACFS

9

Oracle ACFS – Architecture Overview


Multi OS SupportWindows, Oracle Linux, RedHat, SLES, Solaris,

AIX.

10

Generic HardwareOracle Database ApplianceEnterprise Edition (EE)Exadata Database Machine

* Starting with GI 12.1.0.2

Supported database versions: 10.2.0.4/5, 11.2.0.4+, 12.1.0.2+

Oracle ACFS is the primary file system to store all database files

and general purpose data on ODA

Supported database versions:

11.2.0.4+,12.1.0.2+

Platform SupportOracle ACFS - A Complete Oracle Storage Solution


11.2.0.2 11.2.0.3 11.2.0.4 12.1.0.1 12.1.0.2 12.2.0.1 18

LinuxOracle 5

RedHat 5

Oracle 5,6

RedHat 5,6

Oracle 5,6,7

RedHat 5,6,7

Oracle 5,6

RedHat 5,6

Oracle 5,6,7

RedHat 5,6,7

Oracle 6,7

RedHat 6,7

Oracle 6,7

RedHat 6,7

SuSE SLES 10 10,11 10,11 11 11 12 12

Windows2003

2008

2003

2008

2003

2008

2012

20082008

2012

2012

2018

2012

2018

Solaris 10 10,11 10,11 10,11 10,11 10,11 10,11

AIX6.1

7.1

6.1

7.1

6.1

7.1

6.1

7.1

6.1

7.1

6.1, 7.1

7.2

6.1

7.1

11

Oracle ACFS - A Complete Storage SolutionACFS OS Support – initial Grid release shown, also available on prior Grid Releases – For further

information, see My Oracle Support Note 1369107.1

https://mosemp.us.oracle.com/epmos/faces/DocumentDisplay?_afrLoop=216403248385622&parent=WIDGET_RECENTLY_VIEWED&sourceId=2113480.1&id=1369107.1&_afrWindowMode=0&_adf.ctrl-state=pzz2csvxu_53


Creation with ASMCA

12


Creation with ASMCA

13


Program Agenda




Next Generation ACFS

1

2

3

4

14


Oracle ACFS

Oracle ADVM

ADVM Volumes

Database files on ACFS

General purpose files on ACFS


-Dynamic, space efficient, point in time RO & RW sparse copies of ACFS file systems

-Milliseconds to take a snapshot

-Up to 1024 snapshot images per ACFS file system

-Snaps of snaps: Snapshots can be used as the source for a new snapshot

Oracle ACFS

Oracle ADVM

ADVM Volumes

Database files on ACFS

General Purpose files on ACFS


Oracle ACFS Snapshots


Oracle ACFS Snapshot– How to

SnapshotLink

acfsutil snap link –s snap_shot path_to_link

– Creation of a snapshot link (alternate path to an existing snapshot and all of its contents).

SnapshotRemaster

acfsutil remaster –h {snapshot | -c} –f volumes

– Remaster of a file system using a snapshot as a base for the new file system.

SnapshotCreation

acfsutil snap create -r -w -p snap_shot mount_point

– Read only is the default, snaps can be converted from ro to rw and vice versa. Copy-on-write.

SnapshotInfo

acfsutil snap info -t snap_shot mount_point

– Snapshot name, snapshot type, parent name, creation date and space used.

Snapshot Delete

acfsutil snap delete snap_shot mount_point

– Delete command enables snapshots with open filesreferences to be removed from an ACFS file system.


ACFS Snapshot-Based Replication

• Enables efficient replication of ACFS file systems to a remote site• Captures file system changes for a given interval as a set of differences of

successive snapshots on the primary and applies them to the file system on the standby• More efficient than populating and transporting multiple node local

replication logs • Maintains full functionality of the current ACFS Log Capture Replication

solution–Guide for "migration" from old to new method available

• Cross-OS replication among primary and secondary site

17


Oracle Snapshot-Based Replication

PrimaryACFS

Active SiteSSH

Standby Site

Snapshot 1t0

Snapshot 1t1

Snapshot 1t0

Delta

SecondaryACFS

Delta

Snapshot 1t2

DeltaDelta


Oracle ACFS Snapshot based Replication – How to

Replication init on Standby

acfsutil repl init standby –u repluser /sec_mountpoint

Replication init on Primary

acfsutil repl init \ { -i interval | -C} -s repluser@<standby_hostname> \[ -m /sec_mountpoint] /prim_mountpoint

– Specifies that root user will be connecting as specified replication user to replicate changes to the file system

– Specifies that root user will connect to standby to replicate fs in prim_mountpoint to sec_mountpoint.– Interval is optional and establishes –C time units for replication.

For more information please visit our replication whitepaper at www.oracle.com/goto/acfs- All configuration info and upgrade guide from 12.1 to 12.2 & 18c

http://www.oracle.com/goto/acfs


Oracle ACFS NAS Maximum Availability eXtensions�MAX�

• MAX Improves NFS/SMB exported file system availability

• While in use, NFS or SMB daemons will run in High Availability mode– As long as one node in the cluster is available, the protocol (NFS or SMB) will be available

• Extensions provide integration with NAS protocols and the Oracle ACFS stack, meaning ease of use without any additional infrastructure

• Value added to OS NAS protocol implementations without replacing them

HA-NFS & HA-SMB

20


Oracle ACFS NAS Maximum Availability eXtensions (MAX)

• HA-NFS provides uninterrupted NFS

• NFS exports based on Highly Available VIPs (HAVIP)

• Oracle Clusterware ensures that VIPs and NFS exports are always online

• Services are automatically migrated if the current node fails

HA-NFS example

21

Database FilesHAVIP

NFS Export

HAVIP

NFS ExportX

Node A Node B


Oracle ACFS NAS Max eXtensions (NFS) – How to

• srvctl add havip –id hvpExports –address my_havip_name

• Ensure that my_havip_name is mapped in the DNS to the VIP address used by the clients when mounting the file system.

Create a ACFS FS export resourceCreate & Register ACFS FSEnterprise Edition (EE)Add & Register a HAVIP

• Refer to ACFS Admin guide for mkfs & acfsutil manual creation steps or use the asmca command for a guiinterface.

• Register:srvctl add filesystem -device dev_path -diskgroupdisgkgroup_name -mountpathmount_name

• srvctl add exportfs -id hvpExports -path /oracle/cluster1/acfs1 -name hrexport1

• Start havip:srvctl start havip -id havip_name

• For more information please visit our NAS Max eXtensions whitepaper at www.oracle.com/goto/acfs



– Derived from Oracles Database Advanced Compression solution

– Compression for RMAN files, archive logs, data pump files, general purpose files

23

Enterprise Edition (EE)ACFS Compression Enable Compression

acfsutil compress on -a mount_point

Oracle ACFS Compression – How to

Copy / CompressFiles

acfsutil compress –v –f –c –n –r source target_dir

Display Information

acfsutil compress info pathname

– Enables compression on the specified file system

– For files created in the file system previous to enabling compression

– Information such as compression unit size, storage used, savings, etc.


Similar to TDE in the DB

• Protects data-at-rest through encryption

• Encryption using file or volume encryption keys

ACFS AuditACFS SecurityEnterprise Edition (EE)ACFS EncryptionSimilar to DB Vault

• Fine-grained control for higher level security

• Enforces security policiesthrough realms, rules, rule sets and command rules

Similar to DB Audit Vault

• Audit framework for file systems.

• A separate audit trail can be defined for each file system

• Enables separation of duties to be enforced

24

Oracle ACFS – Encryption, Security & Audit


New Features

25


O/S

RACStorageServices

HA/N

FS S

VC &

CIFS

SVC

Network

O/S

RACStorageServices

HA/NFS SVC & CIFS SVC

DomainServicesCluster

O/S

RACStorageServices

O/S

RACStorageServices

MemberCluster

O/S

RACStorageServices

O/S

RACStorageServices

MemberCluster

Storage

• Use of ACFS Remote Services to access database(s) in a separate cluster

• Storage is located in the DSC, never on the client

• Centralized deployment and storage model

26

DB

ACFS Remote


• New capability of reducing ACFS file systems size with acfsutil sizecommand

• Useful when the size of an Oracle ACFS file system is larger than required

• Size reduction works even if it is required to move user data or metadata

• acfsutil info storage shows information how space in an Oracle ASM disk group is being consumed by:• Oracle ASM, Oracle ACFS and Oracle ADVM

• Information contains space, size with mirroring, usable free space, percentages, paths, etc.

• Information displayed is platform independent

27

Enterprise Edition (EE)File System size reduce Enterprise Edition (EE)Storage Usage Info

Oracle ACFS new functionalityNext

GenerationACFS


• Further functionality for acfsutil snap command• Creation/Deletion of snapshots links with

acfsutil snap link command

• New option (-B) for snap apply to specify backup snapshot to be generated

• acfsutil freeze/thaw for temporarily halting/resuming activity in an Oracle ACFS File system

• acfsutil freeze / thaw

• Freeze Freezes modification activity temporarily in the file system. Thaw resumes activity.

• New meta and tune diagnostic commands• acfsutil meta copies metadata from an

Oracle ACFS fs to a separate output file• acfsutil tune displays/sets values of

Oracle ACFS tunable parameters

28

Enterprise Edition (EE)Enhancements to Snapshot commands Enterprise Edition (EE)FS Freeze/Thaw & Diagnostic command

Oracle ACFS new functionalityNext

GenerationACFS


Oracle ACFS Replication Role Reversal

PrimaryACFS

Active SiteSSH

SecondaryACFS

SSH

Standby Site

SecondaryACFS

PrimaryACFS

Next Generation

ACFS


Oracle ACFS Replication Role Reversal – How to

Role reverse on Primary

acfsutil repl reverse primary mountpoint

Role Reverse on Standby

acfsutil repl reverse standby mountpoint

– Ensures standby and primary locations are identical, converts primary to standby and pauses replication.

– Converts standby to primary and then resumes replication

For more information please visit the Oracle ASM Admin guide 18c at www.oracle.com/goto/acfs- All configuration info and upgrade guide from 12.1 to 12.2 & 18c



• Role reversal• Standby assumes Active

Role

ACFS Remote ServiceACFS FS size reduceEnterprise Edition (EE)Role Reversal Replication

• New capability of reducingACFS file systems size

• Native ACFS Functionality on Member Clusters

Temporarily FreezingCommand EnhancementsEnterprise Edition (EE)Storage Usage Info

• acfsutil freeze/thawfor temporarily halting/resuming activity in an ACFS FS

• acfsutil info storage shows information on ASM disk group storage usage by ASM, ACFS and ADVM

• Further functionality foracfsutil snap command• Further functionality for

diagnostics commands

Next Generation ACFS


Program Agenda





1

2

3

4

32



33

ACFSSnapshots

ACFS + Multitenant

gDBCloneODA

• ODA uses ACFS by default

• ACFS + Multitenant provides fast and efficient PDBs snapshot copies

• Script that simplifies management of test and dev provisioning

&ACFS +

MultitenantgDBClone


http://download.oracle.com/otn/samplecode/gDBClone-3.0_ReferenceGuide.pdf


Production Enviroment

DB

Test & Dev Cluster

Oracle ACFSCopy Production DB

DB

RWSnapshot

RWSnapshot

RWSnapshot

TestInstance

TestInstance

TestInstancehttp://www.oracle.com/goto/acfs

Oracle ACFS & gDBClone



– Snapshot Databases are created vastly faster than Database Clones

– gDBClone allows for:• CDB or non-CDB to DB on ACFS for snaps• SI to RAC or RAC-One• non-CDB to PDB• RMAN backup to any of the above

36

Enterprise Edition (EE)Characteristics Enterprise Edition (EE)Characteristics

Oracle ACFS & gDBClone

• Tool developed to provide a fast and efficientmethod for database provisioning in test and development environments

• Leverages Oracle ACFS features and snapshotfunctionality for creating database copies

• Duplicate databases can be used for:– Testing backup and recovery

– Testing upgrades

– Creating Standby DBs

– Etc.


Oracle ACFS & gDBClone - Clone– Clones from production using

RMAN Duplicate from Active

– 3 RMAN channels are used by default but may be overwritten

– Source maybe on any system

– Connection to Remote normally done through SCAN

gDBClone clone

Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | Confidential – Oracle Internal/Restricted/Highly Restricted 38

Oracle ACFS & gDBClone - Snap– Source DB must be stored on

Oracle ACFS

– It can be SI, RAC or RAC-One

– No storage duplication

– Option –standby allows for a standby database as a result


Oracle ACFS & gDBClone - Convert

– Convert a single instance database to a RAC or RAC One Node database Convert

Production

Production

Convert– Convert a 12c non-container database to a

Pluggable database of an specific CDB


Oracle ACFS & gDBClone - ListDBs

– Lists all Databases, DB type and their relationship

– Allows for a tree view of the databases

Remote

Gold

Snapshot0

Snapshot2

Snapshot1

Snapshot3

Clone

Snapshot

Snapshot

SnapshotSnapshot

– Shows:• DB Name

• DB Type (RAC, RACOne, Single, etc)

• DB Home Location

• DB Version

• DB Role (Master, Snapshot)

• Location/Parent


• Standby is continuously refreshed by Data Guard.

• gDBClone snap provides multiple snapshots for test & dev

Clone 11g DB – On LineDatabase Cloning using RMANEnterprise Edition (EE)Test & Dev with Data Guard

• Database clones are created using RMAN backup sets as source.

• Clone from ASM to ACFS while DB is still running.

Many moreClone a DB from RMAN full backup to ACFSEnterprise Edition (EE)Snapshot DB from

diverse sources• DB Upgrade using Transient

Logical Standby

• Clone TDE encrypted DB

• Engineered Systems

• Cloud Migration (OPC to BMC)

• Snapshot DBs can be created from:

• GOLD clone running DBs

• Standby DBs

• Clone the Backup

• Deploy it in ACFS

• Make it a Standby

Oracle ACFS & gDBClone – Use Cases


ACFS + Multitenant

42


Multitenant Architecture Advantages

PDB per application• Apps run unchanged• Fast provisioning (via clones)• Portability (via “pluggability”)

Common operations carried out at a CDB level: Patch, upgrade, HA, backup• Possibility of granular level whenever

is appropiate

Shared process and memory• More applications per server

16


– Tests of new app versions while using

production data.

– Test Scenario exec

– Avoids modifying the production file system

– create pluggable database snap_name from pdb snapshot copy;

44

Enterprise Edition (EE)Characteristics Enterprise Edition (EE)Provisioning

Oracle ACFS & Multitenant

– Starting in 12.1, Oracle ACFS can store

Oracle Database files

– ACFS + Multitenant provides Snapshot Copies• Based on ACFS Copy-on-Write technology

• Create one or multiple snaps of any given PDB



• Considerations:

– All source PDB files must be stored in the same place

– Clonedb parameter must be initialized as False• There is the possibility to start with it initialized as True but then rules change.

– Clones created using SNAPSHOT COPY can not be unplugged, they can only be eliminated with a drop statement.

–Normally the clones will be stored and named with the GUID of the PDB

45


Oracle ACFS & Multitenant• /sbin/acfsutil info fs– number of snapshots: 0– snapshot space usage: 0 ( 0.00 )

• SQL> create pluggable database testPDB_snap1 from testPDB snapshot copy;



• SQL> alter pluggable database testPDB_snap1 open;



46


Oracle ACFS & Multitenant• SQL> show pdbs;

• CON_ID CON_NAME OPEN MODE RESTRICTED

• ---------- ------------------------------ ---------- ----------

• 2 PDB$SEED READ ONLY NO

• 3 testPDB READ WRITE NO

• 4 testPDB_SNAP1 READ WRITE NO



• /sbin/acfsutil info fs– ACFS Version: 12.2.0.1.0

– number of snapshots: 3

– snapshot space usage: 293224448 ( 279.64 MB )

47



http://www.oracle.com/technetwork/database/multitenant/learn-more/pdb-cloudfs-snapclone-2212051.pdf

48

http://www.oracle.com/technetwork/database/multitenant/learn-more/pdb-cloudfs-snapclone-2212051.pdf


Program Agenda


Features and Functionality Highlights



1

2

3

4

49


• Leverages standard UI

• Advanced functionality

Business dataShared homesEnterprise Edition (EE)Oracle Database files

• For DB, Apps and middleware files• HA and simple management

• Business data storage

General purpose filesBackups

• OVM Repository, Golden Gate trail files, trace files, etc.

• Simply host backups using ACFS NFS / CIFS

General ACFS Use Cases

Test & Dev Provisioning

• Leverage of Oracle ACFS snapshot technology


• Access to DB on ACFS on remote clusters

Exadata Cloud ServiceOracle Database ApplianceEnterprise Edition (EE)Remote ACFS

• Default File System on ODA• ODA Leverages ACFS

Snapshot technology

• DB Home on ACFS• General Purpose Files

General purpose filesBackupsEnterprise Edition (EE)Oracle Cloud Infrastructure

• DB on ACFS, RAC• VM images, General purpose

files

General ACFS Use Cases

Enterprise Edition (EE)HA/NFS server

• Leverage existing storage at zero cost


Example Customer ReferenceFinancial Services

• Leading Investment management company in the world.

• Provides services to 15,000 companies through 5000 intermediaries and caters to 20+ million customers.

• Use of competition CFS

SolutionIssuesEnterprise Edition (EE)Company

• License costs reduction.

• Performance improvement needed.

• Complex management of current architecture.

• Software stack simplification required

• RAC, ACFS and ACFS replication for app file systems and Data Guard DB replication.

• Significant performance improvement

• Simplified Management

• Savings in licensing cost52


Oracle on MS Windows with ACFS Windows? Seriously …?

Upgrade to Oracle 12.2 - Real World Customer Cases 53

Von Christian Wolf, www.c-w-design.de, CC BY-SA 3.0 de, https://commons.wikimedia.org/w/index.php?curid=44448489

- Largest independent investment company in the German speaking markets- 650 employees; 70 IT; >400 asset managers globally- Ca. 340 billion € investments- MS shop- Oracle based application- Xentis (core system)- Very resource intense, calculations over periods of >30 years

- SAP- DWH- Other

12.06.18

Universal Investment

8 production Oracle databases; ca. 30 test databases

Eschborn (Standby)Frankfurt (Production + Test)

DWH Databases

OLTP Databases

Old Architecture - Oracle 11.2 with RESTART

12.06.18

20 km

Win2008R260 Core 1,5 TB

16 Core 750 GB

Win2008R2 16 Core 750GB 4 Core 384 GB

+ASM +ASM

+ASM +ASM

NetApp FlashArrays

52 TB 1.2 Mio. IOPS

NetApp FlashArrays

78 TB 900k IOPS

Data Guard 10Gbs

Data Guard 10Gbs

DG4MSQL11.2

EM GridControl 11

ORDS3.0.1

16 Core 750 GBWin2008R260 Core 1,5 TB

Win2008R216 Core 750 GB

4 Core 384 GB

Cold Standby Host

Eschborn (Standby + Test)Frankfurt (Production)

DWH Databases

OLTP Databases

New Architecture with Oracle 12.2 and ACFS

12.06.18

All nodes as GI 12.2, Diskgroup, ASM, ADVM & DB compatibility 12.2

20 km

Win2012R232 Core 6 TB

Win2012R2 32 Core 6 TB

Win2012R2 16 Core 3 TB

Win2012R216 Core 3 TB

+ASM +ASM

+ASM +ASM

NetApp FlashArrays

52 TB 1.2 Mio. IOPS

NetApp FlashArrays

78 TB 900k IOPS

Data Guard 10Gbs

Data Guard 10Gbs

ACFS

DG4MSQL 12.2

EnterpriseManager 13.1

ORDS3.0.9

- Upgrade Path

12.06.18

Upgrade/Migration from Oracle 11.2.0.4 to Oracle 12.2.0.1

Old HWOracle 11.2

New HWOracle 12.2

11.2PROD

11.2PHYS STBY

11.2PHYS STBY

11.2PHYS STBYcatctl.pl12.2

PROD12.2

PHYS STBY

12.06.18

Upgrade/Migration

Some issues - and how we solved them

- Dont‘t forget:- Disable the DG Broker- https://tinyurl.com/DGBROKER

12.06.18

Upgrade/Migration from Oracle 11.2.0.4 to Oracle 12.2.0.1

11.2PHYS STBY

12.2PRODcatctl.pl

12.06.18

ACFS Bug > Blue Screen oracleacfs.sys

Solution: Patch 26007720WINDOWS BLUE SCREEN DRIVER_IRQL_NOT_LESS_OR_EQUAL (ORACLEACFS.SYS)

12.06.18

Only if you are brave enough: Deactivate all SQL Profiles and Baselines

12.06.18

Do you see any issues?

12.06.18

And now?

DECLARE

x NUMBER;

BEGIN

x :=

DBMS_SPM.load_plans_from_cursor_cache (sql_id => ‚gtgvt1jkznvsn',

plan_hash_value => ‚1240834229',

fixed => 'YES',

enabled => 'YES');

END;

/

12.06.18

Define a SQL Plan Baseline

1. 4 x single node cluster instead of 2 x 2 node cluster (Bluescreen OraFenceService.SYS > Doc ID 337784.1)

2. No ACFS at the primary site

12.06.18

Design Change at last minute

2 NodeCluster

2 NodeCluster

ACFS

- Fantastic: Savings in storage comsumption with ACFS

12.06.18

ACFS

ca. 92% space saving after 6 weeks full throttle testing


Any Questions?

12.06.18


Slides Download: https://MikeDietrichDE.com

68

Oracle ASM Cluster File System ACFS

Documents

Transcript of Oracle ASM Cluster File System ACFS