Session ID:

Prepared by:

E-Business Suite Rapid Provisioning Using New Features Of Oracle Database 12c

10619

@ AndrejsKarpovs

Andrejs Karpovs,Lead Oracle Applications DBA

About me

• Lead Oracle Apps DBA / Technical infrastructure architect at Tieto [Latvia]

• Oracle Certified Master• Oracle ACE• Masters Degree in Computer Science• Speaker: Collaborate, UKOUG, OUGH, Oracle• RAC Attack Ninja• Blog: adbaday.wordpress.com

2

Coming from Latvia

3

Tieto

• Tieto is the largest Nordic IT services company

4

Agenda

• ASM – Oracle Cloud Storage• ACFS – Oracle Cloud file system

– Background– EBS on ACFS

• Appl Tier• DB Tier (12c new features)

– Features– Use cases

ACFS definition (I)

• ASM Cluster file system or Automatic StorageManagement Cluster file system

• ASM appeared with 10g• ACFS extension introduced in 11gR2• In January 2011 ACFS has been rebranded to

Oracle Cloud file system– Oracle Cloud Storage = ASM + ADVM + ACFS

6

Cloud Storage Architecture

7

ACFS definition (II)

• Introduced in Grid Infrastructure 11gR2• Built from the traditional ASM disk groups• General purpose file system• Uses standard file related system calls like others

file systems in the market• Disk creation for ACFS is performed with a typical

workflow process• Supported on RHEL/OEL/SLES/AIX/WINDOWS

– ACFS Support On OS Platforms (Certification Matrix). (Doc ID 1369107.1)

8

From ASM to ACFS

9

ACFS Deployment

• Two types of ACFS file systems:– CRS Managed ACFS (for example, ORACLE_HOME)

• Clusterwide by default• Automatically creates a dependency with placed

resource (DB)– General Purpose ACFS

• No structural differences• Starting with Oracle Database 12c

– Database files– Snapshots– NFS HAVIP

10

ACFS Disk Space Usage

• Preallocates large user files to improve performance when writing data

• Keeps local bitmaps available to reduce contention on the global storage bitmap when searching for free space– This disk space is reported as in use by tools such as

the UNIX df command even though some of it may not actually be allocated as of yet

11

Application Tier

12

EBS Appl Tier on ACFS

• https://blogs.oracle.com/stevenChan/entry/choosing_an_ebs_shared_file_system– Does Oracle Certify Storage Systems? - No. Not any

more.– Does the E-Business Suite Division Certify Storage

Systems? – No. Oracle generally suggest that customers either perform their own product testing or consult a trusted consultancy that compares the relative merits of each product against a consistent set of criteria.

– What are the EBS Requirements for a Shared Filesystem? - Shared filesystems must be transparent to the calling application, in this case, the E-Business Suite. In other words, no modifications to the E-Business Suite should be necessary to ensure compatibility with the shared filesystem.

13

Is EBS 11i/R12 certified with ACFS?

• No. Because Oracle Application Server 10g is not certified with ACFS.

• “Customers interested in running Oracle Application Server 10g (and the E-Business Suite) on ACFS are encouraged to log SRs against the Fusion Middleware product to request this certification. If sufficient numbers of customers request it, they may consider it”

• Can you still use it for, for example, $APPLCSF/$APPLLOG, $APPLCSF/$APPLOUT, $PATCH_TOP? – Yes.

14

What about R12.2/FMW/Weblogic?

• http://www.oracle.com/technetwork/database/availability/maa-fmwsharedstoragebestpractices-402094.pdf

• ACFS was found to be a highly performant solution

15

EBS running on ACFS (my experience) (I)

• Since 2009 – not faced a single issue (we are doing our own product testing, right?)– 11g– 12c– Unless there is a clusterware issue

• Multiple audits and performance benchmarking made by Oracle– Oracle never told us we are running on unsupported

configuration– Oracle never told us to migrate from ACFS based

solution– Oracle worked on SR’s related to EBS and ACFS

• Didn’t find a single justification why ACFS cannot be certified with EBS

16

EBS on ACFS (from experience) (II)

• Two clusters: Database and Applications cluster• Both having Grid Infrastructure installed and ASM

used• On Applications cluster EBS application stack is

installed and concurrent managers running• Other nodes are mounting ACFS using NFS• Different mount points (ACFS filesystems) created

for log files, out files, etc.

17

You have options...

• Run EBS Applications stack on two node cluster and utilize NFS for multi-node installation

• Run EBS on three or four (or more) node cluster bypassing NFS

• Do not fully run the applications stack on ACFS, use it for log file, out files, etc.

18

Architecture diagram

19

Fast Provisioning (I)

20

Fast Provisioning (II)

21

Fast Provisioning (III)

22

Fast Provisioning (IV)

• Online resizing, relocation• 256 mounts supported

23

Fast Provisioning (V)

• Or use CLI– scripting– automatic provisioning

• mkdir -p /mnt/acfs• ASMCMD> volcreate -G ACFS -s 5g --column 1 VOL1• mkfs -t acfs <volume_device>• mount -t acfs <volume_device> /mnt/acfs

24

Snapshots (I)

• First Copy-on-Write (FCOW) methodology• Available under .ACFS• Point-in-time view of a file

– ACFS snapshots can be very useful for file-based recovery or for file system backups

• Dynamic and space efficient• Recovery using copy and replace• Use case – ORACLE_HOME patching• Use case – EBS TEST/DEV environments cloning• 1023 snapshots are supported

25

Snapshots (II)

• acfsutil snap create snap_ro /mnt/acfs• acfsutil snap create -w -p snap_rw /mnt/acfs

• acfsutil snap info /mnt/acfs• snapshot name: snap_rw• snapshot location: /mnt/acfs/.ACFS/snaps/snap_rw• parent name: /mnt/acfs• snapshot creation time: Tue Apr 5 11:14:20 2016

26

Replication (I)

• “Data Guard” for general purpose files• Enabled replication of ACFS file systems across the

network to a remote site• Replication logs are used to replicate from primary

to standby asynchronously• Supports ACFS Tagging, needs specific

configuration

27

Replication (II)

28

Tagging

• Assigns a common naming attribute to a group of files

• Selects files with tag for replication– /sbin/acfsutil tag set logfiles -r /u02/logfiles/*.log– /sbin/acfsutil repl init primary -s

standby/standby@standby_site tagname logfiles ...– Avoids entire file system replication

• Requires some additional rpms and configuration

29

12c New Features (I)

• Highly Available NFS (HANFS)

30

12c New Features (II)

• Snapshot enhancements: Oracle ACFS read/write snapshots have been enhanced to support snapshots created from an existing snapshot in the same Oracle ACFS file system (snaps of snaps) and snapshot conversions (read-only to read/write). (use it in cloning/restoring)

• Flex ASM

31

Database tier

32

Background

• Starting with Oracle 12c, ACFS supports Oracle Database data files

• Oracle Database Appliance - databases are hosted on ACFS

Requirements (I)

• Database version >= 11.2.0.4• Grid infrastructure 12c• ASM/ACFS configured• Snapshots require DB EE license• Database parameter filesystemio_options=setall

34

Requirements (II)

• ACFS doesn’t necessarily require a multi-node cluster installation (ASM instance can be created on standalone setup too)

• However, dbca doesn’t like it:

• Workaround: create database manually using CREATE DATABASE statement J

35

EBS installation on ASM

36

...or ACFS

37

Database Rapid Provisioning using Snapshots

• Same setup routine as described before with ApplTier

• Snapshot for consistent copy• Datafile copy for fast media recovery• Database cloning / rapid provisioning• Some scenarios:

http://allthingsoracle.com/?s=ACFS

38

Some drawbacks

• As of now no way to natively restore a snapshot– manual data file copy from the snapshot

• Not recommended to use Production instance as a direct snapshot provider due to performance penalty– Use standby DB– Use secondary standby DB – replicated to test– Production Database copy as a provider

39

Write performance /w snapshots

• http://www.ludovicocaldara.net/dba/oracle-database-on-acfs-a-perfect-marriage/

40

The recommended approach

41

• Rapid DB copies, space-efficient• Source should not be performance critical

Provision new DB from standby

42

gDBClone (I)

• Script that leverages Oracle ACFS snapshot functionality – Space-efficient copies for Test and Dev– Full provisioning cycle

43

gDBClone (II)

• Automatically converts DBs from any type to any type

• http://www.oracle.com/technetwork/indexes/samplecode/index.html

44

gDBClone (III)

• See the WP• Script is for Test/Dev/Educational purposes only• Still being developed• One might use it as a base and enhance/customize

45

General Performance

• Almost the same– slight penalty on ACFS

• SLOB tests:– http://www.ludovicocaldara.net/dba/oracle-database-

on-acfs-a-perfect-marriage/• Oracle response was the same• Under different workloads there might be some

changes in benchmarks• ACFS cannot be faster than ASM

46

Licensing

• Starting with Oracle 12c, no extra license required for ACFS features

• DB Snapshots requires Enterprise Edition

Conclusion: ACFS Benefits

• A better (faster and more reliable) alternative to traditional NFS

• Dynamic management• No need to separate storage administrators and

ASM administrators• Mount it anywhere with NFS• Features• Performance• Database fast and efficient copies

– Rapid cloning and DB provisioning• If configured and setup properly maybe a very

powerful and efficient solution

48

Some last minute tips on ACFS

• Limit of 4 resizes (need to retest in 12c)– There is a limit of 5 extents for the file system's

internal storage bitmap. This causes any attempts to increase the file system to fail after it has been increased four or more times

– Solution - /sbin/fsck -a -v -y -t acfs /dev/asm/acfs-368• When mounting application tier file system to other

node using NFS, use the following nfs parameters:– rw,intr,bg,hard,timeo=600,wsize=32768,rsize=32768,

nfsvers=3,tcp,nolock,acregmin=0,acregmax=0– The above are recommendations based on Oracle

Internal Testing

49

Thank you!

Questions?

My other EBS sessions at #C16LV