Post on 12-Oct-2020
<Insert Picture Here>
Top 10 Things You Always Wanted to Know About Automatic
Storage Management
But Were Afraid to Ask Nitin Vengurlekar
RAC/ASM Engineering
Grid Product Strategy
<Insert Picture Here>
Agenda
• ASM Overview – 2 minute tour
• Top Ten Questions
• ASM Architecture
• ASM Configuration & Performance
• Backups
• ACFS
• Migration
• 3rd Party Software
• Summary
ASM Architecture - Review
• Automatic Storage Management (ASM) instance
• Instance that manages the diskgroup metadata
• Disk Groups
• Logical grouping of disks
• Determines file mirroring options
• ASM Disks
• LUNs presented to ASM
• ASM Files
• Files that are stored in ASM disk groups are called ASM files, this
includes database files
ASM ArchitectureA Brief Overview
Pool of Storage
ASM
Instance
Server
Non–RAC
Database
Oracle
DB Instance
Disk Group Disk Group
ASM Process ArchitectureA Brief Overview
Clustered
Pool of Storage
ASM
Instance
ASM
Instance
ASM
Instance
ASM
Instance
Clustered
Servers
RAC or Non–RAC
Databases Oracle
DB Instance
Oracle
DB Instance
Oracle
DB Instance
Oracle
DB Instance
Database
ASM
Instance
Oracle
DB Instance
DatabaseDatabase
Disk Group Disk Group
<Insert Picture Here>
Top 10 ASM Questions
ASM Architecture
Top 10 ASM Questions
Q. What init.ora parameters does a user need to
configure for ASM instances?
A. The default parameter settings work perfectly for
ASM. The only parameters needed for 11g ASM:
• PROCESSES
• ASM_DISKSTRING*
• ASM_DISKGROUPS*
• INSTANCE_TYPE*
Top 10 ASM Questions
Q. How does the database interact with the ASM
instance and how do I make ASM go faster?
A. ASM is not in the I/O path so ASM does not impede
the database file access. Since the RDBMS instance
is performing raw I/O, the I/O is as fast as possible.
RDBMS and ASM Instance Interaction
Server
Operating System
DATABASE ASM
(1) Database opens file
(1A) OPEN
(1B) Extent Map
(2) Database Reads file
(2A) READ
(2B) I/O
Completes(3) Database Creates file
(3A) CREATE
(3B) Allocates file
(3C) Extent Map
(3D) Initializes file (3D) Commit
Top 10 ASM Questions
Q. Do I need to define the RDBMS
FILESYSTEMIO_OPTIONS parameter when I use ASM?
A. No. The RDBMS does I/O directly to the raw disk
devices, the FILESYSTEMIO_OPTIONS parameter is
only for filesystems.
Top 10 ASM Questions
Q. Don’t I lose all the advanced filesystem features when I
move to ASM; e.g., directIO, write coalescing, and pre-
fetch?
A. Yes, but that’s okay.
• Most of the filesystem features mentioned, though good for
general file data performance, interfere and fractionalize the
benefits inherently provided by the database; e.g.,
• DBWR & LGWR does write coalescing, and user processes do
sequential pre-fetches
• All IO capable processes do un-buffered IO (raw IO) because of ASM
Top 10 ASM Questions
Q. This is cool that ASM can now store Vote and OCR files.
But how does CSS and CRS startup in this configuration?
A. It just does ☺
• There are two keys processes and a lot of crafty coding to
get this to work in the correct startup sequence.
Top 10 ASM Questions
ASMCMD> pwd
+DATA/rst-cluster/OCRFILE
ASMCMD> ls -l
Type Redund Striped Time Sys Name
OCRFILE UNPROT COARSE JUN 25 11:00:00 Y REGISTRY.255.718984285
ASMCMD> lsdg
State Type Rebal Sector Block AU Total_MB Free_MB
Req_mir_free_MB Usable_file_MB Offline_disks Voting_files Name
MOUNTED EXTERN N 512 4096 1048576 203824 193028
0 193028 0 Y DATA/
CSS-CRS-ASM Instance Architecture
ASM
GPnP
1. Get ASM disk string
2. Do ASM disk discoveryto locate vote disk
ASM Disk
CRS
ASM Disk
DATA DG
SPFILE
Vote
OCR
3. Once Vote disk located, startup CSS
OHASD
4. ASM mounts DATA dg
Agents starts GPnP, CSS and ASM
CSS
5. CRS open OCR in well known ASM dg location
<Insert Picture Here>
Top 10 ASM Questions
ASM Configuration
Top 10 ASM Questions
Q. Do I need 11gR2 Grid Infrastructure to use ASM
A. Yes. ASM is now part of Grid Infrastructure, which
includes, Clusterware, ASM and ACFS. So you’ll to
Install GI to use ASM
• In 11gR2 there are two options for install – GI for Standalone
Server (aka Oracle Restart) and GI for Clusterware
Top 10 ASM Questions
Q. We have a 16 TB database. I’m curious about the
number of disk groups we should use; e.g. 1 large
disk group, a couple of disk groups, or otherwise?
What about a database consolidation scenario
A. For VLDBs you will probably end up with different
storage tiers; e.g with some of our large customers
they have Tier1 (RAID10 FC), Tier2 (RAID5 FC), Tier3
(SATA), etc. Each one of these is mapped to a
diskgroup.
Top 10 ASM Questions
Q. What is the best LUN size for ASM
A. There is no best size! In most cases the storage
team will dictate based on their standardized LUN
size. The ASM admin merely has to communicate the
ASM Best Practices and application characteristics to
storage folks :
• Need equally sized / performance LUNs
• Minimum of 4 LUNs
• The capacity requirement
• The workload characteristic (random r/w, sequential r/w) &
any response time SLA
Using this info, and their standards, the storage folks should build
a nice LUN group set for you.
Top 10 ASM Questions
Q. In 11gR2 can my RDBMS and ASM instances run
different versions?
A. Yes. But since ASM is now part of GI stack, it must be
at the highest version. Keep in mind, there’s two
components of compatibility:
• Software compatibility
• Diskgroup compatibility attributes:
• compatible.asm
• compatible.rdbms
Disk Group Compatibility Example
• Need to have compatible.asm set to 11.2.0.1 for
OCR/Vote files and SPFILE in ASM
• ACFS also needs 11.2.0.1 compatible.asm
• Advance compatible.asm• ALTER DISKGROUP data
SET ATTRIBUTE
‘compatible.asm’ = ’11.2.0.1.0’
Top 10 ASM Questions
Q. Where do I run my database listener from; i.e., ASM
HOME or DB HOME?
A.
• For 11gR2, the SCAN listener is run from GI Home,
and database listener from DB HOME.
• For pre-11gR2, it is recommended to run the listener
from the ASM HOME. This is particularly important for
RAC env, since the listener is a node-level resource.
In this config, you can create additional [user] listeners
from the database homes as needed.
<Insert Picture Here>
Top 10 ASM Questions
Backups
Top 10 ASM Questions
Q. How do I backup my ASM instance?
A. Not applicable! ASM has no files to backup
Top 10 ASM Questions
Q. When should I use RMAN and when should I use
ASMCMD copy?
A. RMAN is the recommended and most complete and
flexible method to backup and transport database files
in ASM.
ASMCMD copy is good for copying single files
• Supports all Oracle file types
• In some cases. can be used to instantiate a Data Guard
environment
• Does not update the controlfile
• Does not create OMF files
ASMCMD Copy
ASMCMD> ls
+fra/dumpsets/expdp_5_5.dat
ASMCMD> cp expdp_5_5.dat sys@rac1.orcl1:+DATA/dumpsets/ex
pdp_5_5.dat
source +fra/dumpsets/expdp_5_5.dat
target +DATA/dumpsets/expdp_5_5.dat
copying file(s)...
file, +DATA/dumpsets/expdp_5_5.dat,
copy committed.
<Insert Picture Here>
Top 10 ASM Questions
Migration
Top 10 ASM Questions
Q. We are migrating to a new storage array. How do I
move my ASM database from storage A to storage B?
A.
• Given that the new and old storage are both visible to
ASM, simply add the new disks to the ASM disk group
and drop the old disks. ASM rebalance will migrate
data online.
• For pre-11gR2, See Note 428681.1, which covers how
to move OCR/Voting disks to the new storage array
Top 10 ASM Questions
ASM_SQL> alter diskgroup DATA
drop disk
data_legacy1, data_legacy2,
data_legacy3
add disk
‘/dev/sddb1’, ‘/dev/sddc1’,
‘/dev/sddd1’;
ASM Rebalancing
• Automatic online rebalance whenever
storage configuration changes
• Only move data proportional to storage
added
• No need for manual I/O tuning
Disk Group DATA (legacy disks)
ASM Rebalancing
• Automatic online rebalance whenever
storage configuration changes
• Online migration to new storage
Disk Group DATA
ASM Rebalancing
• Automatic online rebalance whenever
storage configuration changes
• Online migration to new storage
Disk Group DATA
ASM Rebalancing
• Automatic online rebalance whenever
storage configuration changes
• Online migration to new storage
Disk Group DATA
ASM Rebalancing
• Automatic online rebalance whenever
storage configuration changes
• Online migration to new storage
Disk Group DATA (new disks)
Top 10 ASM Questions
Q. Is it possible to unplug an ASM disk group from one
platform and plug into a server on another platform
(for example, from Solaris to Linux)?
A. No. Cross-platform disk group migration not
supported. To move datafiles between endian-ness
platforms, you need to use XTTS, Datapump or
Streams.
<Insert Picture Here>
Top 10 ASM Questions
ACFS
46
3rd Party FSApplication
Extending ASM to Manage ALL Data
Automatic Storage Management (ASM)
ASM Instance Managing
Oracle DB Files
ASM Cluster & Single Node File System
(ACFS)
Database RAC
ACFS Snapshot
ASM Files
Dynamic Volume Manager
Dynamic Volumes
ACFS FSOracle Bin
3rd PartyFS
DB DatafilesDB DatafilesDB DatafilesDB DatafilesDB DatafilesDB Datafiles
OCR & Voting filesOCR & Voting files
ASM
Disk
Group
47
• General purpose scalable file system
• Journaling, extent based
• Single node and cluster
• POSIX, X/OPEN file system solution for UNIX/Linux
• Windows file system solution for Windows platforms
• Accessible through NAS protocols (NFS, CIFS)
• Leverages ASM technology
• Integrated with Oracle Clusterware for cluster support
• Multi OS platform (Linux and Windows at initial release)
• Integrated with Oracle system mgt tools
• Oracle installation and configuration
• Enterprise Manager and ASM Storage mgt tools
• Native OS File System Management tools
What is ASM Cluster File System
(ACFS)
48
ACFS Features• Provides filesystem snapshots (FCOW)
• File system integrity and fast recovery via ACFS metadata
checksums and journaling.
• ACFS designed as a peer to peer, multi-node, shared file
system model and delivers coherent data access
• ACFS file system is installed as a dynamically loadable OS
VFS driver
• Starting with RHEL5, Redhat now supports a 'white list' -
kernel APIs which they commit they will not change in updates
or patches. APIs used by ACFS-ADVM were added to their
'white list'.
• Customers should be able to install an update or patch to the
kernel and our drivers should not be impacted
Top 10 ASM Questions
Q. Is ACFS supported on other platforms besides Linux
A. Yes. Other platforms are forthcoming
Top 10 ASM Questions
Q. Can ACFS be used to store database datafiles?
What about archive logs?
A. No. Currently we will not support database file to be
stored in ACFS. This is due to performance reasons.
Though you can do this in test/Q&A environments
where performance is not essential
Top 10 ASM Questions
Q. Can I use ACFS to store BFILE data or other non-
database related data
A. Yes. ACFS is POSIX compliant filesystem, and thus
can store any file data type (besides database files ☺)
Top 10 ASM Questions
Q. Will ACFS support other Data services, such
advanced cloning, replication, de-dupe, etc..
A. Yes. ACFS Replication will be introduced in the next
patchset release. Other advanced features are part of
the roadmap.
<Insert Picture Here>
Top 10 ASM Questions
3rd Party Software
Top 10 ASM Questions
Q. How does ASM work with multipathing software?
A. It works great! Multipathing software is at a layer
lower than ASM, and thus is transparent.
You may need to adjust ASM_DISKSTRING to specify
only the path to the multipathing pseudo devices.
ASM/RDBMS
/dev/<pseudo>
Multipath driver
/dev/sda1 /dev/sdb1
Controller Controller
Cache
Disk
IO cloud
Top 10 ASM Questions
Q. Is ASM constantly rebalancing to manage “hot
spots”?
A. No…No…Nope!!
I/O Distribution
• ASM spreads file extents evenly across all disks in disk group
• Since ASM distributes extents evenly, there are no hot spots
Average IOPS per disk during OLTP workload
0
50
100
150
200
250
300
Total
Disk
IOPS
FG1: - cciss/c0d2
FG1: - cciss/c0d3
FG1: - cciss/c0d4
FG1: - cciss/c0d5
FG1: - cciss/c0d6
FG2: - cciss/c0d2
FG2: - cciss/c0d3
FG2: - cciss/c0d4
FG2: - cciss/c0d5
FG2: - cciss/c0d6
FG3: - cciss/c0d2
FG3: - cciss/c0d3
FG3: - cciss/c0d4
FG3: - cciss/c0d5
FG3: - cciss/c0d6
FG4: - cciss/c0d2
FG4: - cciss/c0d3
FG4: - cciss/c0d4
FG4: - cciss/c0d5
FG4: - cciss/c0d6
Summary:
• ASM requires very few parameters to run
• ASM based databases inherently leverage raw disk
performance
• No additional database parameters needed to support
ASM
• Mixed ASM-database version support
• Facilitates online storage changes
• RMAN recommended for backing up ASM based
databases
• Spreads I/O evenly across all disks to maximize
performance and eliminates hot spots
ASM Collateral and Content
http://www.oracle.com/technology/asm
• ASM 11g New Features
• ASM Best Practices
• ASM vendor papers
• ASM-RAC Customer Case Studies
<Insert Picture Here>
Top 10 ASM Questions
Extra credit questions
Top 10 ASM Questions
Q. Is ASMLIB required on Linux systems and are there
any benefits to using it?
A. ASMLIB is not required to run ASM, but it is certainly
recommended.
ASMLIB has following benefits:
• Simplified disk discovery
• Persistent disk names
• Efficient use of system resources
Top 10 ASM Questions
Q. Is it possible to do rolling upgrades on ASMLIB in a
RAC configuration
A. ASMLIB is independent of Oracle Clusterware and
Oracle Database, and thus can be upgraded on its
own