Efficient, hot & automatic oracle database cloning Josep Vidal Canet Universtitat de València...
-
Upload
walter-hicks -
Category
Documents
-
view
215 -
download
1
Transcript of Efficient, hot & automatic oracle database cloning Josep Vidal Canet Universtitat de València...
Efficient, hot & automatic oracle database cloning
Josep Vidal Canet
Universtitat de València
“Discover how to clone your production database without disruption, in a totally automated & efficient way”
Motivation
• Useful for the DBA or system administrator who wants to give his developers a full-sized TEST and DEV instance by cloning the PROD instance into the development server system
• To train new, inexpert DBAs• To test backup and recovery strategies, workloads,
software upgrades, migrations, etc …
Universitat de Valencia
• Spain’s third-largest university– 45.000 students– 3.500 teachers– 2.000 workers
• Different RDBMS:– DB2 / ZOS mainly OLTP– The biggest amount of data is stored in Oracle /
Unix / Linux• Both OLTP & OLAP
– Postgres
What’s a database clone?
• A database clone is a complete and separate copy of a database system that includes the business data, applications and the DBMS software (wikipedia)
• The cloned DB is both fully functional and separate in its own right
DB cloning strategies
• Physical copy -> same server architecture & oracle version
• Export / import : different architectures or oracle versions
The Big Picture I: GOAL
Prod Instance
SGA
PMON SMON DBWR
LGWR CKPT Others
Prod Server or LPAR(Source)
DatabaseData Files Control
Files
Redo Log Files
Cloned Instance
SGA
PMON SMON DBWR
LGWR CKPT Others
Test Server or LPAR(Target)
DatabaseData Files Control
Files
Redo Log Files
To produce a physical copy of prod DB
AutomaticallyWithout disruptionEfficient ReliableSecure
pclone.sh
• A script that automates the whole process:– Without disrupting production systems
• Remote Hotbackup• If the OS allows it (AIX, HP-UX, Solaris) WLM controls are imposed
to guarantee QoS – Efficient– Easy to schedule as a crontab job – Only differences (modified blocks) from prod to test database
are copied– Parallel programming techniques are applied to reduce the
amount of time required• The whole process is triggered by target system, which
queries source database catalog and launches a remote hotbackup
• Once completed, it is recovered in target system
The Big Picture II: Procedure
Prod Instance
SGA
PMON SMON DBWR
LGWR CKPT Others
Prod Server or LPAR
DatabaseData Files Control
Files
Redo Log Files
Cloned Instance
SGA
PMON SMON DBWR
LGWR CKPT Others
Test Server or LPAR
DatabaseData Files Control
Files
Redo Log Files
1: Issue a remoteHotbackup
2: Automatically recover it on target system
pclone.sh
• No special hardware or software required• Data is moved using a SSL encrypted
connection• Used at UV to clone/update test instances from
production systems in a wide range of platforms (Linux/x86, Solaris/SPARC, AIX/Power).
• Easy to use: pclone.sh DB user/password• The tool can be downloaded from:
– http://www.uv.es/vijo/pclone.sh• Paper:
– http://www.uv.es/vijo/cloning_oracle_database.pdf
Requirements
• Same ORACLE & OS versions & system architecture
• ssh & rsync tools, configured so you can login without password
• Time synchronization in both servers – NTP (Network Time protocol).
• Archive log mode
• Prod instance reachable from test system with tnsping utility
Main steps
• Automated using bash scripting
• Executed in the target system:1. Queries source database catalog to obtain
physical database objects
2. Copies each database object from source to target system ensuring database consistency
3. Starts the source database in the target system (DB recovery)
Step 1: Database physical layout
• Source DB discovery– Find server & $ORACLE_HOME with
tnsping & /etc/oratab– Test source DB status
• Query target DB to determine physical layout– tablespaces, datafiles, redo logs, archive
logs, init.ora, etc …
Step 1: Database physical layout
ArchivedLog Files
ParameterFile
PasswordFile
Oracle Database
Data Files Control Files Redo Log Files
/data01/oracle/oradata/DB/control01.dbf
• The underlying directory structure must be created on target system, before copying physical objects– mkdir -p `dirname $f`
/data03/oracle/oradata/DB/tbs_03.dbf
/OraHome1/dbs/orapw
/OraHome1/dbs/initDB.ora
Step 2: Copying physical database objects between systems
• Goal: To copy each type of database object from source to target ensuring database consistency
• Data transfer utility: rsync– Capable of copying
files between remote computers by transferring only file differences
rsync
• Utility: To copy files between two systems• Performs a block level comparison of two files• Transfers only modified blocks
– huge benefit if you are transferring large files like dafafiles over a network link
• Speeds up file transfers when the destination file already exists
• rsync remote-update protocol– allows rsync to transfer just the differences between
two sets of files across the network link, using an efficient checksum search
Step 2: Copying physical database objects between systems
• Idea: To launch a remote hotbackup from the target system– Physical database objects are copied from the
source to target ensuring consistency– File paths are kept identical in both systems
rsync -taupog source:/$dir/tbs01.dbf target:/$dir/tbs01.dbf
Oracle database main files
ArchivedLog Files
ParameterFile
PasswordFile
Oracle Database
Data Files Control Files Redo Log Files
• Some files (dump, archives, init.ora) can be copied without taking care of database consistency• Others, like datafiles, must be in a consistent state before being copied
Step 2: Moving physical database objects between systems
remote_backup(){
STATUS=`target_db_status`
if [ "$STATUS" == "OK" ]; thenshutdown_db "IMMEDIATE"
fi
sync_dump_dirssync_initorasync_temporary_datafilessync_db_datafilessync_db_ctrl_and_log_files
}
Moving datafiles
Prod Instance
SGA
PMON SMON DBWR
LGWR CKPT Others
Prod Server or LPAR(Source)
Database
tbs01
tbs02
tbs03
Data Files Control
Files
Redo Log Files
alter TBS begin backup
rsync -taupog source:/dir/tbs01.dbftarget:/dir/tbs01.dbf
Alter TBS end backup
Test Server or LPAR(Target)
DatabaseData Files
sqlplus client
shell
rsync -taupog source:/dir/tbs03.dbftarget:/dir/tbs03.dbf
rsync -taupog source:/dir/tbs02.dbftarget:/dir/tbs02.dbf
Moving datafiles
• The algorithm can be summarized as:– for each Tablespace in the source database do:
• Set tablespace in backup mode • Copy (using rsync) each datafile from source to target system using parallel techniques• End tablespace backup mode
• A similar approach is used to move both control and redo log files
Step 2: Copying datafiles
sync_db_datafiles(){TBS=`get_tablespaces_name ${SOURCE_DB_CONNECT_STRING}`switch_db_logfile_ini # Parallelism related variables & locks initializationfor T in $TBS; do DATAFILES=`get_tbs_datafiles ${SOURCE_DB_CONNECT_STRING} ${T}` begin_tbs_backup ${SOURCE_DB_CONNECT_STRING} ${T} for d in $DATAFILES ; do
_maximum_parallelism_barriermkdir -p `dirname $d`(rsync -e 'ssh -c blowfish' -tapogL$HOST:$d $d; _sub;exit;)&
done
_wait_for_all_children_to_finish_barrierend_tbs_backup ${SOURCE_DB_CONNECT_STRING} ${T}
done}
Parallelism• Cloning large databases can take a long time
• Why not just split the amount of objects needed to be copied among different tasks? – For each tablespace:
• Create a process for each datafile to be copied until a maximum level of parallelism is reached (_maximum_parallelism_barrier)
• Once the maximum parallelism level is reached, a new process is created when a running one ends
• The backup state is released for a tablespace when every task is completed (_wait_for_all_children_to_finish_barrier )
• Do the same for the remaining database files (archived logs, dumps, etc …)
Step 3: Recovering database
• Goal: To restore database integrity in target system• How: by recovering the physical copy of the
source database • Source database catalog is queried to determine which
datafiles need to be recovered• For each datafile a ‘media recovery’ is issued against the
physical copy • Main steps:
– Database is mounted on target system– Media recovery is issued for each datafile– Database is opened
Step 3: Recovering database
recover_db(){export ORACLE_SID=$SOURCE_DBstartup_db "mount"TBS=`get_tablespaces_name ${SOURCE_DB_CONNECT_STRING}`for T in $TBS; do
DATAFILES=`get_tbs_datafiles ${SOURCE_DB_CONNECT_STRING}
${T}`for d in $DATAFILES ; do
recover_datafile $d "AUTOMATIC"done
doneopen_db
}
Conclusions
• Database cloning can be used for different purposes like testing, developing, database maintenance tests or DBA training
• pclone.sh automates the whole process without disrupting production systems in an efficient manner
• The whole process, major steps and the necessary code to implement it, has been discussed
Conclusions II
• Performance– The time needed to complete the whole process
depends on:• database size & available computational resources (CPUs &
disks)• database update rate as well as cloning interval
– Example:• 150 GB accounting production database, running in a p570
server uncapped LPAR (6 power5 CPU’s)• After an initial cloning, no additional synchronization was
performed for 6 months• Afterwards, the test database was synchronized from the
prod one• The whole process took less than an hour