Database Migration Assistant for Unicode (DMU)

Database Migration Assistant for Unicode (DMU)A Real Customer Case

Ludovico CaldaraOracle ACE DirectorTrivadis AG

About Ludovico Caldara

Database Migration Assistant for Unicode2 03/05/2023

■ 17 Years DBA (Not Only Oracle)

▪ I do it everywhere (even Windows)

■ RAC ATTACK Ninja & co-writer

■ President, SOUG & ITOUG Board

■ OCP (11g, 12c, MySQL) & OCE

■ Italian living in Switzerland

■ http://www.ludovicocaldara.net

■ @ludodba

■ ludovicocaldara


Our company.Trivadis is a market leader in IT consulting, system integration, solution engineeringand the provision of IT services focusing on and technologies in Switzerland, Germany, Austria and Denmark.We offer our services in the following strategic business fields:

Trivadis Services takes over the interactive operation of your IT systems.

O P E R A T I O N


COPENHAGEN

MUNICH

LAUSANNEBERN

ZURICHBRUGG

GENEVA

HAMBURG

DÜSSELDORF

FRANKFURT

STUTTGART

FREIBURG

BASEL

VIENNA

With over 600 specialists and IT experts in your region.

14 Trivadis branches and more than600 employees

200 Service Level Agreements

Over 4,000 training participants

Research and development budget:CHF 5.0 / EUR 4 million

Financially self-supporting and sustainably profitable

Experience from more than 1,900 projects per year at over 800customers


Conversion to Unicode


Why

Single-byte character sets cannot represent all the characters

Nowadays it is no more acceptable

Well-known character sets:

– US7ASCII

– WE8ISO8859P1(5)

– WE8MSWIN1252


Issues with Unicode

Variable length per character (1-4 bytes, increase in data size)

Different representation for the non-US7ASCII characters

Data conversion


Conversion to Unicode is Serious!

Be careful, you might corrupt your data!


DMU


DMU identity card

Born in 2011

Last version: Check MOS Patch 21138450 (currently 2.1)

Java-based GUI tool

Replaces csscan and csalter (they are now desupported)

End-to-end migration process


Conversion Flow

AnalyzePrepare Fix / Clean Convert Validate


Prepare


Prepare

Do it on a non-production first!

– Several times, reiterate to be sure the conversion will work

– Use a physical copy of your production

Compile invalid objects ( @?/rdbms/admin/utlrp.sql )

Purge the dba_recyblebin

List invalid objects

– create table pre_dmu_invalid as select owner, object_name, object_type from dba_objects where status != 'VALID' ;


Prepare (cont.)Install the DMU support package SYS.DBMS_DUMA_INTERNAL

– SQL> @?/rdbms/admin/prvtdumi.plbLibrary created.Package created.No errors.Package body created.No errors.

Create a separate tablespace for the DMU repository

– CREATE TABLESPACE DMU_REPO DATAFILE SIZE 100M EUTOEXTEND ONNEXT 100M MAXSIZE 30g;

Extend the Undo Tablespace (it requires a lot of space)

– alter database datafile '/data/orcl/undotbs01.dbf' autoextend on maxsize 30G;


Prepare (cont.)

Save the size of the segments before the migration

– CREATE TABLE dmu_size_before AS SELECT tablespace_name, owner, segment_type, segment_name, bytes, extents FROM dba_segments WHERE tablespace_name NOT IN ('UNDOTBS','TEMP') ;

Why? Bigger character representation

– Migrated rows

– Chained rows


Analyze


Launch DMU

Download and unzip dmu under $ORACLE_HOME/dmu-$version

Launch it:

– export DISPLAY=192.178.56.1:0.0export JAVA_HOME=$ORACLE_HOME/jdkcd /ccv/app/oracle/product/12.1.0.2/dmu-2.1/dmush dmu.sh

Create the connection to the database


Repository installation


Launch the scan


Run the Scan Wizard

Increase according to your server


Scan – Table properties


Fix / Clean


All the blocking issues must be solved

BLOCKING


Find the issues in the scan report


Need conversion


Need conversion

Application Data will be converted during the conversion phase

– No action for you here

Dictionary data needing conversion must be fixed!

– Check on My Oracle Support!

– Tips For and Known Issues With The Database Migration Assistant for Unicode (DMU) Tool version 2.1 (Doc ID 2018250.1)


Dictionary Cleansing for my Customer Case

Cleansing REG$

– truncate table reg$;

Cleansing AWR tables -> recreate AWR SQL> @?/rdbms/admin/catnoawr.sql SQL> @?/rdbms/admin/catawr.sql SQL> execute dbms_swrf_internal.register_local_dbid;

Cleansing scheduler logs exec dbms_scheduler.purge_log;

Cleansing SYS.WRP$_REPORTS* tables (SQL Monitor and R-T ADDM reports) truncate table wrp$_reports ; truncate table wrp$_reports_details ; truncate table wrp$_reports_time_bands ;


Dictionary Cleansing for my Customer Case (Cont.)

Cleansing source$

– Fixed the source code manually

Cleansing col$

– A table column name was containing a special character… OMG…

Scan the dictionary again to make sure that everything has been fixed


Over Column Limit


Fixing Over Column Limit – By Column


Fixing Over Column Limit – Bulk Cleansing


Invalid Representation


Fixing invalid representations – Find the source!

Why there is invalid data?

– Is it because of non-existing characters in current set?

– Or there is a bad configured client application?

– Or there is a bad configured ETL job?


Fixing invalid representations – Cleansing


Fixing invalid representations – Ignoring


Fixing invalid representations – Ignoring

Use it as the last step in your cleansing phase!


Other Fixes

Enable Row Movement on your tables to enable CTAS conversion

– Much faster than standard UPDATE

– No migrated rows after the upgrade

Put your application tablespaces in AUTOEXTEND

– Space increase due to UNICODE conversion

– Space used by Create Table As Select conversion method


Convert


Convert

BEFORECLEANSING

AFTERCLEANSING


Convert (Cont.)

You cannot have connected session to the database

– Alter system enable restricted session;

You may want to ignore other warnings


Convert (Cont.)

My customer had a few issues

– Tables locked by the DMU itself (retry the operation)

– DMU client disconnecting (client on the DB server via VNC is the best solution)

– Tablespaces running out of space (set them in autoextend before converting)


Convert (Cont.)

Restart the database after the conversion


Validate


Install the repository in Validation mode


Scan in validation mode


Timing


A Customer’s Database

500 Application Tables

250 Gb of data segments

Original Charset: WE8ISO8859P1

Not critical in terms of downtime

Very critical in terms of data correctness


Timing

Scan: 10-12 minutes

Conversion: 6-10 hours (depending on parameters)

– Sometimes the elapsed time changes without reason

Validation scan: 8-10 minutes

Allows additional time for backups


Speeding Up

Make sure that big tables are converted with CTAS

Huge tables? Consider:

– Expdp

– Truncate

– Convert

– Impdp

– (Don’t forget to validate the data manually)


Total migration project (for one application)

May 2016: First test on a fake copy

June 2016: Migration of the Test environment

June – July 2016: Data validation and certification of the application for Unicode

July 2016: Migration of the pre-production

August 2016: Refresh prod-test and 2nd migration of the test environment

August 2016: Migration of the Quality environment

September 2016: Migration of the Production

– Downtime reserved from Friday evening to Saturday morning


Alternative methods


Alternative methods

Export/Import with Data Pump

– Bad: No check for conversion issues

Oracle GoldenGate

– Bad: No check for conversion issues + GoldenGate cost

DMU + GoldenGate -> Near Zero Downtime

– Bad: GoldenGate cost,

– But very efficient!

Questions?

03/05/2023 Database Migration Assistant for Unicode55

Ludovico CaldaraACE Director, Senior Consultant

[email protected]

Database Migration Assistant for Unicode (DMU)

Technology

Transcript of Database Migration Assistant for Unicode (DMU)