Getting the Best Out of Data Pump

7/22/2019 Getting the Best Out of Data Pump

1/49

1


2/49

Get the Best Out of Oracle Data Pump Functionality

Dean Gagne (Oracle)Viljo Hakala (Nokia)


3/49

3

Agenda

Moving large amounts of data with TransportableTablespaces

Filtering metadata using the INCLUDE and EXCLUDEparameters

Restarting stopped/failed jobs Hear about Nokia Corporation's database

environment How Nokia uses the REMAP_DATA parameter to

scramble data How Nokia regenerates primary keys without having

to use additional software or scripts


4/49

4

What is Oracle Data Pump?

New feature starting in Oracle Database 10 g Release 1 Enables very fast bulk data and metadata movement between

Oracle databases High-speed, parallel Export and Import utilities (expdp and

impdp) as well as a Web-based Oracle Enterprise Manager interface

Jobs can be restarted without loss of data, whether or not thestoppage was voluntary or involuntary

Jobs support fine-grained object selection. Virtually any type of object can be included or excluded

Supports the ability to load one instance directly from another (network import) and unload a remote instance (network export)


5/49

5

Moving Large Amounts of DataWith Transportable Tablespace


6/49

6

What is Transportable Tablespace?

An Oracle Database feature that allows data filetransfer from a database to another via a simple oscopy and a light specific export/import

Data Pump will move metadata only Data moves with data file copy Much faster than using direct path or external tables


7/49


8/498

Transportable TablespacesSelf Containment Check

Create table part_tab(id number) tablespace apartition by range (id)(partition low value less than (100) tablespace b,

partition hi values less than (1000) tablespace c); Create index part_ind on part_tab.id tablespace d; Transportable tablespace export requires all 4

tablespaces (a, b, c, d)

DBMS_TTS.TRANSPORT_SET_CHECK procedureto verify

accepts a comma separated list of tablespace names SELECT * FROM transport_set_violations;


9/49


10/49


11/49


12/49


13/49


14/49


15/49


16/49


17/49

17

Include Examples:

impdp system/manager tables=hr.employeesvs

impdp system/manager schemas=hr include=table:\"= \'EMPLOYEES\'\"

Same results Includes all objects that have table in the path

vsimpdp system/manager schemas=hr include=table/table:\"=

\'EMPLOYEES\'\ Only includes the table

Some of the TABLE object pathsSCHEMA_EXPORT/TABLE/TABLESCHEMA_EXPORT/TABLE/TABLE_DATASCHEMA_EXPORT/TABLE/GRANT


18/49


19/49


20/49

20

Restarting Stopped Export JobsInitial Export Command:

expdp system/manager job_name=datapump_exp

Restart command: expdp system/manager at tach=datapump_exp Export> start_job

Resets dumpfile pointers to last completed object type Restarts the job after the last completed object


21/49


22/49


23/49


24/49

2424

Nokia

Nokia is a world leader in mobility Head office in Finland; R&D, production, sales, marketing activities

around the world Worlds #1 manufacturer of mobile devices, with estimated 40%

share of global device market in 2009

Mobile device volumes 468 million units Net sales EUR 50.7 billion Operating profit EUR 5.0 billion 128 445 employees at year end (including Nokia Siemens

Networks) Strong R&D presence in 16 countries R&D investment EUR 6.0 billion Sales in more than 150 countries


25/49

2525

Oracle @ Nokia

20 DBAs, 2300 databases Grid Control: used for almost 5 years, #1 tool for all DBAs Applications:

OLTP: 50%, Hybrid databases: 40%, DW: 10% + Teradata About 1800+ Oracle; also MySQL 200+, MS SQL 300+

11g: 4%

10g: 80% 9i: 15% 8i: 1%

Host platforms: Solaris: 35% HP-UX: 35% Linux: 20% Windows 10%


26/49


27/49


28/49


29/49


30/49


31/49

3131

Data Masking with Oracle Data Pump

REMAP_DATA allows transformation of columns data while exporting(expdp) or importing (impdp) by using a remapping function in thedatabase.

REMAP_DATA with Data Pump is usually faster than a customUPDATE statement for masking data.

To mask multiple columns in the same process and command, theREMAP_DATA parameter can be used multiple times.


32/49


33/49


34/49


35/49

3535

Data Masking with Oracle Data Pump:Example

Create a table in the CUSTOMERS schemacalled phones

SQL>CREATE TABLE CUSTOMERS.PHONES(MODELNAME VARCHAR(20) NOT NULL,PHONENUMBER VARCHAR2(50)

);insert into CUSTOMERS.PHONES

values(N900,+3581234567); insert into CUSTOMERS.PHONES

values(N8,+3589817654);

insert into CUSTOMERS.PHONESvalues N7 +3584834819

D t M ki ith O l D t P


36/49

3636

Data Masking with Oracle Data Pump:Example

We then need to create a functionfor remapping

create or replace packagecustomers.maskpkg

asfunction masknumber(phonenumber

varchar2) return varchar2;end;/create or replace package body

customers.maskpkg asfunction masknumber (phonenumber

varchar2) return varchar2 isbegin return

substr(phonenumber,1,4)||round(dbms_random.value (100,999))||lpad(round(dbms_random.value(1,9999)),4,'0');

end;end;

The function masknumber willaccept a varchar2 type andreturns a random phone number in varchar2 type


37/49


38/49

D t M ki g ith O l D t P


39/49

3939

Data Masking with Oracle Data Pump:import/replace table

This example will import the masked dump fileand replace the existing table with masked column data

$ impdp customers/manager \tables=customers.phones \dumpfile=phones_masked.dmp \directory=dumpdir \table_exists_action=REPLACE

Data Masking with Oracle Data Pump:


40/49

4040

Data Masking with Oracle Data Pump:Example, after import

SQL> select * from customers.phones

MODELNAME PHONENUMBER---------- --------------------

N900 +35815499474N8 +3584800578N7 +3581247839

k h l


41/49

4141

Data Masking with Oracle Data Pump:Import

REMAP_DATA parameter can also be used in the import process

Use impdp if you have an existing dump file and you want to mask thedata when loading into the database

$ impdp customers/manager TABLE_EXISTS_ACTION=replacedumpfile=phones.dmp directory=dumpdir remap_data=customers.phones.phonenumber:customers.maskpkg.masknumber

Primary Key Regeneration with


42/49

4242

Primary Key Regeneration withREMAP_DATA

To change primary keys after development phase of an application, useREMAP_DATA parameter.

It is possible to use Data Pumps REMAP_DATA to change the primarykeys within a database by writing a remapping function which follows the

new primary key format.

Primary key conflicts can happen when loading data to an existing table ina database.

With REMAP_DATA, conflicts can be avoided by changing the columnvalues during import.


43/49

Other Methods to Secure Data in Oracle


44/49

44

Other Methods to Secure Data in OracleDatabase 11g Release 2 Oracle Virtual Private Database (VPD aka Fine Grained Access Control)

feature allows filtering of data at row-level for runtime SQL statementsaccording to a defined policy.

Data Masking Pack for Enterprise Manager. Centralized masking solutionwithin Grid Control.

Oracle Advanced Security Option for encrypting data at tablespace or column level.

Oracle Label Security for restricting access to data based on policies.

Oracle Database Vault for separating roles and restricting access to dataper roles.


45/49


46/49


47/49


48/49

48


49/49

Getting the Best Out of Data Pump

Documents

Transcript of Getting the Best Out of Data Pump