What’s New in Oracle Data Pump? -...

Copyright©2016, Oracleand/oritsaffiliates.Allrightsreserved.|

What’sNewinOracleDataPump?OracleDatabase12cRelease2

RoyFSwongerVicePresident,DatabaseUpgrade&UtilitiesDecember6,2016


SafeHarborStatementThefollowingisintendedtooutlineourgeneralproductdirection.Itisintendedforinformationpurposesonly,andmaynotbeincorporatedintoanycontract.Itisnotacommitmenttodeliveranymaterial,code,orfunctionality,andshouldnotberelieduponinmakingpurchasingdecisions.Thedevelopment,release,andtimingofanyfeaturesorfunctionalitydescribedforOracle’sproductsremainsatthesolediscretionofOracle.

2


What’sNewinDataPump?

3

ParallelExport/ImportofMetadataSubstitutionVariables&WildcardsREMAP_DIRECTORYLongIdentifiersupportTRUST_EXISTING_TABLE_PARTITIONSValidation&VerificationoptionsOther12.2FeaturesRecapof12.1Features


ParallelMetadataExport:HowitUSEDTOWork

• StartwithESTIMATEphase– Gathertabledataobjects–Otherworkersremainidleuntildataobjectsaregathered

• Metadataexportedserially• Dataexportedinparallel

4

Pre-12.2


ParallelMetadataExport:HowitWorksNow

• StartwithAnalysisstep–Metadataobjectspassedimmediatelytoworkersastheyarefound– E.g.Worker1findsasetofTABLEdefinitions,theyarehandedofftoworker2

• ESTIMATEphasestillhappens,butmetadatanolongerheldupbyestimate• Notes:–Worksfordumpfile jobs,andfornetworkjobsifdestinationdatabaseis12.2– Transportablejobsarenot(yet)parallelformetadata– ESTIMATEphasenowusesSTATISTICSonly– Restartworksasalways

5

NewFeaturein12.2


ParallelMetadataExport:Logfile• 12.1.0.2 • 12.2.0.1

6


• Pre-12.2:–Oneworkerperpartition/subpartition– PQusedifpartitionsarelargeenough– Packagebodiesloadedinparallel

• Withpatchforbug22273229– Indexesbuiltinparallel– Constraintscreatedinparallel– Availableasbackportto12.1.0.2,11.2.0.4

• Startingwith12.2– Addedparallelimportofmost othermetadataobjects– Someexceptions• Types(duetoinheritance)• Schemas• Proceduralactions

7

ParallelMetadataImport


ParallelMetadataImport:Internals• MetadataisexportedinXMLdocuments– EachXMLdocumentindumpfile containsn objectsofagiventype

• XMLdocumentsareallocatedtoworkers1documentatatime• Example:161userstoimport– Usersareexportedwithupto80usersperXMLdocument– WhathappenswithPARALLEL=8?

• Notes:–Worksforconventional(dumpfile)jobs– Not(yet)fortransportablejobsornetworkmode– Restartworkssameasalways– Statuscommandwillshowmultipleworkersonmetadata

8

XMLDoc1User1User2…User80

XMLDoc2User81User82…User160

XMLDoc1User161

Worker1

Worker2

Worker3

(Workers8-nwouldbeidle)


ParallelMetadataImport:Logfile• ComparisonwithPARALLEL=8 for27586objectgrantsandMETRICS=Y– 12.1.0.2

– 12.2.0.1

9


Performance:ParallelMetadataImport• ExamplesfromE-BusinessSuitetestdatabase

10

ObjectType Count 11.2.0.4PARALLEL=32

12.1.0.2PARALLEL=8

12.1.0.2PARALLEL=32With Patch

12.1.0.2PARALLEL=8

12.2.0.1PARALLEL=32

Comments

OBJECT_GRANT(owner)

27586 49 50 51 10 22 Hardconnectforeachgrant

SYNONYM 43254 105 109 111 25 44

TYPE 4364 108 114 119 111 110 Handledbysingleworker

PROCACT_SCHEMA 606 198 216 214 152 175 Handledbysingleworker

TABLE 33164 923 1160 1298 368 248

OBJECT_GRANT(table)

358649 541 543 578 142 157 Hardconnectforeachgrant

INDEX 53190 6721 5770 360 418 272

PACKAGE 53217 424 476 474 114 54

VIEW 34690 538 583 593 151 184

PACKAGE_BODY 52092 1363 1974 1186 1981 959 Always parallelsince11.2

*importtimeinseconds


SubstitutionVariablesforDumpfile Name• Substitutionvariablesfordumpfile name:– Pre-12.2:%Ugeneratesafixed-width2-digitnumber• e.g.dumpfile=exp%U.dmp

– Newoptionfor12.2expdp orimpdp:• %lor%L:Incrementingnumberfrom01upto2147483646

– Newoptionsin12.2expdp only:• %dor%D:DayofMonthinDDformat• %mor%M:NumberofMonthinMMformat• %yor%Y:YearinYYYYformat• %tor%T:FulldateinYYYYMMDDformat

11

$ expdp system/oracle directory=mydir \filesize=50K dumpfile=exp%T_%L.dmp full=y

...

...

...

. . exported "WMSYS"."WM$METADATA_MAP" 0 KB 0 rowsMaster table "SYSTEM"."SYS_EXPORT_FULL_01" successfully loaded/unloaded******************************************************************************Dump file set for SYSTEM.SYS_EXPORT_FULL_01 is: /home/oracle/exp20160917_01.dmp /home/oracle/exp20160917_02.dmp /home/oracle/exp20160917_03.dmp....../home/oracle/exp20160917_67.dmp /home/oracle/exp20160917_68.dmp /home/oracle/exp20160917_69.dmp /home/oracle/exp20160917_70.dmp /home/oracle/exp20160917_71.dmp /home/oracle/exp20160917_72.dmpJob "SYSTEM"."SYS_EXPORT_FULL_01" successfully completed at Sat Sep 17 23:47:31 2016 elapsed 0 00:03:00


WildcardsforTRANSPORT_DATAFILES• Useawilcard in12.2insteadoflistingeveryfile• Pre-12.2:

• NewFeature:wildcards– *(asterisk)matchesmultiplecharacters– ?(questionmark)matchesasinglecharacter

12

TRANSPORT_DATAFILES=users01.dbfTRANSPORT_DATAFILES=users02.dbf…TRANSPORT_DATAFILES=data1.dbfTRANSPORT_DATAFILES=data2.dbf…

TRANSPORT_DATAFILES=users*.dbfTRANSPORT_DATAFILES=data?.dbf…

$ impdp system/oracle@pdb2 network_link=sourcedb \version=12 full=y transportable=always metrics=y \exclude=statistics \directory=mydir \logfile=pdb2.log \transport_datafiles='/u02/oradata/CDB2/pdb2/user*.dbf’


REMAP_DIRECTORY• AppliestoDDLwheredirectoryspecsareused– E.g.CREATETABLESPACE

• Changedirectoryspecwithout changingfilenames• UsefulwhenmovingbetweenOSplatforms– Example:importingdumpfile createdonOpenVMSintodatabaseonLinuxREMAP_DIRECTORY="'DB1$:[HRDATA.PAYROLL]':'/db1/hrdata/payroll/'"

13


LongIdentifierSupport• Long(128-byte)identifiersaresupportedin12.2by– DataPumpexpdp/impdp– SQL*Loader–ORACLE_LOADERaccessdriver–ORACLE_DATAPUMPaccessdriver– ExternalTables

• Importingdatabasemustsupport128-byteidentifiers– BecarefulwhenexportingwithVERSION=12.1orearlier

14


TRUST_EXISTING_TABLE_PARTITIONS• Pre-12.2– Importingintoexistingtablewasdoneserially– DataPumpcouldn’tbesurethatpartitioninginDBmatchedpartitioningindumpfile

• New12.2Parameter:DATA_OPTIONS=TRUST_EXISTING_TABLE_PARTITIONS– Bigperformanceboost– Ifpartitionsdon’tmatch…error:

15

ORA-31693: Table data object "SH"."SALES_BIG_PT":"SALES_2000" failed to load/unload and is being skipped due to error:ORA-29913: error in executing ODCIEXTTABLEFETCH calloutORA-14401: inserted partition key is outside specified partition


DataValidation&Verification

• DATA_OPTIONS=VALIDATE_TABLE_DATA– Importonly– Validatesdateandnumberformatsoftabledata– Defaultisnovalidation

• DATA_OPTIONS=VERIFY_STREAM_FORMAT– Exportonly– Defaultisnoverification

16

ExtraValidationforThingsThatShouldNeverHappen

Starting "SCOTT"."SYS_EXPORT_TABLE_01": scott/******** tables=t directory=dmpdir dumpfile=t.dmp reuse_dumpfiles=true Processing object type TABLE_EXPORT/TABLE/TABLE_DATAProcessing object type TABLE_EXPORT/TABLE/STATISTICS/TABLE_STATISTICSProcessing object type TABLE_EXPORT/TABLE/STATISTICS/MARKERProcessing object type TABLE_EXPORT/TABLE/TABLE. . exported "SCOTT"."T" 5.570 KB 1 rowsORA-31694: master table "SCOTT"."SYS_EXPORT_TABLE_01" failed to load/unloadORA-02354: error in exporting/importing dataORA-26009: stream verification error: [1], [0], [0], [0]

ORA-02374: conversion error loading table "DPV"."TEST18"ORA-12899: value too large for column C1 (actual: 500, maximum: 498)ORA-02372: data for row: C8 : '


Other12.2Features(1)• Usedirectpathloadinnetworkmode!– SpecifyACCESS_METHOD=DIRECT_PATH withNETWORK_LINK=<dblink>– AllowsnetworkimportofLONG andLONG RAW

• DataPumpavailableinInstantClient– ToolspackageforInstantClient– IncludesSQL*Loader,expdp,impdp,exp,imp

• Viewsthatdescribeavailabletransforms– DBMS_METADATA_TRANSFORMS– DBMS_METADATA_TRANSFORM_PARAMS– DBMS_METADATA_PARSE_ITEMS

17


Other12.2Features(2)• Newinteractivecommands– TRACE parametercanbesetforarunningjob• Noneedtostop/restartjobfortracingtotakeeffect

– STOP_WORKER command• Killanindividualworkeryoubelievetobehungorstuck

– BothwillbedocumentedinMOSnotes

• Enhancedlogfiles–WhenMETRICS=Y• ShowworkerIDforeachitemprocessed• Showaccessmethodforeachtable

– Includecontentsofparfile inlogfile

18

18-SEP-16 15:24:30.950: ;;; **********************************************************18-SEP-16 15:24:30.951: ;;; Parfile values:18-SEP-16 15:24:30.953: ;;; parfile: job_name=md_exp_16_1220118-SEP-16 15:24:30.955: ;;; parfile: reuse_dumpfiles=Y18-SEP-16 15:24:30.957: ;;; parfile: logtime=all18-SEP-16 15:24:30.958: ;;; parfile: metrics=Y18-SEP-16 15:24:30.960: ;;; parfile: parallel=1618-SEP-16 15:24:30.962: ;;; parfile: full=Y18-SEP-16 15:24:30.963: ;;; parfile: logfile=md_exp_16_12201.log18-SEP-16 15:24:30.965: ;;; parfile: dumpfile=md16_12201_%U.dmp18-SEP-16 15:24:30.966: ;;; parfile: directory=EBSIMP18-SEP-16 15:24:30.968: ;;; *********************************************************


Recapof12.1Features• VIEWS_AS_TABLES parameter– Letsyouexportthecontentsofaviewasatable

• TRANSFORM parameteroptions– TRANSFORM=DISABLE_ARCHIVE_LOGGING:Y• Willdisablearchiveloggingduringimportfortablesand/orindexes

– TRANSFORM=LOB_STORAGE:SECUREFILE– TRANSFORM=STORAGE:N– TRANSFORM=TABLE_COMPRESSION:<compression_clause>

• LOGTIME=[NONE |STATUS |LOGFILE |ALL ]parameter–Willwritetimestampsonstatusand/orlogfile messages

19


DataPumpNewsinOracle12c• TRANSFORM optiontoenableAdvanced/HCCCompression– Example:• TRANSFORM=TABLE_COMPRESSION:"compress for query high"

– But: Granularityonlyontheentireimport• Workarounds:– Precreateobjects• Downside:Willslowdownimport!!!or:

– PrecreatethetablespacewithCOMPRESSoption• create tablespace ARCHIGH datafile 'archigh.ora' size 100G default compress for archive high;

– ThenrunDataPumpwithTRANSFORM=TABLE_COMPRESSION:N• Thiswilldropallembeddedcompressionattributesassociatedwiththetables• Nowtablespacecompressionoptionwillbeusedforallnewlycreatedtables

20


ExportingViewsasTables

TABLE1

VIEW1 expdp system/mgrviews_as_tables=scott.view1 ...

impdp system/mgrremap_table=view1:scott.table1 ...

TABLE1

21


EnhancedCompressionAlgorithm• COMPRESSION_ALGORITHM– Definesthecompressionalgorithmwhencompressingdumpfiles

• BASIC Thesamealgorithmusedinpreviousversions.Goodcompression,withoutseverelyimpactingonperformance• LOW: ForusewhenreducedCPUutilizationisapriorityovercompressionratio• MEDIUM: Recommendedoption.SimilarcharacteristicstoBASIC,butusesadifferentalgorithm• HIGH: Maximumavailablecompression,butmoreCPUintensive

– Performance:• Compressionratio• CPUusage

– RequiresAdvancedCompressionOptionlicense

$ expdp scott/tiger tables=emp directory=mydir dumpfile=emp.dmp logfile=expdp_emp.log compression=all compression_algorithm=medium

23


EnhancedCompressionAlgorithm• Customerevaluation– BASICat3.5TB/hour

– MEDIUMat7.0TB/hour

2x

24


DownloadSlidesfromtheDatabaseUpgradeBlog• http://blogs.oracle.com/UPGRADE

25

What’s New in Oracle Data Pump? -...

Documents

Transcript of What’s New in Oracle Data Pump? -...