Rittman Mead Consulting Automatic promotion and versioning ...
48
T : +44 (0) 1273 911 268 (UK) or (888) 631-1410 (USA) or +61 3 9596 7186 (Australia & New Zealand) or +91 997 256 7970 (India) E : [email protected] W : www.rittmanmead.com Rittman Mead Consulting Automatic promotion and versioning with Oracle Data Integrator 12c Jérôme Françoisse DOAG 2014 Konferenz + Ausstellung - 20th September 2014 1 Thursday, 13 November 14
Transcript of Rittman Mead Consulting Automatic promotion and versioning ...
T : +44 (0) 1273 911 268 (UK) or (888) 631-1410 (USA) or +61 3 9596 7186 (Australia & New Zealand) or +91 997 256 7970 (India)
E : [email protected] : www.rittmanmead.com
Rittman Mead ConsultingAutomatic promotion and versioning withOracle Data Integrator 12c
Jérôme FrançoisseDOAG 2014 Konferenz + Ausstellung - 20th September 2014
1Thursday, 13 November 14
T : +44 (0) 1273 911 268 (UK) or (888) 631-1410 (USA) or +61 3 9596 7186 (Australia & New Zealand) or +91 997 256 7970 (India)
E : [email protected] : www.rittmanmead.com
Jérôme Françoisse
•Consultant for Rittman Mead‣Oracle BI/DW Architect/Analyst/Developer
•ODI Trainer•Providing ODI support on OTN Forums•ODI 12c Beta Program Member•Blogger at http://www.rittmanmead.com/blog/•Email : [email protected]•Twitter : @JeromeFr
2Thursday, 13 November 14
T : +44 (0) 1273 911 268 (UK) or (888) 631-1410 (USA) or +61 3 9596 7186 (Australia & New Zealand) or +91 997 256 7970 (India)
E : [email protected] : www.rittmanmead.com
Rittman Mead
•Oracle Gold partner with offices in Europe, US (Atlanta), Australia, India and South Africa
•World leading specialist partner for technical excellence, solutions delivery and innovation in Oracle BI
•Provide consulting, training, global managed services for customers around the world
•120+ consultants including 1 Oracle ACE Director and 3 Oracle ACEs
•All expert in Oracle BI, DW, EPM and Analytics tech
•Skills in broad range of supporting Oracle tools: OBIEE, OBIA, ODIEE, Essbase, Oracle OLAP, GoldenGate, Exadata, Endeca
•Blog : http://www.rittmanmead.com/blog/•Twitter : @rittmanmead
3Thursday, 13 November 14
T : +44 (0) 1273 911 268 (UK) or (888) 631-1410 (USA) or +61 3 9596 7186 (Australia & New Zealand) or +91 997 256 7970 (India)
E : [email protected] : www.rittmanmead.com
ODI 12c Key Components
•ODI Studio (the main development IDE)‣Designer Navigator‣Operator Navigator‣Topology Navigator‣Security Navigator
•ODI Agents (used for executing mappings and packages)
•ODI Console (web-based console for running published packages)
•ODI Repositories (contain definitions of models, interfaces and other objects)
DesignerModel data sources
Define projectsCreate interfaces
Release scenarios
OperatorView running interfaces and
packagesOperate production
RepositoryStandaloneAgent
Orchestrateinterfaces and scenarios
Information Systems(Databases, files, applications)
4Thursday, 13 November 14
T : +44 (0) 1273 911 268 (UK) or (888) 631-1410 (USA) or +61 3 9596 7186 (Australia & New Zealand) or +91 997 256 7970 (India)
E : [email protected] : www.rittmanmead.com
ODI 12c Key Components
•ODI Studio (the main development IDE)‣Designer Navigator‣Operator Navigator‣Topology Navigator‣Security Navigator
•ODI Agents (used for executing mappings and packages)
•ODI Console (web-based console for running published packages)
•ODI Repositories (contain definitions of models, interfaces and other objects)
Security
Topology
Versioning
Master Repository
Models
Projects
Execution
Work Repository(Development)
Execution
Execution Repository(Production)
5Thursday, 13 November 14
T : +44 (0) 1273 911 268 (UK) or (888) 631-1410 (USA) or +61 3 9596 7186 (Australia & New Zealand) or +91 997 256 7970 (India)
E : [email protected] : www.rittmanmead.com
ODI Architecture
6Thursday, 13 November 14
T : +44 (0) 1273 911 268 (UK) or (888) 631-1410 (USA) or +61 3 9596 7186 (Australia & New Zealand) or +91 997 256 7970 (India)
E : [email protected] : www.rittmanmead.com
Physical and Logical Topology
DEV_SCOTT
SH
HR
SCOTT
HR
SCOTT
SH
HUM_RES
DEV_OP
TEST_OP
PROD_OP
Physical Contexts Logical
7Thursday, 13 November 14
T : +44 (0) 1273 911 268 (UK) or (888) 631-1410 (USA) or +61 3 9596 7186 (Australia & New Zealand) or +91 997 256 7970 (India)
E : [email protected] : www.rittmanmead.com
Physical and Logical Topology
DEV_SCOTT
SH
HR
SCOTT
HR
SCOTT
SH
HUM_RES
DEV_OP
TEST_OP
PROD_OP
Physical
SCOTT
SH
HR
Contexts Logical
7Thursday, 13 November 14
T : +44 (0) 1273 911 268 (UK) or (888) 631-1410 (USA) or +61 3 9596 7186 (Australia & New Zealand) or +91 997 256 7970 (India)
E : [email protected] : www.rittmanmead.com
Physical and Logical Topology
DEV_SCOTT
SH
HR
SCOTT
HR
SCOTT
SH
HUM_RES
DEV_OP
TEST_OP
PROD_OP
Physical
SCOTT
SH
HR
Contexts Logical
DEV
TEST
PROD
7Thursday, 13 November 14
T : +44 (0) 1273 911 268 (UK) or (888) 631-1410 (USA) or +61 3 9596 7186 (Australia & New Zealand) or +91 997 256 7970 (India)
E : [email protected] : www.rittmanmead.com
Physical and Logical Topology
DEV_SCOTT
SH
HR
SCOTT
HR
SCOTT
SH
HUM_RES
DEV_OP
TEST_OP
PROD_OP
Physical
SCOTT
SH
HR
Contexts Logical
DEV
TEST
PROD
7Thursday, 13 November 14
T : +44 (0) 1273 911 268 (UK) or (888) 631-1410 (USA) or +61 3 9596 7186 (Australia & New Zealand) or +91 997 256 7970 (India)
E : [email protected] : www.rittmanmead.com
Physical and Logical Topology
DEV_SCOTT
SH
HR
SCOTT
HR
SCOTT
SH
HUM_RES
DEV_OP
TEST_OP
PROD_OP
Physical
SCOTT
SH
HR
Contexts Logical
DEV
TEST
PROD
7Thursday, 13 November 14
T : +44 (0) 1273 911 268 (UK) or (888) 631-1410 (USA) or +61 3 9596 7186 (Australia & New Zealand) or +91 997 256 7970 (India)
E : [email protected] : www.rittmanmead.com
Physical and Logical Topology
DEV_SCOTT
SH
HR
SCOTT
HR
SCOTT
SH
HUM_RES
DEV_OP
TEST_OP
PROD_OP
Physical
SCOTT
SH
HR
Contexts Logical
DEV
TEST
PROD
7Thursday, 13 November 14
T : +44 (0) 1273 911 268 (UK) or (888) 631-1410 (USA) or +61 3 9596 7186 (Australia & New Zealand) or +91 997 256 7970 (India)
E : [email protected] : www.rittmanmead.com
How many repositories do I need?
Security
Topology
Versioning
Master Repository
Models
Projects
Execution
Work Repository(Development)
Execution
Execution Repository(Production)
8Thursday, 13 November 14
T : +44 (0) 1273 911 268 (UK) or (888) 631-1410 (USA) or +61 3 9596 7186 (Australia & New Zealand) or +91 997 256 7970 (India)
E : [email protected] : www.rittmanmead.com
How many repositories do I need?
Security
Topology
Versioning
Master Repository
Models
Projects
Execution
Work Repository(Development)
Execution
Execution Repository(Production)
Execution
Execution Repository(Test)
9Thursday, 13 November 14
T : +44 (0) 1273 911 268 (UK) or (888) 631-1410 (USA) or +61 3 9596 7186 (Australia & New Zealand) or +91 997 256 7970 (India)
E : [email protected] : www.rittmanmead.com
How many repositories do I need?
Security
Topology
Versioning
Master Repository
Models
Projects
Execution
Work Repository(Development)
Execution
Execution Repository(Production)
Execution
Execution Repository(Test)
Security
Topology
Versioning
Master Repository
10Thursday, 13 November 14
T : +44 (0) 1273 911 268 (UK) or (888) 631-1410 (USA) or +61 3 9596 7186 (Australia & New Zealand) or +91 997 256 7970 (India)
E : [email protected] : www.rittmanmead.com
How many repositories do I need?
Security
Topology
Versioning
Master Repository
Models
Projects
Execution
Work Repository(Development)
Execution
Execution Repository(Production)
Execution
Execution Repository(Test)
Security
Topology
Versioning
Master Repository
Execution
Execution Repository(Pre-Production)
Security
Topology
Versioning
Master Repository
11Thursday, 13 November 14
T : +44 (0) 1273 911 268 (UK) or (888) 631-1410 (USA) or +61 3 9596 7186 (Australia & New Zealand) or +91 997 256 7970 (India)
E : [email protected] : www.rittmanmead.com
How many repositories do I need?
Models
Projects
Execution
Work Repository(Development)
Execution
Execution Repository(Production)
Execution
Execution Repository(Test)
Security
Topology
Versioning
Master Repository
Models
Projects
Execution
Work Repository(Hotfix)
Execution
Execution Repository(Pre-Production)
Security
Topology
Versioning
Master Repository
Security
Topology
Versioning
Master Repository
12Thursday, 13 November 14
T : +44 (0) 1273 911 268 (UK) or (888) 631-1410 (USA) or +61 3 9596 7186 (Australia & New Zealand) or +91 997 256 7970 (India)
E : [email protected] : www.rittmanmead.com
How many repositories do I need?
Models
Projects
Execution
Work Repository(Development)
Execution
Execution Repository(Production)
Execution
Execution Repository(Test)
Security
Topology
Versioning
Master Repository
Models
Projects
Execution
Work Repository(Hotfix)
Execution
Execution Repository(Pre-Production)
Security
Topology
Versioning
Master Repository
Security
Topology
Versioning
Master Repository
Execution
Execution Repository(Smoke Test)
13Thursday, 13 November 14
T : +44 (0) 1273 911 268 (UK) or (888) 631-1410 (USA) or +61 3 9596 7186 (Australia & New Zealand) or +91 997 256 7970 (India)
E : [email protected] : www.rittmanmead.com
Export
Security
Topology
Versioning
Master Repository
Models
Projects
Execution
Work Repository(Development)
Execution
Execution Repository(Production)
14Thursday, 13 November 14
T : +44 (0) 1273 911 268 (UK) or (888) 631-1410 (USA) or +61 3 9596 7186 (Australia & New Zealand) or +91 997 256 7970 (India)
E : [email protected] : www.rittmanmead.com
Export
Security
Topology
Versioning
Master Repository
Models
Projects
Execution
Work Repository(Development)
Execution
Execution Repository(Production)
- Manually
15Thursday, 13 November 14
T : +44 (0) 1273 911 268 (UK) or (888) 631-1410 (USA) or +61 3 9596 7186 (Australia & New Zealand) or +91 997 256 7970 (India)
E : [email protected] : www.rittmanmead.com
Export
Security
Topology
Versioning
Master Repository
Models
Projects
Execution
Work Repository(Development)
Execution
Execution Repository(Production)
- Manually
15Thursday, 13 November 14
T : +44 (0) 1273 911 268 (UK) or (888) 631-1410 (USA) or +61 3 9596 7186 (Australia & New Zealand) or +91 997 256 7970 (India)
E : [email protected] : www.rittmanmead.com
Export
Security
Topology
Versioning
Master Repository
Models
Projects
Execution
Work Repository(Development)
Execution
Execution Repository(Production)
- Manually
15Thursday, 13 November 14
T : +44 (0) 1273 911 268 (UK) or (888) 631-1410 (USA) or +61 3 9596 7186 (Australia & New Zealand) or +91 997 256 7970 (India)
E : [email protected] : www.rittmanmead.com
Export
Security
Topology
Versioning
Master Repository
Models
Projects
Execution
Work Repository(Development)
Execution
Execution Repository(Production)
- Manually
15Thursday, 13 November 14
T : +44 (0) 1273 911 268 (UK) or (888) 631-1410 (USA) or +61 3 9596 7186 (Australia & New Zealand) or +91 997 256 7970 (India)
E : [email protected] : www.rittmanmead.com
Export
Security
Topology
Versioning
Master Repository
Models
Projects
Execution
Work Repository(Development)
Execution
Execution Repository(Production)
- Manually
15Thursday, 13 November 14
T : +44 (0) 1273 911 268 (UK) or (888) 631-1410 (USA) or +61 3 9596 7186 (Australia & New Zealand) or +91 997 256 7970 (India)
E : [email protected] : www.rittmanmead.com
Export
Security
Topology
Versioning
Master Repository
Models
Projects
Execution
Work Repository(Development)
Execution
Execution Repository(Production)
- Manually
15Thursday, 13 November 14
T : +44 (0) 1273 911 268 (UK) or (888) 631-1410 (USA) or +61 3 9596 7186 (Australia & New Zealand) or +91 997 256 7970 (India)
E : [email protected] : www.rittmanmead.com
Automation
XKCD.com
16Thursday, 13 November 14
T : +44 (0) 1273 911 268 (UK) or (888) 631-1410 (USA) or +61 3 9596 7186 (Australia & New Zealand) or +91 997 256 7970 (India)
E : [email protected] : www.rittmanmead.com
Automation
XKCD.com
16Thursday, 13 November 14
T : +44 (0) 1273 911 268 (UK) or (888) 631-1410 (USA) or +61 3 9596 7186 (Australia & New Zealand) or +91 997 256 7970 (India)
E : [email protected] : www.rittmanmead.com
Automation
XKCD.com
16Thursday, 13 November 14
T : +44 (0) 1273 911 268 (UK) or (888) 631-1410 (USA) or +61 3 9596 7186 (Australia & New Zealand) or +91 997 256 7970 (India)
E : [email protected] : www.rittmanmead.com
Automation
XKCD.com
17Thursday, 13 November 14
T : +44 (0) 1273 911 268 (UK) or (888) 631-1410 (USA) or +61 3 9596 7186 (Australia & New Zealand) or +91 997 256 7970 (India)
E : [email protected] : www.rittmanmead.com
Export - Automated within a package
18Thursday, 13 November 14
T : +44 (0) 1273 911 268 (UK) or (888) 631-1410 (USA) or +61 3 9596 7186 (Australia & New Zealand) or +91 997 256 7970 (India)
E : [email protected] : www.rittmanmead.com
Versioning - Create a version
19Thursday, 13 November 14
T : +44 (0) 1273 911 268 (UK) or (888) 631-1410 (USA) or +61 3 9596 7186 (Australia & New Zealand) or +91 997 256 7970 (India)
E : [email protected] : www.rittmanmead.com
Versioning - Comparison
20Thursday, 13 November 14
T : +44 (0) 1273 911 268 (UK) or (888) 631-1410 (USA) or +61 3 9596 7186 (Australia & New Zealand) or +91 997 256 7970 (India)
E : [email protected] : www.rittmanmead.com
Versioning - Restore a version
21Thursday, 13 November 14
T : +44 (0) 1273 911 268 (UK) or (888) 631-1410 (USA) or +61 3 9596 7186 (Australia & New Zealand) or +91 997 256 7970 (India)
E : [email protected] : www.rittmanmead.com
SubVersionGIT
22Thursday, 13 November 14
T : +44 (0) 1273 911 268 (UK) or (888) 631-1410 (USA) or +61 3 9596 7186 (Australia & New Zealand) or +91 997 256 7970 (India)
E : [email protected] : www.rittmanmead.com
Export
Models
Projects
Execution
Work Repository(Development)
Execution
Execution Repository(Production)
Execution
Execution Repository(Test)
Security
Topology
Versioning
Master Repository
Models
Projects
Execution
Work Repository(Hotfix)
Execution
Execution Repository(Pre-Production)
Security
Topology
Versioning
Master Repository
Security
Topology
Versioning
Master Repository
Execution
Execution Repository(Smoke Test)
23Thursday, 13 November 14
T : +44 (0) 1273 911 268 (UK) or (888) 631-1410 (USA) or +61 3 9596 7186 (Australia & New Zealand) or +91 997 256 7970 (India)
E : [email protected] : www.rittmanmead.com
Automation
XKCD.com
24Thursday, 13 November 14
T : +44 (0) 1273 911 268 (UK) or (888) 631-1410 (USA) or +61 3 9596 7186 (Australia & New Zealand) or +91 997 256 7970 (India)
E : [email protected] : www.rittmanmead.com
ODI SDK - Groovy Scripting
25Thursday, 13 November 14
T : +44 (0) 1273 911 268 (UK) or (888) 631-1410 (USA) or +61 3 9596 7186 (Australia & New Zealand) or +91 997 256 7970 (India)
E : [email protected] : www.rittmanmead.com
ODI SDK - Groovy Scripting/*(...) Connection to the source repository(...) Creation of a collection of mappings(...) Instanciation of an OdiScenarioGeneratorImpl object*/
25Thursday, 13 November 14
T : +44 (0) 1273 911 268 (UK) or (888) 631-1410 (USA) or +61 3 9596 7186 (Australia & New Zealand) or +91 997 256 7970 (India)
E : [email protected] : www.rittmanmead.com
ODI SDK - Groovy Scripting/*(...) Connection to the source repository(...) Creation of a collection of mappings(...) Instanciation of an OdiScenarioGeneratorImpl object*/
// Generating Scenarios For(Object mapping : mappings) { Mapping odiMap = (Mapping) mapping; String scenName = odiMap.getName(); OdiScenario newScen = gene.generateScenario(odiMap, scenName, newVersion);}
25Thursday, 13 November 14
T : +44 (0) 1273 911 268 (UK) or (888) 631-1410 (USA) or +61 3 9596 7186 (Australia & New Zealand) or +91 997 256 7970 (India)
E : [email protected] : www.rittmanmead.com
ODI SDK - Groovy Scripting/*(...) Connection to the source repository(...) Creation of a collection of mappings(...) Instanciation of an OdiScenarioGeneratorImpl object*/
// Generating Scenarios For(Object mapping : mappings) { Mapping odiMap = (Mapping) mapping; String scenName = odiMap.getName(); OdiScenario newScen = gene.generateScenario(odiMap, scenName, newVersion);}
/*(...) Creation of a collection of the generated scenarios*/
25Thursday, 13 November 14
T : +44 (0) 1273 911 268 (UK) or (888) 631-1410 (USA) or +61 3 9596 7186 (Australia & New Zealand) or +91 997 256 7970 (India)
E : [email protected] : www.rittmanmead.com
ODI SDK - Groovy Scripting// Generating Scenarios For(Object mapping : mappings) { Mapping odiMap = (Mapping) mapping; String scenName = odiMap.getName(); OdiScenario newScen = gene.generateScenario(odiMap, scenName, newVersion);}
/*(...) Creation of a collection of the generated scenarios*/
// Exporting Scenariosfor (Object scen : scenarioCollection) { OdiScenario odiscen =(OdiScenario)scen ; export.exportToXml(odiscen, ExportPath, OverWrite, RecursiveExport, Encoding);}
25Thursday, 13 November 14
T : +44 (0) 1273 911 268 (UK) or (888) 631-1410 (USA) or +61 3 9596 7186 (Australia & New Zealand) or +91 997 256 7970 (India)
E : [email protected] : www.rittmanmead.com
ODI SDK - Groovy Scripting
/*(...) Creation of a collection of the generated scenarios*/
// Exporting Scenariosfor (Object scen : scenarioCollection) { OdiScenario odiscen =(OdiScenario)scen ; export.exportToXml(odiscen, ExportPath, OverWrite, RecursiveExport, Encoding);}
/*(...) Connection to the target repository(...) Creation of a list of XML files.*/
25Thursday, 13 November 14
T : +44 (0) 1273 911 268 (UK) or (888) 631-1410 (USA) or +61 3 9596 7186 (Australia & New Zealand) or +91 997 256 7970 (India)
E : [email protected] : www.rittmanmead.com
ODI SDK - Groovy Scripting// Exporting Scenariosfor (Object scen : scenarioCollection) { OdiScenario odiscen =(OdiScenario)scen ; export.exportToXml(odiscen, ExportPath, OverWrite, RecursiveExport, Encoding);}
/*(...) Connection to the target repository(...) Creation of a list of XML files.*/
// Importing Scenariosfor (String filename : XMLFiles) { import.importObjectFromXml( import.IMPORT_MODE_SYNONYM_INSERT_UPDATE, filename, true);}
25Thursday, 13 November 14
T : +44 (0) 1273 911 268 (UK) or (888) 631-1410 (USA) or +61 3 9596 7186 (Australia & New Zealand) or +91 997 256 7970 (India)
E : [email protected] : www.rittmanmead.com
GIT - Commit and Push
git add . -Agit commit -m "My commit Message"git push
26Thursday, 13 November 14
T : +44 (0) 1273 911 268 (UK) or (888) 631-1410 (USA) or +61 3 9596 7186 (Australia & New Zealand) or +91 997 256 7970 (India)
E : [email protected] : www.rittmanmead.com
...
27Thursday, 13 November 14
T : +44 (0) 1273 911 268 (UK) or (888) 631-1410 (USA) or +61 3 9596 7186 (Australia & New Zealand) or +91 997 256 7970 (India)
E : [email protected] : www.rittmanmead.com
28Thursday, 13 November 14
T : +44 (0) 1273 911 268 (UK) or (888) 631-1410 (USA) or +61 3 9596 7186 (Australia & New Zealand) or +91 997 256 7970 (India)
E : [email protected] : www.rittmanmead.com
Further reading•Blog post by Uli Bethke (@ubethke) :“ODI: AUTOMATING DEPLOYMENT OF SCENARIOS TO PRODUCTION IN ORACLE DATA INTEGRATOR”http://sonra.io/odi-automating-deployment-of-scenarios-in-oracle-data-integrator/
•Blog post by kdevendr :“Export and Importing Scenarios based on Project (or)Folder”http://odiexperts.com/export-and-importing-scenarios-based-on-project-orfolder/
•ODI SDK Public API : https://docs.oracle.com/middleware/1213/odi/reference-java-api/index.html
•David Allan :“ODI - Accelerator Launchpad, getting Groovy” and much morehttps://blogs.oracle.com/dataintegration/entry/odi_accelerator_launchpad_getting_groovy
29Thursday, 13 November 14
I’m speaking at Tech14
Register now to join me: www.tech14.ukoug.org
#ukoug_tech1430Thursday, 13 November 14
