Oracle Database 12c IT WARS - nloug.nl · PDF filecommitment to deliver any material, code ......

OracleDatabase12candDevOpsMariaColganOracleDatabaseSystemsJune,2017@SQLMaria

ThefollowingisintendedtooutlineourgeneralproductdirecNon.ItisintendedforinformaNonpurposesonly,andmaynotbeincorporatedintoanycontract.Itisnotacommitmenttodeliveranymaterial,code,orfuncNonality,andshouldnotberelieduponinmakingpurchasingdecisions.Thedevelopment,release,andNmingofanyfeaturesorfuncNonalitydescribedforOracle’sproductsremainsatthesolediscreNonofOracle.

SafeHarborStatement

• ReputaNon:Fast&Agile

• Primaryfocus:Delivery

•  Favoriteline:“Let’sgetitdone”

Public 3

MeettheDeveloper

Developer

• ReputaNon:Cau=ous&RiskAverse

• Primaryfocus:Longevity&Security

•  Favoriteline:“Let’snotgethacked”

Public 4

MeettheDBA

Together they must build a new web application for the recently

acquired coffee shops chain

Weneedtobuildanewwebapplica=onforthecoffee

shopswerecentlyacquiredinahurry,sowewanttouseaflexibleschemausingJSONdocuments.CanyousetupaMongoDBdatabaseforus?

ShecannotdifferenNateablasterfroma

sNck…

Developer DBA

Ohno!Whatdoeshewant

Whydon’tyoujustuseOracle?

OracleDatabase12casaDocumentStore12.2JSON

SQL> CREATE TABLE orders (order_details BLOB CONSTRAINT is_json CHECK (order_details IS JSON) );

TablecontainingJSONdocuments

Public

•  OraclestoresJSONintablecolumns•  Nospecialdatatype•  CanbeVARCHAR2,BLOBorCLOB

•  JSONsupportedbyallOraclefeatures•  AnalyNcs,EncrypNon,In-Memory,RAC,

ReplicaNon,ParallelSQL,…

•  PluscanindexanyJSONelement

OracleDatabase12casaDocumentStore12.2JSON

SQL> CREATE TABLE orders (order_details BLOB CONSTRAINT is_json CHECK (order_details IS JSON) );

Public

•  OraclestoresJSONintablecolumns•  Nospecialdatatype•  CanbeVARCHAR2,BLOBorCLOB

•  JSONsupportedbyallOraclefeatures•  AnalyNcs,EncrypNon,In-Memory,RAC,

ReplicaNon,ParallelSQL,…

•  PluscanindexanyJSONelementIS_JSONconstraintensuresonlylegalJSONdocumentscanbeinserted

ButtheJSONdocumentisBLOB.That’snotwhatweneed.Weneedtobe

abletoquerytheelementsintheJSON

documents?

Developer DBA

Don’tpanic!OracleoffersnaNveSQL

supportforJSON.Letmeshowyou

NaNveSQLSupportforJSON12.2JSON

Public

•  JSONcanbequeriedusingsimpleSQLdotnotaNon

SELECT

o.order_details.location.city FROM orders o;

Location--------------------BostonLos AngelesNew YorkSan FranciscoWashington

SQL> desc ORDERSNAME TYPE------------------ -----------ORDER_DETAILS BLOB

{"date": "2017-05-03 10:40:36","loyaltyCardNo": 1230, "location": { "city": "Dallas", "state": "TX”}, "salesAmount": 7.55, "order": [ { "item": "Espresso”, …. }}

Afewhourslater….........

Public 11

Developer

12Public

Hey,wehavedecidedtodoeverythingviaRESTandweknowthatMongoDBcanspeakRESTSoitlookslikeweneedadocumentstore

aVerall

Dude,RESTisoutoftheboxwithOracle!Letme

showyou

Conn establishment

Request

Response

Conn close

Request

Response

Conventional communication

RESTful communication

RESTinanutshell

OracleRESTDataServicesHTTP(s)APIApp-DevwithRelaNonalTablesinOracleDatabase

{“loyaltyCardNo”:1001,“name”:“ScooKing”,“LocaNon”:“500Mainstreet,InnovaNonCA”“date”:“Feb27,2017”"salesAmount":4.15“orders”:[{"item":"Mocha","size":"medium","salesAmount":4.15,"coffee_used_gram":15}]}

OracleRESTDataServices

ORDSmapsstandardURIrequeststocorrespondingrelaNonalSQL(notschemaless):e.g.SQLSELECTfromcustomersandorderstable.ORDSalsotransformstheSQLresultsintothehighlypopularJavaScriptObjectNotaNon(JSON),otherformatsincludeHTML,binaryandCSV.

FullycommioedtosupporNnganyandallstandardsrequiredbyFusion/SaaS/FMW;weareacNvelyengagedintheongoingdialog.

URIRequestorHTTP(s)postmappedtoSQLrequest

JDBCConnecNon

JSONSQLresultto

JSONtransformaNon

hops://myhost/myapp/hr/customerorders/1001

ConfiguringRESTwithOracleExadataExpressCloudService

Givetheserviceanameandspecify

theprefix

SpecifytherestoftheURL

orders

Specifythetypeofservice

Specifytheactualcommandtobe

usedbytheservice

RESTEndpointForDeveloper

hops://demo-a425370.db.us2.oraclecloudapps.com/apex/coffeeshop/orders

OracleConfidenNal–Internal/Restricted/HighlyRestricted 19

Developer DBA

Goahead,sendyourRESTcalls

CheckingresultofRESTSELECT Count(*) FROM orders;

COUNT(*)

--------

22Public

Thisisawesome!YoujustreducedourNmetomarketbytwoweeks!Thanksa

million!

Theforceisstrongwithme

You’rewelcome

Developer

Afewhourslater….........

Public 23

24Public

I’mheadinghomeforthedayandIwanttokeep

workingonmycommutebutIwon’thavegoodinternetconnecNvity.Canwebuildthesameenvironmentin

Docker?

Absolutely!YouwillneedtodownloadtheDocker

imageforOracleDatabase12c&Oracle

SQLDeveloper

Developer

Letmeshowyouonmymachine

1.  Clickonthetoolsmenu2.  ClickontheRESTDataServices3.  ClickonInstall

EnablingSQLDeveloperasORDS

1.  UseOracleRESTDataServicethat’sincludedwithSQLDev

2.  SpecifythelocaNonforyourRESTconfigfile

3.  Clicknext

1.  FillinyoutheServiceNameforyourdatabaseindocker

2.  SpecifythepasswordforyourORDS_PUBLIC_USER

3.  Clicknext

1.  ClickRunStandalonemode2.  SpecifyHTTPPORT(8080)3.  Clicknext4.  Clickfinish

RESTEndpointForDocker

hop://localhost:8080/ords/coffeeshop/orders/order

Afewdayslater….........

Public 30

31Public

How’sitgoing?

Isn’ttheCRMdataalreadyinthedatabasewiththeJSONdata?

Yeah,exactly.Nowwejustneedtobuildsomeclassesto

getthatcorrespondingrelaNonaldatafortheJSON

orders

Hey,weareworkingonintegraNngtheCRMdatawiththatJSONdatawe

stored

Developer

32Public

Thereisnoneedtointegrateattheapplevel.YoucanuseSQLtorunaqueryagainsttheJSON&theCRMdata.LetmeshowyoubutfirstI’llneedtocheckwhatthe

structureofyourJSONlookslike.

Developer

DataGuide:UnderstandingYourJSONDocuments

• Metadatadiscovery:discoversthestructureofcollecNonofJSONdocuments– OpNonal:deepanalysisofJSONforListofValues,ranges,sizingetc.

• AutomaNcallyGenerates– Virtualcolumns– RelaNonalviews

•  De-normalizedrelaNonalviewsforarrays

– Reports/SynopsisofJSONstructure

SQL> SELECT JSON_DATAGUIDE(o.order_details) FROM orders o; JSON_DATAGUIDE(O.ORDER_DETAILS) ---------------------------------------------------------------------- [ {"o:path": "$.date", "type": "string", "o:length": 32 }, {"o:path": "$.order", "type": "array", "o:length": 256 }, {"o:path": "$.location", "type": "object", "o:length": 32}, {"o:path": "$.location.city", "type": "string", "o:length": 8}, .... { "o:path": "$.salesAmount", "type": "number", "o:length": 4 }, { "o:path": "$.loyaltyCardNo","type": "number", "o:length": 8}]

DataGuide:UnderstandingYourJSONDocuments

QueryJSONwithRelaNonalDataSQL> SELECT c.c_name,

o.order_details.location.city,

SUM(To_number(o.order_details.salesAmount)) total

FROM customers c,

orders o

WHERE c.c_custid = o.order_details.loyaltyCardNo

GROUP BY c.c_name,

o.order_details.location.city

ORDER BY total Desc

FETCH FIRST 10 ROWS ONLY;

FETCHFIRST10ROWSONLY

•  Newsyntaxtolimitnumberofrowsreturned

•  ReplacesSELECT*FROMWHEREROWNUM<11

Developer DBA

Tomakeiteasierforyou,Icancreateaviewsothedevelopersdon’tneedtoknowallofthe

joincondi=ons

Yeahright,andanybodyis

rememberingthatstuff!

CreaNngaViewonTopofJSON&RelaNonalDataSQL> CREATE OR replace VIEW crm_data AS

SELECT c.c_name,

FROM customers c,

orders o

GROUP BY c.c_name,

o.order_details.location.city

ORDER BY total Desc

FETCH FIRST 10 ROWS ONLY

CreaNngaViewonTopofJSON&RelaNonalDataSQL> SELECT *

FROM crm_data;

Developer DBA

Noway,thatjustleadstoalotofroundtripsbetweenappanddatabase,whichwillmakethingseven

slower.Andbesides,thisproblemhasalreadybeen

solved.EverheardofMaterializedViews?

That'sgreatbuthowwillthatworkwhenwehaveGBsofdatainthere.Surelyjust

selecNngtheJSONfirstandthenretrievingthe

correspondingcustomerdataisfasterandsaferperformance

Ohboy,thisonereallydoesn’tfeeltheforce!

Amaterializedwhat?

Developer DBA

WhatisaMaterializedView?• Amaterializedviewisadatabaseobjectthatcontainstheresultsofaquery•  Thequerycanbebaseontables,vieworothermaterializedviews•  TypicallyusedtoavoidrepeaNngcostlyaggregaNons• OpNmizertransformsqueriesagainstoriginaltablesintoasemanNcallyequivalentqueryagainstthematerializedview

• AtradeoffbetweendiskspaceandCPU

CreaNngReal-TimeMVonTopofJSON&RelaNonalDataSQL> CREATE MATERIALIZED VIEW top_customers_mv

ENABLE query rewrite

ENABLE on query computation

AS SELECT c.c_name,

FROM customers c,

orders o

GROUP BY c.c_name,

o.order_details.location.city;

ENABLEonquerycomputa=on

•  Real-TimeMaterializedView

•  EnsureyoualwaysgetaccurateanswerevenifmaterializedviewisstalebyjoiningtoMVlog

QueriesAutomaNcallyRedirectedtoUseMaterializedView

SQL> SELECT *

FROM crm_data;; --------------------------------------------------------------------------- | Id | Operation | Name | Rows | Bytes | --------------------------------------------------------------------------- | 0 | SELECT STATEMENT | | 10 | 20550 | |* 1 | VIEW | | 10 | 20550 | |* 2 | WINDOW SORT PUSHED RANK | | 1 | 2029 | | 3 | MAT_VIEW REWRITE ACCESS FULL| TOP_CUSTOMERS_MV | 1 | 2029 | ---------------------------------------------------------------------------

I’msNllnotsurethisapproachwillscaleasweneedthequeriestobe

returnedinsub-seconds.Canwetryitwithabiggerdata

Developer DBA

Sure.Whereisthebiggerdataset?

TestMVQueryAgainstLargerDataset3.2millionJSONdocuments

SQL> CREATE MATERIALIZED VIEW top_customers_history_mv

ENABLE query rewrite

ENABLE on query computation

AS SELECT c.c_name,

FROM customers c,

orders_history o

GROUP BY c.c_name,

o.order_details.location.city;

Developer DBA

Thewhat?Ohthat’sgreatfortheknownreportsbutthe

markeNngfolkswanttorunad-hocreportstoo.Whatcanwedotomakesuretheystay

within30seconds?Don’tpanic,wecanput

thetablesintotheIn-Memorycolumn

OracleDatabaseIn-Memory:DualFormatArchitecture

•  BOTHrowandcolumnformatsforsametable

•  SimultaneouslyacNveandtransacNonallyconsistent

•  OLTPusesprovenrowformat

•  AnalyNcs&reporNngusenewin-memoryColumnformat•  SeamlesslybuiltintoDatabase•  Allenterprisefeatureswork

-RAC,Dataguard,Flashback,etc47

BufferCache

NewIn-MemoryColumnStore

SALES SALESRow

FormatColumnFormat

•  FullJSONdocumentspopulatedusinganopNmizedbinaryformat

•  AddiNonalexpressionscanbecreatedonJSONcolumns(e.g.JSON_VALUE)&storedincolumnstore

•  QueriesonJSONcontentorexpressionsautomaNcallydirectedtoIn-Memoryformat

Superfast/MulN-ModelAnalyNcs:In-MemoryJSON

RelaNonal

In-MemoryColumStore

In-MemoryVirtualColumns

In-MemoryJSONFormat

RelaNonal Virtual JSON

{"date": "2016-05-03 10:40:36","loyaltyCardNo": 1230, "location": { "city": "Dallas", "state": "TX”}, "salesAmount": 7.55, "order": [ { "item": "Espresso”, …. }}

SQL> ALTER TABLE orders_history ADD city AS (JSON_VALUE(order_details, '$.location.city'));

SQL> ALTER TABLE orders_history ADD state AS (JSON_VALUE(order_details, '$.location.state'));

SQL> ALTER TABLE orders_history ADD salesamount AS (JSON_VALUE(order_details, '$.salesAmount'));

CreateVirtualColumnsontheOrders_Historytable

SQL> ALTER TABLE customers INMEMORY;

SQL> ALTER TABLE orders_history INMEMORY;

SQL> SELECT count(*) FROM customers;

SQL> SELECT count(*) FROM orders_history;

SQL> SELECT segment_name,

populate_status, bytes_not_populated

FROM v$im_segments;

PlaceTablesintoIMColumnStore

51Public

Wow,that’sfantasNc.WillitautomaNcallystay

In-Memory?Yup,alldoneforyoubythedatabase!

Thisisawesome!That'sexactlywhatyouwantfrom

adatabase.Schemaflexibility,analyNcsand

performance.

Developer

JustWhenyouthinkeverythingisdone…........

Developer Securityguy

HaveyousecuredthecustomersensiNve

Heyonemorething,Chewiehasgivenussomeridiculoussecurityrequirements.Nobodyisallowedtoseetheloyaltycardnumberandstufflikethat.Canwedo

somethingtheretoo?"

Developer DBA

SurewecanLet'simplementdata

redacNon.

DefineaRedacNonPolicytoRedactEmployeeLastNameSQL> BEGIN

dbms_redact.add_policy( object_schema => ’coffeeshop',

object_name => ’customers',

column_name => ’c_custid',

policy_name => 'redact_Loyal_crd',

function_type => DBMS_REDACT.FULL,

expression => '1=1');

Thisisjustawesome,canyoualsogivemeaREST

endpointtothat?

Developer DBA

Ofcourse!

Man,Iwishwewouldhavetalkedtoyousooner.Youpreoymuchgotourgo-livedownfrom6to1week!Hey,canwejustincludeyougoingforwardinourdiscussionsand

projectplanning?

Developer DBA

I'dbedelightedto!Ialwaysthoughtweshouldworkmuchclosertogether.It'salsogreatformetohavebeoerunderstandwhatappsyouarebuildinginorderto

supportthem.

Summary:OracleDatabase12cUniversalPeaceMaker

Summary:

FlexibilityNaNveJSONsupportRESTServices

ScalabilityScale-outandScale-up

Real-TimeAnalyNcsJSONintheIn-MemoryColumnStore

SecurityDataencryptedatrestDataRedacNon

Oracle Database 12c IT WARS - nloug.nl · PDF filecommitment to deliver any material, code ......

Documents

Transcript of Oracle Database 12c IT WARS - nloug.nl · PDF filecommitment to deliver any material, code ......

relaonal( algebra&( calculus(

Adversarial Stas-cal Relaonal AIix.cs.uoregon.edu/~lowd/adversarial-starai-lowd2016.pdf · What is StarAI? “Theore-cally, combining logic and probability in a uniﬁed representaon

The REWASTEE Eco-Innovaon Project · 2017-03-09 · The REWASTEE Eco-Innovaon Project ... deployment and replicaon of a developed technology for recycling steelmaking wastes and the

Cell$Division—Mitosis$ · PDF file3.If(cell(gets(too(big,(it cannot(getenough(nutrients(into(the ... Metaphase Anaphase Telophase Cytokinesis DNA(replicaon,(cell(grows(and(replicates(organelles

Oracle SOA Suite 12c Implementation - nloug.nl of original About Qualogy • What are we in IT-services, Oracle & Java • Since 1998 • Where Rijswijk, Paramaribo • No of employees

Fairnessin( Communication( - University of Alberta · Fairnessin(Communication(A"Relaonal"Fairness"Guide." VeronicaKube"May"2016" Oﬃce"of"the"StudentOmbuds"

Engineered Systems X7 Update - nloug.nl · power off the Exadata Storage Server for services –Turned on automatically in real-time when redundancy is reduced to inform

Pros and Cons of NoSQL Databases - ut · Pros and Cons of NoSQL Databases ... A co‐Relaonal Model of Data for Large Shared Data Banks ... • The RDBMS plus Caching‐Layer Paern

Migration from Oracle Workflow to Oracle BPM 12c - nloug.nl · PDF fileMigration from Oracle Workflow to Oracle BPM 12c By Pedro Gabriel Oracle ADF 11g Certified Oracle BPM 12c Certified

Collec&ve(En&ty(Resolu&on(in(( Relaonal(Data( contd) · 2013-04-11 · Collec&ve(En&ty(Resolu&on(in((Relaonal(Data( contd)! CompSci590.03 Instructor:!Ashwin!Machanavajjhala!! Slides!adapted!from[

The$Evolu*on$of$a$Rela*onal$Database$Layer$over$HBase$phoenix.apache.org/presentations/HBaseCon2015-16x9.pdf · Whatis-Apache-Phoenix?-• A-relaonal-database-layer-for-Apache-HBase-

DNA$Replicaon$/$Prokaryotes$ of DNA.pdf · Photograph$ Reproduced$from$M.$Meselson$and$F.$W.$Stahl,$Proc.&Natl.&Acad.&Sci.& USA 44$(1958):$671/682.$Photo$courtesy$of$Mahew$S.$ Meselson,

Unlocking Hadoop!for!Your!Relaonal!DB! - BigData.bebigdata.be/wp-content/uploads/2014/04/140404... · Whatis! Sqoop? • Apache!TopOLevel!Project • SQl!to!hadOOP! • Tool!to!transfer!datafrom!

Introduc/on to Data Management Lecture #17 SQL NoSQL J · 6/2/17 5 As the Relaonal Era Unfolded • The Spartan simplicity of the relaonal data model made it possible to start tackling

Iframes - nloug.nl · Speaker Mirela Georgescu Date 22-03-2018 Email mirela.georgescu@transfer-solutions.com Iframes when necessity calls for them. 2 “In a close relationship with

PostgreSQL+on+ZFS · SENSING THE PRESENT. SECURING THE FUTURE PostgreSQL+on+ZFS ‣ Replicaon,,Backup,,Human2disaster,Recovery, and,More... ‣ Keith,Paske> ‣ Keith@Paske>.org

Chapter 7 · Chapter 7 Funcons Recap • func.on call statements • setup() and draw() • variables: declaraon, use, assignment ... • various operators (arithme.c, boolean, relaonal,

The$Evolu*on$of$a$Rela*onal$Database$Layer$over$HBase$ · Whatis#Apache#Phoenix?# • A#relaonal#database#layer#for#Apache#HBase # – Queryengine • Transforms#SQL#queries#into#nave#HBase#APIcalls#

We#put#the#SQL#backin# NoSQL* - Apache Phoenixphoenix.apache.org/presentations/HPTS.pdfWhatis#Apache#Phoenix?# • A#relaonal#database#layer#for#Apache#HBase # – Queryengine •

spatial database systemswiki.dpi.inpe.br/lib/exe/fetch.php?media=sds-gribeiro.pdf · DatabaseTechnology& Evoluon & 1960 1970 1980 1990 2000 2010 Object&Oriented&Databases& Object&Relaonal

The$Evoluon$of$a$Relaonal$Database$Layer$over$HBase$phoenix.apache.org/presentations/HBaseCon2015-16x9.pdf · Whatis-Apache-Phoenix?-• A-relaonal-database-layer-for-Apache-HBase-

The$Evoluon$of$a$Relaonal$Database$Layer$over$HBase$ · Whatis#Apache#Phoenix?# • A#relaonal#database#layer#for#Apache#HBase # – Queryengine • Transforms#SQL#queries#into#nave#HBase#APIcalls#