The Forrester Wave™ - Oracledownload.oracle.com/otndocs/products/spatial/pdf/oow_2016/OW2016...The...
Transcript of The Forrester Wave™ - Oracledownload.oracle.com/otndocs/products/spatial/pdf/oow_2016/OW2016...The...
Copyright©2016,Oracleand/oritsaffiliates.Allrightsreserved.|
TheForresterWave™:GeospaEalAnalyEcsToolsAndPlaGorms,Q32016
“WhilehardcoreGISprofessionalsmaystarttheirworkinotherapplicaEons,whentheywanttosolvespaEalproblemsinproducEonandwithweb-andIoT-scaledata,OraclegivesthemtheplaGormtodoso.”
2
Analysts:RowanCurranwithHolgerKisker,Ph.D.andEmilyMillerSeptember1,2016
Geospatial Analytics Tools And Platforms
TheForresterWave™iscopyrightedbyForresterResearch,Inc.ForresterandForresterWave™aretrademarksofForresterResearch,Inc.TheForresterWave™isagraphicalrepresenta>onofForrester'scallonamarketandisplo@edusingadetailedspreadsheetwithexposedscores,weigh>ngs,andcomments.Forresterdoesnotendorseanyvendor,product,orservicedepictedintheForresterWave.Informa>onisbasedonbestavailableresources.Opinionsreflectjudgmentatthe>meandaresubjecttochange.
Copyright©2016,Oracleand/oritsaffiliates.Allrightsreserved.|
BuildApplicaEonsonSpark,Hadoop,andNoSQLwithOracleBigDataSpaEalandGraphJimSteinerVicePresident,ProductManagementSivaRavadaSeniorDirector,ProductDevelopmentZheWuArchitect,ProductDevelopmentSeptember22,2016
Copyright©2016,Oracleand/oritsaffiliates.Allrightsreserved.|
SafeHarborStatementThefollowingisintendedtooutlineourgeneralproductdirecEon.ItisintendedforinformaEonpurposesonly,andmaynotbeincorporatedintoanycontract.Itisnotacommitmenttodeliveranymaterial,code,orfuncEonality,andshouldnotberelieduponinmakingpurchasingdecisions.Thedevelopment,release,andEmingofanyfeaturesorfuncEonalitydescribedforOracle’sproductsremainsatthesolediscreEonofOracle.
4
Copyright©2016,Oracleand/oritsaffiliates.Allrightsreserved.|
ProgramAgenda
5
BigDataSpaEalandGraphfeatureoverview
CommonUseCasesandCodingExamples
BenefitsfortheDeveloper
QuesEonsandAnswers
1
2
3
4
Copyright©2016,Oracleand/oritsaffiliates.Allrightsreserved.|
ACompleteBigDataPorGolio
CREATEVALUEFROMDATA
OracleDataIntegratorOracleGoldenGateOracleBigDataPreparaEon
OracleBigDataSQLOracleBigDataConnectorsOracleNoSQLDatabase
OracleBIEE+VisualAnalyzerOracleBigDataDiscoveryOracleBigDataSpaEalandGraphOracleAdvancedAnalyEcs
OracleBigDataApplianceOraclePublicCloudOracleBigDataCloud
OracleData-as-a-Service
OracleApplicaEons
BIGDATAMANAGEMENT
BIGDATAINTEGRATION
BIGDATAANALYTICS
BIGDATAINFRASTRUCTURE
Copyright©2016,Oracleand/oritsaffiliates.Allrightsreserved.|
BigDataDefined
Big Data: Techniques and Technologies that Enable Enterprises to Effectively and Economically Analyze All of their Data
Copyright©2016,Oracleand/oritsaffiliates.Allrightsreserved.|
RequiresmoredevelopmentresourcesanddatascienEsts
BigDataChallengesBuildyourownenvironmentfromcommodityhardwareandopensourcesoiware.
Copyright©2016,Oracleand/oritsaffiliates.Allrightsreserved.|
MakedevelopersanddatascienEstsmoreproducEve
BigDataSoluEonOpEmized,pre-configuredBigDataApplianceandCloudplaGorms• Popularlanguages,developmentframeworksandserviceAPIs
• In-Memory,parallelOOTBGraphAlgorithms• Massively-scalableGraphDatabase• Pre-built,parallel,MapReduceandSparkSpaEalAlgorithms• RasterandVectorprocessingFrameworks
BigData
Copyright©2016,Oracleand/oritsaffiliates.Allrightsreserved.|
GraphandSpaEalAnalysis–ItisaboutrelaEonships
• ArethingsinthesamelocaEon?Whoisthenearest?Whattaxzoneisthisin?Wherecandeliverin35minutes?Whatisinmysalesterritory?Isthisbuiltinafloodzone?
• WhichsupplieramImostdependentupon?WhoisthemostinfluenEalcustomer?DomyproductsappealtocertaincommuniEes?Whatpamernsarethereinfraudulentbehavior?
Copyright©2016,Oracleand/oritsaffiliates.Allrightsreserved.|
BigDataSpaEalandGraphBigDataoienneedsanOrganizingprinciple–DataHarmonizaEonBigDataanalysisisoienaboutrelaEonshipsnotaggregaEonBigDataplaGormiseconomicallycompellingforworkingwithmassivedatasetsfoundinspaEalandgraphworkflows
Copyright©2016,Oracleand/oritsaffiliates.Allrightsreserved.|
MoEvaEonforOracleBigDataSpaEal
EmergenceofHadoopforspaEalanalysisinbusinessandspaEalworkflowsExisEngHadoop-basedSpaEaltechnologiesareGeoSpaEal-centricnotApplicaEon-centricNosignificantcommercialofferings
Copyright©2016,Oracleand/oritsaffiliates.Allrightsreserved.|
CategorizaEonandfilteringbasedonlocaEonandproximity
PreparaEon,validaEonandcleansingofSpaEalandRasterdata
Visualizinganddisplayingresultsonamap
SpaEalqueryingandanalysisofHadoopdatawithSQL
WhatproblemscanBigDataSpaEalanalysisaddress?
DataHarmonizaEonusinganylocaEonamribute(address,postalcode,lat/long,placename,etc).
Copyright©2016Oracleand/oritsaffiliates.Allrightsreserved.|
Copyright©2016,Oracleand/oritsaffiliates.Allrightsreserved.|
DataenrichmentserviceAPIusingGeoNamesandgeometryhierarchydata
MapReducerouEnesfordistancecalculaEons,PointInPolygon,buffercreaEon,CategorizaEon,KMeansClustering,Binning
SpaEalprocessingofdatastoredinHDFSorNoSQL.RasterprocessingoperaEons:Mosaicandsub-setoperaEons.GeodeEcandCartesiandata
HTML5MapVisualizaEonAPI
HiveSQLAPIQueryfromOracleDBwithBigDataSQL&OracleSQLConnectorsforHadoop
BigDataSpaEalFeatures
Copyright©2016,Oracleand/oritsaffiliates.Allrightsreserved.|
GraphDataModel
Whatisagraph?– Asetoflinksandnodes(andopEonallyamributes)– Agraphissimplylinkeddata
Whydowecare?– Graphsareeverywhere
• Socialnetworks/SocialWeb(Facebook,Linkedin,Twimer,Baidu,Google+,…)• Cybernetworks,powergrids,proteininteracEongraphs• Knowledgegraphs(IBMWatson,AppleSIRI,GoogleKnowledgeGraph)
– GraphsareintuiEveandflexible• Easytonavigate,easytoformapath,naturaltovisualize• Donotrequireapredefinedschema
E
A D
C B
F
Copyright©2016,Oracleand/oritsaffiliates.Allrightsreserved.| 17
PropertyGraph:BigDataSpaEalandGraphFastanalyEcswithhorizontallyscalablestorage
• ParallelIn-memory-basedAnalyst(PGX)– 39built-inmemory-basedgraphanalysisalgorithms– PropertyGraphQueryLanguage(PGQL)– Smartfilteringoflargegraphs
• Flexibleinterfaces– Python,Groovy– Java,Tinkerpop,Blueprints,Gremlin– ApacheLuceneandSolrCloud
• Massively-ScalableGraphDatabase– MulEpleback-ends:NoSQL,Hbase,OracleDatabase– Scalessecurelyto10sofbillionsofnodes/edges
ScalableandPersistentStorage
GraphDataAccessLayerAPI
GraphAnalyEcsIn-memoryAnalyEcEngine
RESTWebService
Blueprints&SolrCloud/Lucene
PropertyGraphStorageonApacheHBaseandOracleNoSQL
Python,Perl,PHP,Ruby,Javascript,…
JavaAPIs
JavaAPIs
Copyright©2016,Oracleand/oritsaffiliates.Allrightsreserved.|
Richsetofbuilt-inparallelgraphalgorithms …andparallelgraphmutaEonoperaEons
ComputaEonalAnalyEcs:Built-inPackage
DetecUngComponentsandCommuniUes
Tarjan’s,Kosaraju’s,WeaklyConnectedComponents,LabelPropagaEon(w/variants),SomanandNarang’sSpacificaEon
RankingandWalking
Pagerank,PersonalizedPagerank,BetwennessCentrality(w/variants),ClosenessCentrality,DegreeCentrality,EigenvectorCentrality,HITS,Randomwalkingandsampling(w/variants)
EvaluaUngCommunityStructures
∑ ∑
Conductance,ModularityClusteringCoefficient(TriangleCounEng)Adamic-Adar
Path-Finding
Hop-Distance(BFS)Dijkstra’s,Bi-direcEonalDijkstra’sBellman-Ford’s
LinkPredicUon SALSA(Twimer’sWho-to-follow)
OtherClassics VertexCoverMinimumSpanning-Tree(Prim’s)
a
d
b eg
c i
f
h
Theoriginalgrapha
d
b eg
c i
f
h
CreateUndirectedGraph
SimplifyGraph
a
d
b eg
c i
f
h
LeiSet:“a,b,e”a d
b
e
g
c
i
CreateBiparUteGraph
ge b d i a f c h
Sort-By-Degree(Renumbering)
FilteredSubgraph
d
bg
i
e
18
Copyright©2016,Oracleand/oritsaffiliates.Allrightsreserved.|
ModelingandAnalyzingTheInternetofThingsMassivescale–TrillionsofConnecEonsDynamicNetworkIncludesmetadataaswellasevents/enricheddataExtensiblebyotherdatasource
19
Copyright©2016,Oracleand/oritsaffiliates.Allrightsreserved.|
• Fast,reliableandsecureserviceonOracleBigDataAppliance
• FullyautomatedserviceforHadoopandSpark
• IncludesBigDataSpaEalandGraphinsubscripEon
OracleBigDataCloudService
Copyright©2016,Oracleand/oritsaffiliates.Allrightsreserved.|
ProgramAgenda
22
BigDataSpaEalandGraphfeatureoverview
CommonUseCasesandCodingExamples
BenefitsfortheDeveloper
QuesEonsandAnswers
1
2
3
4
Copyright©2016,Oracleand/oritsaffiliates.Allrightsreserved.| 23
hmp://semanEcs.us.oracle.com:8988/viewlets/pg_2016_training/oow2016/oow2016_pg_2016_tax_fraud_with_cycle_detecEon_and_visualizaEon/oow2016_pg_2016_tax_fraud_with_cycle_detecEon_and_visualizaEon_viewlet_swf.html
Java,Groovy,BDSG,OracleNoSQLDatabase,PGQL,CytoscapeVisualizaUon,GraphAnalyEcs
Copyright©2016,Oracleand/oritsaffiliates.Allrightsreserved.| 24
Java,Groovy,BDSG,ApacheHBase,Solr,GraphAnalyUcs,GraphNavigaUon,andPython
hmp://semanEcs.us.oracle.com:8988/viewlets/pg_2016_training/oow2016/oow2016_pg_2016_training_recommendaEon_ppr_with_python_chart/oow2016_pg_2016_training_recommendaEon_ppr_viewlet_swf.html
Copyright©2016,Oracleand/oritsaffiliates.Allrightsreserved.|
BuildingSpaEalAnalyEcalApplicaEonswithJavaandPython
25
Copyright©2016,Oracleand/oritsaffiliates.Allrightsreserved.|
RasterDemoMovie
ConfidenEal–OracleInternal/Restricted/HighlyRestricted 26
Copyright©2016,Oracleand/oritsaffiliates.Allrightsreserved.|
ImageProcessingwithfeaturehighlitghing• GivenasetofrasterdataforelevaEon,calculatetheHillShadefromtherasterdata
• Input:SinglebanddigitalelevaEonmodeldata
• Output:Singlebandrasterimageshowingthehillshade
ConfidenEal–OracleInternal/Restricted/HighlyRestricted 27
Copyright©2016,Oracleand/oritsaffiliates.Allrightsreserved.|
• Input:NXMpixelswhereeachpixelisafloaEngpointnumberdenoEngelevaEon• FindtheshadedrelieffromtheDEM• Algorithm
• Lookatthevaluesof8neighborsandthecurrentpixelvalueandgenerateanewpixel• Needstheneighbor’spixelvaluetocalculatethenewpixelvalueforthecurrentpixel
ShadedReliefcalculaEon
Copyright©2016,Oracleand/oritsaffiliates.Allrightsreserved.|
RasterDataAnalysisFramework• TradiEonalalgorithmwillworkverywellinMap-Reduceframework• OncethedataisloadedintoHDFSrestoftheprocessingcanbedoneusingstandardalgorithms
• VeryeffecEveforrasterdataprocessingasmanymap-reducenodescanworktogethertoproducetheresultinashortamountofEme
• Customersdon’tneedtowritemap-reducealgorithmsforimageprocessing• AlltheoperaEonsareperformedonacatalogueofimages
– Customerscanlogicallycombineanynumberofimagesintoacatalogueorintoavirtualmosaic
Copyright©2016,Oracleand/oritsaffiliates.Allrightsreserved.|
RasterDataprocessingworkflow• CopytheimagefilesfromNFStoHDFS
– UsetheprovidedETLtools,noneedtowritecodeforthis• ProcesstheimageswithMap-Reduce
– Weprovidetheframeworktoconvertsinglenodeimageprocessingclassestomap-reduce
– Usersneedtoprovidetheimageprocessingclassforanycustomimageprocessingandalanysis
ConfidenEal–OracleInternal/Restricted/HighlyRestricted 30
Copyright©2016,Oracleand/oritsaffiliates.Allrightsreserved.|
CopytheinputRasterdatatoHDFS
#********Executethescriptwithavalidyarnuserwithaccess************
if__name__=="__main__":
iflen(sys.argv)!=3:
print"usage:imageloader.py<innputLocalFolder><relaEveHDFSPath>where\n"\
"<innputLocalFolder>isacommaseparatedstringwithmulEplefilesorfolderswithimages.\n"\
"<relaEveHDFSPath>HDFSrelaEvepathoftheuserexecuEngthejobtowritetheoutput.\n"
else:
try: printConf()
doLoader(sys.argv[1],sys.argv[2])
listResult(sys.argv[2])
ConfidenEal–OracleInternal/Restricted/HighlyRestricted 31
UsingPYTHONscript
Copyright©2016,Oracleand/oritsaffiliates.Allrightsreserved.|
CopytheinputRasterdatatoHDFS
#*************************************************
#TO-DOConfiguretheseproperEesasnecessary
#************************************************
SHARED_DIR ="/net/host100/scratch/abcduser/test/"
INSTALLATION ="/opt/oracle/oracle-spaEal-graph/"
ALL_ACCESS_DIR =SHARED_DIR+"write/"
GDAL_LIB =ALL_ACCESS_DIR+"gdal"
GDAL_DATA =SHARED_DIR+"data"
RASTER_INST =INSTALLATION+"spaEal/raster/"
LOADER_JAR =RASTER_INST+"jlib/"+"hadoop-imageloader.jar"
#Addmorehadoop_classpathjarsasnecessary
CLASSPATH_JARS =["hadoop-imageprocessor.jar","hadoop-raster-fwk-api.jar","gdal.jar"]
ConfidenEal–OracleInternal/Restricted/HighlyRestricted 32
UsingPYTHONscripts
Copyright©2016,Oracleand/oritsaffiliates.Allrightsreserved.|
CalculatetheslopeprivatedoublecalculateSlope(double[]neighboors,doublepixelWidthX,doublepixelWidthY){
doubleslope=0;
dx=((neighboors[0]+(2*neighboors[3])+neighboors[6])-
(neighboors[2]+(2*neighboors[5])+neighboors[8]))/(8*pixelWidthX);
dy=((neighboors[6]+(2*neighboors[7])+neighboors[8])-
(neighboors[0]+(2*neighboors[1])+neighboors[2]))/(8*pixelWidthY);
slope=Math.sqrt(Math.pow(dx,2)+Math.pow(dy,2));
returnslope;
ConfidenEal–OracleInternal/Restricted/HighlyRestricted 33
Copyright©2016,Oracleand/oritsaffiliates.Allrightsreserved.|
CalculateHillshadedoublezenithRadians=90-alEtude*Math.PI/180;doubleazimuthRadians=azimuth_math*Math.PI/180;clearNeighborhood(neighborhood);slopeRadians=Math.atan(slope*zFactor);aspectRadians=calculateAspectRadians();hillShade=calculateHillShade(zenithRadians,azimuthRadians,slopeRadians,aspectRadians);
finalFloatData[posiEon]=hillShade;
ConfidenEal–OracleInternal/Restricted/HighlyRestricted 34
Copyright©2016,Oracleand/oritsaffiliates.Allrightsreserved.|
ExecuEngtheanalysisprocessasamap-reducejobNowthatthecatalogandmosaicconfiguraEonareset,executethejobusingthehadoopjarcommandwiththehadoop-imageprocessor.jarandalltheopEonsdescribedinthelaststepsandwaitunElitfinishes.
hadoopjar/opt/oracle/oracle-spaEal-graph/spaEal/raster/jlib/hadoop-imageprocessor.jar-catalog/opt/oracle/oracle-spaEal-graph/spaEal/raster/Raster-HOL/data/xml/inputDEM.xml-config/opt/oracle/oracle-spaEal-graph/spaEal/raster/Raster-HOL/data/xml/testSlopeFS.xml-gdal/opt/oracle/oracle-spaEal-graph/spaEal/raster/gdal/lib-gdalData/opt/oracle/oracle-spaEal-graph/spaEal/raster/gdal/data-usrlib/opt/oracle/oracle-spaEal-graph/spaEal/raster/jlib/external-analysis.jar
ConfidenEal–OracleInternal/Restricted/HighlyRestricted 35
Copyright©2016,Oracleand/oritsaffiliates.Allrightsreserved.|
• DatacontaininglocaEonreferencesisprocessedandcategorizedbasedonacustomgeometryhierarchy
• Input:TweetdataforJanuary2016Emeperiod
• Output:categorizeddatathatassociateseachtweettoageometryinthehierarchy
ConfidenEal–OracleInternal/Restricted/HighlyRestricted 36
Categorizedatabasedonahierarchy--Java
Copyright©2016,Oracleand/oritsaffiliates.Allrightsreserved.|
LoadtheinputdataonHDFSforspaEalprocessing• AssumethatthedataisstoredonanNFSfilesystemthatisalsoaccessiblefromtheHDFSserver
• TheinputdataisstoredasatextfileintheJSONformatwithonerecordforonetweethadoopfs-put/opt/oracle/oracle-spaUal-graph/spaUal/vector/HOL/data/tweets.json
/user/oracle/HOL/tweets.json
• ItisassimpleascopyingthedatafromtheNFStoHDFSusinghaddopfilesystemcommands– Nopre-processingrequiredsothedataisreadytogooncethecopyisdone
ConfidenEal–OracleInternal/Restricted/HighlyRestricted 37
Copyright©2016,Oracleand/oritsaffiliates.Allrightsreserved.|
• Ifthedataisinstandardknownformatsusersdon’tneedtowriteanycodeforreadingtherecordsandextracEnggeometryfromit
• Standardformatssupported– JSON,ShapeFiles
• Forothers,usersneedtoprovidecustomcustomRecordInfoReaderclass– WewillshowanexampleofthisusingJSONformat
ConfidenEal–OracleInternal/Restricted/HighlyRestricted 38
HowtoreadTweetdatatoextractgeometryfromit
Copyright©2016,Oracleand/oritsaffiliates.Allrightsreserved.|
RecordInfoProviderExample
ConfidenEal–OracleInternal/Restricted/HighlyRestricted 39
Copyright©2016,Oracleand/oritsaffiliates.Allrightsreserved.|
SetupthecustomhierarchyusingJSONfiles
ConfidenEal–OracleInternal/Restricted/HighlyRestricted 40
Copyright©2016,Oracleand/oritsaffiliates.Allrightsreserved.|
ExampleJSONdataforcustomhierarchy
ConfidenEal–OracleInternal/Restricted/HighlyRestricted 41
Exampleofrecordinthefileeurozone_countries.json:{"type":"Feature","_id":"FRA","geometry":{"type":"Polygon","coordinates":[[1.44136,42.60366,1.47851,42.65168,…,1.44136,42.60366]]},"properEes":{"ConEnent":"EU","Name":"France","Alt_Region":"EMEA","CountryCode":"FRA"},"label_box":[-1.12061,45.13915,6.02255,49.19591]}AndexampleofarecordrepresenEngaFranceprovinceineurozone_provinces.json:{"type":"Feature","_id":"3023519","geometry":{"type":"Polygon","coordinates":[[9.19977,41.36465,9.25876,…,41.36465]]},"properEes":{"Country":"France","ISO":"FRA","StateProvinceName":"Corse","CountryCode_StateProvinceName":"FRA_Corse","CountryName_StateProvinceName":"France_Corse"},"label_box":[8.70974,42.08453,9.53075,42.54732]}
Copyright©2016,Oracleand/oritsaffiliates.Allrightsreserved.|
InvoketheJavaclassestorunthecategorizaEonjobhadoopjar$API_LIB_DIR/sdohadoop-vector.jaroracle.spaEal.hadoop.vector.mapred.job.CategorizaEon-libjars$HADOOP_LIB_JARSspaEalOperaEon=IsInsideinput=/user/oracle/HOL/tweets.jsonoutput=/user/oracle/HOL/euroZoneCatOutputinputFormat=oracle.spaEal.hadoop.vector.geojson.mapred.GeoJsonInputFormatrecordInfoProvider=oracle.spaEal.hadoop.vector.geojson.GeoJsonRecordInfoProvidersrid=8307geodeEc=truetolerance=0.5hierarchyInfo=hol.EuroHierarchyInfohierarchyIndex=/user/oracle/HOL/hierarchyIndexhierarchyDataPaths=file:///opt/oracle/oracle-spaEal-graph/spaEal/vector/HOL/data/eurozone_countries.json,file:///opt/oracle/oracle-spaEal-graph/spaEal/vector/HOL/data/eurozone_provinces.json
ConfidenEal–OracleInternal/Restricted/HighlyRestricted 42
Copyright©2016,Oracleand/oritsaffiliates.Allrightsreserved.|
SimpleHTML5applicaEontoshowtheresultsonamapmap.addLayer(EleLayer);varcs=newOM.style.Color({strokeThickness:1,stroke:"#009400",fill:"#5c85ad",fillOpacity:0.75});varlayer=newOM.layer.VectorLayer("datapack_layer",{def:{type:OM.layer.VectorLayer.TYPE_DATAPACK,url:"saved_results.json",labelColumn:"county"},renderingStyle:cs});
layer.setLabelsVisible(true);layer.setBoundingTheme(true);
ConfidenEal–OracleInternal/Restricted/HighlyRestricted 43
Copyright©2016,Oracleand/oritsaffiliates.Allrightsreserved.|
ImageProcessingwithfeatureExtracEon--Python• GivenasetofrasterdataforelevaEon,calculatetheHillShadefromtherasterdata
• Input:SinglebanddigitalelevaEonmodeldata
• Output:Singlebandrasterimageshowingthehillshade
ConfidenEal–OracleInternal/Restricted/HighlyRestricted 44
Copyright©2016,Oracleand/oritsaffiliates.Allrightsreserved.|
CopytheinputRasterdatatoHDFS
#********Executethescriptwithavalidyarnuserwithaccess************
if__name__=="__main__":
iflen(sys.argv)!=3:print"usage:imageloader.py<innputLocalFolder><relaEveHDFSPath>where\n"\
"<innputLocalFolder>isacommaseparatedstringwithmulEplefilesorfolderswithimages.\n"\
"<relaEveHDFSPath>HDFSrelaEvepathoftheuserexecuEngthejobtowritetheoutput.\n"
else:try: printConf()
doLoader(sys.argv[1],sys.argv[2])
listResult(sys.argv[2])
exceptExcepEonasinst:
printinst.args
printinst
ConfidenEal–OracleInternal/Restricted/HighlyRestricted 45
UsingPYTHONscripts
Copyright©2016,Oracleand/oritsaffiliates.Allrightsreserved.|
CopytheinputRasterdatatoHDFS#*************************************************
#TO-DOConfiguretheseproperEesasnecessary
#************************************************
SHARED_DIR ="/net/host100/scratch/abcduser/test/"
INSTALLATION ="/opt/oracle/oracle-spaEal-graph/"
ALL_ACCESS_DIR =SHARED_DIR+"write/"
GDAL_LIB =ALL_ACCESS_DIR+"gdal"
GDAL_DATA =SHARED_DIR+"data"
RASTER_INST =INSTALLATION+"spaEal/raster/"
LOADER_JAR =RASTER_INST+"jlib/"+"hadoop-imageloader.jar"
OVERLAP ="10"
#Addmorehadoop_classpathjarsasnecessary
CLASSPATH_JARS =["hadoop-imageprocessor.jar","hadoop-raster-fwk-api.jar","gdal.jar"]
ConfidenEal–OracleInternal/Restricted/HighlyRestricted 46
UsingPYTHONscripts
Copyright©2016,Oracleand/oritsaffiliates.Allrightsreserved.|
CalculatetheslopeprivatedoublecalculateSlope(double[]neighboors,doublepixelWidthX,doublepixelWidthY){
doubleslope=0;
dx=((neighboors[0]+(2*neighboors[3])+neighboors[6])-
(neighboors[2]+(2*neighboors[5])+neighboors[8]))/(8*pixelWidthX);
dy=((neighboors[6]+(2*neighboors[7])+neighboors[8])-
(neighboors[0]+(2*neighboors[1])+neighboors[2]))/(8*pixelWidthY);
slope=Math.sqrt(Math.pow(dx,2)+Math.pow(dy,2));
returnslope;
ConfidenEal–OracleInternal/Restricted/HighlyRestricted 47
Copyright©2016,Oracleand/oritsaffiliates.Allrightsreserved.|
CalculateHillshadedoublezenithRadians=90-alEtude*Math.PI/180;doubleazimuthRadians=azimuth_math*Math.PI/180;clearNeighborhood(neighborhood);slopeRadians=Math.atan(slope*zFactor);aspectRadians=calculateAspectRadians();hillShade=calculateHillShade(zenithRadians,azimuthRadians,slopeRadians,aspectRadians);
finalFloatData[posiEon]=hillShade;
ConfidenEal–OracleInternal/Restricted/HighlyRestricted 48
Copyright©2016,Oracleand/oritsaffiliates.Allrightsreserved.|
ExecuEngtheanalysisprocessasamap-reducejobNowthatthecatalogandmosaicconfiguraEonareset,executethejobusingthehadoopjarcommandwiththehadoop-imageprocessor.jarandalltheopEonsdescribedinthelaststepsandwaitunElitfinishes.
hadoopjar/opt/oracle/oracle-spaEal-graph/spaEal/raster/jlib/hadoop-imageprocessor.jar-catalog/opt/oracle/oracle-spaEal-graph/spaEal/raster/Raster-HOL/data/xml/inputDEM.xml-config/opt/oracle/oracle-spaEal-graph/spaEal/raster/Raster-HOL/data/xml/testSlopeFS.xml-gdal/opt/oracle/oracle-spaEal-graph/spaEal/raster/gdal/lib-gdalData/opt/oracle/oracle-spaEal-graph/spaEal/raster/gdal/data-usrlib/opt/oracle/oracle-spaEal-graph/spaEal/raster/jlib/external-analysis.jar
ConfidenEal–OracleInternal/Restricted/HighlyRestricted 49
Copyright©2016,Oracleand/oritsaffiliates.Allrightsreserved.|
DifferenEators--SpaEal• Commercial,supportedsoiware• BigData/MapReduceapproachvs.GIS-relaEonalapproach
– WorksonanydatathatincludeslocaEoninfo,datatypeandfileformat
• GeoEnrichmentservicesincludingglobalgeographichierarchy• Veryrichset(~50)ofspaEaloperatorsandfuncEons• SupportsbothspaEalprocessingandspaEalanalyEcs• BothVectorandRasterservices• IncludesmapvisualizaEon
Copyright©2016,Oracleand/oritsaffiliates.Allrightsreserved.|
DifferenEators--Graph• Commercial,supportedsoiware• “BestofBothWorlds”GraphDB
– In-memorygraphanalysisalgorithms– Distributedgraphdatabasemodel
• SQL-likeParallelGraphQueryLanguage(PGQL)• Dozensofpre-builtin-memorygraphanalysisalgorithms• 10-50xfasteranalyEcsthancompeEtors’offerings
• Analyze20-30BillionedgegraphinmemoryonasingleBDAnode
Copyright©2016,Oracleand/oritsaffiliates.Allrightsreserved.|
SpaEalandGraphatOpenWorld2016SubUtle
OracleSpaEalandGraphProductTeam
ConfidenEal–OracleInternal/Restricted/HighlyRestricted
Copyright©2016,Oracleand/oritsaffiliates.Allrightsreserved.|
The Spatial & Graph SIG User Community The SIG connects and exchanges knowledge via online communities and at conferences and events
• Chat & Coffee at OOW Join us & chat with other S&G users about your experiences! Bring your own coffee J Wednesday at 8:00 am Meet in front of Oracle Bookstore – Moscone South Lobby
• Join us online tinyurl.com/oraclespatialcommunity LinkedIn , Google+ & IOUG SIG groups @oraspatialsig | [email protected]
Copyright©2016,Oracleand/oritsaffiliates.Allrightsreserved.|
UpcomingEvents• OracleSpaEalSummitatBIWA2017OracleHQ,RedwoodShores,CAJan.31-Feb.2,2017www.biwasummit.org*Callforspeakersnowopen!
55
CallforSpeakersNow
Open!
Visitwww.biwasummit.org
Copyright©2016,Oracleand/oritsaffiliates.Allrightsreserved.|
LearnMoreAboutOracle’sSpaUalandGraphTechnologies• www.oracle.com/technetwork/database/opEons/spaEalandgraphwww.oracle.com/database/big-data-spaEal-and-graph
• blogs.oracle.comèoraclespaEalèoracle_maps_blogèbigdataspaEalgraph
CVCSpaEalUpdateforDWR
Copyright©2016,Oracleand/oritsaffiliates.Allrightsreserved.|
ResourcesonBigDataSpaEalandGraph• OracleBigDataSpaEalandGraphonOracle.com:www.oracle.com/database/big-data-spaEal-and-graph
• OTNproductpage(whitepapers,soiwaredownloads,documentaEon,tutorials):www.oracle.com/technetwork/database/database-technologies/bigdata-spaEalandgraph
• OracleBigDataLiteVirtualMachine-afreesandboxtogetstarted:www.oracle.com/technetwork/database/bigdata-appliance/oracle-bigdatalite-2104726.html
• HandsOnLabforBigDataSpaEal:Enyurl.com/BDSG-HOL• Blog–examples,Eps&tricks:blogs.oracle.com/bigdataspaEalgraph
• @OracleBigData,@SpaEalHannes,@JeanIhm
58