Custom Maps in Oracle Big Data Discovery with Oracle's ... · Oracle Spatial Summit at BIWA 2017 ....
Transcript of Custom Maps in Oracle Big Data Discovery with Oracle's ... · Oracle Spatial Summit at BIWA 2017 ....
Oracle Spatial Summit at BIWA 2017
Custom maps in Oracle Big Data Discovery (BDD) with Oracle’s Spatial and Graph technologies David Lapp, Product Manager Oracle Spatial and Graph
Oracle Spatial Summit at BIWA 2017
Safe Harbor Statement
The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle.
2
Oracle Spatial Summit at BIWA 2017
Agenda
• Cloud enabled data lab • Use cases for custom mapping in BDD • Technical overview • Demonstration • Resources
Oracle Spatial Summit at BIWA 2017
Oracle Big Data Discovery The visual face of Big Data, in the Oracle Cloud
find explore transform discover share
Oracle Spatial Summit at BIWA 2017
Oracle BDD built-in map features
“Thematic Map” • Standard administrative boundaries • Drill Country>State/Province>County • Use any metric
“Map” • Point locations using BDD’s geocode attribute type • Render as points, numbered markers, or heatmap • Radius search
Oracle Spatial Summit at BIWA 2017
BDCS Cluster
Cloud Enabled Data Lab
• Cloudera Enterprise • Oracle Big Data Connectors • Oracle Big Data Spatial and Graph • Oracle Data Integrator • Oracle Big Data SQL (add-on)
Oracle Big Data Cloud Service
Oracle Big Data Discovery Cloud Service
Had
oop
Nod
es
• Web Studio • In-memory data index • Data Processing • Sampling, Profiling, Enrichment,
Cataloging, Transforms
Had
oop
Nod
es
Oracle Spatial Summit at BIWA 2017
Oracle Big Data Spatial and Graph 12c (BDSG) Spatial features
• Vector and Raster analysis on Hadoop • Point in polygon, distance, buffer etc • Raster mosaic, subset etc
• Spark and MapReduce processing • Clusters, binning, categorization
• HTML5 map visualization API
Oracle Spatial Summit at BIWA 2017
Oracle BDD with BDSG Use cases
• Mapping with custom layers • Sales territories, flood zones, public safety regions • Transportation routes, utility network, waterway
• Integration and mapping of BDSG processing results • Spatial clusters • Spatial binning • Spatial catergorization
Oracle Spatial Summit at BIWA 2017
Custom Visualization Componant (CVC) BDD feature enabling custom mapping
• Framework for adding custom visualization to BDD
• Invoke any client-side JS library from CVC JS API
• Custom visualizations maintain state with the page
• CVC tutorial available on OTN
Oracle Spatial Summit at BIWA 2017
BDD custom mapping Using CVC with BDSG map visualization library
Oracle Maps JS API
CVC JS API
BDD dashboard
Custom map
Merge BDD results with custom layers
End user interactions
Custom layers
BDD results as JSON
GeoJSON
Standard componants
Oracle Spatial Summit at BIWA 2017
Demo • Using Big Data Lite VM bundled “Movie demo” content • Added “Division” attribute for mapping (based on US Census Divisions)
Oracle Spatial Summit at BIWA 2017
Reference individual BDSG map visualization libraries to avoid conflict with BDD’s libraries.
I used the name in the CVC tutorial. You can
use any name
Use any name for the CVC
Oracle Spatial Summit at BIWA 2017
1st part of code:
• Boilerplate from CVC tutorial
• Collect BDD results
• Create region on the canvas
Oracle Spatial Summit at BIWA 2017
2nd part of code:
• Merges BDD results with GeoJSON layer
• Renders map
Oracle Spatial Summit at BIWA 2017
Next steps for this demo
• Determine the map layer at runtime from the selected dimension
• Drive refinements from map to portal (supported by CVC API)
• Leverage BDSG spatial analytics such as clustering and binning
Oracle Spatial Summit at BIWA 2017
BDD custom mapping Using CVC with and Oracle Spatial and Graph
Oracle Maps JS API
CVC JS API
BDD dashboard
Custom map
Invoke map component
End user interactions
Automatically merge BDD results with custom layers
Spatial data
Predefined styles and themes
WebLogic
Oracle Database
Oracle Spatial and Graph
Map component
BDD results as JSON
Oracle Spatial Summit at BIWA 2017
Alternatives for custom BDD maps using CVC
Oracle Maps JS API only
• For Hadoop environment without Oracle Database (i.e. Oracle Big Data Cloud Service)
• Leverages mapping API of BDSG so no additional technology needed.
• All functionality is JS client
Oracle Spatial and Graph, and map component
• For environment that includes Hadoop and geospatial data in Oracle Database
• Requires deployment and administration of map component (servlet)
• Leverages server features, reducing need for JS client code
Oracle Spatial Summit at BIWA 2017
Resources Demo CVC JS code: Appendix to this presentation
Big Data Lite 4.21 VM: http://www.oracle.com/technetwork/database/bigdata-appliance/oracle-bigdatalite421-2843803.html
Big Data Discovery CVC tutorial: http://www.oracle.com/technetwork/middleware/big-data-discovery/downloads/index.html
Big Data Spatial and Graph sessions: Workshop: Apply Location Intelligence and Spatial Analysis to Big Data with Java Tues 3:34-5:20, Room 202
Presentation: Bring Location Intelligence To Big Data Applications on Spark, Hadoop, and NoSQL Thus 2:30, Room 103
Oracle Spatial Summit at BIWA 2017
Appendix – CVC custom map example code. The following code sample is solely for demonstration purposes.
Oracle Spatial Summit at BIWA 2017
Oracle.BDD.Portlets.Visualization.Renderers.RendererTest = Oracle.BDD.Portlets.Visualization.Renderers.BaseRenderer.extend({ /** * Init function is straight from CVC tutorial */ init: function() { /** * Get the queryConfig for the initial query */ var queryConfig = this.getQueryConfig("eql"); /** * Set default value for num_recs data token if (!queryConfig.getToken("num_recs").getValue()) { queryConfig.getToken("num_recs").setValue("10"); } */ /** * Execute the initial query, indicating renderTest() * as the callback */ this.executeQuery(queryConfig, true, this.renderTest); } ,
Oracle Spatial Summit at BIWA 2017
/** * Callback function to create map */ renderTest: function(queryResults) { var self = this; /** * Get references to the data records and the queryConfig that * defined the EQL query. */ var records = queryResults.getRecords(); /** * Acquire reference to canvas, setting the root node as <div> */ var canvas = this.getCanvas({ tagName: 'div', margins: {top: 0, bottom: 0} }); canvas.clear(); var root = canvas.getRoot(); var divElement = document.createElement("div"); divElement.id = "mapDiv"; divElement.style.width= "800px"; divElement.style.height= "500px"; root.appendChild(divElement);
Oracle Spatial Summit at BIWA 2017
// oraclemaps code OM.gv.setResourcePath("http://localhost:7777/mapviewer/jslib/v2/"); var mpoint = new OM.geometry.Point(-100.45,25,8307); var map = new OM.Map(divElement) ; var tileLayer = new OM.layer.ElocationTileLayer("tilelayer"); map.addLayer(tileLayer); /** Next command adds map layer which requires url to json file. * In BDL VM, the following path maps to http://.../bdd/html * /u01/bdd/v1.1.1/user_projects/domains/bdd-1.1.1.13.11_domain * /servers/bigdatalite.localdomain/tmp/_WL_user/ * /oracle.endecastudio_1.1.1.0.0/b4iezl/war/html * so it’s convenent to put json map layer there */ layer = new OM.layer.VectorLayer("layer1", { def:{ type:OM.layer.VectorLayer.TYPE_DATAPACK, url: "http://localhost:9003/bdd/html/oraclemaps/datapacks/us_divisions.json" } });
// add layer listener to run after layer loads in order to // join the map layer to the BDD results layer.addListener(OM.event.LayerEvent.FEATURES_LOADED, this.afterFeaturesLoaded); // add layer listener to run when layer is clicked to // create pop up window layer.addListener(OM.event.MouseEvent.MOUSE_CLICK, this.featureClicked);
Oracle Spatial Summit at BIWA 2017
// transfer vars to listener fn by adding to layer // i.e. layer.testing = {"myRecords":records, "myDataObj": dataObj}; layer.bdd = {"bdd_records":records}; map.addLayer(layer) ; map.setMapCenter(mpoint); map.setMapZoomLevel(2) ; var legend = new OM.control.MapDecoration( "<table border=0 width=80> " +"<tr><td bgcolor='#0000ff'>    </td><td>  Top Third </td></tr>" +"<tr><td bgcolor='#00ff00'>    </td><td>  Middle Third </td></tr>" +"<tr><td bgcolor='#ff0000'>    </td><td>  Bottom third </td></tr>" +"</table>", { offsetX:10, offsetY:375, draggable:true, contentStyle:{ "border-style":"none", "border-color":"#101418", "border-width":"1px", "font-size":"11px", "color":"#101418", "background-color":"#DAE1E3", "font-family":"Tahoma" } }); map.addMapDecoration(legend); map.init() ; } ,
Oracle Spatial Summit at BIWA 2017
afterFeaturesLoaded: function(event) { // layer is automatically transferred to listener fn as the event target var layer = event.target; var features = layer.getAllFeatures(); // retreive BDD records from layer.bdd var bddRecords = layer.bdd.bdd_records; console.log("---------------"); console.log("BDD data :"); // create hash of BDD results to use for matching to map layer var bddDataHash = {}; for (var n in bddRecords) { var tmp = JSON.stringify(bddRecords[n].groupby); if (tmp) { bddKey = tmp.replace(/\"/g, ""); //get rid of double quotes } // var bddVal = parseInt(JSON.stringify(bddRecords[n].metric)); var bddVal = parseFloat(JSON.stringify(bddRecords[n].metric)); if ( !isNaN(bddVal) ) { bddDataHash[bddKey]=bddVal; console.log(bddKey+":"+bddDataHash[bddKey]); } } console.log("---------------"); console.log("map layer -> BDD matching :");
Oracle Spatial Summit at BIWA 2017
// for each feature find matching BDD data and add its attribute for (var i = 0; i < features.length; i++) { feature = features[i]; mapKey=feature.getAttributeValue("NAME"); var featureAttr; if (bddDataHash[mapKey]){ console.log(mapKey+": match"); featureAttr = {"MAP_KEY": mapKey, "BDD_METRIC":bddDataHash[mapKey]}; } else { //console.log('no match for mapKey = '+mapKey); featureAttr = {"MAP_KEY": mapKey, "BDD_METRIC":null}; } layer.attrNames = ["MAP_KEY","BDD_METRIC"]; features[i].attributes=featureAttr; } var defaultColor = new OM.style.Color({fill:"#000000",strokeThickness:2,stroke:"#000000"}); var color01 = new OM.style.Color({fill:"#ff0000",fillOpacity:.6,strokeThickness:2,stroke:"#000000"}); var color02 = new OM.style.Color({fill:"#00ff00",fillOpacity:.6,strokeThickness:2,stroke:"#000000"}); var color03 = new OM.style.Color({fill:"#0000ff",fillOpacity:.6,strokeThickness:2,stroke:"#000000"}); var equalBucket = new OM.style.BucketStyle({ styleName: 'Equal Bucket', numClasses : 3, classification: 'equal', gradient: "on", styles : [color01,color02,color03] }) ; layer.setRenderingStyle(equalBucket, ["BDD_METRIC"]); layer.redraw(); },
Oracle Spatial Summit at BIWA 2017
featureClicked: function(event) { var map = event.target.parentMap; var point = map.getCursorLocation(); var attributes = event.feature.attributes; var content = attributes.MAP_KEY+"<br>"+attributes.BDD_METRIC; map.displayInfoWindow( point,content); } });
// end of sample code