Adding a Custom Layer (geoJSON) to Visual Analyzer in Oracle BI EE 12c

Contents

Overview ........................................................................................................................................................1

Adding an (ESRI) shapefile as a custom layer ................................................................................................2

Additional steps needed in SampleAppV511 .............................................................................................5

Uploading the GeoJSON file for use with VA .................................................................................................8

Using a custom layer in Visual Analyzer .......................................................................................................13

Overview

This document describes how to add a custom layer to Visual Analyzer (VA) in Oracle BI EE 12c – version

12.2.1.

Note: This doc doesn't apply to maps part of VA in Oracle Business Intelligence Cloud Service or in Oracle

Data Visualization Cloud Service as of February 2016.

The basic steps are: (i) convert the data to a geoJSON file, (ii) place the geoJSON file in the appropriate

directory under a MapViewer home, (iii) add a configuration file, and (iv) restart Mapviewer or instruct it

to re-scan the geoJSON files.

Adding an (ESRI) shapefile as a custom layer

The shapefile must first be converted to geoJSON. The simplest way is using Oracle MapBuilder 12.2.1 or

later. Download it from OTN (it requires JDK1.7 or later). Double-click on MapBuilder to start it or use

the command line “java –jar mapbuilder.jar” optionally providing memory parameters like –

Xmx 1024M.

Note: Find out the spatial reference (coordinate system) used in the shapefile before opening it in

MapBuilder. The spatial reference metadata is contained in the .xml file (e.g. nypp.shp.xml) and will

contain something like <projcsn Sync="TRUE">

NAD_1983_StatePlane_New_York_Long_Island_FIPS_3104_Feet</projcsn>.

Look up the corresponding EPSG spatial reference identifier for this at spatialreference.org. e.g.

http://spatialreference.org/ref/?search=new+york gives:

EPSG:2129: NZGD2000 / Mount York 2000

EPSG:2260: NAD83 / New York East (ftUS)

EPSG:2261: NAD83 / New York Central (ftUS)

EPSG:2262: NAD83 / New York West (ftUS)

EPSG:2263: NAD83 / New York Long Island (ftUS)

among others. We will use 2263 as it's for New York Long Island, the units are Feet, and the datum is

NAD83.

Start MapBuilder. The tools dialog should have an Export to JSON option.

Choose Export to JSON and ignore the error about GDAL.jar. In the wizard choose shapefileSDP as the

source and navigate to the file to import.

Select all the attributes as properties to be included in the JSON file. Choose the Key Column (Precinct in

this case) and click next.

Specify the destination file name and location and the spatial reference id and click next in the dialog.

Then click finish.

Once the file is created view it in some text editor. It should be a text file with content like

{"type":"FeatureCollection",

"collectionName":"customTheme",

"srs":2263,

"geodetic":false,

"bbox":[913175.10901, 120121.88125, 1067382.50842, 272844.29364],

"attr_names":["PRECINCT","SHAPE_AREA","SHAPE_LENG"],

"attr_types":["double","double","double"],

"features":[

{"type":"Feature","_id":"1","geometry": {"type":"Polygon",

"coordinates":[[972081.78821,190733.46741, …

Additional steps needed in SampleAppV511

Due to limitations and bugs in the MapViewer version deployed in OBIEE 12.2.1 of SampleAppV511 the

above process and output JSON file will not work as is with VA. It needs additional steps involving the

database.

The limitations, and bugs, are:

The key column values must be strings.

The SRID (spatial reference id) must be 8307, 4326, or 3857. 8307 and 4326 are SRIDs fpr

Longitude, Latitude coordinates using the WGS84 datum (which is used by GPS). 3857 is the

World Mercator Projections used in Google, Bing, HERE, and Oracle maps.

Note: If some other coordinate system is used then the imported layer can only be displayed as a

Thematic map by itself. That is, it cannot be overlaid on a background map. So if we used

nypp.json with srid 2263 in a newer MapViewer/VA we could only display just the precincts in a

map view. The precincts could not be overlaid on a street level detail background map. Similarly

we could import a floorplan (such as Moscone demogrounds) and show just that as a layer in a

map view. The limitation that only SRIDs 8307, 4326, and 3857 will be supported for layers that

can be overlaid on background maps will continue to exist. There are no plans to support

arbitrary spatial reference systems in the Oracle Maps client.

Since we are likely to want to display NY precincts on a background map the coordinate/spatial

reference system must be 3857 or 4326. We will use the database to convert the NY State Plane (2263)

to World Mercator (3857).

The first step is use import the shapefile into the database using MapBuilder. Add or open the

connection to OBIEE_NAVTEQ. Click on Tools-> Import shapefile. Select the file nypp.shp. Specify 2263

as the SRID when importing it and do not create a spatial index. That is, uncheck the Create/Update

spatial index option.

Once the shapefile is imported connect as obiee_navteq (using sqlplus or sqldeveloper) and do the

following.

Add a new geometry column to hold the transformed geometries.

ALTER TABLE NYPP ADD (GEOM3857 SDO_GEOMETRY);

Add an entry in user_sdo_geom_metadata for this column.

INSERT INTO USER_SDO_GEOM_METADATA VALUES('NYPP', 'GEOM3857',

SDO_DIM_ARRAY(SDO_DIM_ELEMENT('X', -180, 180, 0.5),

SDO_DIM_ELEMENT('Y', -90, 90, 0.5)), 3857);

Transform the coordinates from 2263 to 3857

UPDATE NYPP SET GEOM3857 = SDO_CS.TRANSFORM(GEOMETRY, 3857);

COMMIT;

Next we will modify the precinct values to be strings by prefixing them with 'Precinct '.

ALTER TABLE NYPP ADD (PRECINCT_STR VARCHAR2(16));

UPDATE NYPP SET PRECINCT_STR = 'Precinct ' || PRECINCT ;

COMMIT;

Create a theme on this table using MapBuilder. The geometry column should be GEOM3857. Add

PRECINCT_STR as an info-column with alias Precinct. Specify PRECINCT as the Key Column instead of

ROWID. The theme xml should look like

Now use MapViewer to export this theme to geoJSON. Restart MapViewer and then open Firefox to

issue the following data server request to output the theme as geoJSON.

demo.us.oracle.com:7780/mapviewer/dataserver/obiee_navteq?t=nypp

Save the file shown in the browser as nypp.json. It should look like

{"type":"FeatureCollection",

"collectionName":"nypp",

"srs":3857,

"geodetic":false,

"bbox":[-8266094.6191, 4938300.50611, -8204247.48045, 4999890.73936],

"attr_names":["Precinct"],

"attr_types":["string"],

"columnMap":{"PRECINCT_STR":"Precinct"},

"edit_properties":{"id_attr":"PRECINCT","base_table":"NYPP","spatial_attr":"GEOM3857"

,"spatial_type":"geometry"},

"features":[

{"type":"Feature","_id":"20","geometry": {"type":"Polygon", "coordinates":[[-

8234228.78821,4980723.86729,-8234544.7593,4980899.2926,-8234861.13583,4981075.73046,-

8235014.23157,4981161.04185,-8235026.47702,4981167.79483,-8235038.00389 …

The third step is to modify the spreadsheet (.xlsx) which has the BI data (crime by precinct) to contain

the same key values (i.e. 'Precinct 1' instead of just he number 1) for the key column. The spreadsheet

has a column named PCT with values 1, 2, 5, 7 etc. for the precinct numbers. Add a new column named

PRECINCT and populate it using the formula concatenate(“Precinct “, A2) for the first cell and then copy

and paste the formula into the remaining cells of the new column. Change all the column names (years)

which are just numbers (2001, 2002 etc) to start with a character (e.g. C2001, C2002). This is needed as a

workaround for a possible bug. Save the spreadsheet. It should look like

The next step is to upload the spreadsheet and geojson files for use with VisualAnalyzer.

Uploading the GeoJSON file for use with VA

We need to create a companion metadata (config) file named nypp.config.json that contains some

information needed for display and indexing purposes before we can upload nypp.json.

Note: that the naming convention of the config file is very important. It must end with .config.json, and it must have the same name as the GeoJson file. For instance, if your custom GeoJson file is named mylayer.json, then the companion config file must be named mylayer.config.json. It has a fixed structure. The following important metadata about your custom GeoJson file can be included in the config file:

The unique name for the new GeoJson layer as known to the MapView Suggestion service. This is typically the file name (without the file extension).

The custom Universe definition if the GeoJson layer is non-geographic in nature such as a floor plan or plant schematics.

For polygon shapes, the default rendering styles.

Explicitly lists any column or property in the GeoJson file that represents country name or ISO code.

Specify if any other existing GeoJson map layer(s) should be its parent or child layer. This ensure the new GeoJson layer fits in a neat geographic hierarchy where applicable.

The most important component of the config file is the boost values for all the geospatial names and ID columns. Basically, these are the identifying columns of each record in the GeoJson file. The boost values provide hints to the MapView Suggestion service in terms of how unique and important each column should be treated. Typically the values are integers range from 0 to 9, 0 means the corresponding columns should be completely ignored, 9 means it is of critical importance. As an example, lets say a new GeoJson file contains records about company ABC's sales offices through out the world. Lets also say each record (office) contains columns including "Country", "StateProvince", "City", "ZipCode", "Office Name", "Office ID", "Telephone #" and "Number Of Empolyees". Obviously Office Name and Office ID are two important identifying columns, so they should have a higher boost value. Country, City, StateProvice and ZipCode are also useful information as they can contribute to the auto-suggestion process (such as if offices are represented loosely such as 'Boston, MA, USA' in a business report), so they should also be indexed but probably given a lower boost value. Finally, the Telephone # and Number of Empoloyees are two columns that although might be interesting to the user, do not contribute anything to the process of intelligently matching a record of a particular office to a location on the map, as such their boost values should be set to 0 (to be ignored for map suggestion purposes). So the set of boost values for all the columns in this case might look like an array like below, with Country, StateProvince, City and ZipCode all getting a boost value of 1, Office Name and Office Id both set to 3, and the rest set to 0. The entries in the boost array correspond to each item in the properties array {Country, StateProvince, City, ZipCode, OfficeName, OfficeID, Telephone#, NumberOfEmployees}. OfficeName and OfficeID are preferred identifiers and hence have higher boost values. [1,1,1,1,3,3,0,0]

The content for nypp.config.json is given below. This can be used as a template for other such files describing custom (GeoJSON) layers. {

"init": {

"center": [990000, 200000],

"zoomLevel": 2,

"layers": ["nypp"],

"defaultUniverse": "elocation",

"copyright": {

"anchorPosition": 5,

"textValue": "©2015 Oracle Corp",

"fontSize": 13,

"fontFamily": "Times New Roman"

},

"navigation": {

"style": 3

}

},

"vectorLayers": [

{

"dn": "nypp",

"uri": "nypp",

"keyColumns":["_ID"],

"infoColumns":["Precinct"],

"defaultCountry":"United States",

"boostValues":[3.0],

"cn": ["New York City Precincts"],

"gtype":"polygon",

"styles": {

"normal": "mutedColor1",

"mouseOver" : "mutedColor2",

"selected" : "selectedColor1"

}

}

],

"styles": [

{

"dn": "marker1",

"type": "marker",

"def": {

"width": 13,

"height": 13,

"lengthUnit": "px",

"vectorDef": [{

"shape": {"type": "circle", "cx": 0, "cy": 0},

"style": {"fill": "#257db3", "fillOpacity":0.75,

"stroke": "#000000",

"strokeThickness":2,

"strokeOpacity":0.1}

}

]

}

},

{

"dn": "mutedColor1",

"type": "color",

"def": {

"fill": "#cccccc",

"strokeThickness": 1,

"stroke": "#444444",

"fillOpacity": 0.9

}

},

{

"dn":"mutedColor2",

"type":"color",

"def":{

"fill":"#ff0000",

"strokeThickness":1,

"stroke":"#0000ff",

"fillOpacity":0.4

}

},

{

"dn": "hoverColor1",

"type": "color",

"def": {

"strokeThickness": 2.5,

"stroke": "#ffff00"

}

},

{

"dn": "selectedColor1",

"type": "color",

"def": {

"fill": "#cccc00",

"fillOpacity": 0.9

}

}

],

"universes": [

{

"dn":"usa_albers",

"srid" : 505050,

"bounds" : [-3200000, -500000, 2800000, 3800000, 505050],

"numberOfZoomLevels": 6

}

]

}

When creating your own config.json file, you can model after the one above but make sure to make these important changes:

Change all the layer name occurrences as needed. The above sample is for a layer with the geoJson file nypp.json', whose layer name is specified as 'nypp'. Make sure to change the element in the array 'layers', as well as the value of fields 'dn' (unique name of the layer), 'cn' (descriptive name of the layer), 'uri'(should be same as 'dn') to the new layer's name. The layer name should never include the file extension.

If the geoJson layer is strictly for USA, you can leave the universes definition and the defaultUniverse field as is. Otherwise you may need to make changes to the universes field.

Change the 'center' array to a different map center suitable for the new geoJson layer.

Change the defaultCountry field if necessary; or remove it if your geoJson layer is world-wide or cover multiple countries.

Last but not least, change/set the boostValues array accordingly. Look in your geoJson file, for each feature it should have a properties array. The boostValues array should contain one value for each attribute from that properties array, in the same order.

Now that the config.json file is complete we can upload the GeoJSON and corresponding config file to MapViewer (the map suggestion service) for use in VA. The upload page is at host:port/mapviewer/console/upload.jsp. In SampleAppV511 this is demo.us.oracle.com:7780/mapviewer/console/upload.jsp You will first need to log in as an Administrator (i.e. enter the admin username and password when prompted).

Check the box for “use default tenant”. Enter the layer name (“nypp”). Click on browse and navigate to the GeoJSON file. Do the same for the config.json file. Check the box “Rebuild index immediately”. Click the Upload button. Once the files are uploaded verify that the suggestion service has indexed them by accessing the URL: demo.us.oracle.com:7780/mapviewer/wizard?action=list

Note: The file (and layer) were named ny_precincts_3857 in the SampleAppV511 instance where this was tested. The last step is to upload the spreadsheet and use it in VA.

Using a custom layer in Visual Analyzer

Create a new VA project and upload the modified spreadsheet (nyc_offenses_by_precinct.xlsx) as the data source. Choose a map view and drag and drop Precinct as the Geography and a Year (e.g. C2014) as the measure. The map view should look like the screenshot below.