Multimodal Location Based Serviceslbs2016.lbsconference.org/wp-content/uploads/2016/09/p11_1.pdf ·...
Transcript of Multimodal Location Based Serviceslbs2016.lbsconference.org/wp-content/uploads/2016/09/p11_1.pdf ·...
Multimodal Location Based ServicesSemantic 3D City Data as Virtual and
Augmented Reality
Jochen Wendel Alexander Simons
Andreas Koch
José Miguel Santana Agustín Trujillo
José Pablo Suárez
Collaboration
2
Outline• Project Goals
• Urban and energy-related data models
• Tools: The Glob3 Mobile framework
• Representing a CityGML model in 3D
• Multimodal approach
• Virtual Globe View
• Virtual Reality Mono
• Virtual Reality Stereo
• Augmented Reality
• Use case: Visualising solar irradiation
• Conclusions
3
Project Goals• Prototype an interactive and mobile system for exploring:
• Semantic building / urban representational models (e.g. CityGML)
• Energy related data sets.
• Explore the possibilities of multi-LoD maps, Virtual Reality and Augmented Reality for the visualisation of 3D urban models and temporal data-series.
• Create a iOS app, that serves as demonstration of the service.
4
Urban and energy-related data models• CityGML: Information model for urban objects representation.
• Extension of GML (from OGC)
• Classes, hierarchies, aggregations, relations, and properties.
• Used on 3D virtual cities, analysis, simulation…
• Extendable via Application Domain Extensions (ADE)
• Buildings represented in different levels of detail.
5
• Karlsruhe Model: CityGML LoD 2
• CityStats, Heat D./L.:Energy-related data per building
• SolarB: Temporal data-series representing energy simulations
6
Urban and energy-related data models
Urban and energy-related data models• Karlsruhe model
• +30.000 buildings [LoD2] (from a total set of 80.000).
• +230.000 surfaces [Grounds, Walls and Rooftops]
• Building parameters:
• Heat demand, Volume, GHG Emissions, solar gains, 50 demographic variables…
• High resolution textured models of emblematic buildings and monuments.
7
Glob3 Mobile• Open-source framework for Map & Virtual Globe Applications.
• Native implementations for iOS, Android and Web.
• Developed by CTIM (ULPGC) and IGO Software.
• Available: https://github.com/glob3mobile/g3m
• Automatic Multiplatform Porting
8
G3mFeatures
Manykindsofscenario
9
G3mFeatures2D/2.5D/3DInteractiveSymbology
10
CityGMLto3D
• OpenGL / WebGL only understand triangles!
• The data received via CityGML needs to be translated into a triangulated mesh model to be displayed.
• Model storage can be local or on a remote server.
• Karlsruhe model ~= 400 MB.
• Fetching, parsing and tessellation are executed on parallel threads.
11
12
CityGML Building
Surfaces
Energy DataBounding Vol. and
Render Info
IDs
• SAX Parsing of the CityGML documents:
• CityGMLBuilding class model:
CityGMLto3D:Parsing
STATICDATA
XML SAX Parsing
JSON Parsing
CITYGMLMODEL
CityGML Building
Surfaces
Energy DataBounding Vol. and
Render Info
IDs
• Creating a single triangular mesh of buildings.
• For a high number of models, batched drawing is essential [x49]
• Gouraud shading was used as illumination model.
13
VerticesTessellation
Semantical colouring
Visibility TestIndices
Normal Extraction Vertex Normals
Vertex ColorColour Palette Legend
CityGMLto3D:Tessellation
City Model Mesh
• Visibility test, eliminating shared walls.
• Proximity based limiting the O(n!) complexity.
• EPSG:4326 to Cartesian World Coordinates
• Addition of DEM displacement
• Ear Clipping Algorithm for wall tessellation
• Computationally expensive algorithm [O(n2)]
• 91% of tessellated surfaces are 4 edged.
14http://www.sunshine2k.de/coding/java/Polygon/Kong/
Kong.html
CityGMLto3D:Tessellation
• Classic GIS 3D map approach.
• Multitouch navigation.
• Buildings are selectable showing their associated data.
• OpenStreet Map as base layer. [Mapnik]
15
MultimodalApp:VirtualGlobe
MultimodalApp:VirtualGlobe
16
VIDEO INTERACTING WITH THE MAP
SELECTING BUILDINGAND CHANGING CITY
COLOURING
MultimodalApp:VirtualReality
• Sensors output of iOS, Android and Javascript.
• Control the UI rotations.
• Device location given by GPS.
17
GeolocalizedReferenceSystem
Multiplatformdevice-attituderepresentations
MultimodalApp:VirtualReality
VIDEO OF VR MONO AT WORK
18
MultimodalApp:VRStereo• Stereorenderingofthescene
• Low-levelOpenGLcodingfromscratch.• Stereoparallelprojection.• Usinga1:1scalefortheinteraxial.• Stereofrustumcullingtoimproveperformance.
19
MultimodalApp:VRStereo
• UseofCardboard-likeVRheadsets.
20
MultimodalApp:AugmentedReality
• DeviceAttitudeTracking+RealImageMerging• Phone’scamerahastobemathematicallyrepresented.• IntrinsicparametersobtainedwithOpenCV.
21
IntrinsicC.Matrix ExtrinsicCameraParameters
MultimodalApp:AugmentedReality
22
MULTIFRAMEBUFFERRENDERING
Usecase:SolarB–Solarirradiancemodel
• Numericalsimulationoftheamountofsunlightcapturedbythebuildingsurfaces.• BasedonŠúri,M.andHofierka,J.(2004)
• Finalresult:TemporaldataseriesofSolarradiationpertimeintervalforsurfacesandbuildings[kWh]• Surfacesrepresentedaspointsetswitharesolutionof1p/m2.
23
a)Buildingpart classification b)Pointgrid generation c)Shadowing analysis d)Aggregationof results
Usecase:SolarB–Solarirradiancemodel
• Example:TheEIFERofficesinKarlsruhe(7stories)• 14surfaces[3071points]• 1yearofsolarirradiationdata[8760timestamps]
24
VIDEO SELECTING BUILDING, ACTIVATING POINT CLOUD AND ZOOMING AROUND ON MAP MODE
Conclusions
• Theprototypedemonstratesfeasibletoapplymobilecomputergraphicstechnologiestothestudyofmassiveurbandatasets.
• Theappisabletodirectlyretrievesemanticbuildingdatafromenergysimulations/modelsinastandardisedway(OGC,CityGMLstructure).
• VRandARareusefultechniquesforfieldexpertsonenergyresearchandinfrastructuremaintenance.
• Formediumsizedcities,multi-level-of-detailisnotnecessary.• GPS&CompassaccuracyforVRandspeciallyforARisstillunreliableonmanyurbansituations.• Alternativepositioningtechniquescouldachievebetterresults.
25
Thanksforyourattention
26
Jochen Wendel Alexander Simons
Andreas Koch
José Miguel Santana Agustín Trujillo
José Pablo Suarez