A columnar architecture for modern risk management systems · Voxel-based 3D city model •Storing...
Transcript of A columnar architecture for modern risk management systems · Voxel-based 3D city model •Storing...
![Page 1: A columnar architecture for modern risk management systems · Voxel-based 3D city model •Storing volumetric spaces such as air, water and underground is possible. •Every object](https://reader033.fdocuments.in/reader033/viewer/2022043018/5f3aee4a8584122b5e767a6c/html5/thumbnails/1.jpg)
A columnar architecture for modern risk management systems
Romulo GoncalvesSisi Zlatanova, Kostis Kyzirakos, Pirouz Nourian, Foteini Alvanaki,
and Willem van Hage
![Page 2: A columnar architecture for modern risk management systems · Voxel-based 3D city model •Storing volumetric spaces such as air, water and underground is possible. •Every object](https://reader033.fdocuments.in/reader033/viewer/2022043018/5f3aee4a8584122b5e767a6c/html5/thumbnails/2.jpg)
Risk Management
![Page 3: A columnar architecture for modern risk management systems · Voxel-based 3D city model •Storing volumetric spaces such as air, water and underground is possible. •Every object](https://reader033.fdocuments.in/reader033/viewer/2022043018/5f3aee4a8584122b5e767a6c/html5/thumbnails/3.jpg)
3D city models
![Page 4: A columnar architecture for modern risk management systems · Voxel-based 3D city model •Storing volumetric spaces such as air, water and underground is possible. •Every object](https://reader033.fdocuments.in/reader033/viewer/2022043018/5f3aee4a8584122b5e767a6c/html5/thumbnails/4.jpg)
Data sources
![Page 5: A columnar architecture for modern risk management systems · Voxel-based 3D city model •Storing volumetric spaces such as air, water and underground is possible. •Every object](https://reader033.fdocuments.in/reader033/viewer/2022043018/5f3aee4a8584122b5e767a6c/html5/thumbnails/5.jpg)
Architecture of a service
• Spatial DataBaseManagement System
– Columnar architecture
• Spatial analysis tailored for different use case scenarios
Point Cloud DataVector Data
TOP25RASTER
Semantic data
Storage
![Page 6: A columnar architecture for modern risk management systems · Voxel-based 3D city model •Storing volumetric spaces such as air, water and underground is possible. •Every object](https://reader033.fdocuments.in/reader033/viewer/2022043018/5f3aee4a8584122b5e767a6c/html5/thumbnails/6.jpg)
Looking to the future
• Each use case might require a different resolution– Level of definition on the
semantic data
– Remote sensing data
![Page 7: A columnar architecture for modern risk management systems · Voxel-based 3D city model •Storing volumetric spaces such as air, water and underground is possible. •Every object](https://reader033.fdocuments.in/reader033/viewer/2022043018/5f3aee4a8584122b5e767a6c/html5/thumbnails/7.jpg)
New conceptual scheme
• Voxel-based 3D city
model
– Voxels are the
volumetric
representation of pixels.
![Page 8: A columnar architecture for modern risk management systems · Voxel-based 3D city model •Storing volumetric spaces such as air, water and underground is possible. •Every object](https://reader033.fdocuments.in/reader033/viewer/2022043018/5f3aee4a8584122b5e767a6c/html5/thumbnails/8.jpg)
Voxel-based 3D city model
• Storing volumetric spaces such as air, water and underground is possible.
• Every object is defined by set of voxels– The voxel size depending on the level of detail (LOD)
– voxel’s characteristics e.g. type (wall, glass, roof, door, etc.), color, density
• Simplification to speed up computations– simplifies a range of geometric operations: volumes and areas
– real world objects by a single geometry type (3D cube) instead of collection of polygons/polyhedron
![Page 9: A columnar architecture for modern risk management systems · Voxel-based 3D city model •Storing volumetric spaces such as air, water and underground is possible. •Every object](https://reader033.fdocuments.in/reader033/viewer/2022043018/5f3aee4a8584122b5e767a6c/html5/thumbnails/9.jpg)
Voxelization
• MethodsX paper:– Voxelization Algorithms for Geospatial Applications
• https://github.com/NLeSC/geospatial-voxels/software/voxelGen
• Storage challenges [1]:– With different semantic level of detail and coverage of in- and out-
side empty spaces
– Grid with homogenous cells or heterogenous cells• An entire city will generate a massive 3D grid of voxels at different resolutions
with a large number of semantic attributes attached
[1] Towards 3d raster GIS: on developing a raster engine for spatial DBMS
![Page 10: A columnar architecture for modern risk management systems · Voxel-based 3D city model •Storing volumetric spaces such as air, water and underground is possible. •Every object](https://reader033.fdocuments.in/reader033/viewer/2022043018/5f3aee4a8584122b5e767a6c/html5/thumbnails/10.jpg)
New storage scheme
• Nested column-oriented storage for 3D city models.
• To store nested data structures in flat columnar format [2]– the record-wise versus columnar
representation• In the columnar representation all the
values of a nested field are stored contiguously.
– the schema is mapped to a list of columns• A.B.C can be retrieved without reading
A.E, A.B.D,
• [2] Dremel: Interactive analysis of web-scale datasets.
![Page 11: A columnar architecture for modern risk management systems · Voxel-based 3D city model •Storing volumetric spaces such as air, water and underground is possible. •Every object](https://reader033.fdocuments.in/reader033/viewer/2022043018/5f3aee4a8584122b5e767a6c/html5/thumbnails/11.jpg)
Columnar format for nested structures
• The record structure is defined trough two integers called repetition level and definition level.
• Definition level – The definition level records at which level it
started being null.
• Repetition level. – It is used to define when a new list starts in
a column of values.
• Parquet– https://blog.twitter.com/2013/dremel-made-
simple-with-parquet)
![Page 12: A columnar architecture for modern risk management systems · Voxel-based 3D city model •Storing volumetric spaces such as air, water and underground is possible. •Every object](https://reader033.fdocuments.in/reader033/viewer/2022043018/5f3aee4a8584122b5e767a6c/html5/thumbnails/12.jpg)
Storage scheme
• The LOD is used for the definition level.
• All the voxels inherit the semantics from the parent– if a object has LOD2 semantics, it will
also has LOD1 semantics
• The repetition level is the number of sub-divisions a parent voxel has. – An object is semantically identified as
a building in LOD1
– While in LOD2 it might be composed by a set of sub-voxels to define walls, floor surface and etc.
![Page 13: A columnar architecture for modern risk management systems · Voxel-based 3D city model •Storing volumetric spaces such as air, water and underground is possible. •Every object](https://reader033.fdocuments.in/reader033/viewer/2022043018/5f3aee4a8584122b5e767a6c/html5/thumbnails/13.jpg)
Architecture of a service
• Exploit late materialization– Low memory footprint
• Repetition level– Scan aggregation-
operators
• Definition level– Projections
• Un-nest the data
• Tuple reconstruction
– Blocking operator
– Flat-data integration
Voxel Data
TOP25RASTER
Point Cloud Data Semantic dataVector Data
Storage
![Page 14: A columnar architecture for modern risk management systems · Voxel-based 3D city model •Storing volumetric spaces such as air, water and underground is possible. •Every object](https://reader033.fdocuments.in/reader033/viewer/2022043018/5f3aee4a8584122b5e767a6c/html5/thumbnails/14.jpg)
3D city models
![Page 15: A columnar architecture for modern risk management systems · Voxel-based 3D city model •Storing volumetric spaces such as air, water and underground is possible. •Every object](https://reader033.fdocuments.in/reader033/viewer/2022043018/5f3aee4a8584122b5e767a6c/html5/thumbnails/15.jpg)
“Minecrafted” city
![Page 16: A columnar architecture for modern risk management systems · Voxel-based 3D city model •Storing volumetric spaces such as air, water and underground is possible. •Every object](https://reader033.fdocuments.in/reader033/viewer/2022043018/5f3aee4a8584122b5e767a6c/html5/thumbnails/16.jpg)
Routes
• The available space to define escape trajectory routes
![Page 17: A columnar architecture for modern risk management systems · Voxel-based 3D city model •Storing volumetric spaces such as air, water and underground is possible. •Every object](https://reader033.fdocuments.in/reader033/viewer/2022043018/5f3aee4a8584122b5e767a6c/html5/thumbnails/17.jpg)
Indoor models
![Page 18: A columnar architecture for modern risk management systems · Voxel-based 3D city model •Storing volumetric spaces such as air, water and underground is possible. •Every object](https://reader033.fdocuments.in/reader033/viewer/2022043018/5f3aee4a8584122b5e767a6c/html5/thumbnails/18.jpg)
Summary & future
• Simplification to speed up computations
• The uniqueness of our solution– A voxel-based 3D city models,
– nested column-oriented format to explore the 3D city model at different levels of detail.
– topological and geometric functionality for 3D raster manipulation is part of the relational kernel and not an add-on
• Spatial analysis tailored to different use case scenarios
• Future– Geo-Spark
![Page 19: A columnar architecture for modern risk management systems · Voxel-based 3D city model •Storing volumetric spaces such as air, water and underground is possible. •Every object](https://reader033.fdocuments.in/reader033/viewer/2022043018/5f3aee4a8584122b5e767a6c/html5/thumbnails/19.jpg)
Questions & Ideas?
Image from: http://aboutinterviews.com/5-questions-to-not-ask-in-a-job-interview/
http://www.atlantainjurylawblog.com/files/2016/08/bright-idea.jpg