Global Static Indexing for Real-time Exploration of Very...
Transcript of Global Static Indexing for Real-time Exploration of Very...
CASC VP 1
Global Static Indexing for Real-time Exploration of Very Large Regular Grids
Valerio Pascucci and Randall J. Frankpascucci,[email protected]
Lawrence Livermore National Laboratory
CASC VP 2
Outline• Motivation
• Previous work
• Data layout
• 2n tree indexing
• Performance for slicing large grids
• Conclusions and future work
k
I
i j
(bla bla[vis`01], bla bla bla [sc`00] ……)
CASC VP 3
We must achieve real-time interaction with large datasets on a wide variety of platforms.
The problem
• Extremely large datasets 0.5TB/timestep (8k,8k,8k,(time)).
• Interactive rendering for real-time data exploration.
• Target platforms: desktop,parallel server, cluster.
CASC VP 4
Previous Work• Out-of-core geometric algorithms
[Goodrich, Tsay, Vengroff, Vitter ‘93]
[Vitter ‘00][Matias, Segal, Vitter ‘00]
[Asano, Ranjan, Roos, Welzl ’95][Arge Miltersen ’99]
• Out-of-core visualization[Chiang, Silva ‘97][Sutton, Hansen ‘99]
[Livnat,Shen,Johnson ’96][El-Sana, Chiang’00]
[Bajaj, Pascucci, Thompson, Zhang ‘99]
• Space filling curves(image processing, multidimensional databse, geomtric datastructure …)
[Bandou, Kamata.’99][Balmelli, Kovacevic, Vetterli ’99]
[Parashar, Browne, Edwards, Klimkowski ’97]
[Niedermeier, Reinhardt, sanders ‘97][wise’00]
[Hans Sagan ’94] [Lawder king ’00][Griebel Zumbusch ‘99]
CASC VP 5
We apply three fundamental techniques to the visualization of large simulation data.
Our approach
• Multi-resolution geometric representation:– adaptive view-dependent refinement;– minimal geometric output for selected error tolerance.
• Cache oblivious external memory data layouts:– exploit spatial and resolution coherency;– no need for complicated paging techniques.
• Progressive processing: – continuously improved rendering;– scalability with the resources without budgeting.
CASC VP 6
We focus on the progressive computation of slices (any orientation) of large 3D rectilinear grids.
• Rectilinear grid• Sub-sampling octtree
• 1D order = hierarchical3D Z-order curve
• Coarse to fine slice refinement
CASC VP 7
General Data Layout
Grouping the data by level of resolution
Data coherent Progressive refinement of a hierarchical
geometric data-structure
Grouping the data by geometric proximity
CASC VP 8
General Data Layout
CASC VP 9
General Data Layout
CASC VP 10
General Data Layout
CASC VP 11
General Data Layout
CASC VP 12
General Data Layout
CASC VP 13
General Data Layout
CASC VP 14
General Data Layout
nD to 1D mapping:
I → I*
I → l find the level of resolution lCl (pre)compute the number of elements in the levels coarser than l
I → I’ index of the element within its level of resolution
I* = Cl + I’
C0 C1 C2 C3
CASC VP 15
We exploit the correlation of bin/quad/oct-trees with the Lebesgue space-filling curves.
The Lebesgue curve is also known as Z-order, Morton, …. Curve.Special case of the general definition introduced by Guiseppe Peano in 1890.
CASC VP 16
We turn the recursive definition of the Z-order curve into a hierarchical subsampling scheme.
(a) (b)
(c) (d)
coarse data new level data
CASC VP 17
We obtain a multi-resolution hierarchical representation which is not exactly a 2n-tree.
• Not exactly a quad-tree ……
Level 0
Level 1
CASC VP 18
We obtain a multi-resolution hierarchical representation which is not exactly a 2n-tree.
• Not exactly a quad-tree ……
Level 0
Level 1
Level 2
CASC VP 19
Level 3
We obtain a multi-resolution hierarchical representation which is not exactly a 2n-tree.
• Not exactly a quad-tree ……
Level 0
Level 1
Level 2
CASC VP 20
The 1D index I* can be computed in a simple and efficient way in any dimension.
I*
I
1514131211109876543210
1514131110976532112840Z-order 4
3210Z-order 2
0Z-order 0
210
Cl = 22(l-1) _I_22lI’ =I* = Cl + I’
_I_22(l+1)- - 1
I
i j
CASC VP 21
The 1D index I* can be computed in a simple and efficient way in any dimension.
I*
I
ki j
CASC VP 22
Overall the hierarchical Z-order yields a cache oblivious hierarchical data layout.
Distribution in the grid of each constant size block of data
0 1 2 3 4 5 6 7 8 9
CASC VP 23
Theoretical analysis shows a gain of orders of magnitude independently of the block size.
1
10
100
1000
10000
100000
1000000
10000000
100000000
000000000
1 2 4 8 16 32 64 128
256
512
1024
2048
4096
8192
32*32*32
3-bits shift
1-bit shift
1
10
100
1000
10000
100000
1000000
10000000
100000000
1000000000
1 4 16 64 256
1024
4096
32 3̂-BLOC
3-BITS
1-bit shift
32K blocks 64K blocks
Cache oblivious !!!
CASC VP 24
Real speedup matches theoretical expectations:more than 10x improvement, platform scalable.
• 2048x2048x1920 dataset (we have run up to 8192x8192x7680)
• 20MB memory cache
• Translation and rotation tests (average over 3 primary axis)250Mhz SGI Onyx (1024x1024)
0.001
0.010
0.100
1.000
10.000
100.000
1000.000
1 10 100
Resolution
Seco
nds
ARR-Rot
BIT-Rot
BLK-Rot
ARR-Trans
BIT-Trans
BLK-Trans
500Mhz PIII Laptop (512x512)
0.001
0.010
0.100
1.000
10.000
100.000
1 10 100
Resolution
Seco
nds
ARR-Rot
BIT-Rot
BLK-Rot
ARR-Trans
BIT-Trans
BLK-Trans
CASC VP 25
Conclusions and Future Work
• Results: an implicit scheme for coherent spatial, multi-resolution regular grid data access
• Simple address remapping
• Read/write access
• No additional data overhead
• Supports progressive access
• Near term applications
• Volume rendering
• Time critical iso-contouring
• Future work
• “View dependent” parameterization
• Unstructured/temporal hierarchies
• Improved interpolation
• Distributed implementation
CASC VP 26
UCRL-VG-143542
This work was performed under the auspices of the U.S. Department of Energy by University of California Lawrence Livermore National Laboratory under contract No. W-7405-ENG-48.