Post on 01-Aug-2020
Automatic reconstruction of Building Information Models
Alexandre Boulch
December 19th, 2014
Defense committee:
Dr. Pierre ALLIEZDr. Bruno LEVYDr. Reinhard KLEINDr. Jan BOEHMDr. Jean-Philippe PONSDr. Renaud MARLETDr. Martin DE LA GORCE
2
Building Information Models
The BIM is a digital representation of a building
Building Information Modeling (BIM): Benefits, Risks and ChallengesSalman Azhar et al. (2008)
Model interoperability in buildinginformation modeling Steel, James et al. (2010)
3
Building Information Model
BIM
4
Building Information Model
ParticipantsCoordination
Ifc foundation
BIM
5
Building Information Model
Louis Vuitton Fundation (CSTB)
SimulationParticipantsCoordination
Ifc foundation
BIM
6
Building Information Model
Coriolis Building (ENPC)
Visualization
Louis Vuitton Fundation (CSTB)
SimulationParticipantsCoordination
Ifc foundation
BIM
7
Building Information Model
Coriolis Building (ENPC)
VisualizationLifecycleManagement
Louis Vuitton Fundation (CSTB)
SimulationParticipantsCoordination
Ifc foundation
BIM
8
Renovation of old buildings
● Huge market(e.g., improvement of thermal performance)
● No digital model● Wrong or inexisting
blueprintshttp://www.agrconstruction.fr/
9
Data
● Photogrammetry
10
Data
● Photogrammetry● Depth sensors
11
Data
● Photogrammetry● Depth sensors● Lasers
12
Reconstruct and semantize surfaces
Building materials:● Wall: aerated concrete● Window: double glazing● Window frame: aluminium● Door: wood● ...
Heat capacity:● ...
+Geometry
Semantics
13
Pipeline
Point cloud
Digital model
Structure andinformationenrichment
14
Pipeline
15
Pipeline
Normalestimation
Point cloudPoint cloud +
Normals
Local information
16
Pipeline
Normalestimation
Primitivedetection
Point cloud +Normals
Primitives
Cluster-level information
17
Pipeline
Surfaceextraction
Normalestimation
Primitivedetection
Primitives Surface
Global geometric information
18
Pipeline
Surfaceextraction
Normalestimation
Primitivedetection
Surfacesemantization
Surface Digital model
Global semantic information
19
Pipeline
Surfaceextraction
Normalestimation
Primitivedetection
Surfacesemantization
20
Pipeline
Surfaceextraction
Primitivedetection
Normalestimation
Point cloudPoint cloud +
Normals
Local information
Surfacesemantization
21
Normalestimation Introduction
● Local estimation of the surface orientation
Robustness to:
noise
outliers
sharp edges
anisotropy
Speed
22
Normalestimation Introduction
● Local estimation of the surface orientation
● Robustness to
– noise
outliers
sharp edges
anisotropy
Speed
23
Normalestimation Introduction
● Local estimation of the surface orientation
● Robustness to
– noise
– outliers
sharp edges
anisotropy
Speed
24
Normalestimation Introduction
● Local estimation of the surface orientation
● Robustness to
– noise
– outliers
– sharp edges
anisotropy
Speed
25
Normalestimation Introduction
● Local estimation of the surface orientation
● Robustness to
– noise
– outliers
– sharp edges
– anisotropy
Speed
26
Normalestimation Introduction
● Local estimation of the surface orientation
● Robustness to
– noise
– outliers
– sharp edges
– anisotropy● Speed
27
Normalestimation Previous work
● Regression– Planes: Hoppe et al. Surface reconstruction from unorganized
points. SIGGRAPH, 1992.
– Jets: Cazals & Pouget. Estimating Differential Quantities using Polynomial fitting of Osculating Jets. Symposium on Geometry Processing, 2003.
● Voronoï diagrams– Dey and Goswami. Provable surface reconstruction from noisy
samples. Comput. Geometry, 2006.
● RANSAC– Li et al. Robust normal estimation for point clouds with sharp
features. Computers & Graphics, 2010.
28
Normalestimation Previous work
Regression Planes
Regression Jets
Voronoï diagrams RANSAC
Noise X X X
Outliers XSharp
features X X
Anisotropy X
Speed X X X
29
Normalestimation Randomized Hough transform
Generate hypotheses:pick triplets of points
30
Normalestimation Randomized Hough transform
Generate hypotheses:pick triplets of points
Spacechange
31
Normalestimation Randomized Hough transform
Generate hypotheses:pick triplets of points
Spacechange
Vote in a sphericalaccumulator
32
Normalestimation Randomized Hough transform
Generate hypotheses:pick triplets of points
Spacechange
Vote in a sphericalaccumulator
Select the mostprobable bin
33
Normalestimation Randomized Hough transform
Generate hypotheses:pick triplets of points
Spacechange
Vote in a sphericalaccumulator
Select the mostprobable bin
Generatenormal
34
Normalestimation Randomized Hough transform
Generate hypotheses:pick triplets of points
Spacechange
Vote in a sphericalaccumulator
Select the mostprobable bin
Generatenormal
How manyhypotheses must
be generated ?
35
Normalestimation Robust lower bound
● Filled accumulator of M bins:empirical approximation of a probability distribution
● Estimate the quality of the approximation.
T*, minimum number of triplets to pick such that :
δ
δ: deviation toleranceα: confidence level
36
Normalestimation Robust lower bound
● Filled accumulator of M bins:empirical approximation of a probability distribution
● Estimate the quality of the approximation.
T*, minimum number of triplets to pick such that :
● From Hoeffding inequality:
37
Normalestimation Results
Cazal & Pouget (2003)
Li & al (2010)
Ours
10°
Color code
38
Normalestimation Speed and accuracy
● Same complexity
● As accurate and faster than[Li et al. 2010]
Ours
Time (s), logarithmic scaleNote: [Dey & Goswany 2006] does not appear on the graphic, because the method is not robust to noise
Cylinder, 0.2% noise, 200k points
39
Normalestimation Visuals results: laser scans
40
Normalestimation Visual results: photogrammetric data
41
Normalestimation Conclusion
● Robust normal estimator based on Hough Transform– accurate
– robust to noise, outliers, sharp edges and anisotropy
– fast
● Limitations– smooth normals on wide angles
42
Pipeline
Surfaceextraction
Normalestimation
Primitivedetection
Surfacesemantization
43
Cluster level information
Pipeline
Surfaceextraction
Normalestimation
Primitivedetection
Point cloud +Normals
Primitives
Surfacesemantization
44
Schnabel & al (2007)
Primitivedetection
Extraction
RANSAC
Region growing
Fusion
45
Primitivefusion Practical example
Over-segmentation
46
Primitivefusion Practical example
Over-segmentation
47
Primitivefusion Practical example
Over-segmentation
48
Primitivefusion Existing methods
● Tolerance on parameters– 3 for a plane (2 for orientation, 1 for distance)
– 4 for a sphere (3 for position, 1 for radius)
– …
● A contrario criterion only for planes in depth maps [Bughin, E. 2010.]
49
Primitivefusion Existing methods
● Tolerance on parameters :– 3 for a plane (2 for orientation, 1 for distance)
– 4 for a sphere (3 for position, 1 for radius)
– …
● A contrario criterion only for planes in depth maps [Bughin, E. 2010.]
Our method:– One single indicator: distribution of distances to
primitive
50
Primitivefusion Criterion for fusion● Two surfaces and , associated with point sets
and ● Distance function from point to primitives● Define two sets:
● If then ● Statistical tests
– Mann-Whitney
– Kolmogorov-Smirnov
51
Primitivefusion Results
52
Primitivefusion Results
Nothing changed
53
Primitivefusion Results
Nothing changed
● Very small difference in the distributions rejection
No user control on acceptance
54
Primitivefusion Results
● Very small difference in the distributions rejection
● No user control on acceptance
Nothing changed
55
Primitivefusion Primitive fusion
0.0 0.2 0.4 0.6 0.8 1.0
Distance
0.0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
Den
sity
σ = 0.01
Distribution of X
Distribution of Y
0.0 0.2 0.4 0.6 0.8 1.00.00
0.02
0.04
0.06
0.08
0.10
0.12
0.14
0.16
0.18σ = 0.05
0.0 0.2 0.4 0.6 0.8 1.00.00
0.01
0.02
0.03
0.04
0.05
0.06
0.07
0.08
0.09σ = 0.1
0.0 0.2 0.4 0.6 0.8 1.00.000
0.005
0.010
0.015
0.020
0.025
0.030σ = 0.3
Example: two parallel lines at distance 0.1
Smooth distributions to ensure a controlled fusion
56
Primitivefusion Results
Increasing the level of smoothing
57
Primitivefusion Results
Increasing the level of smoothing
Smoothing level: 0.01
58
Primitivefusion Results
Increasing the level of smoothing
Smoothing level: 0.02
59
Primitivefusion Results
Increasing the level of smoothing
Smoothing level: 0.03
60
Primitivefusion Conclusion
● Criterion for primitives fusion– robustness to noise
– support for any primitive type (plane, cylinder...)
– intuitive user control on merging
● Limits– empirical relation between noise and fusion
distance
61
Pipeline
Surfaceextraction
Normalestimation
Primitivedetection
Surfacesemantization
62
Pipeline
Normalestimation
Primitivedetection
Surfaceextraction
Primitives Surface
Global geometric information
Surfacesemantization
63
Surfaceextraction Objective
Automatic surface reconstruction from laser scan– watertight without self intersection
– piecewise planar
– extended plausibly in hidden region
64
Surfaceextraction Objective
Automatic surface reconstruction from laser scan– watertight without self intersection
– piecewise planar
– extended plausibly in hidden regionwith support for data anisotropy
65
Surfaceextraction Previous work
● Smooth priors– e.g. Poisson reconstruction
● Voxelisation– biased, expensive
● Delaunay tetrahedralization– visible regions only
● Plane adjacency– visible near adjacency
● Manhattan world assumption– too restrictive: 3 directions only
66
Surfaceextraction Chauve et al.
● Plane arrangement– Visible planes
– Hidden plane hypotheses (called ghosts)
● Binary labelization of resulting 3D cell complex (empty or full)– Surface minimization, graph-cut optimization
● Advantages– watertight and non self-intersecting surfaces
– preservation of sharp edges
– extension of primitives far in hidden regions
– more plausible surfaces thanks to hidden plane hypotheses
A.-L. Chauve, P. Labatut, J.P. PonsRobust piecewise-planar 3D reconstruction and completion from large-scale unstructured point data. CVPR 2010
67
Surfaceextraction Chauve et al. (2)● Limitations
– wrong primitive borders with anisotropic point clouds
● Our method– determination of
borders in laser depth image
α-shape
68
Surfaceextraction Chauve et al. (3)● Limitations
– missing plane hypotheses
● Our method– generation of parallel
ghosts (for thin objects without detected thickness)
69
Surfaceextraction Chauve et al. (4)● Limitations
– Holes and truncated corners due to area minimization
● Our method– Edge length and
corner number minimization
70
Surfaceextraction Overview of the method
● Laser point cloud
Primitive extraction
Ghost generation
Volume partition using a plane arrangement
Binary partition using linear programming
Surface extraction
71
Surfaceextraction Overview of the method
● Laser point cloud● Primitive extraction
Ghost generation
Volume partition using a plane arrangement
Binary partition using linear programming
Surface extraction
72
Surfaceextraction Overview of the method
● Laser point cloud● Primitive extraction● Ghost generation
Volume partition using a plane arrangement
Binary partition using linear programming
Surface extraction
73
Surfaceextraction Overview of the method
● Laser point cloud● Primitive extraction● Ghost generation● Volume partition using
a plane arrangement
Binary partition using linear programming
Surface extraction
74
Surfaceextraction Overview of the method
● Laser point cloud● Primitive extraction● Ghost generation● Volume partition using
a plane arrangement● Binary partition using
linear programming
Surface extraction
75
Surfaceextraction Overview of the method
● Laser point cloud● Primitive extraction● Ghost generation● Volume partition using
a plane arrangement● Binary partition using
linear programming● Surface extraction
76
Surfaceextraction Overview of the method
● Laser point cloud● Primitive extraction● Ghost generation● Volume partition using
a plane arrangement● Binary partition using
linear programming● Surface extraction
77
Surfaceextraction Partition
● Partition the volume with a plane arrangement
● Labelization of each cell as empty or occupied
78
Surfaceextraction Energy formulation
● Energy
Data term
Penalizes a surface disagreeing with
observations
Regularization term
Penalizes a complex surface in the hidden areas
79
Surfaceextraction Data term
80
Surfaceextraction Data term
Penalize if xp+
= xp-
81
Surfaceextraction Data term
Penalize if xf+
≠ xf-
82
Surfaceextraction Edge and Corner Regularization
83
Surfaceextraction Edge and Corner Regularization
Shorter edgesSmaller corner number Smaller area
84
Surfaceextraction Edge and Corner Regularization
Shorter edgesSmaller corner number Smaller area
85
Surfaceextraction Edge and Corner Regularization
Shorter edgesSmaller corner number Smaller area
Smaller areaShorter edges
Smaller corner number
86
Surfaceextraction Regularization term
Area penalization
Edge length penalization
Corner number penalization
87
Surfaceextraction Regularization term
Area penalization
Edge length penalization
Corner number penalization
88
Surfaceextraction Edge term
EdgeNo edge
1-1-1+1=0 1-0-1+1=1 1-0-0+1=21-0-1+0=0
89
Surfaceextraction Corner term
90
Surfaceextraction Energy minimization
● Objective
● Very challenging for Markov Random Field– Edges: 4th order potential
– Corners: 8th order potential
– Tree-reweighted Belief Propagation: extremely slow
– Lazy Flipper: local minimum, extremely suboptimal
91
Surfaceextraction Linear relaxation
● Reformulation and relaxation
● This is a standard Linear Program
92
Surfaceextraction Results: meeting room
Area term onlyChauve et al.
93
Surfaceextraction Results: meeting room
Corner term onlyEdge term only
Area term onlyChauve et al.
94
Surfaceextraction Results: meeting room
Edge term only Corner term only
95
Surfaceextraction Results: meeting room
Corner only Edge + corner
96
Surfaceextraction Results: facade
97
Surfaceextraction Conclusion
● A method for piecewise planar surface reconstruction
● Compared to Chauve et al.– handling of anisotropy
– better surface hypotheses in hidden areas
– better regularization on edge length and corner number
● Limits and perspectives– scaling to entire buildings
– need for regularity discovery
98
Pipeline
Surfaceextraction
Normalestimation
Primitivedetection
Surfacesemantization
99
Pipeline
Surfaceextraction
Normalestimation
Primitivedetection
Surfacesemantization
Surface Digital model
Global semantic information
100
Semanticenrichment Building semantization
● Input: surface + semantic priors
● Output:semantized surface Surface Digital model
Method:– Bottom-up
– Based on grammars
101
Semanticenrichment Grammars
Sentence
Subject Verb Complement
NounAdjective
The super computer rules Discovery One
Building
Facade Roof
Window WallArticle
Facade
Expression of hierarchical decompositions
102
Semanticenrichment Constrained attribute grammars
Sentence
Subject Verb Complement
NounAdjective
The super computer rules Discovery One
Building
Facade Roof
Window WallArticle
Facade
Agreement (singular)
adj
Expression of complex relationsbetween elements
103
Semanticenrichment Stairway grammar
Stairway
Polygon
Polygon
104
Semanticenrichment Basic rule
Tread t → Polygon p
Riser r → Polygon p
Step s → Tread t, Riser r
riser
tread
105
Semanticenrichment Basic rule
Tread t → Polygon p
Riser r → Polygon p
Step s → Tread t, Riser r
Riser ?
Tread ?
riser
tread
106
Semanticenrichment Conditional rule
Tread t → Polygon p ( horizontal(p), p.length < 2 )
Riser r → Polygon p ( vertical(p), 0.05 < p.width < 0.25 )
Step s → Tread t, Riser r ( edgeAdj(t,r), above(t,r) )
Step
107
Semanticenrichment Collections
Tread t → Polygon p ( horizontal(p), p.length < 2 )
Riser r → Polygon p ( vertical(p), 0.05 < p.width < 0.25 )
Step s → Tread t, Riser r ( edgeAdj(t,r), above(t,r) )
Stairway w → sequence(Step s, edgeAdj) ms
Stairway
108
Semanticenrichment Combinatorial Explosion
● Enumerating collections may lead to combinatorial explosion
109
Semanticenrichment Combinatorial Explosion
● Enumerating collections may lead to combinatorial explosion
110
Semanticenrichment Combinatorial Explosion
● Enumerating collections may lead to combinatorial explosion
111
Semanticenrichment Combinatorial Explosion
● Enumerating collections may lead to combinatorial explosion
Total: 4 + 3 + 2 + 1 = 10 possible sequences
112
Semanticenrichment Maximal collections
Stairway
In practice, useful = largest collection:
Maximal operators (maxseq, maxcycle,...)
– fast computation
– low number of instances
Tread t → Polygon p ( horizontal(p), p.length < 2 )
Riser r → Polygon p ( vertical(p), 0.05 < p.width < 0.25 )
Step s → Tread t, Riser r ( edgeAdj(t,r), above(t,r) )
Stairway w → maxseq(Step s, edgeAdj) ms
113
Semanticenrichment Results
● Two grammars– Facade
– Stairway
● Experiments– synthetic data: CAD models, simulated laser scans
– real data: laser scans, photogrammetric data
114
Semanticenrichment CAD Models
Triangle soup
115
Semanticenrichment CAD Models
Triangle soup Polygons
116
Semanticenrichment CAD Models
Triangle soup PolygonsSemantized
model
117
Semanticenrichment CAD Models
Triangle soup PolygonsSemantized
model
Semantizedmodel
+ texture
118
Semanticenrichment Real laser scan: stairs
119
Semanticenrichment Conclusion
Grammar-based building semantization method– pure bottom-up parsing
– combinatorial explosion containment
– grammars easy to design
– grammars maintainable by non computer scientist
Limits and perspectives– need of flexibility (missing elements, merged
geometries)
– learning of rule parameters
120
Conclusion
Surfaceextraction
Normalestimation
Primitivedetection
Surfacesemantization
● Pipeline for surface reconstruction and semantization frompoint cloud
● A step towards automatic BIM reconstruction
121
General perspectives
Surfaceextraction
Normalestimation
Primitivedetection
Surfacesemantization
● Arbitrary order– surface useful for
semantization
semantics useful for surface reconstruction
Simultaneous or loop over semantics and surface extraction
Dealing with volumes
122
General perspectives
Normalestimation
Primitivedetection
● Arbitrary order– surface useful for
semantization
– semantics useful for surface reconstruction
Simultaneous or loop over semantics and surface extraction
Dealing with volumes Surfaceextraction
Pointsemantization
123
General perspectives
Normalestimation
Primitivedetection
● Arbitrary order– surface useful for
semantization
– semantics useful for surface reconstruction
● Simultaneous or iterated semantics and surface extraction
Dealing with volumes
Surfaceextraction
andsemantization
124
General perspectives
● Arbitrary order– surface useful for
semantization
– semantics useful for surface reconstruction
● Simultaneous or iterated semantics and surface extraction
Dealing with volumes
125
General perspectives
● Arbitrary order– surface useful for
semantization
– semantics useful for surface reconstruction
● Simultaneous or iterated semantics and surface extraction
● Dealing with volumeshttp://www.bimvision.eu/
126
Thank you for your attention
Fast and Robust Normal estimation for Point Clouds with Sharp Featureswith Renaud MarletSGP 2012Computer Graphics Forum
Semantizing Complex 3D Scenes using Constrained Attribute Grammarswith Simon Houiller, Renaud Marlet and Olivier TournaireSGP 2013Computer Graphics Forum
Statistical criteria for primitive mergingwith Renaud MarletICPR 2014
Piecewise-Planar 3D Reconstruction with Edge and Corner Regularizationwith Martin de La Gorce and Renaud MarletSGP 2014Computer Graphics Forum