Discussions with Wolfgang Forstner, March 19, 2008 1 3D Building Reconstruction from LiDAR Data. Our...

Discussions with Wolfgang Forstner, March 19, 2008

1

3D Building Reconstruction from LiDAR Data.

Our approach A data driven approach that determines building parameters fr

om the LiDAR point cloud. No initial model is assumed, except that the roof consists of pl

anar regions only. A data-mining approach is employed to extract information. The LiDAR point cloud is segmented into different roof plane r

egions. 3D models are reconstructed from segmented planes by using

neighborhood analysis.

Motivation

3D building models provide for a realistic visualization of an Urban area. LiDAR technology directly generates a dense 3D point cloud from which it is possible to extract enough information to generate models of Buildings.

Problems: Manually extracting building models from LiDAR point clouds is extremely inefficient. The size and the nature of the datasets make LiDAR extremely suited to automated algorithmic processes.

Initial point cloud

Segmented point cloud

Reconstructed Building Model

Ajith Sampath


2

3D Building Reconstruction from LiDAR Data.Building Roof SegmentationTo segment LiDAR points into roof segments. generate buil

ding models from Laser scanned point clouds over urban areas.

Assumptions: Planarity of the roofs. The building roofs are piecewise planar.

Main ideas of the approach Curvature and Surface Normals characterize any surface. Under the constraint of planarit

y, surface normals are sufficient. Surface normals are defined only for differentiable and continuous surfaces. Use Eigenvector and Eigenvalue analysis to softly-identify regions of non-differentiability.

Assign a measure between 0 and 1 to each LiDAR return 0 indicates a discontinuous surface with no confidence in the normal vector estimates in that

region 1 indicates complete confidence in the normal vector estimates.

Use subtractive clustering on normal vector estimates from continuous regions to determine estimates of all the plane segments

Refine Clustering using surface normal of all points in a Gaussian Mixture like approach.

Initial point cloud Planar Points

Some Key Issues in Clustering Each feature vector (normal vector)

gets an a-priori confidence measure. Initial Clusters and their locations

approximated using density values around a neighborhood “r”.

Further refinement of cluster centers is based on the feature vector’s distance from cluster center.

Clusters generated for several values of “r” and the similarity measure plotted to determine actual clusters.

The “Elbow Joint” is taken as the actual number of clusters present in the data.

Direction Segmented Points

Further Segmentation Each Cluster may represent several pla

nes in same direction (i.e. same Normal vector)

In the equation Ax+By+Cz+D=0, D is calculated for each point {Xp,Yp,Zp} represented in a cluster.

Points are segmented based on values of “D”.

Planes may be further segmented based on connectivity.

The “Elbow Joint” curve


3

3D Building Reconstruction from LiDAR Data

Building Reconstruction The distance between two planar segments (P & Q) in a roof is defined as:

A neighborhood Matrix is then generated. The matrix shows all mutually intersecting planes.

Any two intersecting planes are selected, and all planes that intersect both of them are enumerated, and solved.

For instance Planes {1,4,10} and {1,4,13} are solved to get the breakline (A-B)as shown.

QqPpqpdQPd iiji ,)),(min(),(

• Building Roof Reconstruction• To Reconstruct the Building model

from the segmented Planes.

Issues and Future Work Segment and Reconstruct non planar

parametric roofs. Extract and map texture from Images


4

Local Convex Hull Approach to Boundary Extraction in LIDAR Point Clouds Objective

•Develop automatic boundary discovery algorithms that works on LIDAR point clouds directly•Avoid high computation-cost global Interpolation or triangulation•Based local topology, no prior assumption on point clouds

Convex Hull•A set in a vector space is called a convex set if the line segment joining any pair of points of lies entirely in•The convex hull of a set of points S in d dimensions is the minimal convex set containing S•Convex hull can quickly capture the rough shape or pattern of the points set

Common Algorithms•Graham's Scan, Gift wrapping, Divide and Conquer, QuickHull, etc.•QuickHull package: supports 2-d, 3-d, and higher dimensions http://www.qhull.org/

Construct a Convex Hull1. Find the lowest point (anchor point)2. Form a nonitersecting polygon by sorting the

points in order around the anchor point3. Remove the non-convex vertex from the polygon

Jun Wang

http://www.qhull.org/




5

Local Convex Hull Approach to Boundary Extraction in LIDAR Point Clouds Local Convex Hull and Non-Boundary Points

•Theorem: for a convex set S, pick up any subset of points contain point p, if p is inside the convex hull of the subset of points, then p is not on the boundary of the convex hull of points S•Detect non-boundary points with local convex hull formed by their nearest neighborhoods

Boundary Labeling AlgorithmInput: {points, n}For all the points

•Pick up n nearest neighbors of point p•Construct the convex hull•Labeling all the points insides the convex hull as “non-boundary” points.

Output: {non-boundary points |boundary points}

Non-Boundary Point

Boundary Point

Design Ideas•Binary classification of the points: boundary point/non-boundary point•Sculpture the unknown surface boundaries by removing the “non-boundary” points defined by local convex hull•Nearest Neighbors based computation, low computation memory requirements


6

Local Convex Hull Approach for Boundary Extraction in LIDAR Point Clouds

0.64

66.4

4

Height

Testing on LIDAR data•R tree supported fast nearest neighborhood search in point clouds•The advanced version to detect boundaries on vertical walls and inside surface patches, such as roof structures is under development.


7

Application of active contour based on level set to identify buildingboundary from LIDAR data

Application of LIDAR data for delineation of building outline and roof topologyAutomation in data acquisition and 3D reconstruction for 3D urban models has been an important research area, and many researchers have proposed and applied various kind of methods using high-resolution satellite images or aerial photographs. In these days, LIDAR data is widely used to reconstruct 3D building models. To delineate the building boundary and roof topology, we aimed to apply “active contour” based on level-set methods into dense point clouds data.

Our approach Application of traditional active contour based on level-set method into dense point clouds data and check the

possibilities for further research. Application of multi-phase level set method to delineate the inner topology of building roof. Propose a new stopping function criterion adequate for roof topology delineation.

KyoHyouk Kim

Rottensteiner


8


Methodology: By Chunming Li et al, 2005

Total energy function :

, where is an external energy and is an internal

Energy. And, external energy function can be described as follows.

, g is a “edge-stopping” function

Therefore, geometrically, the external function represents the length

And the inner area of zero level curve of

The final evolution equation of curve is

, ,( ) ( ) ( )gP

, , ( )g ( )P

, , ( ) ( ) ( )g g gA

( ) ( ) | | , ( ) ( )g gg dxdy A gH dxdy

Initial DSM

300 iteration 600 iteration

756 iteration

[ ( )] ( ) ( ) ( )| | | |

div div g gt

2

1

1 | * |g

G I


9


Problem and possible solution (1) Evolution of curve didn’t stop at outer boundary of building having

gradient roof This problem might be solved if we apply a different stopping

function not based on edge.

Problem and possible solution (2) Traditional level set method can’t delineate the inner boundary of

the gradient roof. We may apply multi-phase motion level set method from inner

seed pixel or initial boundary of each planar roof segment.

Problem and possible solution (3) Computational complexity We may apply “Narrow band” algorithm to achieve computational

efficiency

Gradient roof building

Multi-phase motion

level-set

Simultaneous evolution of curve


10

BUILDING EXTRACTION FROM HIGH RESOLUTION COLOR IMAGERY BASED ON EDGE FLOW DRIVEN ACRIVE CONTOUR AND JSEG

Building extraction and delineation is one of the most salient problems in

cartographic features extraction.

This study presents a semi-automatic framework for reliable and accurate

building extraction from high-resolution color imagery focusing on building

boundary delineation and building roof compositional polygon segmentation.

• The framework for building extraction consists of three steps.

1. Anisotropic diffusion and clustering (pre-processing) for de-noising and color quantization

2. Building boundary extraction using active contours driven by edge-flow.

3. Building roof compositional polygons segmentation by JSEG.

• Many automatic building extraction methods from DEM or multi-spectral imagery suffer from rough

delineation result due to the relatively low resolution of the involved DEM and multi-spectral imagery.

• Low resolution and deficiency of the method cause the fact that several buildings and possibly their

surroundings are extracted as one building.

Therefore, the results can only used as an initial approximation to the final building and must be

enhanced with improved reliability and further refined for better accuracy.

Problems

Strategy for Building Extraction

Yonghak Song


11

BUILDING EXTRACTION FROM HIGH RESOLUTION COLOR IMAGERYBASED ON EDGE FLOW

DRIVEN ACRIVE CONTOUR AND JSEG

J-image generation (Deng et al. (1999) ): measuring local homogeneity

• Apply the k-means clustering • J= the distances between clusters / the distances within each cluster

Edge-flow generation (Total edge energy and likelihood )

Boundary tracking by active contours• Edge flow vector field: external force, measure J: stopping function

Color space conversion : RGB space > CIE L* a*b* space•

CIE L* a*b* space : L* - luminance, a* b* - color. – Separation of achromatic information from chromatic information

– Uniform color space- allow Euclidean distance

– Similarity to human visual system.

Anisotropic diffusion• Selective smoothing : decrease noise and enhance color information•

Use Heat conduction equation (initial: Image, C-coefficient:: gradient)

METHODOLOGY

• Re-clustering with k-mean and regenerate J-image

• watershed segmentation • refinement by merging and

removing segments according to some criteria.

1. Color Space Conversion and Anisotropic Diffusion

2. Building Boundary Delineation using Active Contour

3. Building Roof Polygon Extraction

It (x, y, t) div g I(x, y, t) I(x, y, t)

PT (s,) PL*(s,) Pa*(s,) Pb*(s,) 3

ET (s,) EL*(s,)2 Ea*(s,)2 Eb*(s,)2


12

BUILDING EXTRACTION FROM HIGH RESOLUTION COLOR IMAGERYBASED ON EDGE FLOW

DRIVEN ACRIVE CONTOUR AND JSEG RESULT AND CONCLUSION

• In spite of noisy boundary, proposed framework shows correct localization of the building boundaries. • Also it yields reliable wireframes of building roof. • However, determined roof compositional polygon is incorrect in some cases (bottom left result ).

– Shows over-delineation of building roof due to over-segmentation (red circle) – Lost several wireframes because of under-segmentation. (blue circles)

RESULT

• This study present a framework to segment buildings from high resolution color imagery based on the JSEG method and edge flow driven active contours.

• It shows good result in some case but this result is not consistent. Because the performance of proposed method totally relies on intensity and color information in image, adjacent roof facades which have same reflectance values due to the same incident angles with sunlight can not be separated. This also leads the need of additional information such as DEM that can be incorporated with the proposed algorithm.

CONCLUSION & FUTURE WORKS


13

AUTOMATED ROCK SEGMENTATION FOR MARS EXPLORATION ROVER IMAGERY

Objectives•Make maps showing the locations of different kinds of rocks and soils around the landing sites.•Search for and study many different types of rocks and soils that might hold clues to past water activity.

Prerequisite of mission is to detect and to classify rocks and soils.

Rational of study• Autonomous interactive capability to perform the scientific mission • Image compression by prioritizing rocks for effective data transmission

MER Pancam-Main camera for rock detection• Pancam's imagery is designed to help scientists decide what rocks and soils to analyze in detail, and how to interpret the results.

• Also provides information on the surface features of the area around the rover, the distribution and shape of nearby rocks, and the presence of dunes and features carved by ancient waterways.”

Problem for rock image segmentation• Rock has different intensities due to various reflections

– Segmentation only with intensity leads

Over segmentation Merging procedure is needed

• Rock has more homogeneous in terms of texture than intensity

Image segmentation based on the texture has advantages

• Texture-based segmentation can’t offer sharp rock boundary

Add boundary refinement step using active contours

• 1024x2048 pixel CCD array detectors 1024 x 1024 image

• Effective focal length: 43mm

• FOV : 16 degree by 16 degree

• IFOV: 0.28 mrad/pixel Maintain focus 1.5m~infinity

• 8 filters per camera

Mars Exploration Rover Mission

The Mars Exploration Rover mission is an ongoing robotic mission to explore the Martian surface by rovers, “spirit” and “opportunity”

Two-stage solution for rock segmentation

1. Rock detection using texture based image segmentation

2. Rock boundary refinement by edge-flow driven active contour based on level set method

Pancam image

Yonghak Song


14

AUTOMATED ROCK SEGMENTATION FOR MARS EXPLORATION ROVER IMAGERY Texture based image segmentation

1. Multi-channel approach (A)2. Multi-resolution histogram (B)3. Inter-scale decision fusion (C)

1. The multi-channel approach•Transform an image to a new set of features that offer condensed, classification related information •Exploit Haar wavelet transform to use its four coefficient channels: the approximation, horizontal, vertical and diagonal detail coefficients

2. Multi-resolution histogram •From each channel, the texture features are extracted by measuring the change of histogram with respect to the image resolution relying on the wavelet decomposition level

•Such histogram change reflects the variation of spatial information, i.e., texture measured by the generalized Fisher information content.

3. Inter-scale decision fusion•Integrate the texture features at different scales through their statistical relationships for better texture classification

•Use two adopted k-means algorithms: hierarchical and interactive k-means

Rock Detection using Texture Analysis

Multi-resolution histogramEffect of inter-scale fusion Detected rocks using texture


15

AUTOMATED ROCK SEGMENTATION FOR MARS EXPLORATION ROVER IMAGERY

The texture-based image segmentation yields reliable rock segmentation results but they are too rough to determine accurate rock boundaries. Hence, the rock boundaries are refined, by edge-flow driven active contour based on the level set method, which adopt “edge flow” concept to compensate the problems of traditional active contours that are sensitive to noise and often cause the edge-leaking problem

Edge flow (flow direction + edge energy)• The flow direction determined by the first directional derivatives of the Gaussian kernel

• The edge energy represented as a flow vector by assigning probabilities to its flow direction.

• The edge flow forms the vector field as an external force to enforce the initial boundaries towards the pixels with high probability of rock boundary edges

Boundary refinement using active contours

Edge flow-driven active contour by level set method

• Compute the edge flow vector

• Generate the stopping function from the edge flow vector field by solving

• Propagate initial boundary using following equation

0||),(|| 0

yxgFt

g

Refined rock boundaries

Edge flow vector fieldStopping function

Example of topographic correction


16

A semi-automatic approach to large scale 3D building reconstruction

Our approach The operators ONLY collect topological distinct points from the building roof without rules. Each building will have

an unstructured point cloud set. A building could be a single or multiple point datasets based on the complexity of building structure. Each point set

will be constructed as a building unit. Computational algorithms start from forming the unit boundary, then generating the unit roof. The entire building is the combination of building units.

Complete buildings will be geometric models with topological information, no surface texture.

Current 3D building modeling issues

The demanding of a great amount urban 3D building objects is eager for various applications. The most practical, economic and accurate approaches to reconstruct large scale 3D building models with detail structures are semi-automatic approaches with image-based data. In the semi-automatic approach: operators detect building information; automated tools reconstruct building models. The conventional photogrammetric methods require operators to delineate the building footprints and roof structures.

Problems: manual building detection process is time consuming and labor intense.

§ To improve the efficiency, the detection processes have to be simplified.

§ To increase the successful rate, algorithms for reconstruction process need no constraint on building types.

Sherry Fu


17


Boundary reconstructionThe problem: to construct a compact reasonable representation of the

surface from a given unstructured point set.

Assumptions: building boundary properties. Either the interior angle or the exterior angle of each intersection

tends to be as close to a right angle as possible. If the interior angle or exterior angle is not a right angle, both of

them like to avoid acute angles.

Main idea of the approach Constructing an initial convex polygon. Examining every angle of the polygon. Modify the polygon which edges cause the conflict with the

assumptions. Point selection: pick breakthrough points for edges

modification. Edge selection: pick up the edge needs modification most

The selections are based on scoring their geometric properties:

Height, Length, and Angle. Repeating the process until the polygon can no longer be modified.

Initial point cloud

Building boundary

Problems No unique solution guarantee. The accuracy is influenced by the degree of building complexity. Sensitive to the point measurement quality.


18


Roof reconstructionThe problem: no rule to follow because of the complexity of roof styles.

Assumptions: roof nature based on observation. Roof ridges tend to align with boundary edges. Roof ridges tend to parallel to the ground.

Main idea Naturally form the shape of roofs using triangulation methods.

Work on the triangulation in two-dimension phase.

Approach1. Find roof ridges parallel to the boundary and level to the ground.

2. Make a tin by Delaunay triangulation.

3. Applied Constrained Delaunay triangulation for disagreement edges.

Building boundary with pre-defined roof ridges.

Create a Delaunay triangulation for the entire points.

Remove the edges disagree with each other.

Applied Constrained Delaunay Triangulation to reconstruct conflict part.

Remove redundant edges and merge adjacent triangles if coplanar. The final result is roof polyhedron.

Remaining problems• Need to develop building formulas based

on symmetry, and Euler formula for regular roofs to compensate inaccurate results instead of operator post editing.

• Combination of building units faces adjustment and topological problems.

• Buildings can not be reconstructed by approaches, need an additional process from operators’ auxiliary.

Discussions with Wolfgang Forstner, March 19, 2008 1 3D Building Reconstruction from LiDAR Data. Our...

Documents

Transcript of Discussions with Wolfgang Forstner, March 19, 2008 1 3D Building Reconstruction from LiDAR Data. Our...