Shape Measurement System for Single Point Incremental ... Trino.pdfduring forming. The used metal...

Shape Measurement System for Single Point

Incremental Forming (SPIF) Manufacts by using

Trinocular Vision and Random Pattern

Francesco Setti, Ruggero Bini, Massimo Lunardelli, Paolo

Bosetti, Stefania Bruschi and Mariolino De Cecco

Department of Mechanical and Structural Engineering, University of Trento

via Mesiano 77, 38121 Trento, ITALY

E-mail: [email protected]

Abstract.

Many contemporary works show the interest of scientific community in measuring

shape of artefacts made by Single Point Incremental Forming (SPIF). In this paper we

will present an algorithm able to detect feature points with random pattern, check the

compatibility of associations exploiting multistereo constraints and reject outliers, and

perform a 3d reconstruction by dense random patterns. The algorithm is suitable for

a real-time application, in fact it needs just three images and a synchronous relatively

fast processing. The proposed method has been tested on a simple geometry and results

have been compared with a Coordinates Measurement Machine (CMM) acquisition.

1. Introduction

Both for recognition and measurement of three-dimensional objects, a vision system has

to be able to accurately measure 3D coordinates of several points in the scene, avoiding

badly recognized/matched points, commonly called outliers.

To compute 3D position of a cloud of points belonging to the surface of an object,

we can use different methodologies. Some of these are based on structured light, others

on regular geometric pattern fixed to the body. Neither, however, is generally able to

provide a large number of characteristic points on the objects to be measured with a high

frame rate. To overcome such limitations, in the present work stochastic images will be

projected on the target. In this way, each area of the object becomes morphologically

unique due to the randomness of projected pattern, and it is potentially useful for 3D

reconstruction.

In case of patterns belonging to the target surface, projected or not, the approach

generally used to place each point in 3D space is to exploit the principle of triangulation.

Although stereo vision systems are commonly used for this kind of applications, they can

present some outliers, due to occlusions, parallax, poorly textured regions or periodic

structures [Sabater et al., 2012]. This problem is largely solved by the proposed system

SPIF Manufacts Shape by Trino-Vision 2

by adding a third camera: for this reason we will talk about trinocular -vision instead

of stereo-vision. According with [Hartley and Zisserman, 2000], the stronger geometric

constraint given by three cameras means we will have fewer mismatches, than in the

case of two views. The addition of another point of view will also increase the accuracy

of each point.

The developed system has been optimized to measure the shape of blanks subjected

to the sheet incremental process known as Single Point Incremental Forming (SPIF)

[Ham and Jeswiet, 2008]. This is a novel sheet forming technology that allows to

produce free-form shapes on a metal sheet by moving incrementally a small tool

programmed over a CNC path. This manufacturing process, although slow, allows

then to obtain complex shapes, without using conventional forming dies. It is therefore

very suitable in prototyping phase or in case of flexible production of small lots

[Jeswiet et al., 2005, Jackson and Allwood, 2009]. However, one of the major drawbacks

of such technology is represented by the lack of accuracy of the formed product mainly

due to the elastic springback experienced by the metal blank after each tool pass: in fact,

during the process, the subsequent tool steps induce a localized elastic recovery of the

metal blank, which causes a difference between the planned shape and the obtained one.

A detailed analysis of strain distribution and evolution during SPIF process is presented

in [Eyckens et al., 2011], together with DIC based measure and a finite element analysis.

To overcome this problem, a possible solution is the on-line measurement of the product

shape and the application of a feedback correction to the process parameters.

Several recent works testify the interest of the scientific community in measuring

the shape of parts made by SPIF. [Petek et al., 2009] uses a system consisting of a

single camera and circular patterns on the target surface in order to accurately map the

surface deformation. The target is moved by two micrometric stages, thus requiring the

handling of the piece and therefore it is not usable online. [Mori et al., 1998] employs

a system based on a single camera and a projector of rectangular grids: the shape is

estimated by deformation grids calculated through the Fourier Transform grid method.

[Sasso et al., 2008] employs four cameras and a projector fringes: by detecting the phase,

it estimates the shape extent. The method described in [Decultot et al., 2008] makes

use of two cameras and a 3D-Digital Image Correlation to estimate simultaneously

the product shape as a cloud of points, and the displacement vector of each point of

the undeformed blank (provided the initial pre-processing), which also estimates the

deformation field. A good presentation of 3D-DIC technique applied to a general case

of experimental mechanics is provided by [Orteu, 2009]. A similar approach is used

in [Orteu et al., 2011], but here the authors extend the 3D-DIC method in order to

be used with a general number of cameras. They also propose a system of 4 cameras

based on 3D reconstruction of a target surface on which a random black-and-white

speckle pattern was created by using spray paint. Methods for 3D reconstruction of

generic surfaces have been developed in the past years, both high accuracy marker-

based [De Cecco et al., 2010] and natural landmarks based on Structure-from-Motion

[Baglivo et al., 2011].


In a slightly different field of manufacturing technologies (solar cell wafers),

[Walecki et al., 2008] use a novel structured light approach able to measure the height of

a simil-planar surface with a single image and an appropriate projected pattern. They

can also estimate the stress status of the target.

Compared to the research works mentioned above, the algorithm presented in this

paper has several advantages. It allows you to check the compatibility of associations

through the multi-stereo constraint and thus reduces outliers; it uses a constraint taking

into account the disparity discontinuities; it performs a reconstruction by dense random

patterns. This method is also suitable to be applied on-line, as it consists of just three

images acquisition and a synchronous, relatively fast, processing.

The outline of the paper is as follows: in section 2 the SPIF process will be described

and critical points discussed. In section 3 the developed system will be described, and

the feature extraction and trinocular geometry will be presented in details; while section

4 will present the complete algorithm for 3D reconstruction of point clouds and outliers

rejection. Lastly, section 5 will present some experiments and tests performed in our

labs.

2. Single Point Incremental Forming

The SPIF experiments were carried out on a Deckel-Maho 5-axis milling machine.

During the experiment, the metal blank is mechanically fixed at its periphery by a

proper clamping system and supported by a support plate to prevent any movement

during forming. The used metal blank, 1 mm thick, supplied by Fiat, is made of the

low carbon steel AISI 1009 usually utilized for forming automotive components.

The deformation of the blank is given by a 10 mm dia. cylindrical steel tool with an

hemispherical head; to avoid premature wear of the tool and reduce friction, the milling

machine cutting fluid is used as a lubricant in the center. Figure 1 on the left shows the

configuration of the experimental apparatus.

The proposed case study is a truncated cone angle of the wall varying with depth

and with an arc as the generator, the angle of attack of the cone is equal to 70◦ and the

curvature radius equal to 500mm (Figure 1 right). The set process parameters are as

follows: (a) tool rotational speed equal to 0 (the tool rolls over the blank), (b) tool feed

equal to 1000 mm/min, (c) vertical step size equal to 0.5mm. The adopted toolpath

strategy is the spiral one, which ensures a better surface quality of the formed product.

The process ends when there is failure of the wall of the cone

After forming the cone was measured on a coordinate measuring machine (CMM)

gantry, with the aim of determining the depth at failure and the corresponding angle of

the wall.

The free surface of the cone and the variable inclination of the wall are geometrical

features that affect the accuracy of the product obtained with the measuring system.

The inclination of the field of view of the peculiar manufact reduces accuracy.


Figure 1. Experimental setup (left) and geometry of the case study (right).

3. The Vision System

This section describes the vision system developed in this work. In particular we focus

on principles underlying feature detection and matching, and geometry of trinocular

vision.

3.1. Feature Detection and Matching

Features matching is one of the main steps in this vision application. The technique used

for extracting features from images and to perform an initial matching is based on the

algorithm Scale Invariant Feature Transform (SIFT) [Lowe, 2004]. The main benefit of

this algorithm is that SIFT features are invariant to changes in scale and orientation.

The algorithm also provides a robust matching, even with slight distortions, changes

point of view, presence of noise and variations in lighting.

Figure 2. Sift detection of feature points on a planar surface and projected pattern.

From every image we extract a large number of highly distinctive feature points;


this means that every feature point could have a corresponding feature in another image

representing the same target. SIFT also provides a descriptor, a 128-vector describing

the image gradients around the feature point. Features are then matched by calculating

the euclidean distance between normalized descriptor vectors.

3.2. Trinocular Vision

The study of multi-camera vision systems and the epipolar constraint for n-views has

been a research topic for the past two decades. [Kitamura and Yachida, 1989] first

realized that the relationship within three views of the same point can be characterized

by an epipolar constraint stronger than the one provided by a stereo camera. With

three views and the geometric constraint within feature points, the matching is unique.

Figure 3 shows a picture of trinocular configuration used in this paper.

Figure 3. Trinocular vision system developed for this application and projected

pattern.

Consider three sets F1, F2 and F3 of feature points detected respectively in camera

1,2 and 3. Consider pji and dji respectively the position (rows and columns) and the

SIFT description vector of j-th point onto the i-th camera. Suppose there is at least

one point in 3D space P projected onto the cameras in pi ∈ Fi, i = 1, 2, 3.

Considering point p1, we can compute the matching by solving the optimization

problem:

p2 = argminpj2∈F2‖d1 − dj2‖ (1)

As long as we have a calibrated system, we can improve the estimation of this

matching by using the epipolar constraint both as a hard constraint:

p2 = argminpj2∈F2‖d1 − dj2‖ subject to ‖pj2 − e12(p1)‖ ≤ k (2)

where e12(p1) is the epipolar line from camera 1 to camera 2, ‖p2 − e12(p1)‖ is the

point-to-line distance, and k is a threshold parameter to be tuned.

Anyway, there could be several points who verify the epipolar constraint and have a

similar description vector. So different orientations and noise on the description vector

could generate errors in matching.


The same is also valid for camera 1 and 3. At the same time, both epipolar

constraint and similar description vectors has to be verified also for camera 2 and 3.

This third constraint solves ambiguities in matching, in fact, except for degenerate

configuration, there is just one minimum in:

〈p2, p3〉 = argminpj2∈F2,pj3∈F3

(‖d1 − dj2‖+ ‖d1 − dj3‖+ ‖dj2 − d

j3‖)

subject to ‖pj2 − e12(p1)‖ ≤ k,

‖pj3 − e13(p1)‖ ≤ k,

‖pj3 − e23(pj2)‖ ≤ k

(3)

A graphical explanation of this phenomenon is given in Figure 4

Figure 4. Trinocular constraint.

4. The Proposed Algorithm

The developed system includes a trinocular system and a random pattern generator

using monochromatic projection (Figure 3).

Since SIFT is based on image gradients, we want to emphasize this by a projected

pattern. Random pattern has been generated by starting from a 2D Fourier Transform

of white noise signal, low-pass filtering this transformed signal and applying inverse

Fourier transform. The obtained image is then thresholded to obtain a binary image

(see Figure 2).

By using three cameras and SIFT descriptors, we can accurately, and almost

unambiguously, find the correspondence within three views. But, some feature points

are not detected in every camera. To increase the number of 3D points, we use also each

combination of two cameras in our system. At the end we have a system composed by

one trinocular camera and three stereopairs. This allows us to increase the density of


3D points, but we increase also the percentage of outliers. For this reason we introduce

a post-processing stage in order to filter out the outliers.

The outline of the outliers rejection algorithm is as follows:

(i) reconstruction by trinocular system, using both geometric constraints and SIFT

matching. These points are defined PTRI

(ii) reconstruction by stereo sub-systems, using both geometric constraints and SIFT

matching. These points are defined PCijSTE

(iii) filtering of all reconstruted points from (i) and (ii) considering the minimum

distance between optical rays ‡. Optical rays associated with each camera are taken

into account; if the distance between each pair of its exceeds a fixed threshold, the

point will be rejected

(iv) by using only points PTRI , build a three-dimensional grid and to each voxel assigning

an occupancy status tag

(v) filtering points from (ii) accordingly to status assigned to voxels in (iv). Points

belonging to empty voxels will be rejected

(vi) filtering points by checking the value of disparities compared to the probability

density function (PDF) of the disparity calculated on nearest neighbours. Taken

the projection of a candidate point onto two images, we can compute the disparity

of a set of its neighbours. We can then define the histogram of this disparities

weighted by the inverse of distance from the candidate point. We consider this

distribution as a PDF and compute the probability of candidate point, having a

candidate disparity, to be an inlier

Note about step (vi): If we are looking a planar surface, the disparity histogram

becomes a uniform distribution and the weighted one becomes an hyperbole (ideally).

If we are looking a discontinuity (e.g. two planes) the weighted histograms becomes

the composition of two hyperboles (bimodal distribution). A discrimination criteria

has been used to choose which case is best approximating the real distribution. First

a single-mode Gaussian distribution is assumed, and a regression with this model is

performed. If residuals are lower than a fixed threshold, the process will stop; otherwise

a bimodal Gaussian distribution is considered. This allows to automatically take into

account discontinuities.

5. Experiments

For the experimental part, we developed an experimental set-up composed by three

greyscale cameras and a standard digital videoprojector. We used firewire cameras with

1/3” CCD and pixels resolution 1024× 768; lenses focal length were 4.5mm which gave

a field-of-view of about 400× 500mm at a distance of 500mm. A digital videoprojector

‡ The optical ray of an image point m is the locus of points in space that projects onto m

[Fusiello, 2008].


Cameras ] of points ] of outliers % of inliers σ from plane [mm]

Pair 1-2 1634 15 99.08% 0.185

Pair 1-3 1958 7 99.64% 0.180

Pair 2-3 1740 9 99.48% 0.205

Trinocular 1129 0 100% 0.140

Table 1. Planar reconstruction test results. Comparison within stereo and trinocular.

with resolution 1400 × 1050mm was located at the centroid of cameras system. A

desktop PC managed everything: random pattern image projection, camera acquisition

and image processing.

The experimental set-up has been calibrated. First, optical radial distortions are

estimated and adjusted by rectifying the distorted images. Radial distortion coefficients

are estimated by compensation of the curvature induced by radial distortion on the

calibration grid [Devernay and Faugeras, 1995]. The second step of calibration process

is based on Tsai’s method [Tsai, 1987]. We implemented the method by using a planar

target which translates orthogonal to itself by means of a robotic stage, generating a

three-dimensional grid of calibration points. At a first step the parameters are obtained

using a pseudo-inverse solution of a least-squared problem employing points on the

calibration volume and image points. After this first estimation of the intrinsic and

extrinsic parameters, an iterative optimization is performed in order to minimize the

errors between acquired image points and the projections of the 3D calibration points

on the image plane using the estimated parameters. Accuracy in 3D reconstruction of

calibration grid was σ=0.06mm.

Using one image for each camera we reconstructed point clouds with more than 1000

points using trinocular vision and about 50% more with stereovision. This difference

is mainly due to two causes: first, using three cameras in an equilateral triangle

configuration the field of view is reduced if compared with a stereo-system; and second,

constraints used for trinocular matching are more restrictive.

To verify system accuracy and algorithm efficiency, an aluminium planar surface

(planar tolerance 0.01mm) has been acquired and data have been processed. Cameras

were located in front of the target, and in this test we achieved an accuracy of 0.14mm,

intending with this the standard deviation from a best fit plane computed by RANSAC

[Fischler and Bolles, 1987], which provides also inliers/outliers detection. Table 1 shows

some results. We can appreciate that in this test, even using just stereo cameras the

outliers rate is always less than 1%, and accuracy is still good (σ = 0.2mm). Anyway, by

using trinocular vision results have been improved both in term of outliers and accuracy.

In order to verify the accuracy of the system in a real scenario, we performed a test

with a SPIF manufact. A Coordinates Measurement Machine (CMM) with precision

of 0.004mm has been used to scan this target, and results have been used as ground

truth and compared with results given by our system. In order to superimpose the two


clouds of points obtained, we applied reference markers that have been identified with

both CMM and trinocular system. An image of CMM setup is shown in Figure 5.

Figure 5. SPIF manufact acquired by CMM for comparison with vision system.

In this operation, the main issue is in the fact that point clouds are not referenced

to each other. In order to compare results, we need to reference the clouds without using

comparison data. For this reason we used a cloud of markers and we measured the 3D

positions of this markers with both our system and CMM (see Figure 6). We used Harris

corner detection algorithm [Harris and Stephens, 1988] and trinocular reconstruction.

Given two point clouds Pvis and Pcmm, computed with vision system and CMM

respectively and both registered with respect to its centroid,[Kanatani, 1996] gives us a

method for points clouds registration based on SVD. The problem solved is to find the

3× 3 rotation matrix Rviscmm such as:

Pvis = Rviscmm · Pcmm (4)

Since markers in CMM are manually picked, the position is not accurate. For this

reason we acquired a patch of points nearby each marker and fit this points with a

function of order three. Each point in i-th patch must satisfy Fi(x, y, z) ' 0. Then we

used least squares [Bjorck, 1996] to solve:

minR,t (∑

i Fi(R · Pvis,i + t)) (5)

After this optimization process, standard deviation of ‖Pvis − Pcmm‖ decreases to

0.05mm. A graphical interpretation is shown in Figure 6.

Locally approximating the CMM cloud with cubic equations, we calculated the

distance of each point of trinocular cloud from the fitting function, and therefore the

accuracy of trinocular reconstruction. This test shows an accuracy comparable to the

one computed in planar test (σ = 0.14mm).

Points seen very differently by cameras, for instance areas strongly inclined with

respect to the image planes, may be associated with the matching of SIFT descriptors.

But the identification of marker’s centroid could be different from camera to camera


Figure 6. Comparison with CMM. Left: example image acquired with our system.

Histogram of distances cmm−vis before (center) and after (right) optimization process

for clouds registration.

and this therefore impacts negatively on the reconstruction accuracy. To verify this

phenomenon we performed a test using a planar surface both frontal to cameras

and tilted with and angle of about 70◦ and projecting the same pattern. In frontal

configuration we obtained a standard deviation of 0.22mm, while in tilted one we

obtained a value of 0.43mm. From this test the effect of this phenomenon results evident.

Figure 7. Initially reconstructed point clouds (left) and filtered results (right).

Trinocular reconstructed points are in blue, while stereo reconstructed points are in

green.

Figure 7 shows results obtained by outliers rejection algorithm. Points very far

from the object surface are rejected by three-dimensional occupancy grid (step (v) of

outliers rejection algorithm). Qualitatively, points reconstructed with stereopairs and

tagged as inliers are all correct. We can also appreciate that algorithm works even in

connection area, between planar and conic part of artefact, and this is an evidence of

good performance of bimodal distribution filtering.


6. Conclusions

In this paper we presented a novel system for measurement of generic 3D shapes by using

three cameras and random pattern projection, particularly oriented to SPIF artefacts

reconstruction. We presented a general method for trinocular 3D reconstruction and

outliers rejection. A further improvement could be the investigation of uncertainty of

SIFT features detector that may improve the outliers filtering by means of compatibility

analysis [De Cecco et al., 2009]. An experimental setup have been built and tests have

been performed in order to estimate system performances. In our labs we achieved, in

a workspace of 100×100×50mm a points density of 15points/cm2 and an accuracy of

0.28mm with 95% confidence level.

References

[Baglivo et al., 2011] Baglivo, L., Del Bue, A., Lunardelli, M., Setti, F., Murino, V., and De Cecco, M.

(2011). A method for asteroids 3d surface reconstruction from close approach distances. In 8th

International Conference on Computer Vision Systems (ICVS 2011), Sophia Antipolis, France.

[Bjorck, 1996] Bjorck, r. (1996). Numerical methods for least squares problems. SIAM.

[De Cecco et al., 2010] De Cecco, M., Pertile, M., Baglivo, L., Lunardelli, M., Setti, F., and Tavernini,

M. (2010). A unified framework for uncertainty, compatibility analysis, and data fusion for

multi-stereo 3-d shape estimation. IEEE Transactions on Instrumentation and Measurement,

59(11):2834 –2842.

[De Cecco et al., 2009] De Cecco, M., Pertile, M., Baglivo, L., Parzianello, G., Lunardelli, M., Setti,

F., and Selmo, A. (2009). Multi-stereo compatibility analysis for 3d shape estimation. In IMEKO

XIX World Congress (IMEKO 2009).

[Decultot et al., 2008] Decultot, N., Velay, V., Robert, L., Bernhart, G., and Massoni, E. (2008).

Behaviour modelling of aluminium alloy sheet for single point incremental forming. International

Journal of Material Forming, 1:1151–1154.

[Devernay and Faugeras, 1995] Devernay, F. and Faugeras, O. (1995). Automatic calibration and

removal of distortion from scenes of structured environments. In SPIE, volume 2567.

[Eyckens et al., 2011] Eyckens, P., Belkassem, B., Henrard, C., Gu, J., Sol, H., Habraken, A.,

Duflou, J., Van Bael, A., and Van Houtte, P. (2011). Strain evolution in the single point

incremental forming process: digital image correlation measurement and finite element prediction.

International Journal of Material Forming, 4:55–71.

[Fischler and Bolles, 1987] Fischler, M. A. and Bolles, R. C. (1987). Readings in computer vision:

issues, problems, principles, and paradigms, chapter Random sample consensus: a paradigm for

model fitting with applications to image analysis and automated cartography, pages 726–740.

Morgan Kaufmann Publishers Inc., San Francisco, CA, USA.

[Fusiello, 2008] Fusiello, A. (2008). Elements of geometric computer vision.

[Ham and Jeswiet, 2008] Ham, M. and Jeswiet, J. (2008). Single point incremental forming.

International Journal of Materials and Product Technology, 32(4):374–387.

[Harris and Stephens, 1988] Harris, C. and Stephens, M. (1988). A combined corner and edge detector.

In Proc. Fourth Alvey Vision Conference, pages 147–151.

[Hartley and Zisserman, 2000] Hartley, R. I. and Zisserman, A. (2000). Multiple View Geometry in

Computer Vision. Cambridge University Press.

[Jackson and Allwood, 2009] Jackson, K. and Allwood, J. (2009). The mechanics of incremental sheet

forming. Journal of Materials Processing Technologies, 209(3):1158–1174.

[Jeswiet et al., 2005] Jeswiet, J., Micari, F., Hirt, G., Bramley, A., Duflou, J., and Allwood, J. (2005).


Asymmetric single point incremental forming of sheet metal. CIRP Annals - Manufacturing

Technology, 54(2):88–114.

[Kanatani, 1996] Kanatani, K. (1996). Statistical Optimization for Geometric Computation: Theory

and Practice. Elsevier Science Inc., New York, NY, USA.

[Kitamura and Yachida, 1989] Kitamura, Y. and Yachida, M. (1989). Three-dimensional data

acquisition by trinocular vision. Advanced Robotics, pages 29–42.

[Lowe, 2004] Lowe, D. G. (2004). Distinctive image features from scale-invariant keypoints.

International Journal of Computer Vision, 60:91–110.

[Mori et al., 1998] Mori, K., Otsu, M., Fujiwara, N., and Osakada, K. (1998). Hammering incremental

forming of sheet metal using ccd cameras and database. The Japan Society of Mechanical

Engineering, 97:374–380.

[Orteu, 2009] Orteu, J.-J. (2009). 3-d computer vision in experimental mechanics. Optics and Lasers

in Engineering, 47(3-4):282–291.

[Orteu et al., 2011] Orteu, J.-J., Bugarin, F., Harvent, J., Robert, L., and Velay, V. (2011). Multiple-

camera instrumentation of a single point incremental forming process pilot for shape and 3d

displacement measurements: Methodology and results. Experimental Mechanics, 51(4):625–639.

[Petek et al., 2009] Petek, A., Kuzman, K., and Kopac, J. (2009). Deformations and forces analysis

of single point incremental sheet metal forming. Archives of Material Science and Engineering,

35(2):107–116.

[Sabater et al., 2012] Sabater, N., Almansa, A., and Morel, J.-M. (2012). Meaningful matches in

stereovision. IEEE Transactions on Pattern Analysis and Machine Intelligence, 34:930–942.

[Sasso et al., 2008] Sasso, M., Callegari, M., and Amodio, D. (2008). Incremental forming: an

integrated robotized cell for production and quality control. Meccanica, 43:153–163.

[Tsai, 1987] Tsai, R. Y. (1987). A versatile camera calibration technique for high-accuracy 3d machine

vision metrology using off-the-shelf tv cameras and lenses. IEEE Journal on Robotics and

Automation, 3(4):323–344.

[Walecki et al., 2008] Walecki, W. J., Szondy, F., and Hilali, M. M. (2008). Fast in-line surface

topography metrology enabling stress calculation for solar cell manufacturing for throughput in

excess of 2000 wafers per hour. Measurement Science and Technology, 19(2):025302.

Shape Measurement System for Single Point Incremental ... Trino.pdfduring forming. The used metal...

Documents

Transcript of Shape Measurement System for Single Point Incremental ... Trino.pdfduring forming. The used metal...