2010_AO_3D Scan_telecentric Scanner and Double Wedges

7/28/2019 2010_AO_3D Scan_telecentric Scanner and Double Wedges

1/21

1

Optical Design of a Variable View Imaging System with

Combination of a Telecentric Scanner and Double Wedge

Prisms

Xiaodong Tao1, Hyungsuck Cho,

1,*and Farrokh Janabi-Sharifi

2

1Department of Mechanical Engineering, Korea Advanced Institute of Science and Technology,

373-1, Guseong-dong, Yuseong-dong, Daejeon, Korea

2Department of Mechanical and Industrial Engineering, Ryerson University, 350 Victoria Street,

Toronto,Canada

*Corresponding author: [email protected]

Recent developments in the micro-electro-mechanical system (MEMS) and biotechnology

have presented an emerging need for observation of dynamic targets in three-dimensional

space. Unfortunately, the conventional microscopes with fixed optical parameters are

difficult to supply sufficient vision information because of occlusion, small field-of-view

(FOV) and low depth resolution. This paper introduces the design of a variable view

imaging system which can supply a flexible view with a relative large zenith angle and a

simple kinematics. Due to several performance factors, a multiobjective optimization

process is applied to achieve an appropriate design. A prototype system is developed and

used to verify the proposed design.

OCIS codes: 110.1080, 220.1080, 120.4820, 150.3040, 170.0110, 170.0180, 220.4830.


2/21

2

1. Introduction

The small field-of-view (FOV) and fixed view direction of conventional microscope cannot

supply sufficient visual information needs for observation of dynamic targets in three-

dimensional (3D) space, particularly in the fields of the micro-electro-mechanical systems

(MEMS) and biotechnology applications. In the past, there have been several research efforts to

solve this problem. Multiple fixed microscopes or moving stages are often applied in those

applications [1,2]. Such approaches do not provide sufficient flexibility for observation of

dynamic targets. A smart optical system with adjustable optical parameters becomes a promising

solution. Potsaid has proposed an adaptive scanning optical microscope (ASOM) [3]. Through

the integration of a scanning mirror and deformable mirror, the ASOM can obtain a large FOV

with high resolution. A similar methodology is also applied in a scanning optical telescope [4].

Another system capable of achieving foveated viewing in a large FOV has been demonstrated

using a spatial light modulator (SLM)[5].

In our previous research, a proposal was made for an active optical system which can change

the view position and direction [6,7]. However, because of the coupling effect between the

scanning mirror angle and view angle, the view angle is found to vary with scanning mirror,

causing its zenith angle of view to decrease, making the kinematics complex. In order to solve

those problems, a decoupled design of the system is proposed in this paper, which employs a

telecentric lens. The basic concept of the system is presented in SPIE conference [8]. Since this

design contains a number of parameters that affect the performance of te system, optimal

election of these parameters is necessary. For this, an multiobjective optimization design process

is described in this paper in detail for the first time. Experimental results are presented to show

the validity of the proposed design.


3/21

3

2. Concept of the variable view imaging system

The objective of the design of the variable view imaging system is to change the view state by

control of active optical components in the system. The view state includes view position and

view angles defined as Eq. 1.

T

w wV x y = (1)

where (xw,yw) is the view position in a world coordinate. and are the azimuth angle and the

zenith angle of the view.

In the robot area, by installing a camera on a robot, an active vision system can change the view

state easily. However, considering the size and weight of vision system and speed of scanning, it

is difficult to install a microscope system on a robot. In order to realize this function, the idea is

to design the vision system with active components which mimic the function of robot. Fig. 1

shows the concept of the system. To steer the view angle (, ), double wedge prisms are applied

because the compact size for deflection of light. The function of double wedge prisms is similar

to a camera installed on a pan/tilt stage. It can steer the beam in any direction in a cone. To steer

the view position (xw,yw), the telecentric scanner is applied. Its function is similar to a camera

installed on the translation stage. Therefore by integration of these two active optical components,

the proposed system can steer the both of view position (xw,yw) and orientation (, ). It can be

considered as a camera installed on a robot with four degrees of freedom as shown in Fig. 1.

In our previous research, a scanning mirror and double wedge prisms were used for view steering

[6,7], where the scanning mirror angle was coupled with the zenith angle of the view. The zenith

and azimuth angles changes with the change of the scanning mirror angle. There are several

issues for the old design. The first issue is a small zenith angle. When the target locates near the


4/21

4

optical axis, the motion of scanning mirror decrease the zenith angle of view because the

coupling effect between the scanning mirror angle and view orientation. The second issue is the

complex kinematics relationship. Each variable in the view state is related to all variables in joint

space as V=[fy(),fx(),f(),f()]T, where =[sx, sy, p1, p2]

T is the vector of joint variables

for scanning mirror and wedge prisms, respectively. fy(),fx(),f() and f() are the functions

to calculate the four variables in the view state as shown in Eq. 1. The calculation of joint

variables for a desired view state becomes difficult. And a precise alignment and calibration are

required to obtain an accurate kinematics equations. To decouple the rotation angle of scanning

mirror and view angles, a telecentric lens group is designed and introduced to the system. The

system aperture is projected on the focal point of the telecentric lens. It makes the entrance pupil

infinitely. Therefore in the object side, the chief rays are always parallel to the optical axis.

The layout of the whole system is shown in Fig.2. The 3 rd and 4th lenses integrate a deformable

mirror into the system. The deformable mirror is used for correction of aberration induced by

wedge prisms. The 4th and 5th lenses relay the active aperture of the deformable mirror to the

system aperture. The system aperture is projected onto the scanning mirror, which is located at

the focal point of the telecentric lens. It makes an entrance pupil infinitely. Therefore the rays are

always parallel to the optical axis in the object side. The azimuth angel and zenith angle of view

are only related to the rotation angle of the prisms.

In order to determine view state given the joint variables, a rigorous analysis of the view

position and angle is made by the ray tracing method. The forward kinematics of the system can

be defined as follows,

1 2 1 2 1 2 1 2( , , , ) ( , , , ) ( , ) ( , )

T

x sx sy p p y sx sy p p p p p pV f f f f = (2)
http://en.wikipedia.org/wiki/Entrance_pupilhttp://en.wikipedia.org/wiki/Entrance_pupil


5/21

5

where V(xw,yw, , ) is the vector for view state. And (sx, sy, p1,p2 ) includes the joint variables

for scanning mirror and wedge prisms. The detail derivation can be found in [7]. The inverse

kinematics can be achieved by using a damped least-squares (DLS) method, which can also be

refer to [7].

3. Optimal design of the view steering part

The view steering part include scanning mirror, telecentric lens and wedge prisms. The

performances of the system, such as scanning area, maximum zenith angle, wavefront error are

all related to this part. In this section, the system performance is defined. Because several system

performance factors are involved, a muliobjective optimization process is applied to give an

appropriate design.

A. System performance for the system design

System performance can be identified from the analysis of the functions of the system. The main

function of the proposed system is to avoid the occlusion by changing the view angle. The

effects of system parameters for the occlusion case can be shown in Fig. 3, where the image of a

cubic with a depth D is captured by the proposed system. In the top view, the corner point B is

occluded by the point A as shown in the figure. With a zenith angle , both point A and B can be

observed in the image plane. Because of the telecentric lens group used in the system, the scaled

orthographic projection model can be applied in the system. The distance between A and B in the

image plane is defined as following equation.

AB od Md=

(3)


6/21

6

whereMis the magnification of the system. d0 is projected length of A and B along optical axis,

which is defined as

0 sin( )d D = (4)

In order to avoid occlusion, the distance between A and B need to be observed in the image

plane. Therefore the condition for the proposed system to avoid occlusion is as follows,

sin( )o

r D < (5)

where ro is the optical resolution of the system. We can find that with high resolution r and larger

zenith angle , the system can easily avoid occlusion. Therefore the maximum zenith angle and

the resolution are two important performance factors for occlusion avoidance.

Another function of the proposed system is to increase the field of view by steering the view

position. In the whole scanning area, the area A2 with full azimuth angle with maximum zenith

angle is most important as shown in Fig. 4. In this area, the system can achieve the view with an

azimuth angle from 0 to 2 and a zenith angle from 0 to maximum zenith angle.

As an optical system, image quality is also an important performance factors. Because the

wedge prisms are used in the system, a relative large wavefront error is induced into the system.

In this design, a deformable mirror is applied to correct this aberration. However the wavefront

error correction is limited due to the limited stroke of the actuator in the deformable mirror. In

the proposed system, a micromachined membrane deformable mirror (37ch, D15mm, OKO) is

applied. The limitation of wavefront correction for this mirror is 3.5 (wavelength =0.660 m)

according to [10]. Therefore in system design, the wavefront error should be kept inside this

limitation. The details calculation of these system performances are described as follows.


7/21

7

Maximum zenith angle, max

When two wedges have the same rotation angle, the zenith angle achieves its maximum value as

shown in Fig. 5(a). Therefore for a given v, the maximum zenith angle is defined as

max (0,0)v

f = (6)

Therefore max is a function of the vertex angle of the wedge prisms defined as follows,

maxmax( )

vF = (7)

Fig. 5 (b) shows the final result of the relationship. Larger vertex angle of the wedge prisms can

provide a larger zenith angle.

Radius of view area, r2

The configuration of the system for calculation of radius of A2 is shown in Fig. 6(a). The

radius of the prism is defined as rp. The view will located at the boundary of the A2 when

s px r= , 0sy = , 1 0p = ,

2 0p = . Therefore the radius of A2 for a given v can be obtained

from following equation,

2( ,0,0 ,0 )

vx pr f r

= (8)

r2is a function of the radius and vertex angle of wedge prism, defined as follows,

2 2( , )

r p vr F r

=

.(9)


8/21

8

This relationship is shown in Fig. 6(b). Comparing to the vertex angle v of prisms , the radius

of the prism gives more effects on the change of r2.

Optical Resolution

The Optical resolution is the ability of a microscope to allow one to distinguish between small

objects. The optical resolution ro is depend on the numerical aperture as shown in Eq. 9.[3]

0.61o

rNA

=

(10)

where is the wavelength. NA is the numerical aperture, which characterizes the range of angles

over which the system can accept. It can be defined as following equation.

sin( / 2)NA n = (11)

where n is the refraction index, is the angle of the maximum cone of light that can enter the

objective lens.

Peak-Valley Aberration, WPV

The aberration of the system is variable depending on the wedge prisms angle and scanning

mirror angle. It can be corrected by a deformable mirror which was applied in the system.

However, because the correction limit of the deformable mirror, the peak-valley wavefront error

should small than the correction range of the deformable mirror. For a specific design, the peak-

valley wavefront is depending on the configuration of the prisms and mirror. The worst

configuration with the maximum peak-valley wavefront error can be obtained from Zemax

software as shown in Fig. 7(a), where xs=rp, ys=0, p1=0,p2=180. The maximum peak-valley


9/21

9

wavefront error is a function of vertex angle of prisms, radius of the prism and numerical

aperture defined as follows,

( , , )PV pvW p v

W F r NA= (12)

It is difficult to achieve this relationship analytically. From Zemax software, the warefront

errors for different rp, v and NA can be obtained easily as shown in Fig. 7(b). However, this

discrete result is not accurate enough for the system design. Therefore a Backpropagation (BP)

networks are built for function approximation [11]. In this design, a standard BP networks with 2

hidden layers is applied. Each hidden layer has 10 neurons. The Tan-sigmoid transfer functions

are applied in the networks. The data from the Zemax software was used for training data.

B. Optimized Design of the system

The objective of the optimized design is to achieve best performance of the system. In this

design, there are four system performance factors for optimization. The object functions can be

formulated as follows,

2 max, ,,

max ,max , max , minp vp v v

o pvr NAr NA

r r W

(13)

When the deformable mirror is applied the system, because the wavefront error inside a specific

range can be corrected by the deformable mirror, the Wpv can be a boundary constraint in system

design. Because the relationship between ro and NA defined in Eq. 9, ro can be replaced by NA

to decrease the number of parameters for the design. Therefore the objectives of the system

design can be simplified as following equation,

2 max, ,

max ,max ,maxp v p vr r

r NA

(14)

Subject to ( , , )pv p vW r NA <


10/21

10

where is the correction limit of wavefront error for the deformable mirror. In this design, is

set as 3.5 . As can be seen from Eq. 14, the optimal design becomes a multiobjective

optimization problem. The performance factors are conflicted with each other. For example, The

increase of vertex angle v of the prisms will increase the maximum zenith angle max, and also

increase the wavefront error and decrease the radius of view area, r2. The increase of rp increases

the radius of view area, r2, and also increases the wavefront error. Therefore, we can not get a

single optimized solution from Eq. 14. In order to solve this Multi-Objective optimization

problem, -constraint method is applied in this design according to [12]. This method is to select

one of the objective functions to be optimized and all the other objective functions become

constraint. This method can simplify the multiobject optimization problem to single objective

optimization. In this design, the boundary of the numerical aperture NA, radius of view area r2

can be defined based on the application. The objective of the design is to maximize the zenith

angle max. Therefore the optimization problem can be simplified as

max, ,

maximizep vNA r

(15)

Subject to2 , ,r NA pvr K NA K W <

Kr and KNA are the boundary of the r2 and NA, which can be determined based on the

application by the designer.

In order to solve the inequality-constrained optimization, a Logarithmic barrier method is

applied [34]. The boundary of view area is determined by the application. In the microassembly

application, the Kr is set as 5 mm. KNA is 0.03 in this design. The design result is shown in Table

1. As can be seen, the maximum zenith angle,max

is 22.29. In the real experiment, the in-stock

components are selected with the most similar value achieved in the optimal design. The

parameters for the real experiment setup are shown in Table 2. Based on the catalog of the wedge


11/21

11

prisms, the maximum vertex angle of the wedge prisms are 188, which gives a 19 degree of

zenith angle. The zenith angle in the real experiment setup is only 3.3 less than the optimal

design result.

As discussed in the Section 3.1, the NA and max are two important factor for occlusion

avoidance. By setting different KNA, the relationship between NA vs. max can be achieved from

the optimal design solutions as shown in Fig. 8. As can be seen, when NA decrease to 0.02, 30

degree of zenith angle max can be achieved. When the NA increases to 0.1, only 4 degree of

zenith angle max can be achieved. This tradeoff relationship between NA and max is caused by

the limitation of wavefront error correction for the deformable mirror. The NA and max in the

current experiment system is shown in the Fig. 8, which is near the optical design set.

4. Experiment results

In order to investigate the feasibility of the proposed optical system design, A prototype system

was constructed according to the Table 3. The 37-channel MMDM was supplied by OKO

Technologies. The detailed information for wavefront correction can be referred to [10]. The

deviation angle of one wedge prism was 10. The wedge prisms were installed on the compact

rotation stages. Four achromatic lenses with effective focal length of 150 mm were used in the

system.

A. Experiment for the real object

The ability of the proposed system to change the view without moving the object was

demonstrated by multiple views of micro gears. Three views with different configurations are

shown in Fig. 9(a) ~ (c). Fig. 9 (a) shows the view with a zenith angle =18.9 and an azimuth

angle =90, where the hole in the 2nd gear can be observed. Fig. 9 (b) shows the view with a


12/21

12

zenith angle =18.9 and an azimuth angle =0. The depth of the 2nd gear can be observed. Fig. 9

(c) shows the view with a zenith angle =18.9 and an azimuth angle =270, where the other

side of the 2nd can be observed. In order to compare the imaging results with the previous system

[7], the view state for the previous system is achieved in the proposed system. The max,2, for the

previous system is determined from system kinematics, which equals to 13.8[7]. The three view

states with zenith angle =13.8 is achieved as shown in Fig. 9(d)~(f). The interested features on

the side of gears on the image plane are much smaller than the current system. The heights of

gears, h1, h2, on the image plane for the previous system are 26 and 18 pixels respectively. In the

proposed system, they are 36 and 25 pixels respectively, which is more than 35 percent larger

than the coupled system. During online operation, the joint variables for a desired view state

need be calculated in a short time. The numerical solution based on the damped least-squares

(DLS) method is implemented, which is similar to the previous system [7]. Because of the

simplified kinematics, only 23 iterations are need for calculation of joint variables, which is

approximately 32ms in the current system. However, for the previous system, more than 350

iterations are needed, which will spend 490ms.

B. Experiment for the real application

In the microassembly experiment, a microassembly system including three translational

motions was installed near the proposed system. The preliminary experiment setup was only

intended to evaluate the performance of the system; therefore, the microassembly system was

teleoperated without automation. The task is to insert the micro electrode into the micro slot as

shown in Fig. 10 (a). The widths of slot and electrode are 300um and 250um respectively. The

electrode is held by a gripper. As shown in Fig. 10 (b). At an initial state, the view angle is set as


13/21

13

=0 and =0. The captured image is shown in Fig. 10 (b). The micro electrode will align with

the slot in X direction as shown in Fig. 10 (c). However, the insertion part of the electrode cannot

be observed because of occlusion. In order to align the electrode and slot in Y direction, the view

angle is steered by the proposed system, and Fig. 10 (d) shows the image when a zenith angle

is 19 and the azimuth angle is 0. With the visual information of the side surface, the electrode

is aligned with the slot in Y direction as shown in Fig. 10 (e). The final assembly task is to insert

the electrode into the slot. Fig. 10 (f) shows a final state of the assembly.

5. Conclusion

This paper presents the design of a variable view imaging system which can change the view

angle and view position by actuating active optical components in the system. The proposed

system with a telecentric lens group and double wedge prisms can supply a larger zenith angle of

view and simpler kinematics relationship compare with the previous system [7]. During the

system design, multi performance factors are identified. In order to obtain the best performance,

a multiobjective design process is applied. The experimental results showed the validity of the

proposed system to achieve a larger zenith angle at different azimuth angle. The focus of the

future research will be on the automated operation of the system.


14/21

14

Reference

1. M. Probst, K. Vollmers, B. E. Kratochvil and B. J. Nelson, Design of an advancedmicroassembly system for the automated assembly of bio-Microrobots, presented at 5th

International Workshop on Microfactories, Besancon, France, 2527 October 2006.

2. X. Hui, R. Weibin and S. Lining, A flexible experimental system for complexmicroassembly under microscale Force and vision-based Control, Int. J. Optomechatronics

1, 81102, 2007.

3. B. Potsaid, Y. Bellouard and J. Wen, Adaptive Scanning Optical Microscope (ASOM): amultidisciplinary optical microscope design for large field of view and high resolution

imaging, Opt. Express 13, 65046518, 2005.

4. C. Scott, B. Potsaid and J. Wen, Off-Axis aberration correction for a wide field scanningtelescope, in Proc. SPIE 7266, 72660Y, 2008

5. H. Hua and S. Liu, A dual-sensor foveated imaging system, Applied Optics 47, 317-27,2008

6. X. Tao, D. H. Hong, H. S. Cho, The design of active vision system for variable viewimaging of micro objects, in Proc. SPIE 6376, 637608112, 2006.

7. X. Tao, H. S. Cho and F. Janabi-Sharifi, Active optical system for variable view imaging ofmicro objects with emphasis on kinematic analysis, Appl. Opt. 47, 4121-4132, 2008

8. X. Tao, D. H. Hong, H. S. Cho, Variable view imaging system with decoupling design,International Symposium on Optomechatronic Technologies, Proc. of SPIE, 7266, 72661U-1

- 11, 2008.

9. J. R. Meyer-Arendt, Introduction to Classical and Modern Optics, Prentice Hall, 1995.


15/21

15

10.L. Zhu, P. C. Sun, D. U. Bartsch, W. R. Freeman and Y. Fainman, Wave-front generation ofZernike polynomial modes with a micromachined membrane deformable mirror, Appl. Opt.

38, 60196026, 1999.

11.S. Kumar, Neural Networks: A Classroom Approach, McGraw-Hill, 200512.K. Miettinen, Nonlinear Multiobjective Optimization, Kluwer Academic Publishers, 199913.H. S. Cho, Optomechatronic: Fusion of Optical and Mechatronic Engineering, CRC Press,

2005.


16/21

16

Fig. 1. Concept of the variable view imaging system

Fig. 2. The layout of the proposed system.

Fig. 3. Avoid occlusion in the proposed system

Fig. 4. View area A2 with full azimuth angle with maximum zenith angle

Fig. 5. (a) The configuration of the wedge prisms with maximum zenith angle (b) The

relationship between the vertex angle v (degree) of the wedge prism and max (degree)

Fig. 6. (a) The configuration of the system for calculation ofr2(b)The relationship between

2

, ,p vr r

Fig. 7. (a) The worst configuration with the maximum peak-valley (b) The warefront errors for

different rp, v and NA

Fig. 8. NA vs. max in the optimized design

Fig.9. Multiple views of micro gears: (a)~(c) captured images in three different azimuth angle

with =18.9 and (d)~(f) =13.8

Fig. 10. Microassembly of a micro electrode and a slot, (a) microassembly task, and (b)~(f)

captured images during the assembly.

Table 1 The optimal design result

Table 2 Specification of the experiment setup


17/21

17

Pan/tilt

stageCamera

Scanning

lens

Scanning

mirror

Translation stage

Wedge

prisms

Fig. 1. Concept of the variable view imaging system

camera

telecentric

lens

scanning

mirror

wedge

prisms

deformable

mirror 2nd lens

3rd lens

4th lens

5th lens

6th lens

system

aperture

Xw

Zw

Ow

Yw

Zv

Xv

YvOv

(xw,yw)

view direction

Object plane

Fig. 2. The layout of the proposed system.


18/21

18

A

A

Telecentric lens

groupA

B

B

Image plane

Occlusion

Without occlusion

do

D

di

Fig. 3. Avoid occlusion in the proposed system

Zv

Xv

Yv

Ov

max,2

=0~2

Fig. 4. View area A2 with full azimuth angle with maximum zenith angle

v

max

0 10 20 30 40 50 600

10

20

30

40

50

60

70

80

90

max

v

(a) (b)


19/21

19

Fig. 5. (a) The configuration of the wedge prisms with maximum zenith angle (b) The

relationship between the vertex angle v (degree) of the wedge prism and max (degree)

pri sms

rp

2r2A 5

10

15

20

25

0

10

20

30

-10

0

10

20

30r2(mm)

Fig. 6. (a) The configuration of the system for calculation ofr2(b)The relationship between

2, ,

p vr r

1st wedge prism

2nd wedge prism

Object plane0.1

0.20.3

0.40.5

510

1520

250

10

20

30

40

50

60

70

80

90

rp v

Wpv

() NA=0.1

NA=0.09

NA=0.01

(degree)(mm)

(a) (b)

Fig. 7. (a) The worst configuration with the maximum peak-valley (b) The warefront errors for

different rp, v and NA


20/21

20

0.02 0.03 0.04 0.05 0.06 0.07 0.08 0.09 0.15

10

15

20

25

30max

NA

Curr ent exper i ment

syst em

Fig. 8. NA vs. max in the optimized design

h1

h2

500m 500m 500m

2nd gear

1st gear

h2

(a) =18.9, =90

(b)=18.9

, =0

(c)=18.9

, =270

h1

h2

500m 500m 500m

h2

(d) =13.8, =90 (e)=13.8, =0 (f)=13.8, =270

Fig.9. Multiple views of micro gears: (a)~(c) captured images in three different azimuth angle

with =18.9 and (d)~(f) =13.8


21/21

500um

Gripper

Electrode

Slot

500um

(a) (b) (c)

500um 500um 500um

(d) (e) (f)

Fig. 10. Microassembly of a micro electrode and a slot, (a) microassembly task, and (b)~(f)

captured images during the assembly.

Table 1 The optimal design result

MaximumZenith angle,max

22.29

Scanner area, r2 5.56mmNA 0.0302Wavefront error,WPV 3.498, =685nmVertex angle, v 24.3Radius of prisms, rp 15.15 mm

Table 2 Specification of the experiment setup

Maximum Zenith angle, max 19

Scanner area, r2 4.35 mm

NA 0.033Wavefront error,WPV 2.4, =685nm

Vertex angle, v 18.13

Radius of prisms, rp 12.5 mm

2010_AO_3D Scan_telecentric Scanner and Double Wedges

Documents

Transcript of 2010_AO_3D Scan_telecentric Scanner and Double Wedges