Adaptive Scanning Optical Microscope (ASOM): large field ofview and high resolution imaging using a MEMS deformable

mirror

Benjamin Potsaid, Linda Ivonne Rivera, and John Ting-Yung Wen

Center for Automation Technologies and Systems, Rensselaer Polytechnic Institute,Troy NY, USA

ABSTRACT

For a wide range of applications in biology, medicine, and manufacturing, the small field of view associated withhigh resolution microscope systems poses a significant challenge in practice. This paper describes an opticalmicroscope design, called the Adaptive Scanning Optical Microscope (ASOM), which uses a MEMS deformablemirror working with a specially designed scanning lens to achieve a greatly expanded field of view. Most adaptiveoptics systems (e.g. telescopes and ophthalmology instruments) are designed to achieve near ideal performanceunder nominal operating conditions and primarily use the adaptive optics element to compensate for a timevarying disturbance to the wavefront that is external to the optical system. In contrast to this approach,the deformable mirror in the ASOM is an integral component of the optical system and the static (glass)optical elements have been specifically designed to match the shape correcting capabilities of the deformablemirror. Using a high speed steering mirror coordinated with the deformable mirror actuation voltages, theASOM operates by scanning over the workspace and should achieve diffraction limited imaging over a regionapproximately two orders of magnitude larger in area than a traditional microscope design. With the rapidscanning capabilities allowed by the high speed steering mirror and by acquiring a complete image during eachexposure, the ASOM offers advantages in dynamically reconfigurable and adaptable imaging with no agitationto the workspace. After describing the design and operating principle of the ASOM, we present results from alow cost ASOM prototype.

Keywords: Microscopy, deformable mirror, adaptive optics, optimization

1. INTRODUCTION

In a traditional microscope design, the field of view is limited for a given resolution. Restrictions on the anglesof incidence of the light on the lens surfaces, off-axis aberrations, and manufacturability of the opto-mechanicalassemblies make it extremely difficult to design fixed (static) optical assemblies that simultaneously achieve awide field of view, high numerical aperture, and flat image field. Furthermore, the image must be sampledat or above the Nyquist frequency to avoid aliasing. Thus, the number of pixels in the camera determinesan absolute maximum field size for a given resolution, regardless of the quality of the optics. The relativelysmall field sizes at high resolution that result from these two effects working together is becoming a significanthindrance as optical microscopes are being used in automated systems for advanced biotechnology research,medical diagnostics, robotic micromanipulation, and industrial inspection. These applications are increasinglyrequiring high throughput or challenging spatial-temporal observations where the small field size is often thesource of a bottleneck in the process or prevents observation of the event of interest altogether.

Equipping the microscope with multiple parfocal objectives on a rotating turret, motorized moving stages,zoom microscope designs, and line scanning techniques are all examples of approaches to cope with the small

Further author information: (Send correspondence to B.P.)E-mail: potsab@rpi.edu - Benjamin Potsaid oversees the Smart Optics Lab at the Center for Automation Technologiesand Systems at Rensselaer Polytechnic InstituteE-mail: riverl2@rpi.edu - Linda I. Rivera is a graduate student in the ECSE Department at RensselaerE-mail: wenj@rpi.edu - John T. Wen is the Director of the Center for Automation Technologies and Systems and aProfessor in the Electrical, Computer, and Systems Engineering Department at Rensselaer Polytechnic Institute

MEMS Adaptive Optics, edited by Scot S. Olivier, Thomas G. Bifano, Joel A. Kubby,Proc. of SPIE Vol. 6467, 646706, (2007) · 0277-786X/07/$18 · doi: 10.1117/12.700538

Proc. of SPIE Vol. 6467 646706-1

field of view and are quite common. However, each method suffers from deficiencies of its own in practice.For example, a wide field of view and high resolution can not be obtained simultaneously when using multipleparfocal objectives or a zoom microscope design. Relatively slow dynamic performance and agitation to thespecimen are unavoidable when using a moving stage. And line scanning techniques require extremely brightillumination, which can be damaging to living biological specimens, or require slow scanning speeds when brightillumination is unavailable.

This paper presents a new microscope design, called the Adaptive Scanning Optical Microscope (ASOM),which uses a MEMS deformable mirror in coordination with a high speed steering mirror and specially designedscanner lens to effectively enlarge the field of view without compromising resolution. The basic operatingtheory and system layout of the ASOM have been described in a previously published simulation based paper.1

Demonstrations using a low cost ASOM prototype (built with off-the-shelf optics only) for micro-robotic2 andbiological3 applications illustrate the the advantages of the ASOM for tracking multiple moving objects andobserving large areas at high resolution without agitating the sensitive workspace. This paper focuses on theMEMS deformable mirror shape optimization and realtime control. The lessons learned and experiences gainedserve as a strong foundation for the development of the next generation ASOM prototype, which will incorporatecustom manufactured optics, a high performance steering mirror, and MEMS deformable mirror with longerstroke and a larger number of actuators to fully realize the potential of the ASOM concept.

2. OVERVIEW OF THE ADAPTIVE SCANNING OPTICAL MICROSCOPE (ASOM)

45o39.4o 35.5o

(0.0,0.0mm) (-12.2,0.0mm) (-20.3,0.0mm)Sub field of view in both images is larger

than true scale for illustrative purposes

steering mirror

k x k pixel

sub field

of view

ASOM sub-field

scanning tracking moving

object in time

imaging only

region of interest

rare event

detection

full area

coverage

(b)

(a)

To imaging optics and camera

Figure 1: (a) ASOM post objective scanning design (b) ASOM modes of operation

The ASOM was developed to facilitate challenging spatial-temporal observations and offers a unique setof performance capabilities compared with the existing wide field imaging technologies discussed in Section 1.Referring to Figure 1 (a), the underlying concept of the ASOM is to use a low mass steering mirror located betweenthe scanner lens and the imaging optics to form a post-objective scanning configuration. This configuration allowsa complete k×k pixel sub-field-of-view (not a point as in confocal microscopy) to be quickly scanned throughoutthe workspace, with an image acquired at each scan position. In this way, multiple disjoint or overlappingregions of the workspace can be rapidly visited and imaged sequentially as shown in Figure 1 (b). Additionally,through image warping to remove lens distortion and image mosaic construction, a large and continuous virtualimage of the object can be constructed from the individual image tiles. This configuration has advantages ingenerating a large effective field of view at high resolution with no disturbance to the sample. And the highspeed scanning operation allows both imaging arbitrary regions of the sample and high throughput operation.However, compared to a traditional microscope design or a moving stage based approach, there is extensive

Proc. of SPIE Vol. 6467 646706-2

off-axis imaging (i.e., images are obtained by looking diagonally through the scan lens) in the ASOM, whichintroduces image distortion in addition to contrast degrading and resolution reducing aberrations (e.g. coma,astigmatism, field curvature, etc.)4. We overcome this challenge by using a highly dynamic and active opticaldesign that addresses the off-axis aberrations by:

1. Explicitly incorporating field curvature into the design, which reduces the complexity of the scanner lens.2. Using an actuated deformable mirror (DM) in the optical path to correct for residual aberrations in order

to both expand the diffraction limited field of view and to further simplify the scanner lens design.3. Image processing to remove image distortion.

Figure 2 (a) shows a realistic ASOM layout based on readily available glass types5 and manufacturable lensgeometries. Light from a source illuminates the object (specimen). The role of the scanner lens assembly is tocollect light from the object, form a converging beam, then aim the converging beam to the center of the steeringmirror. In this design, a single mirror with two degrees of steering freedom is used to reduce cross axis scanerror that is inherent in using separate mirrors for each axis6, 7. The specific angle of the steering mirror definesthe location of the center of the field of view on the object as shown in Figure 2 (b). However, especially for theoff-axis field positions, and even to a certain extent on-axis, the scanner lens introduces considerable wavefrontaberrations which blur the image. To manage this blurring, the light from the steering mirror enters a pupilimaging stage to be directed to the deformable mirror, which adapts its surface shape to correct for the wavefronterrors introduced by the scanner lens assembly. Light exiting the deformable mirror is now well corrected andnearly aberration free. After passing through the pupil relay optics and being projected onto the science camera,the image quality is beyond the diffraction limit, which indicates imaging nearly indistinguishable from perfectand comparable to commercial microscope objectives.

Object

Scanner lensassembly

Steeringmirror

Field lens / pupil imaging opticsforward eye-piece

Field lens / pupil imaging opticsinverted eye-piece

Deformablemirror

Science cameraSystemaperture

Final imagingoptics

Scale:50mm

45o39.4o 35.5o

0

1

0

1

0

1

(b)

(a)

Optimal deformable mirror shape

(0.0,0.0mm) (0.0,12.2mm) (0.0,20.3mm)

Figure 2: (a) Preliminary ASOM design (b) Deformable mirror corrects for optical aberrations in the scan lens

The ASOM design shown in Figure 2 (a) was designed around the µDM100 DM from Boston Micromachines(Watertown, MA) with 100 electrostatic actuators, a 3.3mm round aperture, a 2µm stroke, and a 1kHz updaterate8. Figure 2 (b) shows how the DM corrects for the specific wavefront aberration associated with each fieldposition. Over the entire field and for all field positions, the Strehl ratio is much greater than the diffractionlimit of 0.8, resulting in near perfect imaging. Image distortion over all field positions is less than 1 percent.Given that algorithmic image distortion removal has been shown for endoscopes9 and wide angle lenses (fisheye lenses),10 the distortion in the ASOM is well within the proven distortion compensation capabilities ofalgorithmic distortion removal techniques, which will allow for the construction of seamless image mosaics. Moredetailed information about the optical layout and operating principle of the ASOM may be found in a previouslypublished paper1.

Proc. of SPIE Vol. 6467 646706-3

Table 1 lists performance specifications for the ASOM design shown in Figure 2 (a) and Figure 3 comparesthe observable field of view of the ASOM to a traditional microscope using a 1024× 1024 and 4096× 4096 pixelcamera. The 0.38mm size of the ASOM’s sub-field of view is also shown resulting from the use of a 512 × 512camera. For this design, a 512×512 pixel camera was chosen to achieve the high frame rates needed for trackingrapid motile organisms. The ASOM offers diffraction limited (Strehl ratio > 0.8) for all field positions based onhigh fidelity simulation. The field sizes for the fixed microscope designs in Figure 3 assume perfect imaging andwere calculated based on Nyquist sampling theory using a 0.21 numerical aperture with λ = 0.510µm for thewavelength of light for each of the camera pixel counts shown.

Table 1: ASOM performancespecifications

Spec.Field of view diameter 40mmObservable field area 1257 mm2

Numerical aperture 0.21Operating Wavelength 510 nmResolution 1.5 µmMagnification 15.2Camera pixel count 512 × 512Camera pixel size 10µm

3.03mm

4096x4096 pixels

0.76mm

1024x1024 pixels

0.38mm

512x512 pixels

(single sub-field exposure

of preliminary ASOM design)

40.00mm

Virtual field of view

offered by the ASOM

Small sub-field can

be steered within

observable area

shown in gray.

COMMON

TODAY

Figure 3: Field size Comparison

Notice that scanning in the complete observable area shown in gray will require many scan movements.However, even with the small 512× 512 camera, the scan times listed in Table 2 are competitive with or exceedexisting technologies. The table presents the estimated scan time for 100, 250, and 500 frames per second camerarates and for 100%, 50%, and 10% fill factors (percent of total observable area that needs to be imaged). Thesecalculations assume that the total number of scan movements is given by: number of scans = total effectivefield area / sub-field area. Based on our previous work with high speed steering mirrors,11 we estimate thatwe can achieve at least 100 movements per second. Faster frame rates could be achieved if camera exposureswere obtained without waiting for the mirror to completely settle, which would be useful for performing a rapidbackground scan to identify object locations or detect the onset of a biological event, but at compromised imagequality. The system could then image the region of interest at high image quality by stopping the mirror for eachcamera exposure. Currently, large pixel count digital cameras are not fast enough to realize the speeds listed forthe 1024 × 1024 and 4096 × 4096 cameras using camera industry standard interfaces.

Table 2: Estimated ASOM scan time (seconds)Camera frame rate: 100 fps 250 fps 500 fps

Fill factor (%): 100 50 10 100 50 10 100 50 10Proposed 512 × 512 pixel camera 87 44 8.7 35 17 3.50 17 8.74 1.7

1024 × 1024 pixel camera 22 11 2.2 8.7 4.4 0.87 4.4 2.18 0.444096 × 4096 pixel camera 1.4 0.68 0.14 0.55 0.27 0.054 0.27 0.14 0.027

In the ASOM, the static optical components are deliberately designed to be imperfect (i.e., non-diffraction-limited), but in such a way that the wavefront error is within the stroke and shape forming limits of the deformablemirror. This approach significantly decreases the complexity of the static optical elements while simultaneouslyincreasing the capabilities of the instrument. However, the intimate relationship between the lens geometryand the adaptive optics wavefront shape correcting capabilities require that the coupling between the adaptiveoptics components and the static optical components be explicitly considered during the design and optimizationprocess. During the development of the ASOM, we have developed systematic design techniques12 for effectivelysynthesizing such systems, which are based on the field of Multidisciplinary Design Optimization (MDO).

Proc. of SPIE Vol. 6467 646706-4

3. EXPERIMENTAL ASOM PROTOTYPE

Shown in Figure 4, the purpose of the experimental ASOM apparatus was to demonstrate all essential opticalaspects of the ASOM design, but at low cost and with a short development time. As such, off the shelf opticswere used exclusively to avoid the considerable cost of custom fabricated optics and to take advantage of theexisting stock of catalog available items that ship within days. However, most stock lenses are designed to beused in a particular manner (e.g. with infinite conjugates) for generic applications and are offered in a coarserange of focal distances, lens diameters, and glass selections. Considering the atypical imaging characteristics ofthe scanner lens, the experimental ASOM design using off-the-self optics only is far from optimal, and as such,exhibits a noticeably high lens count to achieve 0.1 NA over a nominal 8mm field size. However, even with thelimitation of using off-the-shelf optics exclusively, this experimental apparatus has been carefully designed todemonstrate all of the critical optical characteristics that define the ASOM.

Object Plane

Steering Mirror

Optics in Motion OIM101 Deformable

Mirror

Boston Micromachines

32 actuator Mini-DM

Digital Camera Pulnix TM200System Aperture

Scanner Lens

12 element design

using off-the-shelf

Thorlabs 2 inch optics

Scanning Kohler Illumination 2X Galvo Steering Mirrors

Cambridge TechnologiesExperimental Setup

Figure 4: Low cost ASOM prototype

This initial prototype utilizes a transmitted lighting scheme with a scanning Kohler illumination stage un-derneath the object. And because the current design is very sensitive to chromatic aberration using only BK-7glass, a 510nm wavelength bandpass filter is included in the illumination stage to eliminate much of the lightbelow 500nm and above 520nm. Light transmits through the object contrast pattern and is then collected by thetelecentric 12 element scanner lens assembly. An electromagnetically actuated fast steering mirror (FSM) witha flexure suspension (Optics in Motion OIM101) has two degrees of freedom to steer the sub-field of view withinthe workspace. Optics project the light onto the MEMS deformable mirror (Boston Micromachines Mini-DM).By precisely controlling the shape of the reflective surface of the mirror to be opposite the shape of the wavefronterror (but at half the amplitude), the deformable mirror can correct for the wavefront aberrations to withinthe diffraction limit. This mirror has 32 electrostatic actuators with 400µm actuator spacing, a 2.5µm actuatorstroke, and a 2.0mm diameter actively controlled area. The 2.5µm stroke is capable of correcting for up toseveral waves of aberration, which allows for high image quality even for the off-axis field positions and enablesthe greatly expanded field of view in the ASOM. Additional optics project the final image onto the CCD camera(Pulnix TM200).

Proc. of SPIE Vol. 6467 646706-5

4. DEFORMABLE MIRROR SHAPE OPTIMIZATION AND CONTROL

Allowing for significant aberration in the scanner lens is a key idea in the ASOM to achieve a wide field of viewat a relatively low cost for the static optical elements (lenses). The specific shape and size of the aberrationdepends on the field position, but it is important that the rate of change in the aberration between field positionsbe sufficiently small as to allow a diffraction limited image over one instantaneous field of view (image exposure)to be obtained. In other words, for one particular field position, there is a deformable mirror shape that cancorrect for the aberrations over an entire individual image tile. But, when the steering mirror changes angle, adifferent deformable mirror shape is required for the new field position associated with using a different regionof the scanner lens. For any given field position, however, the aberrations are static as the glass causing theaberration is stationary. And because the electrostatic MEMS deformable mirrors do not exhibit hysteresis,13

an open loop control strategy can be utilized for the deformable mirror control to achieve high speeds duringoperation. First, the appropriate deformable mirror shape must be determined through a calibration procedureand stored. Then, during operation, the actuator voltages to the deformable mirror, V are reconstructed as afunction of the current field position as determined by the steering mirror angles, θx and θy:

V = f(θx, θy). (1)

0 0.5 1.0 1.5 2.0 2.5 3.0

30

40

50

60

70

80

90

100

110

120

130

Actu

ato

r voltages

Iteration number

e4

Optimized set of 32 actuator

voltages on deformable mirror

at field position x = 0, y = 0

Figure 5: Deformable mirror actuator voltage optimization

In simulation, perfect knowledge of the wavefront shape is readily available and it is straightforward tonumerically determine the required mirror shape to correct for the specific aberrations associated with each fieldposition. However, in practice, manufacturing tolerances and assembly errors introduce unmodeled aberrationsinto the optical system. Furthermore, internal residual stresses combined with manufacturing variations result insurface bowing in the MEMS deformable mirror, as well as actuator performance variation14 . Thus, experimentaluse of the deformable mirror requires a method for the in-situ shape optimization of the deformable mirror surface.

Proc. of SPIE Vol. 6467 646706-6

If we consider the inputs of the optimization to be the deformable mirror actuator voltages, V , and Q to be somemetric of the image quality or residual aberration, then the experimental deformable mirror shape optimizationrequires:

• a metric to represent the image quality or residual aberration, Q(V )• an optimization algorithm to minimize Q(V )

The metric, Q(V ), is in general a nonlinear function of the actuator voltages, V , and Q(V ) is defined todecrease with improving image quality. The resulting optimization problem is also subject to upper and lowerbounds on the actuator voltages and there are many possible image quality metrics and optimization methodsthat can potentially be combined.

In practice, the wavefront cannot be measured directly, but must be inferred from a related measurement oflight intensity. Often, a wavefront sensor (e.g. Shack Hartmann or Pyramid) is used. Other common methodsinclude interferometry or algorithmic techniques such as phase diversity using two or more cameras at differentfocal planes, or possibly one camera that can be moved to different focal planes. For the ASOM, it is advantageousfrom a cost and complexity perspective to use a technique based on the image quality alone, not requiring anyadditional hardware or layout reconfiguration. Various metrics have been proposed for automatically assessingthe quality of an image, including image entropy15 and image sharpness16.

The image quality metric used for the optimization in this research is based on maximizing the energy of thehigh frequency image content and the specific formulation of the metric was obtained from17. First, a highpassfilter was convolved with the acquired image. Then the summation of the absolute value of the filtered resultwas calculated as an indication of the high frequency energy in the image. The result is normalized by the imagebrightness to account for light variations. Given an i × j matrix of intensity values to represent the image, I,the specific image quality metric, Q was:

Q = − 1E

∑i

∑j

| [I ∗ K]i,j |, (2)

where E =∑

i

∑j Ii,j , K =

⎡⎣

0 −1 0−1 4 −10 −1 0

⎤⎦, and ∗ indicates convolution.

Optimization of the image quality metric was performed with the parallel stochastic-gradient-descent (PSGD)algorithm, which is based on estimating the gradient ∇V Q(V ) by applying random perturbations to the inputsignals18. It is applicable to systems where there are many input signals and a performance metric that can bequantified, but no model information is available to relate the performance metric to the inputs. A derivationand justification of the approach can be found in18 and the algorithm found in19 is as follows.

Given an initial set of actuator signals, Vn at time step n, and a corresponding objective function value,Q(Vn):

• Generate a set of equal magnitude, but random sign perturbations, δV , to apply to the actuator controlsignals.

• Apply the perturbations to the system and measure the objective function: Q+n = Q(Vn + δV ).

• Reverse the polarity of the perturbations, apply the negative perturbations to the system, and measurethe objective function: Q−

n = Q(Vn − δV ).

• Update the actuator signals with a gain on the step size, α, as:

Vn+1 = Vn + α(Q+n − Q−

n )δV. (3)Figure 5 shows the evolution of the 32 deformable mirror actuator voltages for each optimization iteration

using the image quality metric (2) and update law (3). All actuators start at 85 Volts, which is slightly less thanhalf the voltage supply. The resulting 32 actuator voltages are shown in a 6 × 6 grid pattern missing the corner

Proc. of SPIE Vol. 6467 646706-7

200100

200100

200100

pixels, as is the configuration of the actual deformable mirror. Based on the image of the USAF 1951 calibrationtarget before and after optimization, it is clear that the image quality improves as the shape of the deformablemirror is optimized. The smallest bar pattern can be clearly resolved in the optimized image and represents aresolution of about 4 microns. The theoretical resolution of the low cost ASOM apparatus is about 3.5 microns(0.1NA at λ = 510µm), which indicates that the aberrations have been reduced to close to or exceeding thediffraction limit.

x1 x2

y1

y2

V00

V01 V11

V10

(9,9)

(9,-9)

(-9,9)

(-9,-9)

(a) (b)

Grid sampling

of observable region

Deformable mirror voltage

interpolation scheme

Figure 6: (a) Deformable mirror is optimized for a grid sampling of the observable region (b) Deformable mirrorvoltage interpolation scheme

Because it is not practical to optimize for every possible field position, we optimize a grid sampling of pointsover the entire observable field of view and use interpolation of the deformable mirror actuator voltages for fieldpositions that lie between data points as shown in Figure 6. Due to the constraint of using off-the-shelf optics inthe low cost ASOM prototype, the magnitude and variation of the aberrations introduced by the scanner lensare relatively small, but significant enough to noticeably degrade the image quality. We are therefore able to usea 5 × 5 grid of points to cover the entire observable field of view as shown in Figure 6, with the correspondingoptimal actuator voltages for each sample point shown in Figure 7. The x and y axis represent the position of thesteering mirror over the range of -9V to 9V, which is the full range of the steering mirror in our setup. Bi-linearinterpolation is used to estimate the optimal set of actuator voltages in between the data points, V00,V01,V10,V11

that include the current operating point:

Vi = V00(1 − x)(1 − y) + V10x(1 − y) + V01(1 − x)y + V11xy, (4)

where x = x1/(x1 + x2), y = y1/(y1 + y2) as defined in Figure 6 and Vi is the set of interpolated voltages.

Figures 8 (a), (b), and (c) show the image located at (0.0, 4.5) with zero volts on all actuators, 85 volts onall actuators, and the set of voltages optimized for field position (0.0, 4.5), respectively. Figure 8 (d) shows theimage located at the same position, but using a set of actuator voltages optimized for a different field position at(4.5, 9.0), which is the upper right data point in this grid region. Applying the voltages optimized for (4.5, 9.0)at field position (0.0, 4.5) results in noticeable image degradation (as can be seen in the the smallest bar pattern,especially in the vertical bars), which indicates aberration variation between these nearby field positions. A newimage was obtained at a field position of (2.25, 6.75) using the interpolation scheme of (4), and it is clear fromlooking at Figure 8 (e) that the interpolated voltages result in nearly the same image quality for a point lyingin between sample points as for a point located at a sample point that was explicitly optimized, such as Figure8 (c).

Figure 9 shows an 11× 11 tile image mosaic of the USAF 1951 calibration target obtained using the interpo-lation scheme at each field position. Note that in the low cost ASOM, there are approximately 3.6 frame heightsand 3.4 frame widths between sample points, requiring significant interpolation at most field positions. Imageregistration is not perfect due to image distortion and a nonlinearity in the steering mirror feedback sensor. These

Proc. of SPIE Vol. 6467 646706-8

•óó4

••#

•ó4•

ó•ó•

(-9,9) (9,9)

(-9,-9) (9,-9)

(0,0)

Figure 7: Deformable mirror voltage calibration data for all 25 sample points

issues will be resolved by using distortion compensation algorithms as described in Section 2 and a nonlinearcompensation in the steering mirror control algorithms for calibrated pointing.

In this low cost prototype, an Optics in Motion OIM101 steering mirror was used. This system has separatex and y axis control of a single 1 inch steering mirror suspended on a flexure mount. We found that thedynamic response of this system using the stock analog controller was inadequate because of long settling timesand significant steady state error. For this reason, we replaced the analog controller with our own digitalcontroller, using XPC target from the MathWorks as the realtime operating system and MATLAB/Simulink asthe development environment. First, we identified the spring constant of the flexure mount and used feedbackto cancel out this disturbance. We then identified the dynamic system model and designed a proportional leadcontroller. As is often the case, the performance with a feedback controller alone is not satisfactory, consideringthe need to maintain sufficient stability margins (gain margin and phase margin). For this reason, we augmentedthe feedback controller with a feedforward controller based on an FIR filter to approximately invert the closedloop dynamics for improved tracking control. The coefficients of the FIR filter were obtained using a fit to resultsfrom Iterative Learning Control (ILC) using methods developed for high speed mirror control11 of semiconductorprocessing equipment. All realtime control runs at a 0.15ms sample rate. Figure 10 shows the dynamic responseof the steering mirror system performing coordinated x y moves for a range of move lengths. The y axis movelengths are in multiples of the frame height, from 1 frame height as would be used for constructing mosaics,to 6 times the frame height, which represents a movement of approximately half the entire observable field ofview (refer to Figure 9 for frame dimensions). The two plots that are zoomed in show the y axis response for amove length equal to one frame height (move to 1.44) and 6 times the frame height (move to 8.64). As can beseen in the zoomed-in plots, the time to complete 1 frame height is approximately 2.5 milliseconds and 6 frameheights is approximately 6 milliseconds. This leaves between 4.0-7.5 ms for image exposure (depending on movelength), indicating that this system should be able to achieve at least 100 frames per second during realtimeoperation. Our current camera system is limited to 30 frames per second. We are in the process of integrating

Proc. of SPIE Vol. 6467 646706-9

6 / 111=, "!INE

iii —J7 uIU_—uii' 111=4III ;; UI. —ItIE*m

6

ii:

0 volts on all actuators

85 volts on all actuators

Optimal voltages

for x = 0.0, y = 4.5

at x = 0.0, y=4.5

show good performance

Optimal voltages

for x = 4.5, y = 9.0

at x = 0.0, y = 4.5

show degraded performance

Interpolated voltages

for x = 2.25, y =6.75

show good performance

(0,0)

(-9,-9)

(-9,9) (9,9)

(9,-9)

(a)

(b)

(c) (d)

(e)

Total scannable area

Figure 8: Interpolation results for an intermediate field position

0.3

2m

m475 p

ixels

0.45mm 768 pixels

8488 pixels

5225 p

ixels

Figure 9: 11 × 11 tile image mosaic of a USAF 1951 calibration target

Proc. of SPIE Vol. 6467 646706-10

a high speed camera using the Camera Link data transfer interface to achieve 100 frames per second operationwith the steering mirror settled at each field position, and even faster rates if the steering mirror is not requiredto stop.

0.0 0.4 0.8 1.2-4

-2

0

2

4

6

8

10

Fie

ld P

ositio

n

Time (sec)

Two axis steering mirror

-2 -1 0 1 2 3 4 5-0.20

0.20.40.60.81.01.21.41.6

Fie

ld P

ositio

n

Time (ms)-2 -1 0 1 2 3 4 5 6 7

-10123456789

Fie

ld P

ositio

n

Time (ms)

8

y axis

x axis

Figure 10: Experimental trajectory plots of the high speed steering mirror

5. SUMMARY AND CONCLUSIONSBy replacing optical complexity with active optical components, we have developed a new microscope withunique imaging capabilities, called the Adaptive Scanning Optical Microscope (ASOM). Applications in biology,medicine and industry will benefit from the rapid scanning capabilities that enable observation of challengingspatial-temporal events over a large field of view without any agitation to the specimen. In the ASOM, a MEMSdeformable mirror works with a scanning mirror and custom designed scanner lens to direct the location of thefield of view. A low cost proof of concept prototype demonstrates all of the critical optical aspects of the design,but at a compromised resolution and field size. After calibrating the deformable mirror using an image basedoptimization metric for a grid sampling of operating points over the entire observable field of view, we havesuccessfully demonstrated an open loop (feedforward) control of the deformable mirror using interpolation of thepreviously optimized neighboring data points.

We are now working on developing the follow-on prototype that will make use of custom fabricated opticsand a higher performance deformable mirror to fully realize the potential of this new microscope design. As theASOM, and other systems20, 21 are emerging that use the adaptive optics as a critical and integral component ofthe imaging path to compensate for aberrations inherent to the imaging system itself, there will be an increasedneed for wavefront sensorless calibration and control of adaptive optics systems. While the methods describedin this paper demonstrate the basic concept in a fully functional prototype, we will continue to investigatealternative image based metrics and optimization algorithms in support of systems like the ASOM.

ACKNOWLEDGMENTSThis material is based in part upon work supported by the National Science Foundation under Grant No. CMS-0301827 and by the Center for Automation Technologies and Systems (CATS) under a block grant from theNew York State Office of Science, Technology, and Academic Research (NYSTAR). J.W. is also supported inpart by the NSFC two-bases project (No. 60440420130), and the Outstanding Overseas Chinese Scholars Fundof Chinese Academy of Sciences (No. 2005-1-11), China.

Proc. of SPIE Vol. 6467 646706-11

REFERENCES1. B. Potsaid, Y. Bellouard, and J. T. Wen, “Adaptive scanning optical microscope (asom): A multidisciplinary

optical microscope design for large field of view and high resolution imaging,” Optics Express 13(17),pp. 6504–6518, 2005.

2. B. Potsaid, Y. Bellouard, and J. T. Wen, “Adaptive scanning optical microscope (ASOM) for large workspacemicro-robotic applications,” in Proceedings of IEEE International Conference on Robotics and Automation(ICRA), pp. 1024–1029. (Institute of Electrical and Electronics Engineers, New York, 2006).

3. B. Potsaid, F. P. Finger, and J. T. Wen, “Automation of challenging spatial-temporal biomedical obser-vations with the adaptive scanning optical microscope (asom),” in Automation Science and Engineering,2005. IEEE International Conference on, pp. 37–42, 2006.

4. R. E. Fischer and B. Tadix-Galeb, Optical system design, McGraw-Hill, 2000.5. J. Tesar, “Using small glass catalogs,” Optical Engineering 39(7), 2000.6. D. Weiner, “Design considerations for optical scanning,” in Proc. of SPIE Vol. 3131, pp. 59–64, 1997.7. J. Montagu, “Two-axis beam steering systems, TABS,” in Proc. of SPIE Vol. 1920, pp. 162–172, 1993.8. T. Bifano, J. Perreault, P. Bierden, and C. Dimas, “Micromachined deformable mirrors for adaptive optics,”

in High-resolution wavefront control: methods, devices, and applications IV, (Proc. SPIE 4825), pp. 10–13,2002.

9. R. Shahidi, M. R. Bax, C. R. Maurer, J. A. Johnson, E. P. Wilkinson, B. Wang, J. B. West, M. J. Citardi,K. H. Manwaring, and R. Khadem, “Implementation, calibration and accuracy testing of an image-enhancedendoscopy system,” Medical Imaging, IEEE Transactions on 21(12), pp. 1524–1535, 2002.

10. S. Shah and J. Aggarwal, “A simple calibration procedure for fish-eye (high distortion) lens camera,” inRobotics and Automation, 1994. Proceedings., 1994 IEEE International Conference on, 4, pp. 3422–3427,(San Diego, CA), May 1994.

11. B. Potsaid and J. T. Wen, “High performance motion tracking control,” in Proceedings of IEEE Conferenceon Control Applications, pp. 718–723. (Institute of Electrical and Electronics Engineers, New York, 2004).

12. B. Potsaid, Y. Bellouard, and J. T.-Y. Wen, “A multidisciplinary design and optimization methodology forthe adaptive scanning optical microscope (asom),” in Novel Optical Systems Design and Optimization IX,pp. 62890L–1–62890L–12, 2006.

13. Y. Zhou and T. Bifano, “Characterization of contour shapes achievable with a mems deformable mirror,”MEMS/MOEMS Components and Their Applications III 6113(1), p. 61130H, SPIE, 2006.

14. J. W. Evans, K. Morzinski, S. Severson, L. Poyneer, B. Macintosh, D. Dillon, L. Reza, D. Gavel, D. Palmer,S. Olivier, and P. Bierden, “Extreme adaptive optics testbed: performance and characterization of a 1024-mems deformable mirror,” MEMS/MOEMS Components and Their Applications III 6113(1), p. 61130I,SPIE, 2006.

15. M. B. Garvin, M. T. Gruneisen, R. C. Dymale, and J. R. Rotge, “Image entropy as a metric for iterativeoptimization of large aberrations,” Advanced Wavefront Control: Methods, Devices, and Applications III5894(1), p. 58940Y, SPIE, 2005.

16. L. P. Murray, J. C. Dainty, J. Coignus, and F. Felberer, “Wavefront correction of extended objects throughimage sharpness maximisation,” 5th International Workshop on Adaptive Optics for Industry and Medicine6018(1), p. 60181A, SPIE, 2005.

17. M. Cohen, G. Cauwenberghs, and M. A. Vorontsov, “Image sharpness and beam focus vlsi sensors foradaptive optics,” IEEE Sensors Journal 2(6), pp. 680–690, 2002.

18. M. A. Vorontsov and V. P. Sivokon, “Stochastic parallel-gradient-descent techniques for high-resolutionwave-front phase-distortion correction,” Optical society of america A 15, pp. 2745–2758, Oct. 1998.

19. G. Reimann, J. Perreault, P. Bierden, and T. Bifano, “Compact adaptive optical compensation systemsusing continuous silicon deformable mirrors,” in High-resolution wavefront control: methods, devices, andapplications III, (Proc. SPIE 4493), pp. 35–40, 2001.

20. M. T. Gruneisen, M. B. Garvin, R. C. Dymale, and J. R. Rotge, “Agile-field telescope system with diffractivewavefront control,” in Proc. SPIE, Advanced Wavefront Control: Methods, Devices, and Applications III,pp. 589414–1–589414–6, 2005.

21. D. V. Wick and T. Martinez, “Adaptive optical zoom,” Optical Engineering 43, pp. 8–9, Feb. 2004.

Proc. of SPIE Vol. 6467 646706-12