Toward a standard for remote microscope control systems

7
SCANNING VOL. 20, 110–116 (1998) Received November 25, 1997 © FAMS, Inc. Accepted January 29, 1998 Summary: Remote access to scientific instruments provides a way to share a variety of valuable resources with a world- wide audience. By pooling these resources and providing a common network and user interfaces to these resources, sci- ence researchers and educators will have capabilities that no one institution could afford. The Microscope And Graphic Imaging Center (MAGIC) at California State University, Hayward (CSU Hayward) is developing a model for such a system. We are developing model software for interactive re- mote-shared access to an unmodified Philips XL 40 scanning electron microscope (SEM). Located within the Department of Biological Sciences, MAGIC provides facilities and exper- tise in specimen preparation as well as providing remote ac- cess to researchers and educators over the Internet. We have used a wide range of networking technologies, including mo- dem, ISDN, Ethernet, T1, and ATM to control the SEM, and a wide range of image transmission technologies, including closed circuit TV, compressed video over ATM, and digital imaging (Smith et al. 1996a,b,c). We have taken a modular approach to software control which allows us to arrange key pieces in a variety of ways on a number of different platforms. Results indicate that under the correct conditions, access to expensive and sensitive commercially available scientific equipment can be shared by large research, educational, and industrial communities. However, care must be taken to ad- dress issues of security, robustness, and performance. Plans are to incorporate remote access into university, community college, and high school use starting in 1998. Based upon this experience, we propose a framework for standardizing re- mote access to visual scientific equipment. This framework consists of three layers: (1) clients, (2) servers, and (3) instru- ments. This approach will allow a wide variety of equipment to be shared in a flexible and uniform manner. Key words: telemicroscopy, telepresence microscopy, re- mote operations, microscopy Introduction Computer-controlled visual scientific instrumentation is an exciting example of a shareable digital resource. With the ad- vent of the Internet, digital resources can be shared through- out the entire earth. Media coverage of NASA’s control of the Pathfinder Mars probe attests to a high level of public interest in access to such resources, even those on other planets. Remote microscopy is an active area of research and devel- opment, with work taking place at national laboratories, su- percomputer centers, and several universities. Microscope manufacturers such as Philips, JEOL, Topcon, LEO, and R. J. Lee recognize its importance and are developing computer interfaces to their instruments. Remote access to microscopes goes by a number of names. Telemicroscopy is a convenient term attributed to Mark H. El- lisman’s group at the National Center for Microscopy and Imaging Research at the University of California at San Diego (Fan et al. 1993). TelePresence Microscopy is used by Zaluzec at Argonne National Laboratory (Zaluzec et al. 1996). Remote access was demonstrated as early as the SIG- GRAPH 1992 conference in Chicago and the Supercomput- ing 92 conference in Minneapolis, connecting with an inter- mediate voltage electron microscope in San Diego (Fan et al. 1993). At Argonne National Laboratory’s Material Science Division, Zaluzec (1996) has set up a web site (http://www.amc.anl.gov) and CU-SeeMe reflector that demonstrates and discusses the idea of TelePresence Mi- croscopy. Edgar Voelkl (1996a,b 1997), Lawrence Allard, and others at Oak Ridge National Laboratory’s High Temper- ature Materials Facility are working with Al Geist of Oak Ridge’s Computer Science and Mathematics Division to de- velop custom and web-based software for remote control of several types of electron microscopes, for example; http://www.epm.ornl.gov/~geist/ java/applets/uscope/ and http://www.ccs.ornl.gov/SC95/ Voelkl.html. At Lawrence This study was supported by the following grants: Pacific Bell: External Technology Program. National Science Foundation: 9451318 Tullis, Smith, and Fegan, “An Interactive Science Curriculum Using Computer Networking and Scanning Electron Microscopy,” Undergraduate Instru- mentation & Laboratory Improvement Program. CSU DELTA program: Distributed Learning Resources Project. Address for reprints: Christopher L. Morgan Department of Mathematics and Computer Science California State University, Hayward Hayward, CA 94542, USA Toward a Standard for Remote Microscope Control Systems CHRISTOPHER MORGAN, DON P ARDOE,* NANCY SMITH* Department of Mathematics and Computer Science, and * Department of Biological Sciences, California State University, Hayward, California, USA

Transcript of Toward a standard for remote microscope control systems

Page 1: Toward a standard for remote microscope control systems

SCANNING VOL. 20, 110–116 (1998) Received November 25, 1997© FAMS, Inc. Accepted January 29, 1998

Summary: Remote access to scientific instruments providesa way to share a variety of valuable resources with a world-wide audience. By pooling these resources and providing acommon network and user interfaces to these resources, sci-ence researchers and educators will have capabilities that noone institution could afford. The Microscope And GraphicImaging Center (MAGIC) at California State University,Hayward (CSU Hayward) is developing a model for such asystem. We are developing model software for interactive re-mote-shared access to an unmodified Philips XL 40 scanningelectron microscope (SEM). Located within the Departmentof Biological Sciences, MAGIC provides facilities and exper-tise in specimen preparation as well as providing remote ac-cess to researchers and educators over the Internet. We haveused a wide range of networking technologies, including mo-dem, ISDN, Ethernet, T1, and ATM to control the SEM, anda wide range of image transmission technologies, includingclosed circuit TV, compressed video over ATM, and digitalimaging (Smith et al. 1996a,b,c). We have taken a modularapproach to software control which allows us to arrange keypieces in a variety of ways on a number of different platforms.Results indicate that under the correct conditions, access toexpensive and sensitive commercially available scientificequipment can be shared by large research, educational, andindustrial communities. However, care must be taken to ad-dress issues of security, robustness, and performance. Plansare to incorporate remote access into university, communitycollege, and high school use starting in 1998. Based upon thisexperience, we propose a framework for standardizing re-mote access to visual scientific equipment. This framework

consists of three layers: (1) clients, (2) servers, and (3) instru-ments. This approach will allow a wide variety of equipmentto be shared in a flexible and uniform manner.

Key words: telemicroscopy, telepresence microscopy, re-mote operations, microscopy

Introduction

Computer-controlled visual scientific instrumentation is anexciting example of a shareable digital resource. With the ad-vent of the Internet, digital resources can be shared through-out the entire earth. Media coverage of NASA’s control of thePathfinder Mars probe attests to a high level of public interestin access to such resources, even those on other planets.

Remote microscopy is an active area of research and devel-opment, with work taking place at national laboratories, su-percomputer centers, and several universities. Microscopemanufacturers such as Philips, JEOL, Topcon, LEO, and R. J.Lee recognize its importance and are developing computerinterfaces to their instruments.

Remote access to microscopes goes by a number of names.Telemicroscopy is a convenient term attributed to Mark H. El-lisman’s group at the National Center for Microscopy andImaging Research at the University of California at SanDiego (Fan et al. 1993). TelePresence Microscopy is used byZaluzec at Argonne National Laboratory (Zaluzec et al.1996).

Remote access was demonstrated as early as the SIG-GRAPH 1992 conference in Chicago and the Supercomput-ing 92 conference in Minneapolis, connecting with an inter-mediate voltage electron microscope in San Diego (Fan et al. 1993). At Argonne National Laboratory’s Material Science Division, Zaluzec (1996) has set up a web site(http://www.amc.anl.gov) and CU-SeeMe reflector thatdemonstrates and discusses the idea of TelePresence Mi-croscopy. Edgar Voelkl (1996a,b 1997), Lawrence Allard,and others at Oak Ridge National Laboratory’s High Temper-ature Materials Facility are working with Al Geist of OakRidge’s Computer Science and Mathematics Division to de-velop custom and web-based software for remote control ofseveral types of electron microscopes, for example;http://www.epm.ornl.gov/~geist/ java/applets/uscope/ andhttp://www.ccs.ornl.gov/SC95/ Voelkl.html. At Lawrence

This study was supported by the following grants: Pacific Bell: ExternalTechnology Program. National Science Foundation: 9451318 Tullis,Smith, and Fegan, “An Interactive Science Curriculum Using ComputerNetworking and Scanning Electron Microscopy,” Undergraduate Instru-mentation & Laboratory Improvement Program. CSU DELTA program:Distributed Learning Resources Project.

Address for reprints:

Christopher L. MorganDepartment of Mathematics and Computer ScienceCalifornia State University, HaywardHayward, CA 94542, USA

Toward a Standard for Remote Microscope Control Systems

CHRISTOPHER MORGAN, DON PARDOE,* NANCY SMITH*

Department of Mathematics and Computer Science, and * Department of Biological Sciences, California State University,Hayward, California, USA

Page 2: Toward a standard for remote microscope control systems

Berkeley National Laboratory’s National Center for ElectronMicroscopy (NCEM), Bahram Parvin, Michael O’Keefe, and others are developing remote access software for a high-voltage electron microscope (O’Keefe 1996 andhttp://ncem.lbl.gov/ncem.html). LBNL’s NCEM offers work-shops on microscopy that include remote operations with theHF-2000 at Oak Ridge (http://ncem.lbl.gov/ ncem.html)(summer 1997). A consortium called the Materials Mi-crocharacterization Collaboratory (see http://tpm.amc.anl.gov/MMC) has formed to develop a system for shared ac-cess to microscopes. Participants include scientists from OakRidge, Argonne, Pacific Northwest, and Lawrence BerkeleyNational Laboratories together with representatives from theUniversity of Illinois and from the National Institute of Stan-dards and Technology, and with industrial representativesfrom Jeol, Philips, Gatan, and R. J. Lee. In the fall of 1997,they held a workshop in Oak Ridge National Laboratory todevelop an architecture for remote access to electron micro-scopes; two of the authors attended this workshop.

The World Wide Laboratory (WWL) at the University ofIllinois’ Beckman Institute, with the Biomedical MagneticResonance Laboratory and the National Center for Super-computing Applications, has online a transmission electronmicroscope and a magnetic resonance imaging system (seehttp://vizlab.beckman.uiuc.edu/WWL/).

Universities such as Iowa State (Chumbley et al. 1995,1996), the University of Michigan (Mansfield 1996, and see:http://emalwww.engin.umich.edu/people/jfmjfm/tsem/imp.html), San Diego State University (http://www.sci.sdsu.edu/EM_Facility), and Arizona State (Razdan 1997, and see:http://invsee.eas.asu.edu/) are integrating remote control ofmicroscopes into their curriculum.

The advantages to shared remote access to instrumentssuch as electron microscopes are well documented (Fan et al.1993). Remote access opens specialized equipment to a muchlarger audience. Many institutions cannot afford even a singleinstrument, few institutions can afford a full complement ofinstruments, and even for institutions with necessary equip-ment, direct access is usually limited because microscopesare often located in cramped quarters.

Remote shared access to visual instrumentation providesspecial challenges for two reasons: (1) image transmission islimited by the bandwidth of the communications channel, and(2) much of this instrumentation was designed to be used byexperts. Specific challenges include (1) anticipating and re-stricting access to undesirable actions, (2) navigating and fo-cusing through a wide range of magnifications under restrict-ed bandwidths, and (3) dealing with unpredictabletransmission delays.

Commercially available remote control software such asPC Anywhere® and Timbuktu® allows a remote user to takecontrol of an entire computer. This gives direct access to ven-dor-supplied software which is often complex and not de-signed for general public use. Some commands can actuallyput the instrument into unacceptable situations, such as vent-ing a vacuum system or even crashing or locking up the in-strument and/or the controlling computer.

C. Morgan et al.: Toward a standard for remote microscope control 111

Internet-based video conferencing systems such as CU-SeeMe (see: http://cu-seeme.cornell.edu/) provide a popularway to distribute images digitally, but at a loss of quality and alack of integration with the control software.

Developing custom software provides a way to coordinatethe number and time of users and the commands they can use,tailor use to different levels of expertise, and allow users totrade image quality for navigational efficiency.

We have developed a model with demonstration softwarefor remote control of a Philips XL 40 scanning electron mi-croscope (SEM). This model, termed interactive remoteshared access to visual and analytic scientific instrumentation(IRSA/VASI), uses a minimum amount of custom software atthe equipment site, but provides remote users access throughstandard computers and communications links that are af-fordable by and in place at most research and education sites.The IRSA model is adaptable to a wide range of bandwidths.It works well at low bandwidths, but increases in capability asmore bandwidth becomes available. IRSA provides uniformaccess for multiple users to multiple resources, including avariety of types of microscopes and analytic scientific equip-ment. Plans include developing software for a scanning probemicroscope (SPM) and a laser scanning confocal microscope(LSCM).

Materials and Methods

We are implementing IRSA as a distributed client/server/resource system. The model is to have multiple clients con-nect via the World Wide Web to a system of servers that putsclients in control of the instrument they select. The server co-ordinates user authentication, filters and channels user com-mands to the selected instrument, and integrates imaging withcontrol (Fig. 1).

The Network Connection

This model depends upon the fact that each instrument canbe connected to a network. In the case of the Philips XL 40

FIG. 1 The basic architecture: Clients, server system, and instruments.

Client

Client

Client

Client

Client

Server

Instrument

Instrument

Instrument

Instrument

Instrument

Instrument

Server

Page 3: Toward a standard for remote microscope control systems

112 Scanning Vol. 20, 2 (1998)

SEM, an MS Windows-based computer (PC) controls the ac-tual instrument through a SCSI cable. This PC can be con-nected to a network such as an Ethernet Local Area Networkvia commonly available methods. A vendor-supplied appli-cation program on the control PC calls a vendor-supplied dy-namic link library (DLL) which causes commands to be is-sued over the SCSI cable to control the instrument. Philipssupplied us with a description of the functions in their DLL(called XCLLIB). With this information and a spirit of adven-ture, we developed a small client/server system for effectiveplacement of the DLL functions at a remote computer. Wecall this approach DLL tunneling because the network acts asa tunnel between a DLL on the remote client and the DLL onthe server. In general, vendors supply either the DLL or theclient/server software.

The tunneling arrangement that we use gives quite a bit offlexibility. Our remote client software can run remotely or lo-cally (directly on the control PC), and we can place manage-ment software in between client and server as a kind of trafficcheckpoint in the middle of the tunnel. This approach makesdevelopment and testing easier and allows for selected imple-mentation and availability of functionality.

We implemented the client/server system (the tunnel) inthe customary way as a set of remote procedure calls (RPC).An RPC is a function that is invoked (called) on one machine(the client) and is performed on another machine (the server).The network facilitates the transfer of the function’s parame-ters from client to server and the function’s return value fromserver to client. In our case, we wanted to make the machineaccessible from anywhere in the world, so we chose the Inter-net TCP/IP suite of protocols as the lower-level transportmechanism. We chose to implement our own RPC protocolso that we would have maximum platform/vendor indepen-dence, but other RPC protocols (Sun XDR, Microsoft COM,Java RMI, CORBA) are quite feasible and are under consid-eration. Our next approach will be to use CORBA to be con-sistent with efforts at the National Laboratories.

The Client

Our first client software program was designed for use bypeople who are not expert with the instrument. It has controlsto turn the beam on and off, to change the magnification, to setthe beam probe diameter (spot size), to set the image quality,and to move the stage. It is written in C++ for the 16-bit ver-sion of the Windows Application Program Interface (API).We have developed a Java applet that will run from a WorldWide Web page. Figure 2 shows its user interface. Advancedclients with more capabilities will be developed as needed.We are working with microscopists to develop the featuresthat are needed.

Management Software

Management software coordinates user access and reducesdemand on the actual instrumentation. It serves as an interme-

diary between the remote users and the actual instrumenta-tion. It provides authentication of users, coordination amongusers, support for imaging, and integration of imaging withcontrol. It also serves as a command switch to direct com-mands between remote users and various instruments. Wefound that it was easy for this server to separate remote usercommands into a stream of pure control functions and astream of pure image capture commands. The first streamgoes directly to the control PC for the instrument and the sec-ond stream goes to the image capture system. The manage-ment software that separated them easily synchronizes the re-sults from these two streams.

Imaging

Imaging can be done in several ways. The instrument itselfgenerates a video signal that goes to a closed circuit TV sys-tem. This signal drives TV monitors in classrooms, laborato-ries, and offices on campus and can be fed to the local com-munity cable channel and to compressed video for distancelearning classes. We now capture images from this closed cir-cuit video signal as part of the IRSA software system.

Initially, we used the image capture capabilities built intothe microscope. However, capturing an image in this way andshipping it through the control PC produced an anticipatedbottleneck (producing a 2.5 s delay), so we set up a separatevideo capture server on a Sun SPARCstation using a Sunvideo adapter. This system can capture video signals in vari-ous formats including NTSC and can produce digitized im-ages in the form of single frames or sequences of frames informats including raw data, JPEG, and MPEG. It can bufferseveral frames to increase throughput, it can perform opera-tions on images such as convolution and amplitude scaling,and it can synchronize image-processing operations.

We used Sun’s XIL image processing library, which con-sists of an extensive collection of image capture and imageprocessing functions. XIL is well documented XIL (1994a),XIL (1994b), and Pratt (1997). Its functions control all of theabove (and more) and can be accessed by C, C++, Java, etc.,

FIG. 2 The client user interface.

Page 4: Toward a standard for remote microscope control systems

programs running on the Sun workstation. Although we arecurrently handling imaging on the same machine as manage-ment software, imaging could be placed on a separate com-puter (as in Fig. 1).

The User Interface and the Bandwidth

Operators of a typical SEM are accustomed to TV rate vi-sual feedback, allowing them to focus and move the sample inreal time. Remote users who have closed circuit TV or highbandwidth digital networks can enjoy this same approach.Users with lower bandwidth networks (even 10 mbps Ether-net) cannot.

We designed our system to use a kind of dead reckoning toallow users to locate the desired position in the sample. In ef-fect, users travel in three dimensions: x, y for the stage posi-tion and z for magnification. When users are seeking a newposition/magnification, the software presents a dialog with asufficiently large still image view (384 ×384 pixels) that userscan see where they are. Users with direct video or high speeddigital networks can glance at bigger images at this point.They can either use a mouse to click on a new (x, y) point inthe image or they can select a new magnification. The soft-ware sends the appropriate commands to the instrument tocenter on the new position or change the magnification, andupdates the image and data values on the user screen. Thisworks well up to about 2,000 magnification. With highermagnifications, beam shifting should be used. An importantconsideration is that the image only changes when the user se-lects a new position/magnification, and thus does not needconstant updating. This is reasonable because of the static na-ture of SEM samples. With other sample conditions, constantupdating may be needed.

To reduce image bandwidth requirements, we needed toreduce the amount of data associated with each image. Thiscan be done by compressing the image data or by reducingthe quality of the image. The popular CU-SeeMe softwaredeveloped at Cornell University (see http://cu-seeme.cor-nell.edu/) uses a variety of techniques to reduce bandwidth re-quirements for images. JPEG (ISO/IEC JTC1 SC29 1994) isa well-known standard for image compression which com-presses images with an optional, user-selectable reduction inimage quality. MPEG (ISO/IEC JTC1 SC29 1993-) is a seriesof standard formats for image streams that builds on JPEGand allows for differential image compression over time. Asimple form of wavelet analysis provides an efficient way todo this that is acceptable for imaging while a user is navigat-ing. With this method, the intensities for a block of pixels areaveraged and sent as one number. At the other end, this onevalue is replicated over the corresponding block of pixels atthe destination. At LBNL, Parvin et al. (1996) are usingwavelet analysis to compress images. Our remote softwareprovides 1:1, 2:1, 4:1, and 8:1 pixel averaging. Since imagesare two-dimensional, the transmission time is reduced qua-dratically, yielding a 64-fold increase in speed for our lowestimage quality. Once remote users locate areas of interest, theycan slow down and examine detail. This can happen automat-

C. Morgan et al.: Toward a standard for remote microscope control 113

ically with a gradual unfolding of detail as an image is dis-played. Users can even download a 768 × 484 image that is1:1. This image can be saved and printed out. The full imagecontains 371,712 bytes. Without compression, this full imagetakes over a minute to download over a fast modem. In con-trast, the 384 ×384 navigational image with 8:1 pixel averag-ing consists of 2304 bytes and requires less than a second totransmit over a modem. Alternatively, with the Sun imagingsystem we provide the more compressed JPEG images forviewing over WWW at the same time as users are using oursoftware for navigation.

Focusing

Focusing a visual instrument remotely presents a specialchallenge. A local operator normally scans through a largenumber of images while gradually changing the focus to de-termine the best value for the focal distance. Alternatively, theoperator may invoke an automatic focusing procedure such asis available on the Philips XL 40. However, this has provedproblematical for remote use and for on-site users. We areworking with Philips to get better control of this procedure.

Even with high bandwidth imaging, it may not be possibleto vary the focus as smoothly as a local operator would. Wehave developed a method for focusing under low bandwidths(Fig. 2). The remote user opens up a dialog with five small im-ages (60 ×90 pixels) at different values for the focal distance.The user acts as an oracle, choosing the best image in a se-quence of steps until the best focus is obtained. The gap be-tween focal distances is a uniform number inversely propor-tional to the magnification. If there is no clear winner, the usershould double the gap. Selecting an image at the extreme endsretains the gap size, but moves the focal values so that the se-

FIG. 3 The focusing dialog.

Page 5: Toward a standard for remote microscope control systems

lected image now appears in the center. Selecting one of thethree middle positions divides the gap in half and centers thechoices at the selected value.

Using this system, we are able to focus on images that havea significant depth of field throughout a wide range of magni-fications.

Congestion and Delay

Ideally there should be no perceivable delay between thetime a user issues a command and the time that the user seesthe result of the command. On a real data network, communi-cation speeds are limited and often irregular, creating notice-able and unpredictable delays. The irregularity is due to irreg-ular input from many users and to the resulting networkcongestion.

Network congestion is a phenomenon that is analogous tovehicular congestion on a transportation network. Like atransportation network, a data network consists of links(roads) and nodes (interchanges) that are implemented as spe-cial purpose computers called gateways. Gateways that directtraffic among several paths are often called routers. Some pre-dominant kinds of networks, namely local area networks suchas Ethernet, act like gateways. They consist of a single chan-nel fed by several competing sources that contend for use ofthe channel.

Data travel in units called frames or cells that act somewhatlike vehicles. Each frame contains information logically orga-nized into what is called a packet (the payload). Thus thepackets are like the passengers and freight while the framesare like the vehicles.

Different types of information generate different demandson the network, even greater than the difference between a bi-cycle and an 18-wheel truck. For example, a person typingcontrol commands can generate several tens of bits per sec-ond, but a sequence of images can require several million bitsper second. Often network traffic comes in bursts as userssend chunks of data such as images.

To get from source to destination, information may travelacross several links. If networks were not shared, then thespeed at which information travels over the entire trip wouldbe determined by the speed at which it travels over the slowestlink. However, networks generally carry traffic from manyusers at a time, each producing an irregular flow of informa-tion. Just like a transportation network, sharing creates thepossibility of congestion.

The analogy to a transportation network breaks down inthe way congestion is handled. On a data network, once thetraffic is admitted to a data link, it tends to move at a constantrate that is determined by the nature of the link (ordinary tele-phone lines at tens of thousands of bits per second, Ethernet attens of millions of bits per second, or optical fibers at speedsup to tens of billions of bits per second). The congestion of adata network instead occurs at the nodes. Information mayhave to wait at the gateway or entrance to a shared communi-cation channel for other traffic before it can proceed. Thesebottlenecks often occur where local area networks join the In-

ternet at a service provider. When a packet waits, it is stored inmemory (buffered) until it can get to the next segment of itsjourney. This produces irregular and often lengthy delays. Ifthe traffic demand exceeds the memory capacity of the gate-way, information may be discarded. In a transportation net-work, excess traffic tends to be stored on the road itself, fillingup available space as gaps narrow between vehicles, until traf-fic stops and there is no more room for vehicles to enter. Thisis the well-known “highway becomes parking lot” syndrome.On the other hand, when the node on data network runs out ofmemory, it discards packets, a strategy that is looked uponvery unfavorably for transportation networks.

Congestion causes a real challenge for the design of net-worked interface for remote operation of scientific equip-ment. The temptation is to send a command several timeswhen no response is received from the instrument. This tendsto make the situation worse. It adds more traffic and maycause duplicate commands to be received by the instrument.One precaution against the effects of duplicate commands isto require that every command is idempotent, that is, execut-ing it once is equivalent to executing it several times. For ex-ample, commanding an instrument to move so many millime-ters in a certain direction is not idempotent; however,commanding it to move to a specified location is. There canbe some difficulty in determining when the instrument is ac-tually at a specified location, especially if floating point num-bers are used to specify position and when measurement ofposition is subject to error and does not yield repeatable re-sults. For our SEM, the vendor-supplied command to movethe stage to a specified location first performs a backlash op-eration that moves the stage some distance away from the lo-cation and then back again toward the target. Backlash is de-signed to move the system to a known state (to compensatefor mechanical hysteresis, a lagging of a physical system be-hind its controls) and then bring it to the target position in amore predictable manner. This might be considered to beidempotent, but it has undesirable side effects, at least formoving relatively small distances. Our system requires a setof idempotent commands without such side effects.

Web Serving

One issue that we encountered was delivering a view ofone person’s actions to a large number of people. We foundthe World Wide Web to be a convenient way to do this. TheSun workstation handles both a web server and our manage-ment program. We added some code to the management pro-gram on the Sun to produce a JPEG file of the image eachtime the client called for a new image, and we developed asimple common gateway protocol (CGI) program (written inC) that would push a new copy of the image each time the im-age changed its time and date. A person could view the cur-rent image with a web browser that attaches to the URL of theCGI program. The only connection between the IRSAclient/server and the web is through the operating system’sfile system.

114 Scanning Vol. 20, 2 (1998)

Page 6: Toward a standard for remote microscope control systems

Results

Our project has demonstrated the feasibility and the chal-lenges of sharing high-quality visual scientific instrumenta-tion over the Internet. We have developed a model for re-mote access of visual scientific equipment that can be usedby industry to facilitate uniform access by a wide variety ofscientific instrumentation. It merely requires that each man-ufacturer supply a DLL or RPC network interface to theirsystem.

We are now in the process of deploying the IRSA/VASItechnology to neighboring institutions, including other CSUcampuses and nearby community colleges and high schools.We are working with other CSU campuses in Fresno andStanislaus to include other instruments in the IRSA modelwith the intention of creating a unified network of micro-scopes within the CSU system. Starting in 1998, we plan tohave students at Ohlone Community College in Fremont,California, and Jefferson High School in Daly City, Califor-nia, access the SEM as part of their science classes. We havealready begun experimental use from these sites.

Discussion

As a result of our experience, we propose an architecturalframework for a standard way to interface visual scientific in-struments to the Internet. This model applies to a wide varietyof instruments including electron and optical microscopes aswell as telescopes and even particle beam accelerators and de-tectors (Wright 1997).

We propose a simple architecture consisting of clients,servers, and instruments. Clients provide remote users with auser interface to the instrumentation. Servers provide a way toisolate clients from device and network protocol dependen-cies and to authenticate and schedule users. Instruments pro-vide the actual images and manipulation of specimens andtheir viewing.

An instrument can be any device that produces an image,either still images on demand or a stream of images, and thatcan be controlled to set a viewing position and viewing para-meters such as magnification, focus, brightness, contrast, andimage quality. In this system, each instrument has a networkinterface consisting of a set of capabilities. Particular require-ments for each instrument are: (1) a network connection overwhich images and control commands can be sent; (2) an im-age production system that produces digital images, eitherstill or in a stream; and (3) a command system that positionsthe instrument’s view of a specimen or target, and changes pa-rameters such as magnification, focus, brightness, contrast,and image quality.

The interface to each instrument can be in the form of aDLL or a set of RPCs. The RPC interface must be one of sev-eral types that are now available including Sun RPC, COR-BA, RMI, and COM. A DLL interface is connected to one ofthese network interfaces. This interface offers a set of capabil-ities (positioning, magnification, focusing, etc.). Each capa-

bility has the possibility of “get” and “set” operations and willbe guarded by a boolean variable that will tell if it is available.Parvin (1997) has suggested a “software bus” that providesthese capabilities in an object-oriented fashion. Some capa-bilities will have domains of valid values associated withthem. For example, position may have a range of values for xand y, and magnification may have a minimum and maxi-mum value. Details of how this should be done are under de-velopment through the MMC Consortium. MAGIC is a part-ner in this development.

We propose a server system that interfaces between clientsoftware and the instruments. The simplest example would bea single server that receives requests from client software andpasses them on to a selected instrument. A more complicatedsituation would be to have clients connect to their local serverwhich would patch them through to an instrument at a remotesite. Servers would provide a variety of services including (1)translation from one type of protocol to another, (2) user au-thentication, and (3) scheduling.

We propose that clients communicate to servers via a sim-ple RPC protocol. Candidates are Sun RPC, CORBA, RMI,and COM. Another possibility is HTML.

Acknowledgments

The authors are grateful to Pacific Bell, the National Sci-ence Foundation, and the CSU DELTA program for fundingand to CSU Hayward’s Instructional Computing Services, In-structional Media Services, and the Department of Mathe-matics and Computer Science for data and video networkingsupport. They also wish to thank Computer Science studentsDennis Andersen, Rocky Hillyard, Sonja Ellefson, and TimTreem.

References

Chumbley LS, Meyer M, Fredrickson K, Laabs FC: Computer net-worked scanning electron microscope for teaching, research, andindustrial applications. Microsc Res Techn 32, 330 (1995).

Chumbley LS, Meyer M, Frederickson K, Labbs FC: The instructionalSEM laboratory at Iowa State University. In Proc Microscopy andMicroanalysis 1996 (Ed. Bailey GW, Corbett JM, Dimlich RVM,Michael JR, Zaluzec NJ). San Francisco Press (1996) 396–397

Fan G, Mercurio P, Young S, Ellisman M: Telemicroscopy. Ultrami-croscopy 52, 499–503 (1993)

ISO/IEC JTC1 SC29: Coding of moving pictures and associated audiofor digital storage media at to about 1.5 megabits per second.ISO/IEC 11172 , parts 1 through 5 (1993–1995)

ISO/IEC JTC1 SC29: Information technology—Digital compressionand coding of continuous-tone still images: Requirements andguidelines. ISO/IEC 10918-1 (1994)

Mansfield J: The teaching SEM: An example of real-time remote controlSEM. In Proc Microscopy and Microanalysis 1996 (Ed. BaileyGW, Corbett JM, Dimlich RVM, Michael JR, Zaluzec NJ). SanFrancisco Press (1996) 394–395

O’Keefe MA, Taylor J, Owen D, Crowley B, Westmacott KH, JohnstonW, Dahmen U: Remote on-line control of a high-voltage in-situtransmission electron microscope with a rational user interface. InProc Microscopy and Microanalysis 1996 (Ed. Bailey GW, Cor-

C. Morgan et al.: Toward a standard for remote microscope control 115

Page 7: Toward a standard for remote microscope control systems

116 Scanning Vol. 20, 2 (1998)

bett JM, Dimlich RVM, Michael JR, Zaluzec NJ). San FranciscoPress (1996) 384–385

Parvin B: Distributed virtual microscopy and microcharacterization.A white paper, May 1997, http://www-itg.lbl.gov/MMC/white_paper.html (1997)

Parvin B, Taylor J, Crowley B, Wu L, Johnston W, Owen D, O’KeefeMA, Dahmen U: Telepresence for in-situ microscopy. IEEE Inter-national Conference on Multimedia Systems and Computers, June(1996)

Pratt WK: Developing Visual Applications. Sun Microsystems Press,Mountain View, California (1997)

Razdan A: Remote scientific visualization and operation using scan-ning probe, microscope. MMC Architectural Workshop, OakRidge National Laboratory, October 1997,http://www.ornl.gov/doe2k/workshop/html/talks.html

Smith NR, Fegan N, Tullis RE, Morgan CL: Remote operation of anSEM from distant classrooms. In Proc. Microscopy & Microanaly-sis 1996 (Ed. Bailey GW, Corbett JM, Dimlich RVM, Michael JR,Zaluzec NJ) San Francisco Press (1996a) 414–415

Smith NR, Pardoe D, Pelote C, Honer M, Morgan CL: Remote controlof a scanning electron microscope using asynchronous transfermode. Microscopy Today 96-2 (1996b) 18

Smith NR, Tullis RE, Pardoe D, Pelote C, Fegan N, Morgan CL: Remoteaccess to a computer operated scanning electron microscope(SEM). Proc 4th CSU EM Microscopy Colloquium, San Diego,May 4–5 (1996c)

Voelkl E: Imaging needs for electron microscope distance learning andremote operation. Adv Imag 11, 31 (1996)

Voelkl E, Allard LF, Bruley J, Williams DB: Undergraduate TEM in-struction by telepresence microscopy. J Micros 187, 139–142(1997)

Voelkl E, Allard LF, Nolen TA, Hill D, Lehmann M: A simple and inex-pensive route to remote electron microscopy. In Proc Microscopy& Microanalysis 1996 (Ed. Bailey GW, Corbett JM, DimlichRVM, Michael JR, Zaluzec NJ) San Francisco Press (1996)386–387

Wright M: Personal communication (1997)XIL Programmer’s Guide, Sun Microsystems Press, Mountain View,

California (1994a)

XIL Reference Manual, Sun Microsystems Press, Mountain View, Cali-fornia (1994b)

Zaluzec N: Tele-Presence microscopy/LabSpace: An interactive col-laboratory for use in education and research. In Proc Microscopyand Microanalysis 1996 (Ed. Bailey GW, Corbett JM, DimlichRVM, Michael JR, Zaluzec NJ). San Francisco Press (1996)382–383

Zaluzec N, Stevens R, Evard R, Disz T, Olson R, Kuhfuss T: Tele-Pres-ence Microscopy & the ANL LabSpace (eLab) Project, Power-Grid, 1.01, http://www.powergrid.com/1.01/telemicro-wp.html(1996)

World Wide Web Sites Referenced

CU-SeeMe at Cornell University: http://cu-seeme.cornell.edu/

Electron Microscope Facility at San Diego State University:http://www.sci.sdsu.edu/EM_Facility

Geist, Al: http://www.epm.ornl.gov/~geist/java/applets/uscope

Mansfield J: World Wide Web http://emalwww.engin.umich.edu/people/jfmjfm/tsem/imp.html

Materials MicroCharacterization Collaboratory:http://tpm.amc.anl.gov/MMC

MMC Architectural Workshop, Oak Ridge National Laboratory, Octo-ber 1997: http://www.ornl.gov/doe2k/workshop/index.html

The National Center for Electron Microscopy: http://ncem.lbl.gov/ ncem.html

Voelkl E: Remote Operation of an Electron Microscope via the Internet: http://www.ccs.ornl.gov/SC95/Voelkl.html

World Wide Laboratory (WWL) at the University of Illinois’ BeckmanInstitute: http://vizlab.beckman.uiuc.edu/WWL

Zaluzec N: Telepresence Microscopy Laboratory:http://tpm.amc.anl.gov/TPMSelect.html

Zaluzec N, et al: Tele-Presence Microscopy & the ANL LabSpace(eLab) Project, PowerGrid, 1.01,http://www.powergrid.com/1.01/ telemicro-wp.html (1996)