Digital Microscopy: New Paradigm's for Teaching

Microscopic Anatomy andPathology

Michael Feldman, MD, PhDAssistant Dean IT

Assistant Professor PathologyUniversity of Pennsylvania Health System

Feldmanm@mail.med.upenn.edu

Agenda

• Introduction (Feldman)• Aperio (TBA)• Microbrightfield (Jack Glaser)• Break• Aurora (John Stinson/Kemp Watson)• Zoomify – (Dave Urbanic)• Panel Discussion

Objectives

• Define Digital Microscopy• Hardware solutions for creating slides• Software environment’s for:

– File formats and storage– Server options– Viewer options– Annotation tools

• Educational Pedagogy – how to best use these tools?

Digital Imaging in Education

• How many use Digital images in Pathology or Histology Education?

• How are the images used?• Have you been satisfied with the

technology?• Limitations?• How does it fit your educational pedagogy

for teaching histology/pathology?

Penn Answer’s• How are the images used?

– Static - Powerpoint lectures, web based small group teaching• Have you been satisfied with the technology?

– Resolution of images at low magnification– Variability of images from different sources– No interactivity

• Limitations?– Time to take all these images– Static

• Pedagogy– Static– Passive– Limited self exploration

Static Microscopic Imaging in Pathology

CCD – chip 6 um pixel

Histology Slide

Filter for RGB

2X 4X

10X 40X

Digital Slide• Entire slide or large area of slide imaged at sub

micron resolution (0.2-0.4 um/pixel) to simulate the action of a microscope zoom from 1X – 40X and slide movement XY directions

• Resulting images are HUGE! 50-100K pixel on edge for a 1-1.5 cm2 piece of tissue

• Uncompressed 5-10 Gb, compressed 300-500 Mb• Can’t download these babies using standard Html!

TeachingDigital slide processScan slideAperioMicrobrightfieldBacusMedia Cybernetics

Display ImageJava appletActive XExecutableHtmlQuicktime

Annotate ImageMicrobrightfieldAuroraZoomify

Store ImageFlashpixJPEG2000ZoomifyMscope Image Server

IseemediaAperioZoomifyAurora – MscopeMicrobrightfield

How are Slides captured?

• Tiling– Manually take pictures and assemble them with

software• Panovue Image Assembler ($79, XY tiling, small slides only 5-

7K/side)• Mac Quicktime VR

– Robotic tiling• Microbrightfield (http://www.microbrightfield.com/)• Bacus (http://www.bacuslabs.com/)

– Striping• Aperio Scanscope (http://www.aperio.com/home.asp)

Manual Tiling

• Images captured with static digital camera and manual stage

• Works for small areas of a slide (up to 50 images for a 7x7 field of capture)

• Stitched with Panovue ImageAssembler http://www.panavue.com/products/index.htm

• Demonstration of small slideshttp://130.91.96.203/index.html

• Images served using ZoomServer from Iseemediahttp://www.iseemedia.com/

Digital Slide Stitching

- A 1 cm2 piece of tissue imaged at 40X (0.25 mm FOV at 40x)

- Creates a 40x40 grid = 1600 Images (25k x 25k pixels)

- Total Image Tiled together is 1.6 Gbuncompressed

- With compression 100-200 Mb

Striping

Aperio- Instead of using individual images, scanner works like fax and scans a stripe (width of scanning element) at very high resolution

- Linear array is 54,000 pixels/inch using 20X lens (0.4 um/pixel)- Using 40X, array achieves 108,000 pixels/inch (0.2 um/pixel)- Very fast scan times (10-15 minutes with 40X lens for a 1.5 cm portion tissue)

Image storage/format

• Flashpix – pyramidal format• JPEG2000 - Aperio• Mscope – Aurora• Zoomify - pyramidal

Pyramidal format

FlashpixZoomify

ThumbnailHigh resolution

Webpage

Pixel on Demand

• Do not retrieve entire image, only data need to view image in a given window at a given magnification

• Return image as tiles (you can see the blocks come in as they are retrieved by the server)

• Makes for efficient viewing of HUGE files over reasonable bandwidth

Aperio and Aurora

• JPEG2000 Aperio and Mscope Aurora formats– More compression than jpeg (wavelet)– Rather than viewing images from a selected

subfile, wavelet reconstructs the image at the desired resolution

– Image is not stored as pyramidal format.– More compression without as much loss of

image quality than jpeg

How are slides Served (Accessed)

• Central server houses and serves images (Image server)

• Images are retrieved from server by client using pixel on demand as image tiles:

• Servers vary – How send data (tile size)– How data is cached (server, client or both)– How much CPU is used, how much disk is used– How many concurrent processes are used by server– Combine to define how many tiles of data can be pulled

off disks and served out to concurrent clients

Image Server (du Jour…)

• So many flavors:• Zoomserver (www.Iseemedia.com) - flashpix• Zoomify – (www.zoomify.com)• Microbrightfield – Zoomserver or zoomify• Aurora – Mscope proprietary, flashpix &

jpeg2000 in near future• Aperio – jpeg2000• Bacus – proprietary

Client side viewing• What software is used to pull images from server

– Java Applet from web page (Aurora, Microbright, Bacus)

– ActiveX (Zoomify, Bacus)– Stand alone application (Aperio)– HTML (Aperio)– Quicktime (Zoomify)– Flash (Zoomify)

Advantages of Applet

• Whether Flash, Java, Quicktime– Cross platform viewing Mac and PC– Easier to support across enterprise for software

updates– Portable to devices including (PDA’s)– Lightweight (<100 Kb)

Annotation

• Serving up standardized microscope quality high resolution images is start– Uniformity– No bad slides, cracked, not representative– No slide handling and management– Slides always available

• Even better if we can author content on slides and with the slides into educational lessons

Annotation Cont’d• Ability to annotate image can be used in multiple

manners– Simply label slide – simple answer

– Prior to lesson - Author material with questions and no answer’s

– After lesson – provide same images with answers and pointers

– Allow students to label images with questions independent of faculty annotations

• Questions• Interactive – like a threaded discussion that embeds digital

slides

– Which fit’s your pedagogy?

How do you teach with glass vs. digital?

• Glass: Search and identify with you assisting and facilitating in a small group

• Digital: Same• Digital: Compare and contrast multiple slides at once• Digital: Before and after scenario• Digital: support’s asynchronous learning, threaded

discussion– Student to student – Student to faculty

• Digital: expert learning system with intelligent tutor (Rebecca Crowley, University Pittsburgh)

• Digital: distributed multiheaded microscope• Digital: share content

Virtual Slide Examples

• Iowa Slides• Penn Med• Comparative slides Penn• Penn MBR sight• Aperio web• Microbrightfield• Aurora Mscope

Usage Data from Penn Pilot

• Usage stats from one course section (4 labs):– 1.6 million hits (each time move slide registers as hit– Only 12 manually tiled slides used– Avg 22 minutes per visit– More 50% viewed outside campus– 150 students viewed slides from > 200 computers– Pushed > 2 Gb data during these 4 small group sessions

Feedback from Pilot

• Uniformity of image• Always there to study• Supported group study• Image quality excellent• Student’s liked it, more comfortable than

search and find on microscope• Wanted more of it

Future direction• Integrate multi-role annotation tool to support

different instances of annotation of the same image – faculty, student

• Integrate threaded discussion• Use interactivity to make education sessions

dynamic (educational traces vs expert trace)• Create a distributed multi-headed client-server for

real time interaction• Awaiting delivery of Aperio scanner• Create archives for UME and GME (planned

scope thousands of images over 3-5 year period)