Brief CommuniCationhttps://doi.org/10.1038/s41592-018-0291-9

A robust and versatile platform for image scanning microscopy enabling super-resolution FLIMMarco Castello1,6, Giorgio Tortarolo   1,2,6, Mauro Buttafava   3, Takahiro Deguchi4, Federica Villa3, Sami Koho1, Luca Pesce4, Michele Oneto4, Simone Pelicci4, Luca Lanzanó   4, Paolo Bianchini4, Colin J. R. Sheppard4, Alberto Diaspro   4,5, Alberto Tosi   3 and Giuseppe Vicidomini   1*

1Molecular Microscopy and Spectroscopy, Istituto Italiano di Tecnologia, Genoa, Italy. 2Dipartimento di Informatiche, Bioingegneria, Robotica e Ingegneria dei Sistemi, University of Genoa, Genoa, Italy. 3Dipartimento di Elettronica, Informazione e Bioingegneria, Politecnico di Milano, Milan, Italy. 4Nanoscopy and NIC@IIT, Istituto Italiano di Tecnologia, Genoa, Italy. 5Dipartimento di Fisica, University of Genoa, Genoa, Italy. 6These authors contributed equally: Marco Castello, Giorgio Tortarolo. *e-mail: giuseppe.vicidomini@iit.it

SUPPLEMENTARY INFORMATION

In the format provided by the authors and unedited.

NATuRe MeThODS | www.nature.com/naturemethods

Supplementary Figure 1

The SPAD array detection system developed for image scanning microscopy.

Photograph of the SPAD array detection system. The photograph shows the two stacked printed circuit boards (PCBs). Inset shows the 5 × 5 SPAD array. (b) Photon detection efficiency (PDE) of the central element of the 5 × 5 SPAD array, at different excess bias voltages. (c) Temporal response (or impulse-response function (IRF)) of the central element (Vex = 6 V excess-bias) of the SPAD array to a pulsed laser source at 850 nm (20 ps FWHM) when all the other pixels are turned ON. The long tail on the right side of the IRF is due to the optical cross-talk with the other SPAD elements.

Supplementary Figure 2

Custom image scanning microscopy schematic setup.

LPD, laser-pulsed diode; HWP, half-wave plate; QWP, quarter-wave plate; AOM, acoustic optical modulator; PMF, polarized maintaining fiber; PC, personal computer; TDCs, time-to-digital converters; TTL, transistor–transistor logic; 3A-S, three-axis stage; 3A-PS, three-axis piezo stage; DM, dichroic mirror; GMs, galvanometer mirrors ; SL, scan lens; OL, objective lens.

 Supplementary Figure 3

Image scanning microscopy with adaptive (A) pixel reassignment (PR).

(a) Side-by-side comparison of the effective PSFs for ‘ideal’ confocal (0.2 AU), ‘open’ confocal (1.4 AU), uncorrected ISM (the pixel- reassignment method uses theoretical shift vectors), and ISM (the pixel-reassignment method uses estimated shift vectors). Scale bar, 500 nm. (b) Fingerprint maps superimposed with the shift vectors for the uncorrected ISM (left) and the true ISM (right). Scale bars, 50 nm. (c) Radial PSFs obtained from the intensity profiles of (a). The top panel shows the un-normalized PSFs, and the middle and bottom panels show the normalized PSFs, together with the Gaussian fitted data. Pixel-dwell time, 50 µs. Pixel-size, 5 nm. Image format, 400 × 400 pixels. Scale bars, 500 nm.

Supplementary Figure 4

Image scanning microscopy on nuclear pore complexes (NPCs).

(a) Comparison of large-field-of-view images of the nuclear pore scaffold structures stained with Alexa Fluor 488: ‘ideal’ confocal (top) and APR-ISM (bottom). Image format, 1,500 × 1,500 pixels; scale bars, 10 µm. (b) Side-by-side comparison of single-cell images, ideal confocal (0.25 AU), open confocal (1.7 AU), APR-ISM and deconvolved ISM++ (five iterations), respectively. Notably, to highlight the property of the proposed ISM implementation to generate high-resolution large-field-of-view images, all images are a digital zoom of the white boxes in (a). Image format, 400 × 400 pixels; scale bars, 1 µm. (c) Magnified views of the white boxes in (b). Image format, 100 × 100 pixels; scale bars, 1 µm. (d) Line intensity profiles across two closely packed NPCs at the position of the arrowheads for the different imaging modalities. For all images: pixel dwell time, 100 µs; pixel size, 66.6 nm; excitation power Pexc, 840 nW. The FRC-based resolution values for the 1,500 × 1,500 pixel images are 213 nm, 253 nm and 207 nm for ideal confocal, open confocal and APR-ISM, respectively. Data are representative of n = 10 experiments.

 Supplementary Figure 5

Image scanning microscopy of nuclear lamin.

(a) Side-by-side comparison of HEK cell nuclear lamin stained with Alexa Fluor 488: ‘ideal’ confocal (0.25 AU), open confocal (1.7 AU), APR-ISM and deconvolved ISM++ (five iterations). Image format, 600 × 600 pixels. (b) Magnified views of the regions outlined by white boxes in (a). (c) Line intensity profiles across a nuclear invagination (nucleoplasmic reticulum) at the position of the arrowheads in (b) for the different imaging modalities. For all images: pixel dwell time, 100 µs; pixel size, 50 nm; excitation power Pexc, 840 nW. Scale bars, 1 µm. The FRC-based resolution values for the images are 238 nm, 263 nm, and 206 nm for ideal confocal, open confocal and APR-ISM, respectively. Data are representative of n = 10 experiments.

Supplementary Figure 6

Image scanning microscopy of fluorescent beads.

(a) Side-by-side comparison of ‘ideal’ confocal, ‘open’ confocal, APR-ISM and deconvolved ISM++ (ten iterations) images (500 × 500 pixels, 40-nm pixel size) of 20-nm red fluorescent beads. Pixel dwell time, 50 µs. Pixel size, 40 nm. Image format, 500 × 500 pixels. Excitation power Pexc, 56 nW. Scale bars, 1 µm. (b) Magnified views of the regions outlined by white boxes in (a). Scale bars, 1 µm. (c) Line intensity profiles across two close fluorescent beads at the position of the arrowheads in (b) for the different imaging modalities. Data are representative of n = 5 experiments.

Supplementary Figure 7

FRC-based resolution scaling on ISM with increasing excitation power.

(a) Series of tubulin (labeled with Abberior STAR red) images of the same area for ‘ideal’ confocal (top), ‘open’ confocal (middle) and APR-ISM (bottom) as a function of the excitation beam power. Bleaching was negligible across the whole imaging experiment. Pixel dwell time, 100 µs. Pixel size, 37.5 nm. Image format, 400 × 400 pixels. Excitation power Pexc: 50, 55, 62, 70, 90, 110, 170, 220, 250, 350, 520, 700, 890, 1,000, 1,300 and 2,000 nW. Scale bars, 1 µm. (b) Fourier-ring-correlation curves for different excitation beam powers and different imaging modalities: ideal confocal (top), open confocal (middle) and APR-ISM (bottom). The fixed 1/7 threshold value is also reported with the curves. Resolution (based on the FRC analysis and the 1/7 threshold value) as a function of excitation power for the three different imaging modalities is reported in Fig. 1f. Data are representative of n = 2 experiments.

     Supplementary Figure 8

Image scanning microscopy for live-cell mitochondria imaging.

(a) Side-by-side comparison of ‘ideal’ confocal (top), ‘open’ confocal (middle-top), APR-ISM (middle-bottom) and deconvolved ISM++ time-lapse (3 min, 24 frames) of mitochondria labeled with MitoTracker Deep Red in a live cell. For each imaging modality, three representative frames are shown (0 s, 90 s and 180 s). Pixel dwell time, 30 µs. Pixel size, 40 nm. Image format, 500 × 500 pixels. Excitation power Pexc, 140 nW. Scale bars, 1 µm. (b) Maximum intensity projections (color-coded by time) of the time-lapses for the different imaging modalities. These projections allow one to identify the fraction of mitochondria with minimal mobility (white) from the mobile one (colored). Data are representative of n = 10 experiments.

Supplementary Figure 9

Image scanning microscopy for live-cell tubulin imaging.

(a) Side-by-side comparison of ‘ideal’ confocal (top), ‘open’ confocal (middle), and APR-ISM (bottom) time-lapses (4.5 min, 19 frames) of tubulin labeled with SiR-tubulin in a live HeLa cell. For each imaging modality, three representative frames are shown (0 s, 142 s and 264 s). All frames are normalized in intensity to the maximum and minimum of the first frame of the respective time-lapse. This allows one to appreciate the negligible photobleaching. Pixel dwell time, 50 µs. Pixel size, 57.2 nm. Image format, 700 × 700 pixels. Excitation power Pexc, 2 µW. Scale bars, 1 µm. (b) Magnified view of the regions outlined by white dashed boxes in (a). Data are representative of n = 3 experiments.

Supplementary Figure 10

Photon-arrival-time histograms for ISM.

Comparison between the histograms of the photon-arrival times for the different imaging modalities: ISM, ‘open’ CLSM (1.4 AU) and ‘ideal’ CLSM (0.2 AU). We obtained the histograms by integrating all the pixels from the 3D datasets of the tubulin experiment (Supplementary Fig. 11). Clearly, because the pixel reassignment simply shifts photons from one pixel to another, the ISM and open CLSM histograms are identical. This is also an indication that our extension of the pixel-reassignment method to fluorescence lifetime imaging does not introduce artifacts. The reduced SNR of the ‘close’ CLSM appears clear in the comparison between the histograms. We report also the impulse response function (IRF) of the scanning microscope obtained by measuring the reflection (488 nm) from a gold bead.

Supplementary Figure 11

Fluorescence lifetime image scanning microscopy on tubulin labeled with Abberior STAR red.

(a) Side-by-side comparison of the fitting-based ‘ideal’ confocal FLIM (left; 1.4 AU), FLISM (center) and ‘open’ confocal FLIM (right) images of tubulin labeled with Abberior STAR red. Purple pixels correspond to out-of-range fluorescence lifetime values, which are probably the result of artifacts during the time-domain (fitting) analysis. All pixels were analyzed. (b) The respective intensity images are also reported. Pixel dwell time, 100 µs. Pixel size, 30 nm. Image format, 500 × 500 pixels. Excitation power Pexc, 500 nW. Scale bars, 1 µm. Data are representative of n = 4 experiments.

Supplementary Figure 12

Fluorescence lifetime image scanning microscopy of membrane labeled with ANEP.

Side-by-side comparison of the fitting-based FLIM (left; 0.25 AU) and FLISM (right) images of membrane labeled with ANEP. Purple pixels correspond to out-of-range fluorescence lifetime values, which probably resulted from artifacts during the time-domain (fitting) analysis. All pixels were analyzed. Pixel dwell time, 300 µs. Pixel size, 62.6 nm. Image format, 800 × 800 pixels. Excitation power Pexc, 3 µW. Scale bars, 1 µm. Data are representative of n = 4 experiments.

Supplementary Figure 13

Commercial-system-based image scanning microscopy setup.

(a) Schematic describing the connections between the different hardware components of the image scanning microscope based on a commercial Nikon A1R system. Carma microscope control registers the photons collected by the SPAD array and is in charge of the detector initialization. The synchronization with the scanning system and with the other actuators of the Nikon A1R, which allows generation of the scanned images, is obtained through communication with the Nikon microscope controller. In the case of imaging with galvanometer mirrors, the carma controller provides the analog scanning signal to the Nikon controller, which successively communicates with the galvanometer mirrors located in the confocal scan head (i.e., carma is the master). In the case of imaging with a resonant mirror (for the fast axis), the carma microscope unit receives the synchronization signals (pixel, line and frame clocks) from the Nikon microscope (i.e., carma is the slave). Both the Nikon and the carma microscope controls communicate with the personal computer (PC). In particular, the PC hosts the carma software, which visualizes, analyzes and processes (deconvolution, pixel reassignment and Fourier-ring correlation) the data. (b) Simplified scheme of the Nikon scan head. Only the important elements for the ISM implementation are reported. The excitation beam (blue) is sent to the galvanometer mirrors (GMs) or resonant mirror thanks to a first dichroic mirror (DM1). The beam is scanned on the specimen/object plane thanks to the scanning lens (SL), the tube lens and the objective lens; tube and objective lenses are not shown in the scheme. The fluorescence (green) is collected by the objective lens and descanned by the GMs. The SL and the TL generate a second conjugate image plane, the pinhole plane, with magnification M2 = 3.9 × M1. M1 is the magnification on the first image plane (not shown in the scheme), which corresponds to the nominal magnification of the objective lens (10×, 20× or 60× in our experiments). A second dichoric mirror (DM2), which usually deflects the fluorescence to the conventional single-point detector (light green), is removed, and the pinhole is completely opened during ISM. The zoom lens (i) is positioned on a five-axis stage (5A-S) to align the fluorescence beam with respect to the SPAD array; (ii) conjugates the pinhole plane on the SPAD array and adds an extra magnification (M3) on the detector plane; and (iii) allows a projected size of the SPAD array detector on the object plane equal to 1 Airy unit.

Supplementary Figure 14

Shift vectors as a function of the depth of imaging.

Module of the estimated shift vectors for the 3D (x,y,z) dataset of Fig. 3(a) (20× objective lens; top) and Fig. 3(b) (10× objective lens; bottom). The estimated shift vectors were calculated via the phase-correlation approach. Only the values of the four direct-neighbor elements are shown.

Supplementary Figure 15

Image scanning microscopy combined with fast resonant scanning.

Side-by-side comparison between ‘ideal’ confocal, ‘open’ confocal and APR-ISM images of tubulin stained with Alexa Fluor 546 (format, 256 × 256 pixels; resonant frequency, 7.9 kHz; zoom factor, 8×, which results in a pixel size of 103 nm and a minimum pixel dwell time of about 70 ns; 64 line integrations). Insets show magnified views of the regions outlined by white boxes. Scale bars, 1 µm. Data are representative of n = 10 experiments.

 

1 Single-Photon-Avalanche-Diode Array

In recent years, single-photon avalanche diodes (SPADs) have become in-creasingly popular in the field of time-resolved spectroscopy, thanks to theirphoton-counting nature [1, 2]. With performance comparable (or even superior)to photo-multiplier-tubes (PMTs) in terms of detection e�ciency, noise and tem-poral response, SPADs benefit from the great scalability, robustness, flexibilityand reliability o↵ered by the microelectronic fabrication technology. Comparedto their vacuum based counterparts, multi-pixel microelectronic single-photondetectors also contribute to reduce cost, overall size and system complexity. Allthese properties currently make the SPAD array one of the best technologies todevelop a real photon-counting two-dimensional detector array for fluorescencemicroscopy.We have designed and developed a novel single-photon detector array, specifi-cally tailored to implement image scanning microscopy. The array is composedof a square matrix of 5⇥5 SPADs [1], having 75 µm distance (pixel pitch) and50 µm side length (pixel size), with 5 µm corner radius (rounded-square active-area shape, as shown in Supplementary Fig. S1a). This geometry results in afill-factor (i.e., the ratio between photosensitive area and total detector area) of⇠ 44%, considering the external frame, otherwise ⇠ 50%.The overall detection system is composed by the custom-made SPAD array andtwo stacked printed circuits boards (PCBs), dedicated to system managementand connectivity (Supplementary Fig. S1a). The SPAD array ASIC (appli-cation specific integrated circuit) is fabricated using a 0.35 µm high-voltageCMOS technology by Fraunhofer IMS (Duisburg) and integrates the 25 SPADdetectors, which design is extensively described in [3], together with 25 dedi-cated quenching and readout circuits, described in [4]. This ASIC provides 25low-time-jitter digital outputs, whose rising edges are synchronous to photon de-tections. By means of a dedicated serial communication interface, the user canselectively switch either ON or OFF any single SPAD, allowing for choosing dif-ferent detection patterns. A first printed circuit board (called front-end board)hosts the detector ASIC, which is directly glued and electrically wire-bondedto it (circuit-on-board mounting technique), together with programmable sup-ply voltage generators (about 6 V for SPAD excess-bias, 3.3 V for readout andquenching circuits, about 25 V for SPAD biasing) and a digital-to-analog con-verter (DAC) for globally setting the hold-o↵ time duration. The 25 digitaloutput signals are passed to a second printed circuit board (called connectionboard) through high-bandwidth connectors and then are provided to a set of 25low-time-jitter bu↵ers, able to drive 50 ⌦ impedance cables. These output ca-bles are finally connected to an external FPGA-based system for collecting thephoton counts. The connection board also includes a 8-bit microcontroller forsystem initialization and power management. Power supply comes from a single5 V � 1 A external source. Individual pixels can be selectively enabled/disabledby the user using three external digital signals: two of them for serial communi-cation (implementing a serial peripheral interface - SPI - communication) andone for global enable. The detection system can be directly mounted on a multi-axis positional stage for a precise and reliable optical alignment.We chose a relative small number of pixels (i.e., 25) since our theoretical studiesshow that, for a fixed 1 A.U. projected-size of detector array, a higher numberof elements would provide a marginal improvement on spatial resolution [5]. On

the other hand, a higher number of elements translates into more complex data-readout architectures, hindering a fully parallel and independent operation ofeach pixel (thus reducing both speed and versatility of the device). To this pur-pose, in our SPAD array each of the 25 elements can deliver a fully-independentdigital signal each time a photon is collected. Concerning the fill-factor, it isimportant to highlight that: (i) in the ISM application, the fluorescent photonsprojected on the detector array are not uniformly spread across the whole de-tector, for example, for a fluorophore located in the center of the SPAD arraydetector (projected in the specimen plane), the fluorescent photons distributeaccording to the emission PSF, thus the odd-numbered elements of the detec-tor makes the overall probability that a photon reaches the active area higherthan the fill-factor itself; (ii) increasing the fill-factor, by reducing the spac-ing between elements, will likely deteriorate performances in terms of opticalcrosstalk between adjacent pixels; (iii) the collection e�ciency can be substan-tially improved by using a micro-lenses array (MLA) in front of the detector.We are currently working on the fabrication of a high fill-factor MLA directlyon the SPAD array chip [6], expecting an increase of the equivalent fill-factorto above 78% (i.e., above the theoretical value predicted by using a rectangulararray of circular micro-lenses).We characterized the SPAD array in terms of photon detection e�ciency (PDE),dark-count-rate (DCR), temporal response, optical cross-talk and afterpulsingprobability. In Supplementary Fig. S1b we show the measured PDE for thecentral pixel, as a function of wavelength and for di↵erent excess-bias voltages(Vex) (other pixels exhibit similar performance). The PDE decreases when in-creasing the wavelength, ranging from about 45% at 480 nm down to 20-15% inthe 600-700 nm region (both at 6 Vex excess-bias). Higher PDE values could beachieved by using di↵erent fabrication technologies, as recently demonstratedwith the Bipolar-CMOS-DMOS (BCD) technology [7]. The dark count rate hasbeen measured at 25 � for each array element, resulting in an average DCRvalue around 200 counts per second (cps). The detector temporal response isshown in Supplementary Fig. S1c for the central pixel only (similar resultsare obtained for all the 25 elements). It has been acquired using an exter-nal TCSPC board (SPC-630, Becker&Hickl) and a pulsed diode laser (32 ps ofFWHM, 1 MHz of repetition rate and 850 nm of wavelength, Advanced LaserSystem), with all the other pixel turned ON, resulting in a time jitter below200 ps FWHM. The optical cross-talk probability between pixels is lower than2% among closest neighbors (in the orthogonal direction). Finally, the after-pulsing probability ranges from 6.5% when enforcing a SPAD hold-o↵ time of50 ns, down to 1.4% with 200 ns hold-o↵ time. Increasing the hold-o↵ time isbeneficial for the reduction of afterpulsing probability, but as a drawback, itcorrespondingly reduces the maximum count-rate of the detectors. However,the maximum count rate of the SPAD array module is in general much higherthan single SPAD, thanks to the full parallel reading of each element and dueto the fact that in a point-scanning microscope all fluorescent photons collectedfrom a specific region of the sample are spread across all elements of the SPADarray. This property is also particularly important in the context of fluores-cence lifetime, which historically has been strongly limited by the relative smallmaximum count-rate supported by single-photon timing detectors.In the context of integration of the SPAD array into an existing confocal micro-scope, it is important to note that the overall size of the SPAD array sensitive

area is 350 ⇥ 350 µm2, thus the magnification values requested to obtain aprojected size of ⇠1 A.U. in the sample plane are workable. For example, forimaging with the 10⇥ CFI Plan Apo 10⇥C Glyc NA 0.5 objective lens in thegreen spectral range (520 nm) the size of 1 A.U. is ⇠ 1.3 µm, thus an extramagnification of ⇠ 27⇥ is requested. In the case of the ISM implementation onNikon A1R, it is obtained thanks to the extra magnification inside the scan-head(3.9⇥) and the zoom-lens system (1.3-8.7⇥).

2 Pixel-Reassignment

In ISM, the most straightforward method to recombine the scanned imagesinto a high-resolution image is pixel-reassignment (PR). Since the PR-ISM imageresults from the sum of the scanned images, it is still expressed in photon countsand it is linear with the fluorophore concentration, which make this approachfully compatible with a quantitative analysis [8]. It is necessary to describe theimage formation process of the image scanning microscope to understand the PRmethod. Since a fluorescent image scanning microscope can be considered as alinear and space-invariant system, the relation between the expected (noise-free)scanned image gi,j(x) � associated with the element (i, j) of the detector array� and the object/specimen function f(x) can be described by a convolutionoperator Hi,j

gi,j(x) = [Hi,j(f)](x) =

Zhi,j(x� y)f(y)dy = (hi,j ⇤ f)(x)

with i = 1, .., 5, j = 1, .., 5,(S1)

where hi,j is the e↵ective PSF associated to the element (i, j), y is the positionin the sample and x is the position in the image back projected into the sample,i.e., the scanning position. Here, we consider a magnification equal to 1 betweenthe object and image planes and a detector array with 5⇥5 elements. Assumingthat the projected size of each element of the detector array is much smallerthan 1 Airy unit (A.U.), the e↵ective PSF of each element reads

hi,j(x) = hexc(x)hem(x� di,j),(S2)

where hexc and h

em are respectively the excitation and emission PSF of a con-ventional scanning microscope, and di,j = (dxi.j , d

yi.j) is the vector describing the

displacement between the (i, j) element and the central element (i = 3, j = 3).For a non-negligible element size, the emission PSF h

em have to be first con-volved with the function describing the geometrical shape of the element.If we assume for the sake of simplicity that both excitation and emission PSFsare identical, as would be the case for no fluorescent Stokes shift � excitationand emission wavelengths are the same �, the e↵ective PSF is a peak func-tion whose maximum is located at the position midway between the excitationhexc(x) and the shifted emission h

em(x� di,j) PSFs’ maxima, i.e., at the posi-tion si,j = di,j/2. Since the signal recorded by the element (i, j) is most likelyto have originated from the position si,j , it can be “reassigned” to its originalposition, following the PR concept. Performing this “reassignment” for eachelement corresponds to scaling the image by a factor of 2, the so-called pixel

reassignment factor ↵. From the point of view of imaging, having a shifted PSFmeans the generation of a shifted image. Thus, every scanned image is shifted,with respect the central one, by the shift vector si,j and the pixel-reassignmentmethod can be implemented by shifting-back and adding-up the single scannedimages.This is the main strategy that we implemented within this work. In particular,the shift is implemented in the Fourier domain also allowing for sub-pixel shiftvectors si,j . The shifting vectors can be calculated theoretically according toa simple geometrical model, i.e. the physical distance between the center ofthe detector elements d

ri,j , divided the magnification of the microscope on the

detector plane (M3, Figure 1), and divided by the pixel-reassignment factor ↵,

si,j =dri,j

M3⇥ ↵(S3)

However, in practice the shift vectors calculated using this model are signifi-cantly di↵erent from the real ones. A first source of deviation is the Stokes-shift,i.e. the excitation and emission PSFs are not identical, but they have di↵erentwidth, meaning that the position of the maximum of the e↵ective PSF is notexactly located midway the excitation PSF and the shifted emission PSF, andthe PR factor is di↵erent from 2. A PR factor compensating for the Stokes-shiftcan be estimated a-priori [9]. Another important source of deviation is the dif-ferent aberrations which e↵ectively change the shape of both the excitation anddetection PSFs, and which are di�cult to estimate a priori. These aberrationsinfluence the PR factor, and more generally the shift vectors. For these reasons,we implemented a method to estimate the shift vectors directly from the seriesof scanned images, gi,j . Clearly, this method is an a posteriori approach � nota real-time approach, as in the all-optical ISM implementations, where the finalimage is built-up pixel-by-pixel as a conventional confocal microscope � but ito↵ers the important ability to compensate for system- and sample-dependentdistortions. Notably, in the case of an all-optical implementation based on flu-orescent re-scanning, two di↵erent PR factors can be implemented along thetwo-axis, but these factors need to be known a-priori.We estimated the shift vectors si,j for the PR using a phase correlation ap-proach, which is typically used to estimate the drift between two images. Be-fore describing the phase correlation approach, we need to introduce the dis-crete notation for the scanned images. Indeed, images are usually acquired ona regular 2-dimensional raster scanning grid. If we identify each pixel by itsindex n = (nx, ny), we can denote the Nx ⇥Ny scanned image as gi,j(n) withnx = 1, ..., Nx and ny = 1, ..., Ny.Phase correlation estimates the shift between two similar images relying on afrequency-domain representation of the data, which in our implementation isobtained through fast Fourier transform (FFT). To calculate the phase correla-tion between the two di↵erent scanned image gi,j and g3,3, we first define theso-called correlogram ri,j

ri,j = FFT�1

0

@ FFT�gi,j)FFT (g3,3

�⇤���FFT

�gi,j)FFT (g3,3

�⇤���

1

A ,(S4)

and subsequently find the maximum of the correlogram, whose position denotesthe drift/shift between the two scanned images gi,j

(sxi,j , syi,j) = argmax

(nx,ny)(ri,j(n)).(S5)

The position of the maximum can be obtained using a Gaussian-fitting algorithmor a centroid algorithm in order to obtain sub-pixel values.This approach, i.e. evaluating the drift between the scanned images to obtainthe shift vectors, is based on the assumption that the e↵ective PSFs, associatedto the di↵erent elements, do not changed appreciably. However, this assumptionmay fail. For example, for aberration-free Airy disks, the centre of the e↵ectivePSF is not a global maximum for detector elements farther more than 1.35A.U. from the optical axis. These changes in the shape of the PSFs are evidentin Figure 1b. On the other side the contribution of the images associated toelements at a distant grater than 1.35 A.U. are minimal, for this reason nosignificant artifacts have been introduced in the restored image. More robustapproaches to estimate the shift vectors may introduce constraints based on thegeometry of the detector.Interestingly, the shift vectors can be used to estimate the magnification (M3)of the microscope on the detector array. In absence of Stokes-shift, such asin reflection microscopy, the shift vectors are equal to half of the displacementvalue, si,j = di,j/2. Since the displacement value depends on the projectedphysical distances of the element of the SPAD array, which are well knownvalues, the magnification can be calculated as

M3 =4p

2P

kx/y={�1,1} |s3+kx,3+ky |,(S6)

where p is the pixel-pitch, i.e. 75 µm in our SPAD array. Only the shiftvectors linked to the first-order neighbors of the central element are used, sincetheir estimation is more robust, i.e., they are associated to higher SNR scannedimages. We used reflection imaging of gold beads and this approach to calibratethe magnification for all our experiments. For example, from imaging in Figure1b we estimate a magnification of 456⇥, which is fully in agreement with theset of lenses included in the custom setup (M3 = 450⇥).

3 Multi-image deconvolution for ISM

Multi-image deconvolution is another approach for recombining the scannedimages into an high-resolution image. In comparison to PR, deconvolution needshigher computational e↵ort and prior-information, such as the PSFs of thescanned images, but can provide higher SNR and higher e↵ective resolution[10]. In this Note, we derive the multi-image deconvolution algorithm followinga maximum-likelihood (statistical) approach [10] and using a discrete notationfor the object function, the PSFs and the digital images.If we denote the discretized object function, the expected scanned images andthe PSFs with the one-dimensional vectors f(n), gi,j(n), and hi,j(n) (withn = nyNy + nx), we can write the image formation process (Eq. S1) as

gi,j = Hi,jf,(S7)

where the Hi,j are the convolution matrices (NxNyNxNy sized) associated withthe convolution (linear) operatorHi,j and the PSF hi,j (Eq. S1). Moreover, withthe discretization of convolution integral of Equation S1 using cyclic convolutionand periodic extension of the pixel values of f and hi,j reduces Hi,j to a circularmatrix, hence the transformation

Hi,jf = hi,j ⇤ f,(S8)

can be easily computed by means of the FFT. Here and in all subsequent equa-tions multiplication and division of one vector by another is taken to meanpixel-by-pixel.The measurement process is dominated by shot noise and count rates are usuallyin the range of zero to a few hundred photons per pixel. Thus, for each pixel nand each scanned image (i, j), the measured value gi,j(n) is the realization of aPoisson random variable with its expectation value given by gi,j(n). Because,each pixel is statistically independent from the other, the probability to recordthe series of scanned images g for a given specimen f is given by

P (g|f) =Y

(i,j)

Y

n

poi[g(i,j)(n)|g(i,j)(n)] =Y

(i,j)

Y

n

e�(Hi,jf)(n) ((Hi,jf) (n))

g(i,j)(n)

gi,j(n)!.

(S9)

Since we assume to knowledge of the probability density P (g|f) of the data andthe specimen f appears as a set of unknown parameters, the problem of deconvo-lution can be approached as a classical problem of parameter estimation, whichcan be solved by the standard maximum likelihood (ML) estimation approach.We introduce the likelihood function Lg, defined by

Lg(f) = P (g|f),(S10)

which is a function of f only, since the series of scanned image g is given. Then,the ML-estimate of the unknown object f is any object f

⇤ that maximize thelikelihood function

f⇤ = argmax

f

Lg(f).(S11)

Since in our application the likelihood function is the product of a very largenumber of factors (Eqs. S10 and S9), it is convenient to take the logarithm ofthis function; moreover, if we consider the negative logarithm, the maximizationproblem is transformed into one of minimization. By introducing the so-calleddiscrepancy functional J , the deconvolution problem reads

f⇤ = argmin

f

�B lnLg(f) + C = argminf

J(f;g),(S12)

where B and C are suitable constants that can be introduced in order to simplifythe expression of the functional. By using simple mathematics the discrepancyfunction of our algorithm reads

J(f;g) =X

i,j

X

n

⇢gi,j(n) ln

gi,j(n)

(Hi,jf) (n)+ (Hi,jf) (n)� gi,j(n)

�.(S13)

For the solution of Equation S12 we chose the split-gradient-method (SGM) [11,12] due to its robustness, the simplicity of its implementation and its capabilityto enforce non-negative constraint, i.e. f > 0, in a natural fashion. For ourdiscrepancy function (Eq. S13) the SGM iterations are given by

fk+1 = f

kX

(i,j)

1

HTi,j1

HTi,j

gi,j

Hi,jfk

!,(S14)

where HTi,j is the transpose of the operatorHi,j and 1 is the vector whose entries

are all equal to 1. In practice, the matrix-vector multiplication HTi,j1 generates

a vector whose elements are the sum of Hi,j across its columns. Since the matrixHi,j is cyclic, HT

i,j1 is a vector whose entries are all equal to the sum of thediscretized PSF hi,j

wi,j =X

n

hi,j(n),(S15)

and the SGM algorithm (Eq. S14) reduces to

fk+1 = f

kX

(i,j)

w

�1i,j hi,j ?

gi,j

hi,j ⇤ fk

!,(S16)

where, for the sake of simplicity, we move to a vector notation and ? denotes thecorrelation operation, which as with convolution can be implemented through aFFT. The algorithm in Equation S16 can be considered as an extension of theRichardson-Lucy algorithm for solving the multi-image deconvolution problem.Indeed, for a single image, the algorithm reduces to the well-known RL algo-rithm.Finally, it is important to discuss how we calculated the PSFs throughout ourmanuscript. We used a simplified Gaussian-based model. A more rigorous modelbased on vectorial focusing theory [13] can be used, but they are based on pa-rameters di�cult to know. For each element (i, j) we calculated a normalized(the integral is equal to 1) Gaussian PSF centered on si,j (Eq. S5). We used thesame full-width at half-maximum for all the elements, but we scaled each PSFfor a factor wi,j which takes into account the expected di↵erent SNRs of theassociated scanned images. As scaling factors we used the values of the normal-ized fingerprint map, i.e. wi,j = a(i, j). We estimated the FWHM directly fromthe images, by fitting the line intensity profile of single isolated sub-di↵ractionstructure in the brighter scanned image with a Gaussian function.

4 Fingerprint map

In this Note we introduce the concept of the fingerprint map and we showits ability to encode information about the status of the optical scanning micro-scope. Such information is normally discarded in a conventional scanning mi-croscope, since single-photon detectors can register the time at which a photonreaches the sensitive area but not its position. In particular, we demonstratethat, from the series of scanned images, it is possible to extract information

about the alignment of the ISM system, and more importantly, informationabout its PSF, regardless of the specimen observed.Given the series of scanned images g, we define as fingerprint map a the totalnumber of photons collected by each element of the detector array during theregistration of the scanned images

a(i, j) =X

n

g(i,j)(n),(S17)

and we demonstrate that a is proportional to the correlation of the emissionand detection PSFs of the scanning microscope.To demonstrate this proportionality we analyze the fingerprint map in the con-tinuous domain. We consider a detector array composed by infinitesimal ele-ments and we observe that the image gx0,y0 registered by the element at theposition (x0

, y0) 2 R2 reads

gx0,y0(x, y) = (hx0,y0 ⇤ f)(x, y),(S18)

where hx0,y0 denotes the PSF associated with the detector element in the position(x0

, y0). Thus the fingerprint map a(x0

, y0), defined respect to the coordinates

of the detector array, reads

a(x0, y

0) =

ZZ

x,ygx0,y0(x, y)dxdy =

ZZ

x,y(hx0,y0 ⇤ f)(x, y)dxdy.(S19)

Applying the integration property of convolution, the fingerprint image reads

a(x0, y

0) =

ZZ

x,yhx0,y0dxdy ·

ZZ

x,yf(x, y)dxdy = �

ZZ

x,yhx0,y0dxdy,(S20)

where � is the total flux of photons from the sample. Interestingly, a(x0, y

0) issample independent, at the condition � > 0, but it is strictly connected to thePSF of the microscope. Recalling that the PSF of the infinitesimal element is

hx0,y0 = hexc(x, y) · [hem(x, y) ⇤ �(x� x

0, y � y

0)]

= hexc(x, y) · hem(x� x

0, y � y

0),(S21)

and substituting in the Equation S20, it is possible to obtain

a(x0, y

0) = �

ZZ

x,yhexc(x, y) · hem(x� x

0, y � y

0)dxdy / (hexc? hem),(S22)

where ? denotes the correlation operator. To summarize, the fingerprint imageis instrument-dependent and not sample-dependent, and it depends simultane-ously on both the excitation and the emission PSFs.In this work we used the fingerprint map to align the ISM system: to co-align theemission PSF with the excitation PSF we maximize the intensity of the centralelement (i=3,j=3) of the fingerprint map. Future directions will be to use thefingerprint map to implement an adaptive optics (AO) feedback system, whichuses the fingerprint map as a figure of merit to modify light shaping devices,such as spatial-light-modulators (SLMs) and/or deformable mirrors (DMs). Ina nutshell, since aberrations lead to a change of the emission and/or excitation

PSFs, the fingerprint map will reflect such changes and its analysis can helpin retrieving the aberrations, which can be compensated by the SLMs and/orDMs.Another area of application of the fingerprint map is in image processing, suchas image deconvolution. Conventional deconvolution needs knowledge of themicroscope PSF (see 3) which is not always easy to obtain. An estimation ofthe PSF of the system can be decoded from the fingerprint map.

Bibliography

[1] Zappa, F., Tisa, S., Tosi, A. & Cova, S. Principles and features of single-photon avalanche diode arrays. Sensors and Actuators A: Physical 140,103–112 (2007).

[2] Becker, W. (ed.) Advanced Time-Correlated Single Photon Counting Ap-plications (Springer International Publishing, 2015).

[3] Villa, F. et al. CMOS SPADs with up to 500 µm diameter and 55% detec-tion e�ciency at 420 nm. Journal of Modern Optics 61, 102–115 (2014).

[4] Bronzi, D. et al. Fast sensing and quenching of CMOS SPADs for mini-mal afterpulsing e↵ects. IEEE Photonics Technology Letters 25, 776–779(2013).

[5] Castello, M., Sheppard, C. J. R., Diaspro, A. & Vicidomini, G. Imagescanning microscopy with a quadrant detector. Opt. Lett. 40, 5355 (2015).

[6] Surdo, S., Carzino, R., Diaspro, A. & Duocastella, M. Single-shot laser ad-ditive manufacturing of high fill-factor microlens arrays. Adv. Opt. Mater.6, 1701190 (2018).

[7] Sanzaro, M. et al. Single-photon avalanche diodes in a 0.16 µm BCDtechnology with sharp timing response and red-enhanced sensitivity. IEEEJ. Sel. Topics Quantum Electron. 24, 1–9 (2018).

[8] The quest for quantitative microscopy. Nature Methods 9, 627–627 (2012).[9] Roth, S., Sheppard, C. J. R., Wicker, K. & Heintzmann, R. Optical photon

reassignment microscopy (opra). Opt. Nanoscopy 2, 5 (2013).[10] Bertero, M., Boccacci, P., Desidera, G. & Vicidomini, G. Image deblurring

with poisson data: From cells to galaxies. Inverse Probl. 25, 123006 (2009).[11] Vicidomini, G., Boccacci, P., Diaspro, A. & Bertero, M. Application of

the split-gradient method to 3D image deconvolution in fluorescence mi-croscopy. J. Microsc. 234, 47–61 (2009).

[12] Vicidomini, G. Three-Dimensional Image Restoration inFluorescence Mi-croscopy. Ph.D. thesis, DISI, University of Genoa (2008).

[13] Leutenegger, M., Rao, R., Leitgeb, R. A. & Lasser, T. Fast focus fieldcalculations. Opt. Express 14, 11277–11291 (2006).

Figure sample dwell Time (µs)

excitation power (nW)

format (pixels)

pixel size (nm/pixel) notes

Fig. 1, dfixed human Hela cells, tubulin filaments stained with Aberrior STAR red

50 280 400 x 400 37.6 fixed sample

Fig. 1, g SiR-Tubulin in live human Hela cells 50

500 (ten frames)

700 x 700 57.2time lapse (20 frames) of live

cells5000 (ten frames)

Fig. 2, afixed human Hela cells, tubulin labelled with Aberrior STAR red 100 500 500 x 500 30 fixed sample

Fig. S11

Fig. 2, b, c Live Human Prostate Carcinoma (PC-3) cells stained with ANEP membrane biomarker

100 3000 800 x 800 62.5 live cellsFig. S12

Fig 3, a, b Whole cleared brain of Thy1-eYFP-H transgenic mouse 30 - 512 x 512

250 (10x objective) cleared brain of

transgenic mouse 123 (20 x

objective)

Fig 3, c

BSC-1 cells from African green monkey kidney, microtubules stained with Alexa 488, mitochondria with Alexa 568

30 - 512 x 512 41 fixed sample

Fig. S3, a fixed sample of 80nm gold beads 50 - 400 x 400 5 fixed sample

Fig. S4Hos cells, nuclear pore labelled with secondary antibody Alexa 488

100 840 1500 x 1500 66.6 fixed sample

Fig. S5 Hek cells, lamin labelled with secondary antibody Alexa 488 100 840 600 x 600 50 fixed sample

Fig. S6 fixed sample of 20 nm red fluorescent beads 50 56 500 x 500 40 fixed sample

Fig. S7fixed human Hela cells, tubulin filaments stained with Aberrior STAR red

10050 < Pexc

< 2000400 x 400 37.6 fixed sample

Fig. S8live human embryonic kidney cells, mitochondria labeled with Mito Tracker Deep Red

30 140 500 x 500 40time lapse (3 minutes, 24

frames) of live cells

Fig. S9 live human Hela cells, tubulin labelled with SiR-Tubulin 50 2 700 x 700 57.2

time lapse (4.5 minutes, 19

frames) of live cells

Fig. S15 fixed human Hela cells, tubulin filaments stained with Alexa 546

0.07 (minimum),

64 line integrations

500 256 x 256 103 fixed sample

Supplementary table 1: Acquisition parameters for all data used in this work.