2174 IEEE TRANSACTIONS ON ULTRASONICS...

13
2174 IEEE TRANSACTIONS ON ULTRASONICS, FERROELECTRICS, AND FREQUENCY CONTROL, VOL. 63, NO. 12, DECEMBER 2016 Lamb Wave Multitouch Ultrasonic Touchscreen Kamyar Firouzi, Amin Nikoozadeh, Thomas E. Carver, and Butrus (Pierre) T. Khuri-Yakub Abstract— Touchscreen sensors are widely used in many devices such as smart phones, tablets, and laptops with diverse applications. We present the design, analysis, and implementation of an ultrasonic touchscreen system that utilizes the interaction of transient Lamb waves with objects in contact with the screen. It attempts to improve on the existing ultrasound technologies, with the potential of addressing some of the weaknesses of the dominant technologies, such as the capacitive or resistive ones. Compared with the existing ultrasonic and acoustic modalities, among other advantages, it provides the capability of detecting several simultaneous touch points and also a more robust per- formance. The localization algorithm, given the hardware design, can detect several touch points with a very limited number of measurements (one or two). This in turn can significantly reduce the manufacturing cost. Index Terms— Lamb waves, learning method, multitouch localization, piezoelectric transducers, training method, ultrasonic touchscreen, ultrasound. I. I NTRODUCTION T OUCHSCREEN sensors are widely used in many devices such as smart phones, tablets, and laptops. There are many different types of modalities that enable sensing the touch. The dominant technologies on the market are the capac- itive, resistive, acoustic or ultrasound, and optical touch sys- tems. None of these technologies is perfect and each has some advantages and disadvantages. Overall, the main difficulties of the current touch technologies are the cost of manufacturing, complexity of the hardware/software, power consumption, and multitouch capability. These have tremendously impeded their widespread applications for large screens. Capacitive touch technologies are the most common in the industry. However, they suffer from hardware complexity, high manufacturing cost, and high power consumption. They may cause problems by affecting other functionalities of the device in which they are installed, such as reducing the optical performance and transparency of the screen and introducing cross talks with other electronics in the device. They work based on conductivity of the touch object, so any nonconduc- tive object cannot be sensed [1]. The main stream ultrasound touch technologies are surface-acoustic waves (SAWs) [2], acoustic pulse recognition (APR) [1], [3], and dispersive signal technology (DST) [1]. The main advantages they offer are simplicity in hardware and low manufacturing cost. They operate based on utilizing SAW or bending waves Manuscript received August 9, 2016; accepted September 9, 2016. Date of publication September 13, 2016; date of current version December 1, 2016. This work was supported by Intel Corporation. The authors are with the Edward L. Ginzton Laboratory, Stanford University, Stanford, CA 94305 USA (e-mail: kfirouzi@stanford.edu). Digital Object Identifier 10.1109/TUFFC.2016.2608781 (APR and DST). Despite the advantages, they share less than 1% of the market. SAWs are highly leaky (into the adjacent medium) or highly attenuating along the path of propagation, thus making SAW technologies extremely sensi- tive to any surface contamination. Bending wave technologies are more robust. However, they require a tap and thus a high activation force to produce enough bending waves to be detected. Overall, ultrasound technologies mainly suffer from lacking robustness (i.e., sensitivity to environmental, mechanical, and thermal noise), multitouch capability, and smooth touch response, making them uncompetitive to analog resistive and capacitive ones. SAWs and bending waves are subclasses of a larger group of guided waves called Lamb waves. Among academic lit- erature, transient Lamb waves induced by finite piezoelectric transducers have been previously attempted in ultrasonic touch systems, where a tactile object is localized through its inter- action with Lamb waves. These involve the interaction of a tactile object such as a human finger with the waves in a solid substrate in either passive [4]–[6] or active [2], [7]–[9] forms. In active designs, the finger acts as an object perturbing the wave field (for example, SAW), whereas in the passive one, the touch object acts as the source of the wave field (for example, APR and DST). The key advantage of the active designs is that they typically have much higher touch sensitivity and smooth response. The other major difference between these proposals is in the localization algorithm. Despite the differences, they all suffer from lack of robustness and multitouch capability. We present the design, analysis, and implementation of an ultrasonic touchscreen system that utilizes the interaction of transient Lamb waves with the objects touching the screen. It attempts to improve on the existing ultrasound technologies, with the potential of addressing some of the weaknesses of dominant technologies, such as capacitive or resistive ones. Compared with existing ultrasonic modalities, among other advantages, it provides the capability of detecting several simultaneous touch points and also a more robust performance. Furthermore, it demands a much less hardware complexity resulting in higher yield, less manufacturing cost, and less operating power consumption. It is sensitive to any touch object that can reflect or absorb sound waves such as a finger, a gloved finger, and a pen. It is flexible to support a wide range of screen sizes from a watch to a projection screen. It works based on sensing the interaction of ultra- sound wave fields with the touch objects. For the proposed mechanism, we present localization algorithms that can detect several touch points with a very limited number of measure- ments (one or two). This in turn significantly reduces the manufacturing cost. 0885-3010 © 2016 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.

Transcript of 2174 IEEE TRANSACTIONS ON ULTRASONICS...

Page 1: 2174 IEEE TRANSACTIONS ON ULTRASONICS ...stanford.edu/group/khuri-yakub/publications/2016_firouzi...2176 IEEE TRANSACTIONS ON ULTRASONICS, FERROELECTRICS, AND FREQUENCY CONTROL, VOL.

2174 IEEE TRANSACTIONS ON ULTRASONICS, FERROELECTRICS, AND FREQUENCY CONTROL, VOL. 63, NO. 12, DECEMBER 2016

Lamb Wave Multitouch Ultrasonic TouchscreenKamyar Firouzi, Amin Nikoozadeh, Thomas E. Carver, and Butrus (Pierre) T. Khuri-Yakub

Abstract— Touchscreen sensors are widely used in manydevices such as smart phones, tablets, and laptops with diverseapplications. We present the design, analysis, and implementationof an ultrasonic touchscreen system that utilizes the interactionof transient Lamb waves with objects in contact with the screen.It attempts to improve on the existing ultrasound technologies,with the potential of addressing some of the weaknesses of thedominant technologies, such as the capacitive or resistive ones.Compared with the existing ultrasonic and acoustic modalities,among other advantages, it provides the capability of detectingseveral simultaneous touch points and also a more robust per-formance. The localization algorithm, given the hardware design,can detect several touch points with a very limited number ofmeasurements (one or two). This in turn can significantly reducethe manufacturing cost.

Index Terms— Lamb waves, learning method, multitouchlocalization, piezoelectric transducers, training method,ultrasonic touchscreen, ultrasound.

I. INTRODUCTION

TOUCHSCREEN sensors are widely used in many devicessuch as smart phones, tablets, and laptops. There are

many different types of modalities that enable sensing thetouch. The dominant technologies on the market are the capac-itive, resistive, acoustic or ultrasound, and optical touch sys-tems. None of these technologies is perfect and each has someadvantages and disadvantages. Overall, the main difficulties ofthe current touch technologies are the cost of manufacturing,complexity of the hardware/software, power consumption, andmultitouch capability. These have tremendously impeded theirwidespread applications for large screens.

Capacitive touch technologies are the most common inthe industry. However, they suffer from hardware complexity,high manufacturing cost, and high power consumption. Theymay cause problems by affecting other functionalities of thedevice in which they are installed, such as reducing the opticalperformance and transparency of the screen and introducingcross talks with other electronics in the device. They workbased on conductivity of the touch object, so any nonconduc-tive object cannot be sensed [1]. The main stream ultrasoundtouch technologies are surface-acoustic waves (SAWs) [2],acoustic pulse recognition (APR) [1], [3], and dispersivesignal technology (DST) [1]. The main advantages theyoffer are simplicity in hardware and low manufacturing cost.They operate based on utilizing SAW or bending waves

Manuscript received August 9, 2016; accepted September 9, 2016. Date ofpublication September 13, 2016; date of current version December 1, 2016.This work was supported by Intel Corporation.

The authors are with the Edward L. Ginzton Laboratory, Stanford University,Stanford, CA 94305 USA (e-mail: [email protected]).

Digital Object Identifier 10.1109/TUFFC.2016.2608781

(APR and DST). Despite the advantages, they share lessthan 1% of the market. SAWs are highly leaky (into theadjacent medium) or highly attenuating along the path ofpropagation, thus making SAW technologies extremely sensi-tive to any surface contamination. Bending wave technologiesare more robust. However, they require a tap and thus ahigh activation force to produce enough bending waves tobe detected. Overall, ultrasound technologies mainly sufferfrom lacking robustness (i.e., sensitivity to environmental,mechanical, and thermal noise), multitouch capability, andsmooth touch response, making them uncompetitive to analogresistive and capacitive ones.

SAWs and bending waves are subclasses of a larger groupof guided waves called Lamb waves. Among academic lit-erature, transient Lamb waves induced by finite piezoelectrictransducers have been previously attempted in ultrasonic touchsystems, where a tactile object is localized through its inter-action with Lamb waves. These involve the interaction of atactile object such as a human finger with the waves in a solidsubstrate in either passive [4]–[6] or active [2], [7]–[9] forms.In active designs, the finger acts as an object perturbing thewave field (for example, SAW), whereas in the passive one, thetouch object acts as the source of the wave field (for example,APR and DST). The key advantage of the active designs is thatthey typically have much higher touch sensitivity and smoothresponse. The other major difference between these proposalsis in the localization algorithm. Despite the differences, theyall suffer from lack of robustness and multitouch capability.

We present the design, analysis, and implementation of anultrasonic touchscreen system that utilizes the interaction oftransient Lamb waves with the objects touching the screen.It attempts to improve on the existing ultrasound technologies,with the potential of addressing some of the weaknesses ofdominant technologies, such as capacitive or resistive ones.Compared with existing ultrasonic modalities, among otheradvantages, it provides the capability of detecting severalsimultaneous touch points and also a more robust performance.Furthermore, it demands a much less hardware complexityresulting in higher yield, less manufacturing cost, and lessoperating power consumption. It is sensitive to any touchobject that can reflect or absorb sound waves such as afinger, a gloved finger, and a pen. It is flexible to supporta wide range of screen sizes from a watch to a projectionscreen. It works based on sensing the interaction of ultra-sound wave fields with the touch objects. For the proposedmechanism, we present localization algorithms that can detectseveral touch points with a very limited number of measure-ments (one or two). This in turn significantly reduces themanufacturing cost.

0885-3010 © 2016 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.See ht.tp://ww.w.ieee.org/publications_standards/publications/rights/index.html for more information.

Page 2: 2174 IEEE TRANSACTIONS ON ULTRASONICS ...stanford.edu/group/khuri-yakub/publications/2016_firouzi...2176 IEEE TRANSACTIONS ON ULTRASONICS, FERROELECTRICS, AND FREQUENCY CONTROL, VOL.

FIROUZI et al.: LAMB WAVE MULTITOUCH ULTRASONIC TOUCHSCREEN 2175

TABLE I

MECHANICAL PROPERTIES OF GLASS

We present a learning (training)-based technique to localizethe touch contacts. Training or learning methods have beenpreviously applied in different contexts such as localizationof tactile objects in contact with a plate [4], [8], [9] andlocalization and classification of defects and flaws in solidsubstrates [10]–[12]. In such methods, generally, the systemis looked at as a black box and any (a posteriori) measureddata are matched with a set of a priori measured data. They areadvantageous in localization problems where the wavefield isseemingly random, chaotic, and thus, extremely complicated,such as the wave propagation in a reverberant or highlyheterogeneous domain.

The organization of this paper is as follows. In Section II,the governing physics and mechanism are explained. Thedesign and prototyping procedures are described in Section III.Following that, in Section IV, the localization algorithms witha multitouch capability are presented. Robustness is discussedin Section V. The experimental tests and verifications are givenin Section VI.

II. GOVERNING PHYSICS

The basic governing principle revolves around the propa-gation of guided elastic waves in a bounded space such asa plate (e.g., a glass screen). One feature it heavily relieson is the propagation of Lamb waves in the screen andtheir leakage upon interfacing with a field-perturbing object(such as a human finger). The second feature is the longtimebehavior of the Lamb waves inside a bounded domain, whichis reminiscent of a reverberant field and transient wave chaos.In this paper, the main interest is in utilizing the time evolutionof the high-frequency modes of a bounded elastic structure.

Lamb waves are guided elastic waves that propagate inthin elastic media. Lamb waves are multimode and dispersivewith a very complicated nature. Dispersion results in severalorthogonal modes. They are classified based on the symmetryof the mode shapes into symmetric (S) and asymmetric (A)modes. The characteristics of Lamb waves in a plate are afunction of thickness, Young’s modulus, Poisson’s ratio, andfrequency. The dispersion curves of an 830 μm thick glassscreen representing the lowest order Lamb modes and thedesired frequency range are plotted in Fig. 1. The correspond-ing mechanical properties are given in Table I.

The lowest order symmetric and asymmetric modes aregenerally abbreviated as, respectively, the S0 and A0 modes.They have unique properties that make them ideal for appli-cations such as nondestructive monitoring of solid substrates.Among these properties, the most useful one is the fact thatthey exist in the entire frequency spectrum, whereas higherorder modes have certain frequency cutoffs, below which theycannot exist [13]. This makes them ideal when the applicationis limited to a bandwidth below the cutoff frequencies of the

Fig. 1. Dispersion curves of an 830 μm thick glass plate representing thelowest order Lamb modes and the desired frequency range. (a) Frequencyversus phase velocity. (b) Wavenumber versus frequency.

higher order modes, as there would be no mode conversioninto other modes. This in turn reduces the complicationsarising from significant dispersion related effects that occurupon utilizing higher order modes. Furthermore, the dispersivebehaviors of these modes are well tolerable compared withthe higher order ones, in particular, in low frequencies, theS0 mode is almost nondispersive and the A0 mode verywell matches the behavior that is predictable using simplereduced order models such as the classical plate theory [14].In addition, as one goes lower in frequency, the phase veloc-ities of the S0 and A0 modes separate more; hence, the wavepackets generally can be separated and analyzed more easilyand accurately. The A0 and S0 modes at the lower end of thespectrum are also much less lossy and more scratch resistantcompared with the higher end of the spectrum (i.e., Rayleighwaves).

In many practical applications, it is favorable to selectivelyexcite the Lamb modes. This can, however, be very challeng-ing, and in this regard, upon isolating the frequency band, theA0 and S0 modes can be robustly and selectively excited usinga proper transducer design [15]. Because of these reasons, theA0 and S0 modes at the lower end of the spectrum are morefavorable than the other modes and those at the higher end ofthe spectrum for the touch-sensing mechanism.

Lamb waves propagating adjacent to a fluid can leakdepending on the velocity of propagation relative to the sur-rounding medium. These waves are called leaky Lamb waves(also called generalized Lamb waves) [13]. They are muchmore complicated in behavior. A human finger to ultrasoundwaves at around a megahertz frequency regime appears as acompressible fluid with negligible shear effects and with a

Page 3: 2174 IEEE TRANSACTIONS ON ULTRASONICS ...stanford.edu/group/khuri-yakub/publications/2016_firouzi...2176 IEEE TRANSACTIONS ON ULTRASONICS, FERROELECTRICS, AND FREQUENCY CONTROL, VOL.

2176 IEEE TRANSACTIONS ON ULTRASONICS, FERROELECTRICS, AND FREQUENCY CONTROL, VOL. 63, NO. 12, DECEMBER 2016

speed of sound at about 1500 m/s, which can in turn leadto the leakage of the Lamb waves into the finger. A glassplate and human finger have a significant impedance mismatchwith air. Lamb waves can also leak into air, however, withmuch less efficiency. This principle makes a human finger (orany object with a close acoustic impedance) create a muchmore pronounced effect on the Lamb waves compared withthe surrounding environment such as air. This property laysout a key feature for a human touch to perturb the Lamb wavesupon interfacing with the glass screen.

Wave propagation in enclosures can lead to mixing of thewave energy, ultimately leading to an incoherent spreading ofinformation. This is the manifestation of a reverberant field,which makes the localization problem very challenging. Rever-berant fields in enclosures can potentially carry useful informa-tion, however, in an incoherent way. Incoherency comes fromconsecutive reflections of the wave energy several times in thedomain. This along with diffraction and dispersion effects canultimately lead to mixing of the wave energy in a seeminglyrandom way. However, spreading of the wave energy in areverberant field can lead to multiple interrogations of eachpoint in the enclosure. This suggests that upon registering alongtime response of the system at only a few locations inthe domain, any substructural changes in the enclosure can besensed with sufficient information carried by the wave energyflow. The Lamb wave touchscreen attempts to reconcile thesekey features of the reverberant field with the benefits of thelowest order Lamb modes.

This, thus, motivates a system consisting of small trans-ducers integrated with a plate. The transducers are pulsedselectively and repeatedly to create propagating Lamb wavesinside the plate. The field is then measured at a selection of thetransducers (which can include the transmitters as well). Uponhaving a touch, a local perturbation is created at the touchedregion, and hence, a portion of the wave field is absorbedthrough the touch(es). This absorption alters the base signals(i.e., the signals measured when there is no touch) in manyways such as by reducing the energy and introducing phaseshifts. Corresponding to different positions of touches, numberof touches, and contact areas, different signatures are inducedon the base signal, making a touch configuration distinct fromother possible touch configurations. As a result of a largenumber of reflections from the boundaries of the plate, aftera while, the whole screen is interrogated several times by thewaves. This implies that every point of the plate is met bythe waves multiple times so that a touch is guaranteed to haveaffected the wave field in a unique way. Furthermore, sincethe geometry is bounded, no information can escape fromthe domain. Thus, all the information will be preserved andaccessible through measurements at the edges leading to themain hypothesis that sufficient information of the perturbedfield can ultimately be registered at a few fixed locationsin or at the boundaries of the plate. We present localizationschemes that benefit from the reverberant field and can reducethe required number of spatial measurements.

In order to verify this concept, we have implemented a full3-D finite-element model that demonstrates the mechanismof the proposed touch system for a 100 mm × 60 mm ×

Fig. 2. Lamb waves induced by the S0 transducer. The colormap scale isin decibels and visualizes the amplitude of the displacement vector field.(a) and (b) Propagation at t = 19 μs with and without a touch object.(c) Registered voltages. Solid line with a no-touch object is shown. Dashedline with a touch object in the middle of the screen is shown. (a) With notouch object, at t = 19 μs. (b) With a touch object, at t = 19 μs. (c) Voltageoutputs with and without the touch object.

0.83 mm screen (see Fig. 2). The details of the model can befound in [16].

A. A0 Versus S0 Modes for Localization

Through studying the forward physics of the system [16],it became apparent that since the A0 mode has considerablymore out-of-plane displacement than S0 mode, it has muchmore touch sensitivity, i.e., a touch contact leaks around 50%of the A0 wave energy compared with around 5% leakage ofthe S0 mode. Furthermore, it is slower than the S0 mode andthis provides a shorter wavelength and thus a better diffraction-limited resolution. This makes this mode ideal for conventionalimaging techniques such as the tomographic approach. For theproblem in hand, sustaining the field reverberations for a longtime window is a key factor. Therefore, it is desired to have

Page 4: 2174 IEEE TRANSACTIONS ON ULTRASONICS ...stanford.edu/group/khuri-yakub/publications/2016_firouzi...2176 IEEE TRANSACTIONS ON ULTRASONICS, FERROELECTRICS, AND FREQUENCY CONTROL, VOL.

FIROUZI et al.: LAMB WAVE MULTITOUCH ULTRASONIC TOUCHSCREEN 2177

a gentle touch sensitivity in order to ensure that the touchmoderately leaks the wave energy and in longtime. Moreover,the S0 mode is faster and hence has the potential of setting upthe reverberant field faster. These, thus, suggest utilizing theS0 mode for localization.

III. DESIGN AND PROTOTYPING

A. Design MethodFor protoytyping, a 20 in × 12 in 830 μm thick glass

plate, as a standard component in manufacturing of tablets [1],was used. The touch system consists of small piezoelectrictransducers glued to the glass plate. An ideal transducer modelis a 1-D piezoelectric element of a finite thickness (andinfinitely thick in the other directions), with two opposingsurfaces metalized in order to provide: 1) electrical outputs and2) a desired electrical field inside the material. In practice, thealignment between the polarization direction and the metal-ization surface determines the mode of operation of a specificdesign. A fundamental arrangement for exciting longitudinalmotions adjacent to media in the front or back is realized whenthe polarization vector is aligned with the electric field.

Piezoelectric materials generally have higher nominal acou-stic impedances with respect to glass. The nominal acousticimpedance is defined as the product of the nominal speedof sound and the mass density. This suggests that theyare the most efficient as half-wavelength resonators [15].Thus, the principle dimension L is chosen to aim for ahalf-wavelength resonator at the resonance frequency fo,resulting in

L = v p

2 fo, v p =

√C D

ρ(1)

C D33 = C E

33(1 + K 2), K 2 = e233

C E33ε33

(2)

where C E33, e33, and ε33 of the corresponding components of

the elasticity, coupling matrix, and permittivity matrices, allrepresented in the Voigt notation. ρ is the mass density.

The coupling efficiency of a piezoelectric material is gen-erally characterized by a coupling coefficient known as k2

T .It quantifies the efficiency of a piezoelectric material inconverting the electrical energy into the mechanical energy,and vice versa. It is given as

k2T = K 2

1 + K 2 . (3)

PZT-5H is among the most efficient piezoelectric materi-als with kT ≈ 0.5 [15]. This gives C D

33 = 157 GPa(v p = 4575 m/s).

The above-mentioned design assumes that the piezoelectrictransducer is infinite in the lateral directions. The real-worldtransducers are, however, finite in size. Even though, accordingto the procedure above, they can be designed to achievea desired thickness-mode performance, the coupling of thelateral modes arising from their finite dimensions in thedirections other than the principle one can have dramaticallyspurious effects on the desired performance. The exact reso-nance frequencies of these modes are difficult to predict using

Fig. 3. S0 bonding configuration, realized by attaching the longitudinaltransducer to the edge of the screen.

the full piezoelectric theory due to the complex coupling of theelastic properties. Nevertheless, the in-plane lateral dimensionH is chosen such that the coupling of the lateral mode to theprinciple mode is minimized, following the method of [17].This theory assumes that only two coupled thickness-moderesonances exist and the other modes are neglected. This inturn results in a biquadratic relation between the two modescharacterized by a coupling coefficient. In the case of PZT-5H,for the aspect ratio G = H/L < 0.6, the mode separation islarge enough to have a safe single mode operation. This leadsto the choice of G ≈ 0.5.

The bandwidth of a piezoelectric transducer to a large extentis determined by the medium in the back and front mechanicalports. Considering an air-backed design bonded to a glassplate, with around a 3:1 impedance mismatch at the frontport, would provide around a 35% bandwidth, according to theKLM model [15]. This suffices to limit the performance belowthe cut-offs of the higher Lamb modes for the present glassprototype; however, it is wide enough to register an enoughbandwidth of information for the localization purpose.

The out-of-plane lateral dimension dictates the diffractioneffects. It is thus kept at around the plate thickness to achievea uniform directivity pattern and minimize the coupling ofthe corresponding lateral mode. Finally, the bonding configu-ration of the longitudinal transducer will be the determiningfactor in the selective excitation of the S0 mode. The properS0 configuration is schematically depicted in Fig. 3.

Following the considerations above, the designed trans-ducers are 1.66 mm × 1 mm × 0.83 mm PZT-5H cuboidelements, with 1.66 mm being the dimension governing theideal thickness-mode resonance. They have the ideal resonancefrequencies at 1.38 MHz, with around a 35% bandwidth. Thisdesign will lead to the predominant propagation of S0 waveswith a typical wavelength of around 4 mm.

B. Prototyping and Assembly

The design has been prototyped by the Microfab Shop ofthe Stanford Nano Shared Facilities at Stanford University.A prototyped touchscreen is shown in Fig. 4. The processincludes the following steps.

1) Polishing the Glass Plate: In this process, the circum-ference of the plate is ground and polished to ensurethe edges are flat. This is essential for a proper contactcondition after the PZT-5H transducers are bonded.

2) Metallization: The edges of the glass plate are metalizedusing Cr and Au. This provides the ground shared by allthe PZT-5H transducers. After the transducers are dicedto the desired dimensions, they are metalized on two of

Page 5: 2174 IEEE TRANSACTIONS ON ULTRASONICS ...stanford.edu/group/khuri-yakub/publications/2016_firouzi...2176 IEEE TRANSACTIONS ON ULTRASONICS, FERROELECTRICS, AND FREQUENCY CONTROL, VOL.

2178 IEEE TRANSACTIONS ON ULTRASONICS, FERROELECTRICS, AND FREQUENCY CONTROL, VOL. 63, NO. 12, DECEMBER 2016

Fig. 4. (a) Attachment of the PZT-5H crystals to the edge of the screen.(b) Fabricated touchscreen prototype.

the faces with Cr and Au (Au over a Cr adhesion layer).One shares the ground (and will be glued to the plate)and the other is connected to an electrical connector,through which the response is measured.

3) Bonding Process: The PZT-5H transducers are bondedto the plate using a low viscosity epoxy mixture(HYSOL RE2039+HD3561). For this, the metalizedPZT-5H crystals are polished down to match the thick-ness of the plate and the plate is sandwiched betweentwo slabs of ultrahigh-molecular-weight polyethylene,which provide for accurate alignment. The transducersare then pushed down toward the edge of the plateby a rod and a weight to glue. The bonding is leftat room temperature for 24 h to cure. This bondingprocess results in a very thin bond line (less than 1 μmthick) where there is enough metal-to-metal contactbetween the transducers metal electrode and the Cr/Aumetalization on the edge of the glass plate to make agood electrical connection between the two faces. Theprocess should be repeated for each transducer.

4) Assembling Process: The plate (with the PZT-5H crys-tals attached to it) is sandwiched at each corner betweentwo rubber washers with a thin sleeve of Teflon inbetween them to protect the edges of the glass platefrom making contact with the metal studs, which supportthem. This mounting arrangement is also for protect-ing the contact metalization around the perimeter ofthe glass plate. The whole setup is then mounted onthe aluminum standoffs provided by the housing. Theglass plate is floating at a fixed distance above thealuminum backing plate. The backing plate is coveredwith a machined square grid pattern, which aids in thepositioning of the finger(s) during testing. The aluminumbacking plate also serves as a limiter to protect the

glass from breaking in case the glass plate is pusheddown with excessive force during testing. The distancebetween the glass and the aluminum backing plate wasarrived at by determining the amount of bow the glassplate could tolerate safely without breaking.

5) Electrical Connection: The metalized face of eachPZT-5H (the one that is not bonded to the plate) isconnected to the connectors (SMA or BNC) using tinplated copper wires, which are bonded to the PZT-5Htransducers using silver epoxy. Similarly, the ground isprovided by bonding the wires to the metalized face ofthe plate.

C. Implementation

The system was implemented using a National InstrumentsNI-PXI5024 digitizer. In order to test the hardware, a functiongenerator was used to pulse the traducers, with a 10 V squarepulse with a 630 ns pulse width. The main lobe of this pulseis bandlimited below 2 MHz to assure negligible excitation ofthe higher order modes. The transducer design as explainedin the previous section assures a dominant excitation ofthe S0 waves. There could, however, be a small contributionof A0 waves, coming from slight coupling of the lateral modeof the transducers and mode conversion at the boundaries.The responses are then measured at the other transducers. Theacquisition setup is shown in Fig. 5(a). In order to register theresponses, a customized acquisition program was developedin the National Instruments LabVIEW 2012 programmingenvironment. Registered signals at a receiver at the right edgein response to a source at the opposite edge are presentedin Fig. 5(c) [and the spectra in Fig. 5(d)], without any touchobject and with a human finger in the middle of the screen,confirming the system is functional. The measurement setupis shown in Fig. 5(b).

Some of the expected wave features can be observed,namely, the sound is diffusive in a longtime scale (at about2 ms), with high-frequency oscillations at around 1 μs.As can be seen, a human touch perturbs the registered wavefield weakly and randomly at different times. The other featureis the bandwidth of the response, which is about 35% andlimited by −50 dB below the cutoff frequencies of the higherorder Lamb modes (below 2 MHz for the present glassprototype).

The acquisition rate depends on the amount of time–datathat must be acquired, which in turn depends on the reverber-ant time and how much of which is deemed adequate for thelocalization. For the present prototype, a 2 ms time window ofdata and about 8 ms processing time based on the algorithmto be presented in the subsequent section lead to a 100 Hzacquisition rate.

IV. LOCALIZATION ALGORITHM

We propose a learning (training) method to localize thetouch contacts. The learning method provides a black-box treatment of the system, implying that the entirealgorithm can be implemented experimentally. The learn-ing method, upon an experimental implementation, con-sists of two steps. i) Training Step: The screen is touched

Page 6: 2174 IEEE TRANSACTIONS ON ULTRASONICS ...stanford.edu/group/khuri-yakub/publications/2016_firouzi...2176 IEEE TRANSACTIONS ON ULTRASONICS, FERROELECTRICS, AND FREQUENCY CONTROL, VOL.

FIROUZI et al.: LAMB WAVE MULTITOUCH ULTRASONIC TOUCHSCREEN 2179

Fig. 5. Test measurement and acquisition setup. (a) Hardware implementa-tion. (b) Measurement setup. (c) Registered waveforms at an S0 transducer.(d) Fourier spectra of the registered waveforms.

at selective points with controlled uniform contact areas.The corresponding measurements to each test along with thewaveforms of the no-touch condition are stored in the memory.ii) Localization Step: Upon having a touch, the measureddata at each receiver is matched with the training set thatcorresponds to the same transducer. These steps are furtherelaborated in the following sections.

A. Training Step

For a given transmit–receive pair, the screen is touchedusing an ultrasound-absorptive phantom (i.e., a material withan acoustic impedance close to that of a touch object such as ahuman finger) over a set of points arranged over a rectangulargrid. A suitable material for this purpose is Sylgard-160 witha 1.6 MRayls impedance close to that of soft tissues. It is castas a cylinder with a slightly curved end to assure a propercontact radius around 4–5 mm once placed on the screen, upona 1–2 N force. The corresponding signals are acquired andstored in a hard drive. The size of the phantom as well asthe system parameters such as the sampling rate, number ofacquired samples, and spacing between the training pointsdepend on the size of the screen, frequency content of theinput, and accuracy and resolution of interest. After storingthe raw signal, several processing techniques are performedincluding, but not limited to, filtering. The training wave-forms as column vectors are stacked together in an Nt × Nc

matrix M, where Nt is the number of acquired time samplesand Nc is the number of training points (i.e., spatial samples).The training waveforms construct a training set.

B. Localization Step

Upon having a touch interaction, the measured signal at thereceiver undergoes a similar signal processing to that ofthe training set. The measured signals are then corrected forthe drift and noise of the system (see Section V). The intuitiveidea behind the learning approach emanates from projectingthe touch contact absorptivity or reflectivity �(x) over a finitebasis set of simple functions, that is, suppose

�(x) ≈Nc∑

i=1

θiχxi (x, ai), θi ∈ R (4)

where χxi (x, ai ) is an indicator function centered at xi with|supp(χxi (x, ai ))| ≈ a2

i , i.e., the induced absorption or reflec-tion by a collection of objects can be constructed by summingover the induced effect of some reference objects. Nc is asuitably chosen number. It can then be shown that

δd ≈Nc∑

i=1

θiδdi (5)

where δdi is the system response to the χxi (x, ai) as a touchcontact and δd is the system response to the total touchfunction �(x). This implies the measured data due to thepresence of an unknown object can be considered as a linearcombination of a set of measurements corresponding to priorlocations of objects and with the same projection coefficients.The resolution of such an approximation obviously depends onhow well the parameter functions can be approximated by theassumed set of simple functions. Ideally, the training touchcontacts should have a finer resolution (i.e., smaller contactareas) than the tests and also should be nonoverlapping andcover the entire domain in order to have the best reconstruc-tion. This, however, may not be the best choice from thepractical point of view because the size of the data and/orhardware limitations. Note that this method merely requires

Page 7: 2174 IEEE TRANSACTIONS ON ULTRASONICS ...stanford.edu/group/khuri-yakub/publications/2016_firouzi...2176 IEEE TRANSACTIONS ON ULTRASONICS, FERROELECTRICS, AND FREQUENCY CONTROL, VOL.

2180 IEEE TRANSACTIONS ON ULTRASONICS, FERROELECTRICS, AND FREQUENCY CONTROL, VOL. 63, NO. 12, DECEMBER 2016

measurements (observations). This implies the system can belooked at as a black box, for which a limited knowledgemay be available. This offers an experimental approach tothis problem; in cases that computing reference measurementsis difficult, the learning theory can be utilized to teach theoperator M by training the system by a prior set of referencemeasurements, which will be henceforth referred to as thetraining set.

Mathematically, this method is reminiscent of consideringthe references as bases for a vector space spanned by thetraining set and then trying to find the projection of anarbitrary measurement in that space. The operator M can bethought of as a matrix with N columns and infinite rows(experimentally very large, ≈ 105), i.e., a matrix with thereference measurements as the columns. We also remark thatthe reference measurements may not generally be orthogonal(for a weak object, in fact, they can be very close). Let δd(t)be a measurement and δd(t) = δd‖(t)+δd⊥(t), the orthogonaldecomposition of it, where δd‖(t) ∈ D, δd⊥(t) ∈ L2\D andD = span{δdi (t)}N

i=1. The projection operator in terms of thedata matrix is M(M†M)−1M†, where M† is the adjointof M. Hence

δd‖ = M(M†M)−1M†δd. (6)

Upon projecting an arbitrary measurement onto the trainingdata space, we expand it as a linear combination of the bases(i.e., the reference measurements), that is, to write δd‖(t) =∑N

i=1 θiδdi (t) = M�, � ∈ RN , � = 〈θ1, · · · , θN 〉†.

Combining this with the previous equation gives

� = (M†M)−1M†δd (7)

which is equivalent to solving a least-squares problem

min�∈RN

1

2‖M� − δd‖2

L2([0,T ]). (8)

When there exist a number of sources and receivers (say Ns

and Nr , respectively), we can extend the above formulation to

min�∈RN

1

2

∑r,s

μr,s‖Mr,s� − δdr,s‖2L2([0,T ]) (9)

where μr ’s are the weighting parameters. Mr,s and δdr,s arethe data matrix and the measured signal at the r th receiverin response to the sth source. The proposed learning method,upon utilizing the entire reverberant field and longtime data,requires a very limited number of spatial measurements(one or two). Furthermore, since the linear combination ispointwise in time, the entire representation is independent ofaliasing in time. Hence, it offers a great flexibility in terms ofthe sampling requirements in space or time.

C. Variations and Improvements

As aforementioned, the measurements can be close to oneanother (in the energy norm) in the training data space.Furthermore, measuring or constructing the perturbation δdgenerally is not a robust approach, since it should be subtractedfrom do (the measurement with no field-perturbing objectspresent in the medium). A more robust alternative is to

augment the data space by the base measurement do, withthe corresponding projection coefficient θo, in which case itresults in a new constraint

N∑i=0

θi = 1. (10)

The underlying physics motivates to enforce a positivityconstraint. This is because of the positive definiteness andstability of the system [18]. The physical interpretation is thatthe entire system (including the object and medium) eitherconserves or loses the total wave energy. This, in turn, leadsto a positivity constraint: θi ≥ 0, for all i .

Furthermore, when it is believed that the distribution ofobjects is sparse, the optimization problem can be penalizedby a sparsity promoting constraint. This is practically imposedby penalizing the problem through the l1 norm [19], [20].However, given the constructed constraints, the overall schemecan be conveniently implemented as

min�∈RN

1

2

∑r,s

μr,s‖Mr,s� − dr,s‖2L2([0,T ]) (11a)

s.t. (11b)

θi ≥ 0, for all i (11c)

μ

N∑i=1

θi = 1 (11d)

where μ is a penalty parameter. This variant can improve thesuccess of the localization. However, another more powerfulvariant can be introduced by reformulating the problem in theimage space as opposed to the data space. The space spannedby all possible configurations of � is called the image space,denoted by I. This suggests posing the localization problemas a minimization in the image space with essentially the sameconstraints as before. That is

min�∈RN

1

2

∑r,s

μr,s‖� − (M†r,sMr,s)

−1M†r,sdr,s‖2

L2(I)(12a)

s.t. (12b)

θi ≥ 0, for all i (12c)

μ

N∑i=1

θi = 1. (12d)

This algorithm can be implemented as a two-step method.

Step 1: Solve the original unconstrained least squares

�∗r,s = arg min

�r,s ∈RN

1

2‖Mr,s�r,s − dr,s‖2

L2([0,T ]). (13)

Step 2: Solve a constrained least squares as follows:

min�∈RN

1

2

∑r,s

μr,s‖� − �∗r,s‖2

L2(I)(14a)

s.t. (14b)

θi ≥ 0, for all i (14c)

μ‖θ‖l1 =N∑

i=1

θ∗i . (14d)

Page 8: 2174 IEEE TRANSACTIONS ON ULTRASONICS ...stanford.edu/group/khuri-yakub/publications/2016_firouzi...2176 IEEE TRANSACTIONS ON ULTRASONICS, FERROELECTRICS, AND FREQUENCY CONTROL, VOL.

FIROUZI et al.: LAMB WAVE MULTITOUCH ULTRASONIC TOUCHSCREEN 2181

We remark that all the above-mentioned processing stepscan be implemented using the Fourier transformed data.Since the system is bandlimited, this would lead to a signifi-cant reduction in the computation time and memory, once onlythe in-band information is utilized in the inversion process.In addition, the training set may be constructed using acomputational model. However, the computation process maylack robustness, and be cumbersome and intense.

V. ROBUSTNESS

Imaging systems in bounded domains have a finite band-width, which is reminiscent of the quality factor of thesystem itself or due to the transducers [15], whence only alimited bandwidth of the information is registered. There couldgenerally be two types of noise sources. 1) Additive noise,which appears as high-frequency fluctuations with generallya normal probability distribution. This noise can be easilyfiltered by a basic IIR or FIR filter [21]. 2) Multiplicative noise,which can be viewed as a convolution of a random functionwith the underlying true response of the system (which will behenceforth referred to as drift). Filtering this type of noise canbe challenging. The second type of the noise is to a large extentunknown and uncertain and cannot be estimated or controlledto the precision required for the inversion process.

Many factors can potentially contribute to this noise type,such as temperature, temperature gradient, mechanical noise,and uncertainties coming about due to stresses and fatiguein time. This motivates to construct a methodology for ablind estimation and compensation of the drift, which canbe applied to adapt an a posterior training set to the priorone. Let {di }Nc

i=0 and {di }Nci=0 be, respectively, the posterior

(drift-affected) and prior (drift-free) training sets. This processmay go under different names such as restoration, registration,and deblurring [22]. Suppose Dr is the operator that mapsthe prior base (background) measurement (with no objects orperturbations) to the corresponding posterior one. This is anexample of a regularized inverse filtering. Now we can alsoadd an additive noise term n(t) to the system, which can bethought of as the difference between the white noise in theprior and posterior data models

do(t) = (Dr do)(t) + n(t). (15)

A Wiener filter attempts to construct Dr such that the expectedvalue of the energy of the error n(t) is minimized

Dr = arg minDr

E[n]2. (16)

This gives [22]

Dr (ω) =( eo

eo

)Sdd∣∣ eo

eo

∣∣Sdd + Snn(17)

where Sdd and Snn are the (auto)power spectral densities of themeasurement and noise, respectively, and Dr (ω) is the Fourierkernel of Dr . (·) is the complex conjugate of (·).

In practice, upon measuring of the background field, thedrift operator is constructed as shown above. Next, an arbitrary

Fig. 6. In-laboratory implementation of the learning algorithm.

measurement that corresponds to an unknown object is mappedto a corresponding prior model using the drift operator

d(t) = (Dr d)(t) (18)

where d(t) can now be used against the prior training library.

VI. EXPERIMENTAL RESULTS

A. Experimental Setup

Fig. 6 represents the in-laboratory implementation of theexplained procedure. The training procedure consists of onetransmitter and one receiver. The domain enclosed in the boxwas chosen as the training domain. A set of grid lines witha half-inch grid spacing were patterned underneath the glassscreen on the aluminum substrate in order to provide guidancefor the training procedure. The screen was then trained on theregions indicated by solid discs, which approximately form aclose nonoverlapping touch contact areas on the order 0.5 cm2

covering the entire training domain. This forms a totalof 91 training measurements in addition to the data corre-sponding to the no-touch case. The system was implementedusing a National Instruments NI-PXI5024 digitizer, witha 12 b vertical resolution. A function generator was used topulse an S0 transducer, with a square pulse with a 630 ns pulsewidth. The transmitter at the right edge is pulsed using thefunction generator and the response is measured at the receiverat the opposite edge. The data were acquired at 50 MS/scorresponding to a 50 MHz sampling frequency and witha 2 m time window, resulting in 105 time samples. Thelocalization algorithms were implemented at this samplingfrequency.

B. Comparison With the Existing Algorithms

Among the literature, with a similar detection mechanism,two types of algorithms can be found: the correlation-basedlocalization [4] and localization using the Manhattan (l1)norm [8], [9]. Applying the projection algorithm (8), a com-parison of the three different algorithms for a case of a singletouch test and a three-touch test are shown in Figs. 7 and 8,respectively. The results notably demonstrate a better perfor-mance of the proposed method compared with the previousmethods.

Page 9: 2174 IEEE TRANSACTIONS ON ULTRASONICS ...stanford.edu/group/khuri-yakub/publications/2016_firouzi...2176 IEEE TRANSACTIONS ON ULTRASONICS, FERROELECTRICS, AND FREQUENCY CONTROL, VOL.

2182 IEEE TRANSACTIONS ON ULTRASONICS, FERROELECTRICS, AND FREQUENCY CONTROL, VOL. 63, NO. 12, DECEMBER 2016

Fig. 7. Comparison of the projection method with the existing methodsfor a single-touch test. (a) Exact touch location. (b) Projection method.(c) Manhattan method [8]. (d) Correlation method [4].

Fig. 8. Comparison of the Projection method with the existing methodsfor a three-touch test. (a) Exact touch location. (b) Projection method.(c) Manhattan method [8]. (d) Correlation method [4].

C. Projection Versus Image Space Methods

As pointed out earlier, increasing the number of the touchpoints degrades the performance of the projection algorithm,for which case the image space algorithm was presented (12)as the substitute. For a case of 5- and 11-touch tests, theperformance of the projection versus image space algorithmsis demonstrated in Figs. 9 and 10. Further test results for thecases of 9, 10, and 12 touches are, respectively, presented inFigs. 11–13. The image space method relies on a regularizationparameter. The effect of this parameter on the location resultof the 11-touch test is presented in Fig. 14. We remarkthat it is very likely that one or several of the test fingers

Fig. 9. Localization using the projection method versus the image spacemethod for a five-touch test. (a) Exact touch locations. (b) Projection method.(c) Image space method.

were misplaced. In the worst case scenario, if a finger ismisplaced by, for example, 1/4 of an inch, i.e., half the traininggrid, the localized touch would be a linear combination for theadjacent points, which will show nonzero amplitudes (highcontrast with respect to the background). For example, inthe case the 12-touch test (Fig. 13), one of the fingers wasmisplaced by half the training grid and because of this at thelocation of this finger, two grid points are showing contrastcompared with the background.

D. Robustness

In order to evaluate the influence of the multiplicativenoise and the performance of the compensating algorithm,we conducted a controlled experiment. We considered twodifferent scenarios. As the first scenario, the screen was kept ata drift-inducing environment for two weeks and a three-touchmeasurement was collected. Next, for the second scenario, wecreated a temperature gradient (around 45 in2 in area) withthe peak temperature at around 42 °C in a close proximityof the receiver (see Fig. 16) and then collected a three-touch

Page 10: 2174 IEEE TRANSACTIONS ON ULTRASONICS ...stanford.edu/group/khuri-yakub/publications/2016_firouzi...2176 IEEE TRANSACTIONS ON ULTRASONICS, FERROELECTRICS, AND FREQUENCY CONTROL, VOL.

FIROUZI et al.: LAMB WAVE MULTITOUCH ULTRASONIC TOUCHSCREEN 2183

Fig. 10. Localization using the projection method versus the image spacemethod for an 11-touch test. (a) Exact touch locations. (b) Projection method.(c) Image space method.

measurement. In both cases, the training set explained abovewas used in localizing the touch points. In Figs. 15 and 17,for both scenarios, the dramatic effect of the drift on thelocalization results are shown side by side the localizationresults attained by using the drift-compensating algorithm.Through experimentation, it appeared that temperature gradi-ents are more detrimental to the localization performance thana uniform temperature elevation. One likely explanation is thatin addition to slowing down or speeding of the impinging rays,it can also potentially bend the rays. The average wavelength inthis particular prototype is about 4 mm and the area coveredis several wavelengths across. Furthermore, since we utilizeall the reverberant data, the area over which the temperaturegradient is imposed is seen many times by the waves in theprocess of reverberation and incoherent propagation, leadingto further complications in the measured data.

E. Remark on Resolution

A natural question arising in the context of any imagingtechnique is about the resolution limit, identified by the

Fig. 11. Localization of nine simultaneous touches using the image spacemethod. (a) Exact touch locations. (b) Image space method.

Fig. 12. Localization of ten simultaneous touches using the image spacemethod. (a) Exact touch locations. (b) Image space method.

minimum size of a resolvable (identifiable) object. It is essen-tially diffraction limited and generally in the order a half-wavelength in classical techniques [23]. The answer to thisquestion, however, is more subtle for the proposed learningalgorithm, as the entire reverberant field is treated.

Page 11: 2174 IEEE TRANSACTIONS ON ULTRASONICS ...stanford.edu/group/khuri-yakub/publications/2016_firouzi...2176 IEEE TRANSACTIONS ON ULTRASONICS, FERROELECTRICS, AND FREQUENCY CONTROL, VOL.

2184 IEEE TRANSACTIONS ON ULTRASONICS, FERROELECTRICS, AND FREQUENCY CONTROL, VOL. 63, NO. 12, DECEMBER 2016

Fig. 13. Localization of 12 simultaneous touches using the image spacemethod. (a) Exact touch locations. (b) Image space method.

Fig. 14. Effect of the regularization parameter on the image space method.(a) γ = 0.75. (b) γ = 1.25.

A second measure of resolution can be considered as theminimum distance by which an object (e.g., a training basis)can be offset and yet results in a unique identification of thesaid change. This limit can be well below a wavelength as longas the object size is in the resolvable regime discussed above.This is by virtue of utilizing the entire reverberant field.

Fig. 15. Effect of the environmental and mechanical noises on the localizationand performance of the compensation algorithm. (a) Exact touch locations.(b) Without compensation. (c) With compensation.

Fig. 16. Temperature map using an infrared camera in the controlled studyof the effect of the thermal noise on the performance of the localizationalgorithm.

As long as an object can create sufficient perturbations,through the action of longtime propagation and reverberation,a set of distinct features can be registered that can suffice todistinguish it from its subwavelength neighboring locations.

Page 12: 2174 IEEE TRANSACTIONS ON ULTRASONICS ...stanford.edu/group/khuri-yakub/publications/2016_firouzi...2176 IEEE TRANSACTIONS ON ULTRASONICS, FERROELECTRICS, AND FREQUENCY CONTROL, VOL.

FIROUZI et al.: LAMB WAVE MULTITOUCH ULTRASONIC TOUCHSCREEN 2185

Fig. 17. Effect of the thermal noise on the localization and performance of thecompensation algorithm. (a) Exact touch locations. (b) Without compensation.(c) With compensation.

VII. CONCLUSION

This paper presented a successful design and implementa-tion of an ultrasonic touchscreen system capable of detectinga multiple simultaneous touch contacts and with a high touchsensitivity. It attempts to reconcile the benefits of Lambwaves and field reverberation in the screen as the governingmechanism. It relies on the longtime reverberation of thewaves inside the screen, where potentially any informationinduced by a field-perturbing object such as a touch contactinterrogates the entire screen several times before reachingout to the receiver(s). The proposed technology utilizes theminimum number of transducers for a successful localization.Adding more transducers can help improve the quality of thelocalization. It offers a cost-effective technology with a simplehardware architecture. It is sensitive to any touch object thatcan reflect or absorb ultrasound such as a finger, a glovedfinger, and a pen. It is flexible to support a wide range ofscreen sizes, from a watch to a projection screen.

A proper design strategy was presented to achieve a desiredperformance. The main design features include a proper

identification of the lowest order Lamb modes and selec-tive excitation of them upon proper transducer designs. Theproposed design utilizes a learning method to localize thetouch contacts. The chief advantage the algorithm offers isthe capability of reducing the number of spatial measurementsby virtue of utilizing the temporal information beyond theclassical limit, in both coherent and incoherent phases ofpropagation. The learning algorithm benefits from the entirereverberant field leading, in turn, to merely a single source–receiver pair. The learning method relies on a prior setof measurements and is constructed based on finding theprojection of any arbitrary measurement in the space spannedby the prior set. This is particularly important in systems witha limited available knowledge and immense uncertainties. Thealgorithm calls for the minimum knowledge of the system, andfor the most part, looks at it as a black box. Several differ-ent improvements of the algorithm were presented based onmotivations from the physics or operational conditions of thesystem. A methodology to compensate for the environmentaland thermal noise was also presented, aiming at improving thestability of the learning algorithm. Investigating the stabilityof the learning algorithm to surface contaminants is the nextkey step and is left as a future direction.

The presented prototype is just one of many possible combi-nations of transducer types and orientations, source–receivercombinations, and frequencies of operation. In practice, thescreen is integrated with other components such as differentlayers of thin films. They can change the effective thickness,material properties, and boundary conditions, which essen-tially may affect the characteristics of the propagating Lambwaves and the reverberant field. This opens up many directionsfor the future works to understand how the performance isaffected by the said variations and how the system hereincan be optimized for the best performance. Furthermore, thesystem probably is not at the best combination of transducersfor the best performance of the localization methods. Theperturbed field due to a touch at different locations maynot be equal in behavior as the statistical properties of theevolving perturbed modes can be different and the localizationalgorithms utilize the coherent phase of propagation as wellas the incoherent one, and the touch contacts in the vicinity ofthe direct propagation path between a transmitter–receiver paircreate more pronounced perturbations. On-chip implementa-tion of the proposed system and integration with electronicsis yet another important direction for the future works, posingkey questions of optimizing the power consumption, amount ofmemory required for the optimal performance, required framerate, and so on.

REFERENCES

[1] G. Walker, “A review of technologies for sensing contact location on thesurface of a display,” J. Soc. Inf. Display, vol. 20, no. 8, pp. 413–440,2012.

[2] R. Adler and P. J. Desmares, “An economical touch panel using SAWabsorption,” IEEE Trans. Ultrason., Ferroelect., Freq. Control, vol. 34,no. 2, pp. 195–201, Mar. 1987.

[3] K. North and H. D’Souza, “Acoustic pulse recognition enters touch-screen market,” Inf. Display, vol. 22, no. 12, pp. 22–25, 2006.

Page 13: 2174 IEEE TRANSACTIONS ON ULTRASONICS ...stanford.edu/group/khuri-yakub/publications/2016_firouzi...2176 IEEE TRANSACTIONS ON ULTRASONICS, FERROELECTRICS, AND FREQUENCY CONTROL, VOL.

2186 IEEE TRANSACTIONS ON ULTRASONICS, FERROELECTRICS, AND FREQUENCY CONTROL, VOL. 63, NO. 12, DECEMBER 2016

[4] R. Kiri, N. Quieffin, S. Catheline, and M. Fink, “In solid localization offinger impacts using acoustic time-reversal process,” Appl. Phys. Lett.,vol. 87, no. 20, p. 204104, 2005.

[5] S. Reis et al., “Touchscreen based on acoustic pulse recognition withpiezoelectric polymer sensors,” in Proc. IEEE Int. Symp. Ind. Electron.,Jul. 2010, pp. 516–520.

[6] V. G. Reju, A. W. H. Khong, and A. B. Sulaiman, “Localization of tapson solid surfaces for human-computer touch interfaces,” IEEE Trans.Multimedia, vol. 15, no. 6, pp. 1365–1376, Oct. 2013.

[7] Y. Liu, J. P. Nikolovski, N. Mechbal, M. Hafez, and M. Vergé, “A multi-touch plate based on Lamb wave absorption,” Procedia Chem., vol. 1,no. 1, pp. 156–159, 2009.

[8] Y. Liu, J. P. Nikolovski, N. Mechbal, M. Hafez, and M. Vergé, “Anacoustic multi-touch sensing method using amplitude disturbed ultra-sonic wave diffraction patterns,” Sens. Actuators A, Phys., vol. 162,no. 2, pp. 394–399, 2010.

[9] J.-P. Nikolovski, “Moderately reverberant learning ultrasonic pinchpanel,” IEEE Trans. Ultrason., Ferroelect., Freq. Control, vol. 60, no. 10,pp. 2105–2120, Oct. 2013.

[10] D. Chetwynd, F. Mustapha, K. Worden, J. A. Rongong, S. G. Pierce,and J. M. Dulieu-Barton, “Damage localisation in a stiffened compositepanel,” Strain, vol. 44, no. 4, pp. 298–307, Aug. 2008.

[11] D. Chetwynd, J. A. Rongong, S. G. Pierce, and K. Worden, “Damagedetection in an aluminium plate using outlier analysis,” Fatigue FractureEng. Mater. Struct., vol. 31, no. 8, pp. 629–643, Aug. 2008.

[12] C. Larrosa, K. Lonkar, and F.-K. Chang, “In situ damage classificationfor composite laminates using Gaussian discriminant analysis,” Struct.Health Monitor., vol. 13, no. 2, pp. 190–204, Mar. 2014.

[13] B. A. Auld, Acoustic Fields and Waves in Solids, vol. 2, 2nd ed.Melbourne, FL, USA: Krieger, 1990.

[14] R. D. Mindlin, An Introduction to the Mathematical Theory of Vibrationsof Elastic Plates, J. Yang, Ed. Hackensack, NJ, USA: World Scientific,2006.

[15] G. S. Kino, Acoustic Waves: Devices, Imaging, and Analog SignalProcessing. Englewood Cliffs, NJ, USA: Prentice-Hall, 1987.

[16] K. Firouzi, A. Nikoozadeh, and B. T. Khuri-Yakub, “Numerical mod-eling of ultrasonic touchscreen,” in Proc. IEEE Ultrason. Symp.,Sep. 2014, pp. 753–756.

[17] M. Onoe and H. F. Tiersten, “Resonant frequencies of finite piezoelectricceramic vibrators with high electromechanical coupling,” IEEE Trans.Ultrason. Eng., vol. 10, no. 1, pp. 32–38, Jul. 1963.

[18] L. C. Evans, Partial Differential Equations. Providence, RI, USA: AMS,1998.

[19] S. Boyd and L. Vandenberghe, Convex Optimization. Cambridge, U.K.:Cambridge Univ. Press, 2004.

[20] A. Chai, M. Moscoso, and G. Papanicolaou, “Robust imaging oflocalized scatterers using the singular value decomposition and �1minimization,” Inverse Problems, vol. 29, no. 2, p. 025016, Feb. 2013.

[21] A. V. Oppenheim, Discrete-Time Signal Processing. Upper Saddle River,NJ, USA: Pearson, 2010.

[22] T. F. Chan and J. Shen, Image Processing and Analysis: Variational,PDE, Wavelet, and Stochastic Methods. Philadelphia, PA, USA: SIAM,2005.

[23] A. J. Devaney, Mathematical Foundations of Imaging, Tomography andWavefield Inversion. Cambridge, U.K.: Cambridge Univ. Press, 2012.

Kamyar Firouzi received the M.S. degree inmechanical engineering from University CollegeLondon (UCL), London, U.K., in 2010, and thePh.D. degree in mechanical engineering from Stan-ford University, Stanford, CA, USA, in 2016. Forhis Ph.D. dissertation, he studied localization ofobjects in chaotic and reverberant enclosures, basedon which he developed a Lamb-wave multitouchultrasonic touchscreen system.

From 2010 to 2011, he was a Research Assis-tant with UCL, where he developed a predictive

computational tool for the evaluation of photoacoustic imaging techniquesfor the detection of brain tumors. He has worked on numerous problemsin ultrasound/MEMS technologies, including the modeling and designingof ultrasonic transducers, photoacoustics, microbubbles, wave propagation,and numerical methods. His current research interests include transcranialultrasound, ultrasound neuromodulation, ultrasonic flow-measurement, andultrasound signal processing and inverse problems.

Amin Nikoozadeh received the B.S. degree in elec-trical engineering from Sharif University of Tech-nology, Tehran, Iran, in 2002, and the M.S. andPh.D. degrees in electrical engineering from Stan-ford University, Stanford, CA, USA, in 2004 and2010, respectively, where he designed and devel-oped fully-integrated ultrasound imaging cathetersfor forward-viewing intracardiac imaging usingcapacitive micromachined ultrasonic transducers(CMUTs).

From 2011 to 2014, he was a Research Associateand from 2014 to 2015, he was a Senior Research Engineer with theEdward L. Ginzton Laboratory, Stanford University. His previous and currentresearch interests include ultrasound imaging, image-guided therapeutics,MEMS, and circuit design, with a main focus on design, modeling, fabrication,integration, and applications of CMUTs.

Thomas E. Carver received the B.A. degree inindustrial design from San Francisco State Univer-sity, San Francisco, CA, USA, with an emphasis onaudio and acoustics.

He has worked for Stanford University, Stanford,CA, USA, for over 30 years, first with the GinztonLaboratories, and is currently with Stanford NanoShared Facilities as a Science and Engineering Asso-ciate II. He manages the SNSF Flexible Cleanroomand the SNSF Microfab Shop. His specialties are inthe fields of thin film deposition, process develop-

ment, design and fabrication work, equipment design and fabrication, andother technical consultation services. He holds four patents in the field ofAFM and STM tip fabrication, and one patent pending.

Butrus (Pierre) T. Khuri-Yakub received the B.S.degree in electrical engineering from the AmericanUniversity of Beirut, Beirut, Lebanon, the M.S.degree in electrical engineering from DartmouthCollege, Hanover, NH, USA, and the Ph.D. degreein electrical engineering from Stanford University,Stanford, CA, USA.

He is currently a Professor of Electrical Engineer-ing with Stanford University. He has authored over550 publications and holds 93 U.S. and internation-ally issued patents. His current research interests

include medical ultrasound imaging and therapy, ultrasound neurostimulation,chemical/biological sensors, gas flow and energy flow sensing, micromachinedultrasonic transducers, and ultrasonic fluid ejectors.

Dr. Khuri-Yakub was awarded the medal of the City of Bordeaux in1983 for his contributions to nondestructive evaluation, the DistinguishedAdvisor Award of the School of Engineering at Stanford University in 1987,the Distinguished Lecturer Award of the IEEE UFFC Society in 1999, theStanford University Outstanding Inventor Award in 2004, the DistinguishedAlumnus Award of the School of Engineering of the American Universityof Beirut in 2005, the Stanford Biodesign Certificate of Appreciation forcommitment to educate, mentor, and inspire Biodesign Fellows in 2011, andwas the 2011 recipient of the IEEE Rayleigh Award.