12 Computer P u b l i s h e d b y t h e I E E E C o m p u t e r S o c i e t y

I N D U S T R Y T R E N D S

I n the movie Minority Report,actor Tom Cruise’s computerinterface consisted of transpar-ent screens set in space. He usedhis hands to play the touch

screens, sliding one piece of datahere, bringing up a program there.

Until recently, touch-screen tech-nology has not been nearly soadvanced but instead has been lim-ited to a handful of narrow-use appli-cations with limited functionalitysuch as automated teller machines,gas pumps, museum displays, airline-ticket kiosks, and PDAs.

No one is predicting that touchdisplays will soon replace the tradi-tional keyboard-and-mouse inter-face for general-purpose computing.However, they are beginning tomove into mainstream technology.

Instead of single-purpose termi-nals, multifunctional touch screensare now being included in mobilephones such as Apple’s recently intro-duced iPhone, as well as Microsoft’ssoon-to-be-released Surface tabletopcomputer.

Chinese companies such as Deeda,Hua Long, and Meizu ElectronicTechnology are also planning to selltouch-screen mobile phones.

And other mobile devices, such asglobal positioning systems like theNextar Q3, and universal televisionremote controls, like Logitech’sHarmony 1000, are appearing withmultifunction touch screens.

PCs such as Hewlett-Packard’snew TouchSmart IQ770 desktop arealso using the technology.

Touch screens offer the promise ofa more intuitive interface that makesit easier for multiple users to worktogether on the same machine andthat can save valuable space by elim-inating physical keyboards on smallmobile devices.

For the technology to take off,though, it needs a catalyst, saidStuart Robinson, director of hand-set-component technologies forStrategy Analytics, a market-researchfirm.

“The iPhone will probably do thejob. It’s already spawning copycatsand the touch-screen industry is see-ing much greater demand than dur-ing 2006,” he explained.

He predicted that touch screenswill appear in 40 percent of mobiledevices in 2012, up from just 3 per-cent in 2006.

Meanwhile, researchers are work-ing on new touch-screen approachesand new ways to use the technology.However, they must also overcome

technical and marketplace barriersto widespread adoption.

PUSHING THE TOUCH SCREENThe falling price of touch-screen

technology, due largely to mass pro-duction and reduced failure rates,may be the biggest factor driving itsincreased adoption, according toJennifer Colegrove, senior analystfor iSuppli, an electronics-marketresearch firm.

The ability of touch screens tohave interfaces for multiple applica-tions and to create interfaces that areeasier to use, more intuitive, andmore customizable than those basedon keyboards and mice are alsomaking the technology more popu-lar, she added.

For example, when viewing avideo on the Apple iPhone, most ofthe controls disappear to makeroom for the display and only thebuttons needed to work with thefile remain.

In addition, operating systems,including Apple’s Mac OS X andMicrosoft’s Vista, are providingmore comprehensive support fortouch-screen operations. They dothis by adding APIs to map thetouching of onscreen buttons tohardware functions.

Other fundamental OS changeshave also been necessary, said ChiaShen, senior research scientist withthe Mitsubishi Electric ResearchLabs. For example, operating sys-tems must be designed to handlesimultaneous input from multipleusers, interpret the commands theyrepresent, and send the results to thehost device’s display.

By providing faster speeds andmultithreaded applications, newerCPUs can better handle the multipletouches and interactions among sev-eral users that today’s touch screensenable.

Manufacturers also like the abilityof touch screens to free up the lim-ited amount of real estate on mobiledevices for purposes such as provid-ing larger displays and enabling cus-tomizable controls.

New Interfaces at the Touch of a FingertipSteven J. Vaughan Nichols

August 2007 13

NEW DEVELOPMENTSTouch-screen technology has been

the subject of research since the early1970s.

Building touch screensThere are several principal ways

to build a touch screen. The keygoals are to recognize one or morefingers touching a display, to inter-pret the command that this repre-sents, and to communicate thecommand to the appropriate appli-cation.

In the most popular technique,called the capacitive or resistiveapproach, manufacturers coat thescreen with a thin, transparentmetallic layer. When a user touchesthe surface, the system records thechange in the electrical current thatflows through the display.

Dispersive-signal technology,which 3M created in 2002, mea-sures the piezoelectric effect—thevoltage generated when mechani-cal force is applied to a material—that occurs when a chemicallystrengthened glass substrate istouched.

There are two infrared-basedapproaches. In one, an array of sen-sors detects fingers touching oralmost touching the display, therebyinterrupting light beams projectedover the screen. In the other, bottom-mounted infrared cameras recordscreen touches.

In each case, the system deter-mines the intended command basedon the controls showing on thescreen at the time and the positionof the touch. It then sends the infor-mation to the OS to generate theappropriate response.

To make touch-screen systemswork, designers must ensure that thelarge surface of a user’s finger canmap to a single screen point thatcauses an action—such as making amenu appear or minimizing a screen.This isn’t a problem for cursor- orstylus-operated systems. To com-pensate, designers must, for exam-ple, make touch-screen controlslarger.

Multitouch screensMultitouch technology has been

around since early research at theUniversity of Toronto in 1982.

With multitouch devices, one ormore users activate advanced func-tions by touching a screen in morethan one place at the same time. Forexample, a person could expand orshrink images by pinching the edgesof the display window with thethumb and forefinger of one hand,explained Microsoft principalresearcher Bill Buxton.

Users could also, while in contactwith a point on a map, touch othercontrols to make the system displayinformation, such as nearby restau-rants, about the area surroundingthe indicated location.

This is accomplished much as ithas been in PCs for years. Forexample, desktop users can pressthe Alt and Tab keys at the sametime to toggle between open win-dows. The OS translates the simul-taneous keystrokes into a singlecommand.

Industry observers say tabletopcomputers are likely to become apopular multitouch-screen imple-mentation. Because multiple users at

different positions will work withtabletop systems, the computersmust be able to display material indifferent parts of the screen andmove controls around to keep themfrom blocking reoriented content.

The systems can determine users’locations based on the positionsfrom which they input commands ordata. The computers then orienttheir displays toward the tabletopedge nearest to the user.

Vendors are beginning to releasecommercial multitouch systems. Forexample, Mitsubishi Electric ResearchLaboratories’ Diamond Touch table,which includes a developer’s kit, canbe used for small-group collaboration.

Perceptive Pixel. Jeff Han, con-sulting research scientist at NewYork University, is doing innovativework with large multitouch screens.He has founded Perceptive Pixel, acompany building multitouch andmultiuser tabletop displays.

Han said his screens use frustratedtotal internal reflection technology,which Figure 1 shows, also utilizedin biometric fingerprint-imageacquisition.

The display’s surface is a six-mil-limeter-thick piece of clear acrylic,

LED

Baffle Diffuser Source: Perceptive Pixel

Scattered light

Acrylic pane

Total internalreflection

Figure 1. Perceptive Pixel’s touch screens work via frustrated total internal reflection

technology.The acrylic surface has infrared LEDs on the edges. When undisturbed, the

light passes along predictable paths, a process known as total internal reflection.

When one or more fingers touch the surface, the light diffuses at the contact points,

changing the internal-reflection pathways. A camera below the surface captures the

diffusion and sends the information to image-processing software, which translates it

into a command.

14 Computer

which interprets the intended com-mands. A digital-light-processingprojector then generates the appro-priate images on the display.

Microsoft is initially selling Sur-face to Starwood Hotels for guestcheck-in, Harrah’s Entertainmentfor video gambling, and T-Mobilefor providing customers with tech-nical information about the com-pany’s mobile phones.

HapticsResearchers are working with hap-

tics—the science of applying tactilesensations and controls to interac-tions with computer applications—in developing touch screens that, forexample, vibrate when users pushkeys on a virtual keyboard.

This provides the feedback thatmany users don’t get from mosttouch screens unless they are care-fully watching all their interactions,which can be difficult in dark con-ditions or when a user is otherwiseoccupied.

In its SCH-W559 touch-screenphone, recently released in China,Samsung will use Immersion Corp.’sVibeTonz haptics technology, whichmakes a virtual key vibrate whenpressed.

Touching different onscreen ob-jects can produce distinct context-sensitive sensations. For example,pressing a number key might notfeel the same as pressing the volumekey.

This haptics approach typicallyworks via mechanical activatorsunder the display that prod the skinor vibrate when a particular part ofthe screen is touched, said ImmersionCEO Vic Viegas.

Mobile phonesIn 1993, BellSouth and IBM

released the Simon smart phone, the

with infrared LEDs on the edges.Left undisturbed, the light passesalong predictable paths within theacrylic, a process known as totalinternal reflection.

When objects such as fingerstouch the surface, the light diffusesat the contact point, causing theacrylic’s internal-reflection pathwaysto change. A camera below the sur-face captures the diffusion and sendsthe information to image-processingsoftware, which can read multipletouches simultaneously and trans-late them into a command.

The system sends informationabout screen touches to applicationsvia the lightweight Open SoundControl protocol, utilized for net-work-based communication betweencomputers and multimedia devices,and User Datagram Protocol data-transport technology. The applicationsthen take the appropriate actions.

Perceptive Pixel, which has built aprototype that measures 36 � 27inches, is still working on applica-tions for its displays, Han noted.They could be used for collaborativework on design-related and otherprojects, perhaps in place of inter-active whiteboards, he said.

Microsoft Surface. Microsoft hasbeen working since 2001 on table-top touch-screen displays, in both its Touchlight and PlayAnywhereresearch projects, according to thecompany’s Buxton.

Using this work, the companyplans to release its Surface tabletopPC later this year. The computer hasa 30-inch display that can simultane-ously recognize dozens of movementssuch as screen touches and user ges-tures, and it can work with multipleparticipants at the same time.

Surface runs on Windows Vista;uses an off-the-shelf graphics card,a 3-GHz Pentium 4 processor, and 2gigabytes of memory; and supportsWi-Fi, Bluetooth, and Ethernet con-nectivity.

Underneath the scratchproofacrylic surface are five infraredcameras that detect screen touchesand send the information to Vista,

first to use touch-screen technology.Now, Apple’s recently unveiled andheavily hyped iPhone has givenmobile touch-screen technology amuch higher profile.

For most commands, the iPhoneuses its touch interface, based in a 3.5-inch diagonal screen. Its functionsinclude those of a cell phone, musicplayer, video display, and Internetbrowser, with the screen commandschanging for each application.

The device comes with a 2-mega-pixel camera; 4 or 8 gigabytes offlash memory; and cellular, Wi-Fi,and Bluetooth connectivity.

The iPhone’s display is coveredwith a thin metallic skin. Contactfrom a finger disrupts the slightelectrical charge flowing throughthe screen’s skin, which the systemregisters as the touch of a button.

TOUCHY SUBJECTAlthough touch-screen technology

is beginning to experience morewidespread commercial use, it stillfaces some noteworthy challenges.

For example, simple and commonoccurrences such as dust, smudges,and scratches can cause problems.Small dust specks threw off touchrecognition in early Microsoft Touch-light and PlayAnywhere prototypes,noted company software developerNigel Keam.

Manufacturers are trying to ad-dress these problems. For instance,Synaptics makes its ClearPad touchscreens of durable clear plastic.

Because of the additional technol-ogy necessary to recognize and inter-pret screen contact, touch screensare more expensive than regularones, which only have to displayinformation.

For example, Microsoft is ex-pected to sell the Surface initially forbetween $5,000 and $10,000, pri-marily to large companies. If suc-cessful, Microsoft plans to sell lessexpensive versions to consumers.

Touch screens for mobile phoneswill have to drop from their currentprice of about $500 before they willbe inexpensive enough for wide-

I N D U S T R Y T R E N D S

Touch screens offer the promise of a

more intuitive andmultifuntional interface.

August 2007 15

spread popularity, according toStrategy Analytics’ Robinson.

Because touch screens must act asboth interface and display, they con-sume more energy than regularscreens. This is a significant problemfor battery-powered mobile devices.

Also, Microsoft’s Buxton added,touch is not the best way to input alltypes of information. “Even on largesurfaces, writing or drawing with thefinger is generally not as effective as itis with a stylus,” he explained. “Andon small devices, it is virtually uselessto try to take notes or make drawingsusing a finger rather than a stylus.”

U BS Investment Research analystBen Reitzes predicted thattouch-screen technology will be

so successful that Apple will expandits use from the iPhone to othermajor hardware products, includingMacs, within five years.

However, said Shiv K. Bakhshi,director of mobility research at mar-ket-analysis firm IDC, cell phoneswith touch screens may be tooexpensive, at least at first, to bewidely successful.

These same issues exist for largetouch-screen displays. While theremay be applications that use the

approach, they might not be com-pelling enough to make users wantto pay the additional initial cost fordevices that use the technology. ■

Steven J. Vaughan-Nichols is a free-lance technology writer based in Arden,North Carolina. Contact him at sjvn@vna1.com.

Editor: Lee Garber, Computer,l.garber@computer.org

IT Professional magazine gives builders andmanagers of enterprise systems the “how to” and“what for” articles at your fingertips, so you candelve into and fully understand issues surrounding:

• Enterprise architecture and standards

• Information systems

• Network management

• Programming languages

• Project management

• Training and education

• Web systems

• Wireless applications

• And much, much more …

Giving Youthe Edge

www.computer.org/itpro