1

Su-ting, Chuang11OutlineIntroductionRelated workHardware configurationDetection systemOptimal parameter estimation frameworkConclusion22Introduction3OutlineIntroductionRelated workHardware configurationDetection systemOptimal parameter estimation frameworkConclusion44Related WorkFTIR (Frustrated Total Internal Reflection)

J. Y. Han, Low-cost multi-touch sensing through frustrated total internal reflection," in Proceedings of the 18th annual ACM symposium on User interface software and technology (UIST '05). New York, NY, USA: ACM Press, 2005, pp. 115-118.

5Related WorkDI (Diffused Illumination)J. Rekimoto and N. Matsushita, Perceptual surfaces: Towards a human and object sensitive interactive display," Workshop on Perceptural User Interfaces (PUI'97), 1997.

6An IR camera with IR illuminators to observe hands

6OutlineIntroductionRelated workHardware configurationDetection systemOptimal parameter estimation frameworkConclusion77Hardware configurationTable setup

8 IR IR cam8Hardware configurationOrder of diffuser layer and touch-glass layer9Diffuser layerIR illuminatorIR cameraspotIR illuminatorIR cameraTouch-glass layerIR cameraspotIR camera

(decay intensity)9Hardware configurationProblem: IR rays will be reflected by the touch-glass and resulting IR spot regions in camera viewsSolution:Use other cameras to recover the regions which are sheltered by IR spots

10

camera10OutlineIntroductionRelated workHardware configurationDetection systemOptimal parameter estimation frameworkConclusion1111Detection systemIR camPre-processingImageprocessingFingerDetectionDataAssociationDataTransmissionIR camGPUCPU12Todo: cam 12Detection systemPre-processing

Image processing

ImageFusion(Blend)IR CameraIR cameraUndistortionUndistortionHomoWarpHomoWarpBackgroundSubtractionNormalizationSimpleHighpassMonoThreshold13I (x,y) = a x I1(x,y) + (1-a) x I2(x,y)Undistortion ? () mono??

13Pre-processingUndistortionUndistort camera imageWarpUnify finger size among different position of tableImage fusionIncrease intuition of visionSimplify foreground object matching among cameras

14matching()14Pre-processingAdvantage of implementing on GPUIncrease performanceHigh frame rate Preserve CPU for application computationEnable detection system and interactive application on the same computerReduce unsynchronized problem among different computers1515Image processingNormalizationMotivationEliminate influence due to ununiform lighting conditionVarious finger touch responseHard to decide a good thresholdMethodModel each pixels dynamic rangeUsing specific material to simulate foregroundStretch dynamic range to 0-255

1616Image processingFinger DetectionConnected componentFinger analyzerfinger size evaluation

17Data associationFingertip matchingMatching fingertips among framesUsing bipartite algorithmFingertip trackingSmooth detected results and fix lost resultsUsing Kalman filter1818OutlineIntroductionRelated workHardware configurationDetection systemOptimal parameter estimation frameworkConclusion1919Software architecture20Detection systemSamplesetTrainingParameter SetDetection ResultGroundTruthOptimalParameter SetVerifyNext Parameter SetGeneratorDetectionResultGroundTruthError RateParameter SetOptimal parameters estimation framework for finger detectionMotivationParameter setProcedureCollect samplesVarious finger sizeHard press and soft pressSearch exhaustivelyVerify performance of all possible parameter combinations

21Parameter setSubtract value from foreground imageSimple high pass kernel sizeScale valueThreshold Finger Size

21Optimal parameters estimation framework for finger detectionTaskSoft /Hard touchVertical/Oblique touchVarious fingersSample setEach task has 2x2x5 samplesSample collectionStep-by-step instructionStraightforward UI design

Finger touch position5timerInstructions.2222MethodExhaustive searchTest various parameter combination in each setStepEach parameter combinationDetect finger touchCalculate precision and error rate

23Step -> algorithm

23OutlineIntroductionRelated workHardware configurationDetection systemOptimal parameter estimation frameworkConclusion2424Backup2525Sample collectionHard/Soft vertical touch

Finger touch position5timer2626BackgroundSubtractionNormalizationSimpleHighpassMonoThreshold2727ImageFusion(Blend)IR CameraIR cameraUndistortionUndistortionHomoWarpHomoWarp2828DetectionModuleVerifyNext Parameter SetGeneratorDetectionResultGroundTruthError RateParameter SetParameter SetParameter Set2929

3031Detection systemSamplesetTrainingParameter SetDetection ResultGroundTruthOptimalParameter Set32VerifyNext Parameter SetGeneratorDetectionResultGroundTruthError Rate33Detection systemSamplesetTrainingParameter SetDetection ResultGroundTruthOptimalParameter SetVerifyNext Parameter SetGeneratorDetectionResultGroundTruthError RateParameter Set34