PupilWare Petra 2015

download PupilWare Petra 2015

of 48

  • date post

    07-Aug-2015
  • Category

    Documents

  • view

    74
  • download

    0

Embed Size (px)

Transcript of PupilWare Petra 2015

  1. 1. eric c. larson | eclarson.com PupilWare Assistant Professor Computer Science and Engineering towards pervasive cognitive load measurement using commodity devices
  2. 2. PupilWare Suku NairSohail Raqi Chatchai Wangwiwattana Ephrem Fernandez Jasmine Kim Team:
  3. 3. sclerairispupil pupil: regulate light Pupillometry Primer
  4. 4. Pupillometry Primer macro changes: light reex drug impairment concussions
  5. 5. sclerairispupil pupil: regulate light autonomic nervous system physiological response to stress, arousal Pupillometry Primer
  6. 6. Subtle Pupillary Response 1940 1950 1960 1970 1980 1990 2000 2010 2015 1943 Blood alcohol proportional to pupil size 1947 First scientic studies of light reex 1950 Pupil size decreases with age 1959 Pupil size projector invented, by Hess and Polt 1960 Pupil size and visual stimuli 1961-1969 Indicator of preference, fatigue 1966 Pupil size & memory 1967 Pupil size & processing difculty 1968 Pupil size maintained during thought 1983 First digital device for capturing pupil size 1989 Apparatus for desktop imaging of pupil 1995 First handheld digital pupillometer 2005-2012 Validation of eye trackers for pupil measurements 2008 Framework for measuring digital interruption cost
  7. 7. 1940 1950 1943 Blood alcohol proportional to pupil size 1947 First scientic studies of light reex 1950 Pupil size decreases with age Skoglund, 1943 Thompson 1947 & Delauney 1949 Birren, 1950
  8. 8. 1950 19 udies x 1950 Pupil size decreases with age 1959 Pupil size projector invented, by Hess and Polt Hess and Polk, 1959-1960 12 mm 4 mm 4.2 mm
  9. 9. 1960 59 Pupil size projector ented, by Hess and lt 1960 Pupil size and visual stimuli preference, f Hess and Polk, 1959-1960 Hess and Polk, 1960
  10. 10. 1960 1970 ctor d 1960 Pupil size and visual stimuli 1961-1969 Indicator of preference, fatigue schizophrenia and neurotic diagnosis (Rubin 1964) sexual orientation (Hess 1965) political preference (Hess 1965) pleasant taste (Hess 1965) infant object recognition and preferential looking (Fitzgerald et al.1967) musical pitch interpretation (Kahneman and Beatty 1967) fatigue, alertness, and sleep deprivation (Lowenstein and Loewenfeld 1964, Bartlett et al. 1967) Consensus: -pupil dilation indicated something pleasing -not a strict measure of emotion -related but not proportional to arousal
  11. 11. 1966 Pupil size & memory 1967 Pupil size & processing difculty 1968 Pupil size maintained during thought Kahneman and Beatty, 1966 Hess and Polk, 1967 Bradshaw, 1968 Eventually the processing of the brain and use of working memory came to be understood as cognitive load
  12. 12. 1966 Pupil size & memory 1967 Pupil size & processing difculty 1968 Pupil size maintained during thought Kahneman and Beatty, 1966 Hess and Polk, 1967 Bradshaw, 1968 Eventually the processing of the brain and use of working memory came to be understood as cognitive load klingner, 2012 Digit Span Task
  13. 13. 1960 1970 1980 ctor d 1960 Pupil size and visual stimuli 1961-1969 Indicator of preference, fatigue 1966 Pupil size & memory 1967 Pupil size & processing difculty 1968 Pupil size maintained during thought 1983 First digital device for capturing pupil size
  14. 14. 1980 1990 1983 First digital device for capturing pupil size 1989 Apparatus for desktop imaging of pupil Jones and Smith, 1983 Carter, 1989
  15. 15. 2000 1995 First handheld digital pupillometer mea 2 Carter, 1995 $10,000 USD $4,000 USD
  16. 16. 2000 2010 2005-2012 Validation of eye trackers for pupil measurements 2008 Framework for Klingner et al., 2005-2012 $500+ USD
  17. 17. 2010 20152008 Framework for measuring digital interruption cost Iqbal and Bailey, 2008 Use of cognitive load for real time user awareness is becoming more of a possibility
  18. 18. can cognitive load be measured ubiquitously?
  19. 19. the webcam ~15 x 15 pixels dilations are mostly sub-pixel highly affected by noise and lighting HD 1280x720
  20. 20. Study Can baseline cognitive load be measured, compared to a gold standard pupillometer? Can subtle variations be measured, compared to a remote eye tracker? Age: (mean=23, range=19-38) Sex: 7 Male, 5 Female Color: Lighter Eyes: 7, Brown: 5 Vision: Corrected-to-normal, 2 12 participants
  21. 21. Experimenter Laptop Control View Participant Camera iPad Application Survey & Self Report Remote Gaze Tracker Focal point Calipers measurement Pupillometer Measurement DesignStudy
  22. 22. 5 Digits 6 Digits 7 Digits 8 Digits 9 Digits x 5 iterations over 200 iterations total 2 7 9 3 13 Digit Span Task
  23. 23. Algorithm Overview
  24. 24. Algorithm grayscale conversion face detection 1280x720 ~350x370
  25. 25. approximate eye location~125x125
  26. 26. darkness threshold morphology means of gradient
  27. 27. iris bounding box histogram equalization median ltering
  28. 28. modied starburst algorithm nd strong edges ~45x45
  29. 29. eliminate measurements with RANSAC t ellipse to points calculate distance between pupil centers
  30. 30. Algorithm Overview
  31. 31. Algorithm Overview 10 FPS
  32. 32. Post Processing 0 10 20 30 40 50 60 15 16 14 17 13 seconds ellipsediameter,inpixels
  33. 33. Post Processing 0 10 20 30 15 16 14 13 seconds ellipsediameter,inpixels average
  34. 34. Post Processing 0 10 20 30 15 16 14 13 seconds ellipsediameter,inpixels 12 11 10 average median smooth
  35. 35. Post Processing 0 10 20 30 seconds 5.0 5.2 4.8 4.6 ellipsediameter,inmm 4.4 4.2 4.0 5.4 average median smooth millimeters
  36. 36. average median smooth millimeters Light Eyes Dark Eyes Reflection 3 participants removed from analysis
  37. 37. Baseline Results 3.00 4.00 5.00 6.00 7.00 P1 P2 P3 P4 P5 P6 P7 P8 P9 Pupillometer PupilWare GZ-C = GZ-C and PupilWare different (p