Capture: Preliminary Experiment

Instrumentation:– 2 Kinects (1 overhead and 1 side view)– 1st person audio/video: GoPro glasses on examiner– Boom microphone (fixed)– Consumer video camera– EDA on child and examiner

Observations:– Activities with examiner/child take place across the whole room

• Placement of fixed sensors must be studied carefully or activities more controlled

– 1st person video • Go Pro $300 price point offers exciting opportunities for in-home use and wide

deployment with examiners and families• Challenges with respect to stabilization for viewing/coding• Offers the examiner’s view at any time (both a positive and negative)

Kinect vs. HD Video• Multiple Kinects

– Positives: • Inexpensive (~5-10K for system with Kinects, PCs, and 4 Kinects)• Gives RGB + depth • Body pose estimates (in side views)

– Negatives: • Overlap between Kinect views yields errors in depth streams• Each Kinect intermittently drops video frames at different times• Capture is asynchronous, which is challenging for multi-view analysis

• High-res video capture (using Camera-Link)– Positives:

• high-definition, high optical quality• Synchronized at time of capture

– Negatives:• Expensive (~$50K for system with capture equip, PC, and 4 cameras)• Does not include depth sensing

WorkflowStep in workflow Software Hardware

Capturing parallel streams Open source, homebrew, or turnkey capture software?

Kinect, consumer video, high quality video, 1st person camera(s), Affectiva, mic

Editing streams Elan?

Annotating/coding sessions Elan?

Storing of data and Elan files Backups? Mirroring? Disk array with fault tolerance and mirror

Sharing Web server – secure?

• Where in the workflow does synchronization of streams occur?• What method(s) will be used for synchronization?• Is synchronization needed?