August 30, 2016 Sam Siewert

CS415Human Computer Interaction

Lecture 1 – IntroductionPart-2

The Basic ElementsHuman – Models of Perception and Cognition

– Brain Models [Simulations]– Brain Scans [fMRI, EEG, fNIRS]– Brain Processing of Sensory Information [Perception]

Computer – Theory of Computation and Devices– What Can Be Computed? What Can’t? How Fast?– Traditional ALU, Stored Program Computer: Von Neumann

Architecture, Von Neumann Bottleneck– Turing Machines [Wikipedia], Church-Turing Thesis

[Wikipedia], Turing Test– Hypercomputation (Myth, Reality?)– Quantum Computing, D-Wave, ANN, Analog “Real”

Computing, Neuromorphic, Deep Learning (GPU)

Interaction – Physical Devices and Interfaces– Physics, Time and Space, Chemistry– Acoustics, Haptics, Optics, Olfaction, Gustation– 5 Senses – Hear, Feel, See, Smell, Taste– Fusion of Senses – Proprioception [Body movement]– Extended – Chronoception [Time], Nocioception [Pain]– 20+ Senses: Internal [interoceptive], External [exteroceptive]– Superhuman Sensing [E.g. Thermal imaging, Magnetic field]

Sam Siewert 2

Why is Human Vision > Computer?

Sam Siewert 3

Approximately 100+ Mega-Pixel(Rod/Cone Count)

James Cutting & Peter Vishton –Perceiving Layout

Cortex=10 Billion Neurons(High fan-out)

Total=100 Billion Neurons

Neuroscience. 2nd edition.Purves D, Augustine GJ, Fitzpatrick D, et al., editors.Sunderland (MA): Sinauer Associates; 2001.http://www.ncbi.nlm.nih.gov/books/NBK10848/

Red Epic 64563 Mega-Pixel

I/O Bus (x16 5Gbps = 8GB/sec)

CameraLink

Interface Card

Local Bus

CPU CPU

MemoryController

5 To 10 billion transistors1. Neuron > Transistor2. Better Programming? ROM?3. More Richly Interconnected4. Storage + Processing

> 1 Trillion Synapses

http://bluebrain.epfl.ch/Human Brain ModelsProject StatusVisualization

Biological Vision vs. Machine Vision(Why A Honey Bee is Better than HPC for CV)

Humans - 100 million Photoreceptors

– 10 billion Neurons (Cerebral Cortex)– Brain with 100 billion Neurons– Millisecond Transfer – Massively Parallel Analog + Digital Computation

Synapse Match is a Challenge– 7000 Connections from 10 Billion Neurons– 3 Year Olds Have 1015 Synapses

CPU to Digital Camera/HDD– Connects 10’s of millions of pixels – to Several Billion transistors – Through Sequential Logic and I/O Bus

Sam Siewert 4

960K Neurons in flight:Learns locations,complex odors,colors, and shapes;with high efficiency (500 Watt/Kg), 0.218g

Brain plasticity for learning, connectedness, concurrency, integrated sensing, power efficiency, and resiliency

2016 – 16 billion?

NVIDIA GK11028nm, (7.1 billion)

Pascal – 15 billion

Intel MICA 22nm(5 billion)

http://en.wikipedia.org/wiki/List_of_animals_by_number_of_neurons

Moore’s Law Graphic - Wikipedia

3D PerceptionWe Live in an Animated 3D World

We still Compute with 2D Desktops (WIMP)– Tablets, MS Surface, Notebooks, Laptops,

Desktops– Mobile SmartPhones, iWatch, Android Gear– Enabled by Key Devices (Pointer – Mouse,

Stylus, Touch-screen, Raster Graphics – CRT, LCD, OLED, VR CAVE)

3D Limited to CAD, VR Worlds, Movies, Visualization So Far …

3D Batch Processing – Printers and Scanners

What About Interactive 3D?

Sam Siewert 5

https://en.wikipedia.org/wiki/OLED

https://en.wikipedia.org/wiki/3D_film

1950’s

Real World 3D PerceptionMany Visual Cues Provide 3D Perception – Open Research (Perceiving Layout)

HCI Inventors Want to Exploit What Matters Most– Stereopsis and Disparity (Left Eye and Right Eye View)– Accommodation (Near and Far Field)– Correspondence (Common Features Noted with Each Eye)

Sam Siewert 6

http://www.ibm.com/developerworks/library/bd-mdasecurity/, Sam Siewert

“Play” with RGB Depth MappersScene Depth Mapping + RGB Planar Image– Scene Segmentation (in Z as well as XY)– Skeletonization– Hand Tracking and Gesture Recognition– Interaction with 3D Virtual Worlds

Sam Siewert 7

Data Glove vs. RGB Depth MapperHand and Finger Tracking Based on SegmentationDepth (Skeletonization)Processing is Somewhat IntenseNothing to Wear

Data GloveRotational Rates, Accelerometer, Finger Flexure Sensors

Sam Siewert 8

CreativeCamWith Intel RealSense

3D SDK

M.S. research, CU Boulder,“A Hybrid Sign Language Recognition System”, Van Culver, 2004OpenCV and Monovision Camera with 5DT Data Glove

Interactive 3D Devices I/O?3D Input – Active Depth Mappers, Passive Stereopsis– ASUS Xtion Interactive Depth Mapper– PrimeSense Chip [Apple now] – Revitalized– Many Competing Chips and Devices Now– Passive – Compute Intensive, Less Robust

(Flat Wall Problem)– Time-of-Flight, LIDAR (Distance, Outdoor)

3D Output [Left, Right Eye Disparity]– 120 Hz, Left / Right Eye Parallax– Filtering, Polarization (Left/Right Eye)– Active Shutter Glasses– Entertainment and Visualization Success,

Interactive, Less So … Sam Siewert 9

“Fool the Eyes!”

3D Alternatives?Augmented Reality (Interactive Mix of Real/Virtual)

1. Project Virtual World onto Real World2. View Real World Through Goggles with Annotation3. Project Virtual Images into retina as Light-field Overlay

Heads-Up-Display – Aviation [Gunsight Generalization]

Pranav Mistry, MIT Media Lab, Samsung– Project onto Real World– We’ll Study Later

Google Glass VRD – Failed, “Glassholes”

MS Hololens [View Real World with Annotation]

Magic Leap [VRD] – MIT Tech Review Article

Oculus Rift – VR Headset [Dual OLED, Head Tracking]

Wearable AR/VR Doomed? Limited? Unhealthy?

Sam Siewert 10

C130 HUD

Before We Go FurtherStart with Basics

Move to 2D

3D Animation

Interaction Theory and Guidelines

Cognition and Perception Models

Then, Back to the Future! Sam Siewert 11

August 24, 2004 Sam Siewert

Linux Skills

Introduction SessionPart-2

Linux How-ToUbuntu LTS 14.04 on Virtual Box and Jetson– Local How-To Resources

LinuxLinux-Development-Getting-Started.pdfLinux-Basic-Makefile-by-Example.pdfLinux-Setting-up-VBOXSF-Automount-from-Host.pdfLinux-Programming-Top-Errors.pdf

– https://ubuntu-manual.org/– https://www.youtube.com/watch?v=SGGj3hc

6R8Q

Sam Siewert 13

Virtual Box VM Installation ofa New OS Like Ubuntu Linuxis a Safe Way to Learn on thePlatform you Know Best

Server/Desktop, VM, EmbeddedA Key Advantage of Ubuntu is that One Linux Distribution Can be Used for Server/Desktop, VM and Embedded Use– Server: PRCLab (prclab.pr.erau.edu)– VM: Virtual Box with Ubuntu 14.04

LTS– Embedded: Jetson TK1 Developer

System

Linux Kernel supports the NDK Layer in the AOS (Android Operating System)– https://developer.android.com/ndk/inde

x.html– https://developer.android.com/sdk– https://developer.android.com/tools/sd

k/ndk/index.html

Sam Siewert 14

A few Comments on OptionsNative Linux (Jetson) – Best Performance for Interactive Applications

VM Linux – Nice Option for Convenient Development and Test

Server Linux – Safe Haven that is Highly Available, Backed-Up

Moving Code Can be Done with SFTP or SCP– SFTP Requires ZIP or Archive to Move Whole Directories– SCP – Moves Structure Automatically

Sam Siewert 15