August 28, 2016 Sam Siewert

CS415 Human Computer Interaction

Lecture 1 - Introduction

Dr. Sam Siewert UC Berkeley – Philosophy/Physics University of Notre Dame, BS - Aerospace/Mechanical Engineering Johnson Space Center, U. of Houston – UHCL Computer Engineering, Mission Control Center U. of Colorado, Boulder, MS/PhD – JPL, Colorado Space Grant, Computer Science

CU Boulder Senior Instructor, Adjunct Professor, CTO, Architect, Developer/Engineer in Local Start-ups

U. of Alaska, Anchorage, Assistant Professor, Computer Systems Engineering, Alaska Space Grant Embry Riddle Prescott, Assistant Professor, CESE

Sam Siewert 2

1984-85

1985-89

1989-92

1992-2012

2012-14

Sam Siewert 3

Related Industry Background

General Experience (~25 Years in Embedded and Scalable Systems) – 12 Years NASA JSC, NASA JPL / CU, Ball Aerospace – 12+ Years Commercial Telecomm, Storage/Networks, Embedded,

Digital Video

Instrumentation and Machine Vision – Spitzer Space Telescope – Unmanned Aerial Systems – Robotics at CU-Boulder, Arctic Sensor Systems at U. of Alaska

Anchorage

Software Engineering – NASA Johnson and JPL (Shuttle Ascent/Entry Guidance, Deep

Space) – Intel, Emulex, Start-ups

Consulting – Graphics, Storage and Networking, UAS/UAV

Course Goals and Outline Human Computer Interaction, 3rd Edition, Alan Dix, Janet Finlay, Gregory Abowd, and Russell Beale HCI – GUI, CLI, Visualization, Services, Tasks, Status, Monitoring, 2D/3D, I/O Devices, Shared-Control, Configuration, Collaboration, Social Networks, Animation, VR, AR … ? http://mercury.pr.erau.edu/~siewerts/cs415/ Monday / Wednesday – Lecture and Discussion Fridays – Q&A and Assignment Discussion

Sam Siewert 4

http://www.hcibook.com/e3/

At Dawn of Computing – Limited Interaction Batch Processing Punch Card Input/Output (Decks) Line Printer Output Status Lights, Tones Not Much to Go on… Computer Time Was More Important than User Time!

Sam Siewert 5

Punch Card Batch I/O

IBM Line Printers

Batch Processing Evolves and Continues …

Tape (Analog, Digital) Graphic Printers (Laser, Ink-jet – 1980’s to 90’s) Scanners (OCR) – Xerox Document Services 3D Scanners and Printers (3D Systems Corp.)

Publishing on Demand, Self-Publishing Sam Siewert 6

http://us.xyzprinting.com - $500

6250 bpi Tape

http://store.makerbot.com/digitizer

Software View of Interaction (Approximate Milestones from Computer History)

1. ENIAC – Electronic Computer - 1946 2. Punch Cards, Switches, LEDs, Line Printer – 1950 3. Artificial Intelligence – 1956 4. Serial Terminals and Minicomputers - 1960 5. Computer Vision - 1966 6. Command Line Interface (Unix) - 1970 7. Scripting (Unix) - 1970 8. Graphic Visualization, Animation, Flight Simulators – 1976 9. Word Processing, WYSIWYG – 1971, 1975 10. Home PC – 1976 (Apple I, PET) 11. MS DOS - 1981 12. WIMP (Windows, Icons, Menus, Pointers) - 1985 13. Virtual Reality, 3D Graphics – 1990 14. Palm Computing (Apple Newton), Web Browser – 1993 15. GhZ Computing, ILOVEYOU Virus, Y2K - 2000 16. Mobile WIMP (PDA, mobile touchscreen, voice recognition) – 2005 17. SmartPhone, Tablet PC, SmartTV, VR CAVE, Gesturing - 2010 18. Augmented Reality, Wearable Computing, IoT - 2015 19. Future?

Sam Siewert 7

Brief History of Interactive – 1970 to 2000 Binary I/O, Switches & LEDs, Joystick Command Line Interface (CLI) – ASCII Terminals or Shells – Canonical Mode (Buffer until Return) – Non-Canonical Mode (Capture and Transmit

Each Character) – Echo or No-Echo, e.g. password – Cmd.exe, Power Shell (MS), BASH (Linux,

OS-X), Terminal

2D Mouse and Bit-mapped Window Manager (GUI) – Xerox PARC (Palo Alto Research Center) –

1970 – Mac OS with Toolbox (1979) – Jef Raskin – X-Windows (MIT) and Windows 3.x, NT,

Win7, 8 …,10 (Microsoft)

Web, Mobile and Wearable … Sam Siewert 8

Apple Macintosh

X-windows

Xerox Alto

Intel-based Microcomputer

Apple Newton

MS Windows

Game Console

MS-DOS

Linux Watch

Jef Raskin Canon Cat

NCSA Mosaic

Interactive I/O Devices – Brief History Trackball (Tom Cranston – 1952) Mouse (1970 – US Patent 3,541,541) – Douglas Engelbart – Stanford Research Institute (SRI)

Newer Devices – Data Gloves – Trackers, Motion Capture

http://www.polhemus.com/ Leap Motion

– Gesture Recognizers (PrimeSense) MS Kinect Creative Cam Occipital (3D Scanner)

– Resistive Touch Screen (1975 … Modern Tablet) – Voice Command (Smartphones, Siri) – 3D, Data Gloves/Suits, Haptic Feedback,

VR, AR, Holographic

Sam Siewert 9

Trackball - (Prior to Mouse)

Engelbart’s Mouse 5DT Data Glove (Ultra)

Depth Mappers

MS Surface

Aviation to Games (1908)

3D Animation and Motion Trackers Used for Animation in Digital Cinema – Motion Capture Compared to Stick Figure with Degrees of Freedom and Kinematics (Avars) – e.g. Toy Story Robotics & Kinematics (joint rotations to define position) and Inverse Kinematics (math optimization to get joint to position in X,Y,Z space)

Sam Siewert 10

Motion Capture for Digital Cinema – E.g. Avatar

Personal Robotics

Jibo Personal Assistant

Virtual and Augmented Reality HMDs – Monocular or Binocular – Replace 2D (3D rendered desktop frames)

with Immersive Device – Changes Scene with Head Motion – Stereo – Frames specific to each Eye

CAVE (Cave Automatic Virtual Environment) Rudimentary AR - Hold Phone with Camera and Display Wearable Camera and Display – Google Glass Explorer Program First Person Sensing - Drones

Sam Siewert 11

Immersive VR

Wearable Camera+Display

Virtual Reality Caves

FPS Drones

MS Hololens

VM or PRClab - Linux Option #1 – Use PRClab, prclab.pr.erau.edu via SSH – Recommend Putty connection with SSH and X11 forwarding – Code Development (GCC/g++, Make, etc.) -

http://mercury.pr.erau.edu/~siewerts/se420/documents/Linux/ – Debugging C/C++ Source Code using DDD

(http://www.gnu.org/software/ddd/manual/html_mono/ddd.html ) – Verification and Validation of C/C++ Implementations – General Linux System (RHEL 6.5)

Option #2 – Use Virtual-Box Linux with Centos 6.5 Install of Ubuntu 14.04 LTS (Both Supported) – Must Have Windows, Macintosh or Linux Personal PC – http://mercury.pr.erau.edu/~siewerts/se420/documents/Linux/Lin

ux-Development-Getting-Started.pdf

Sam Siewert 12

Embedded Linux Jetson TK1 – Tegra SoC – 4 64-bit ARM Processor Cores – 192 Vector Co-processor Cores (Graphics Processing Unit)

SoC Used in Google’s Project Tango, NVIDIA Shield https://developer.nvidia.com/jetson-tk1

Sam Siewert 13

Logon and create your own account Logon as “ubuntu” with PASSWORD (given in class) 9 Stations – Shared with CEC450

Sam Siewert 14

Why HCI is Critical Going Forward…

Non-Linear Growth in Computing Capability and Complexity

Cooperative / Supervisory Automation – Safety Improvement

Super Intelligence = AI + Human Intelligence

Intelligent Transportation - Safety Dr. Clark Chapman (SWRI - http://www.boulder.swri.edu/clark/ ) Motivation for Driverless Car - Safety – Sebastian Thrun Safety HCI Monitors -> Shared Control -> Fully Autonomous

Sam Siewert 15

1. Driver to Driver Interaction

2. Fatigue

3. Distraction

4. Human Error

5. No Autopilot?

How We’ll Do It 1/3 Theory – Lectures/Reading (On-going) – Lectures related to HCI Textbook and Instructor’s Experience – History of HCI and Current Best Practices – Discussions

1/3 Practice – Jetson, PRClab, VB-Linux – Linux Coding (C and C++ or Java if you wish) – Basic HCI Coding Skills – HCI Applications - Original Implementations, Walk-throughs,

Design Validations

1/3 Project – Group Project to Build HCI Proof-of-Concept Application – Present Design and Prototype

Sam Siewert 16

Sam Siewert 17

Administrivia Introductions – Instructor (Office Hours) – Fall 2016 – Students (Introductions) – Please do Collaborate, but cite well! – Policies - http://mercury.pr.erau.edu/~siewerts/cs415/policies/

ERAU Canvas ( https://ernie.erau.edu/ , https://erau.instructure.com/ ) – Primarily for Assignments and Assignment Grading – Mercury Website - http://mercury.pr.erau.edu/~siewerts/cs415/

Course Information – Attendance & E-mail list (please sign up on sheet being passed around) – Lecture Notes at http://mercury.pr.erau.edu/~siewerts/cs415/documents/ – Fall 2016 Syllabus

Must have PRClab account OR VB-Linux, Access to Control Systems Lab I highly recommend all 3 if possible, but PRClab and Jetson is sufficient

August 24, 2004 Sam Siewert

Linux Skills

Introduction Session

Sam Siewert 19

Linux Getting Started Help Jetson Boards Run Ubuntu LTS Linux – 14.04 You Can Run the Same on VB-Linux on Windows, Max OS-X, etc. Linux is the NDK Layer in the AOS (Android OS) and Java is the SDK Much of Interactive Computing has Gone Mobile! Either iOS or AOS How-To’s 1. Development-Getting-Started.pdf 2. Makefiles by Example 3. Integrated Development Environments on Linux

Next Time … HCI History – Come to Class Prepared to Discuss – Gurus (e.g. TBD) – Organizations for HCI – IEEE Systems, Man, Cybernetics – HCI,

ACM SIGCHI

Assignment #1 Discussion – I will Post Every Other Wednesday, We’ll Discuss, Due

Following Week on Friday – Late Assignments – 10% Penalty for Monday Turn-in, After

Monday, only with Instructor Permission

Sam Siewert 20