VR / AR Fundamentals

Michael Naimark Visiting Associate Arts Professor

Interactive Media Arts NYU Shanghai

INTM-SHU  280C - 1   Topics in New Media & Entertainment Spring 2018

4:15pm - 6:00pm Mondays and Wednesdays

Room 933/950

VR / AR Fundamentals

I. The Class 1. Who are You? 2. Who Am I? 3. Class structure 4. Assignments & grading 5. Being “positively critical”

II. What are VR / AR fundamentals?

Week 1: Introduction

VR / AR Fundamentals

I. The Class 1. Who are You? 2. Who Am I? 3. Class structure 4. Assignments & grading 5. Being “positively critical”

II. What are VR / AR fundamentals?

Week 1: Introduction

Please say a bit about yourself and why you are interested in VR and AR!

1. Who are You?

VR / AR Fundamentals

I. The Class 1. Who are You? 2. Who Am I? 3. Class structure 4. Assignments & grading 5. Being “positively critical”

II. What are VR / AR fundamentals?

Week 1: Introduction

VR / AR Fundamentals

I. The Class 1. Who are You? 2. Who Am I? 3. Class structure 4. Assignments & grading 5. Being “positively critical”

II. What are VR / AR fundamentals?

Week 1: Introduction

2. Who am I?

“Michael Naimark” is easy to find online so let’s look at stuff.

\

MIT / SF Museum of Modern Art “Displacements” 1980 / 1984

“Displacements” (1980 / 1984 / 2005), CAVS MIT / SFMOMA / Art Center College of Design, Pasadena

\

Art Center College of Design Pasadena “Displacements” 2005

Claire Denamur Toben Seymour (dir)

“Prince Charmant” 2009

Architecture Machine Group MIT “Aspen Moviemap” 1978-80

Exploratorium “Golden Gate Flyover” 1987

ZKM | Centre for Arts and Media “Karlsruhe Moviemap” 1991

ZKM | Centre for Arts and Media “Karlsruhe Moviemap 2009” 2009

Banff Centre for the Arts Interval Research Corporation

“See Banff” kineotscope

1993-94

Banff Centre for the Arts Interval Research Corporation

“See Banff” kineotscope 1993-94

w/ H Baker, J Woodfill, P Debevec, L Villareal Interval Research Corporation

“Dimensionalization Studies”

1994

“Be Now Here,” Michael Naimark, Interval Research and UNESCO, 1995 3D panning motion picture and rotating floor

“Be Now Here Triptych”, Michael Naimark, USC, 2008 3D, motion, sound, but same footage spatially offset

3 screens over 180 degrees

USC Cinema

“Be Now Here Triptych” 2008

USC Cinema

“Be Now Here Triptych” 2008

http://medium.com/@michaelnaimark/vr-cinematography-studies-for-google-8a2681317b3

September 2015 “Big Chairs” Park, Google

Jump VR camera Area taped off with 12 “hour” radials in concentric rings

Our intention was to explore how people are represented in VR.

Teaching this class

and (my current projects):

1) Community-based Interactive Hyper-Images

2) VR Virtual Travel “Super StreetView”

3) First-Person VR via a Wearable VR Camera

and maybe reformatting (older work):

“See Banff” and “Be Now Here” for projection VR

2. Who am I?

VR / AR Fundamentals

I. The Class 1. Who are You? 2. Who Am I? 3. Class structure 4. Assignments & grading 5. Being “positively critical”

II. What are VR / AR fundamentals?

Week 1: Introduction

VR / AR Fundamentals

I. The Class 1. Who are You? 2. Who Am I? 3. Class structure 4. Assignments & grading 5. Being “positively critical”

II. What are VR / AR fundamentals?

Week 1: Introduction

3. Class Structure

This is a new(ish) and experimental class

The first half semester will be learning the basics and seeing what’s out there

The second half semester will be producing short, entertaining, “studies” relevant to VR / AR today

First half semester

The weekly routine will be:

Mondays: Michael’s “fundamentals” presentations (with lots of time for Q & A)

Wednesdays: Students assignments presentations (and posts on IMA Documentation blog)

Mondays and Wednesdays: VR / AR “News of the Week”

3. Class Structure

Second half semester

The weekly routine will be conceiving, producing, and documenting short “studies.”

Like the Google Cinematography Studies, they’ll typically be under one minute each.

We will work as a team.

The goal is to make them public.

3. Class Structure

Second half semester

Last Semester . . .

“Make short VR sequences with real people in constructed environments, with the people doing synchronized movement and sound, interactively.”

3. Class Structure

VR / AR Fundamentals

I. The Class 1. Who are You? 2. Who Am I? 3. Class structure 4. Assignments & grading 5. Being “positively critical”

II. What are VR / AR fundamentals?

Week 1: Introduction

VR / AR Fundamentals

I. The Class 1. Who are You? 2. Who Am I? 3. Class structure 4. Assignments & grading 5. Being “positively critical”

II. What are VR / AR fundamentals?

Week 1: Introduction

4. Assignments and Grading

Assignments initially will be to find, critique, rate, and present

1) relevant VR / AR content 2) relevant content from other media

All assignments must be documented on the IMA Documentation blog http://ima.nyu.sh/documentation/

VR Content will be added to ongoing VR Catalog.

(I will help with pointers and suggestions!)

4. Assignments and Grading

Grading Components

First Half-Semester Finding, presenting, documenting: 60% In-class critical discussion: 30% Readings and video assignments: 10%

Second Half-Semester Production team member: 60% In-class critical discussion: 30% Readings and video assignments: 10%

VR / AR Fundamentals

I. The Class 1. Who are You? 2. Who Am I? 3. Class structure 4. Assignments & grading 5. Being “positively critical”

II. What are VR / AR fundamentals?

Week 1: Introduction

VR / AR Fundamentals

I. The Class 1. Who are You? 2. Who Am I? 3. Class structure 4. Assignments & grading 5. Being “positively critical”

II. What are VR / AR fundamentals?

Week 1: Introduction

5. Being “positively critical”

Note written by Marie Sester over dinner with Red and me, April 3, 2009, NYC:

Red Burns - Her Way - April 09 1. Am always direct.  2. I have a good bullshit detector. 3. Nobody intimidates me - Am never afraid of anybody.

5. Being “positively critical”

Note written by Marie Sester over dinner with Red and me, April 3, 2009, NYC:

Red Burns - Her Way - April 09 1. Am always direct.  2. I have a good bullshit detector. 3. Nobody intimidates me - Am never afraid of anybody.

VR / AR Fundamentals

I. The Class 1. Who are You? 2. Who Am I? 3. Class structure 4. Assignments & grading 5. Being “positively critical”

II. What are VR / AR fundamentals?

Week 1: Introduction

VR / AR Fundamentals

I. The Class 1. Who are You? 2. Who Am I? 3. Class structure 4. Assignments & grading 5. Being “positively critical”

II. What are VR / AR fundamentals?

Week 1: Introduction

What are VR / AR fundamentals?

VR / AR fundamentals

Prologue

1. The Dark Side! - “The Fog of VR” 2. Some Comments about VR 3. Some Comments about AR 4. My Approach

What are VR / AR fundamentals?

What are VR / AR fundamentals?

The Dark Side! - “The Fog of VR”

Delirium! Naiveté! Hype!

What are VR / AR fundamentals?

The Dark Side! - “The Fog of VR”

Delirium!

“I saw the best minds of my generation destroyed by madness.” Alan Ginsberg, “Howl”

What are VR / AR fundamentals?

The Dark Side! - “The Fog of VR”

“I saw the best minds of my generation destroyed by madness.” Alan Ginsberg, “Howl”

Twice.

Delirium!

What are VR / AR fundamentals?

The Dark Side! - “The Fog of VR”

“Virtual” “Reality”

Delirium!

What are VR / AR fundamentals?

The Dark Side! - “The Fog of VR”

“Virtual Reality”

Delirium!

What are VR / AR fundamentals?

The Dark Side! - “The Fog of VR”

Didn’t meet market predictions Nonsense forums (“VR in 100 Years”) Vacuum Chamber Had its share of “tech bro” scandals Current state of Silicon Valley Current state of the US

Delirium!

What are VR / AR fundamentals?

The Dark Side! - “The Fog of VR”

Delirium!

alternative from

the past

What are VR / AR fundamentals?

The Dark Side! - “The Fog of VR”

Naiveté!

“Beginner’s Mind” ironies

2010

(hi my name is) mono view

1380 x 460 409 x 460

Romalyn

690 x 388

(the question game)

Zach on left moves closer to Todd on right.

Both maintain eye contact.

[audio]

1: Zach and Todd are directly opposite the camera.

2, 3, 4: Zach and Todd are offset from the camera.

1

2

3

4

1

2

3

4

Creative Connections

What are VR / AR fundamentals?

The Dark Side! - “The Fog of VR”

Hype!

4X ROI is “not interesting.”

“We want 40x ROI.”

What are VR / AR fundamentals?

The Dark Side! - “The Fog of VR”

Delirium! Naiveté! Hype!

1. The Dark Side! - “The Fog of VR” 2. Some Comments about VR 3. Some Comments about AR 4. My Approach

What are VR / AR fundamentals?

1. The Dark Side! - “The Fog of VR” 2. Some Comments about VR 3. Some Comments about AR 4. My Approach

What are VR / AR fundamentals?

VR Cinema VR Games

2D frames

Hollywood Silicon Valley

linear interactive

3D models

get it right in production fluidity in post-production

Two Different Cultures Two Different Lineages

1. The Dark Side! - “The Fog of VR” 2. Some Comments about VR 3. Some Comments about AR 4. My Approach

What are VR / AR fundamentals?

2016

What are VR / AR fundamentals?

Some comments about AR

Opacity Mobile v Immersive (v Projection Mapping)

The End-Game

1. The Dark Side! - “The Fog of VR” 2. Some Comments about AR 3. My Approach

What are VR / AR fundamentals?

A Funny Question

What are VR / AR fundamentals?

My Approach

How do you know I’m not a movie?

What are VR / AR fundamentals?

My Approach

Mixing Board Analogy

2. What are VR / AR fundamentals?

Begin with our sensors (our “inputs”)

• sight & sound • touch • smell & taste • (mind)

2. What are VR / AR fundamentals?

Begin with our sensors (our “inputs”)

• sight & sound • touch • smell & taste • (mind)

And our effectors (our “outputs”)

• fingers & hands • head position & gaze • feet & body • voice • (mind?)

2. What are VR / AR fundamentals?

What does it take to “fool” our sensors and effectors into believing a representation is real?

2. What are VR / AR fundamentals?

What does it take to “fool” our sensors and effectors into believing a representation is real?

Start here . . .

1 - Resolution and FidelityJiayi Wang

2 - Spatiality and ImmersionJiayi Wang

2 - Spatiality and ImmersionJiayi Wang

2 - Spatiality and ImmersionJiayi Wang

3 - Other Senses (Haptic, Smell, Taste, Mind)

Jiayi Wang

4 - Input and InteractivityJiayi Wang

5 - Social and LivenessJiayi Wang

2. What are VR / AR fundamentals?

Different communities and applications have different priorities for these fundamentals

• Cinema or Games (“Hollywood or Silicon Valley”)

• VR or AR

• Home or Location-based or Outdoors

• Immersive or “Casual”

• Cheap or Expensive

Week 1 - Introduction to VR and AR

Week 2 - FUNDAMENTALS 1: Audiovisual Resolution and Fidelity Fooling a single static eye and single static ear experiencing a framed representation.

Week 3 - FUNDAMENTALS 2: Audiovisual Spatiality and Immersion Fooling two static eyes and ears, then two moving eyes and ears, then removing the frame.

Week 4 - FUNDAMENTALS 3: Other Senses (Haptic, Smell, Taste, Mind) Fooling the other senses.

Week 5 - FUNDAMENTALS 4: Input and Interactivity Using our effectors as inputs and how they shape interactive experiences.

Week 6 - FUNDAMENTALS 5: Social and Liveness Being immersed with other people and live events.

Week 7 - Current Dilemmas This is a lead-in, “pre-production,” to the second half-semester.

First half semester

VR / AR Fundamentals

Read:

Alvin Wang Graylin, “16 Lessons for a VR-First Future From ‘Ready Player One’” (2017)

Week 1: Assignment for Wednesday

VR / AR Fundamentals

Read:

Alvin Wang Graylin, “16 Lessons for a VR-First Future From ‘Ready Player One’” (2017)

Post and Present:

Of the 16 Lessons, which one you MOST agree with and which one you LEAST agree with. The posts can be very short, like one or two sentences for each.

Week 1: Assignment for Wednesday