The Profitable Collision of Real and Virtual Worlds · The Profitable Collision of Real and Virtual...

20
The Profitable Collision of Real and Virtual Worlds Steve Eglash Executive Director, Strategic Research Initiatives, Computer Science, Stanford University T HE AI S UMMIT S AN F RANCISCO , 27 S EPTEMBER 2017

Transcript of The Profitable Collision of Real and Virtual Worlds · The Profitable Collision of Real and Virtual...

Page 1: The Profitable Collision of Real and Virtual Worlds · The Profitable Collision of Real and Virtual Worlds. Steve Eglash Executive Director, Strategic Research Initiatives, Computer

The Profitable Collision of Real and Virtual Worlds

Steve EglashExecutive Director, Strategic Research Initiatives,

Computer Science, Stanford University

T H E A I S U M M I TS A N F R A N C I S C O, 2 7 S E P TE M B E R 2 0 1 7

Page 2: The Profitable Collision of Real and Virtual Worlds · The Profitable Collision of Real and Virtual Worlds. Steve Eglash Executive Director, Strategic Research Initiatives, Computer

The AI and data science revolution

Page 3: The Profitable Collision of Real and Virtual Worlds · The Profitable Collision of Real and Virtual Worlds. Steve Eglash Executive Director, Strategic Research Initiatives, Computer

Some trends in AI and data science

• Data mining and machine learning• Algorithms that learn from data• Clustering• Unstructured and dark data• Anomaly detection in real-time streams• Probabilistic approaches• Computer vision• Natural language processing• Decision making in complex environments• Merging of real and virtual worlds

Page 4: The Profitable Collision of Real and Virtual Worlds · The Profitable Collision of Real and Virtual Worlds. Steve Eglash Executive Director, Strategic Research Initiatives, Computer

The merging of real and virtual worldsPreviously, artistic animation primarily for

entertainment and gamingNow, physics-based animation allows

predictive modeling and creation of realistic virtual objects

Page 5: The Profitable Collision of Real and Virtual Worlds · The Profitable Collision of Real and Virtual Worlds. Steve Eglash Executive Director, Strategic Research Initiatives, Computer

Artistic animation versus physics-based animation

Doug James

Page 6: The Profitable Collision of Real and Virtual Worlds · The Profitable Collision of Real and Virtual Worlds. Steve Eglash Executive Director, Strategic Research Initiatives, Computer

Physics-based animation, simulation, and predictive modeling• Based on physical principles of materials and forces• If the models are correct, then provides accurate images, dynamics, and

interactions• Allows creation of images of things that no one has ever seen• Enables investigation of events in a simulated environment• How?

– Create virtual objects that accurately reflect properties of real physical objects

• Why?– Useful for entertainment, e-commerce, education, experimentation,

and much more– Can generate data– Can narrow range of likely solutions

Page 7: The Profitable Collision of Real and Virtual Worlds · The Profitable Collision of Real and Virtual Worlds. Steve Eglash Executive Director, Strategic Research Initiatives, Computer

Component parts• Rendering: light transport, 3D scene → 2D image• Geometry: creating 3D geometry for a scene• Animation: moving 3D geometry around• Simulation: automated animation

Page 8: The Profitable Collision of Real and Virtual Worlds · The Profitable Collision of Real and Virtual Worlds. Steve Eglash Executive Director, Strategic Research Initiatives, Computer

ApplicationsEntertainment and gaming, obviouslyVirtual reality and augmented realityTextiles, garments, and fashionRetailing and e-commerceEducation and trainingRoboticsHealthcare and medicineInsurance and automotive collision repairDisaster scenario responseManufacturingand much more

Page 9: The Profitable Collision of Real and Virtual Worlds · The Profitable Collision of Real and Virtual Worlds. Steve Eglash Executive Director, Strategic Research Initiatives, Computer

Visual cloth details

Page 10: The Profitable Collision of Real and Virtual Worlds · The Profitable Collision of Real and Virtual Worlds. Steve Eglash Executive Director, Strategic Research Initiatives, Computer

Virtual cloth for e-commerce• Apparel is an important category for

e-commerce• However, it is hard to decide the fit or imagine

the look without actually trying on the clothing• Returns due to the mismatch between

expectation and reality can incur logistical loss and hurt customer satisfaction

• Virtual try-on can make online clothes shopping a smoother process for both vendors and shoppers

Ron Fedkiw, Jenny Jin, Stanford University

Page 11: The Profitable Collision of Real and Virtual Worlds · The Profitable Collision of Real and Virtual Worlds. Steve Eglash Executive Director, Strategic Research Initiatives, Computer

Cloth simulation is hard• The goal is to simulate high quality and visually

interesting virtual cloth with realistic folds and wrinkle patterns under dynamic motion

• It is very difficult to simulate real world cloth faithfully—even the best CG cloth for movies isn’t good enough compared to real cloth

Page 12: The Profitable Collision of Real and Virtual Worlds · The Profitable Collision of Real and Virtual Worlds. Steve Eglash Executive Director, Strategic Research Initiatives, Computer

Improved cloth simulationThe inequality cloth simulation model on the right produces better visual results and improved data augmentation

Ron Fedkiw, Jenny Jin, Stanford University

Page 14: The Profitable Collision of Real and Virtual Worlds · The Profitable Collision of Real and Virtual Worlds. Steve Eglash Executive Director, Strategic Research Initiatives, Computer

14

Faces and the uncanny valley

We, as humans, subconsciously detect something wrong with near human-like characters, particularly in their face

Characters that are obviously fake and non-human provoke a negative response

This effect occurs when one pushes the boundaries of realism, which is where the visual effects industry is at right now

Very talented artists spend lots of time tweaking results for each shot by hand

Extremely hard to fix, because we often don’t know what’s wrong

Ron Fedkiw, Michael Bao, Stanford University

Page 15: The Profitable Collision of Real and Virtual Worlds · The Profitable Collision of Real and Virtual Worlds. Steve Eglash Executive Director, Strategic Research Initiatives, Computer

15

Building a realistic anatomical model

Create a highly accurate model of the face and simulate it using volumetric muscles

Compute muscle activations from motion capture data by solving an inverse problem

Preserves physical properties: Volume Preservation Contact and CollisionProblems: Lacks the expressiveness of traditional

facial blendshape animations Hard for artists to control Still uncanny; not good enough!

circa 2005

Page 16: The Profitable Collision of Real and Virtual Worlds · The Profitable Collision of Real and Virtual Worlds. Steve Eglash Executive Director, Strategic Research Initiatives, Computer

Hybridizing data and simulation

Page 17: The Profitable Collision of Real and Virtual Worlds · The Profitable Collision of Real and Virtual Worlds. Steve Eglash Executive Director, Strategic Research Initiatives, Computer

Yarn-level cloth simulation

Doug James (Stanford), Steve Marschner (Cornell)

Page 18: The Profitable Collision of Real and Virtual Worlds · The Profitable Collision of Real and Virtual Worlds. Steve Eglash Executive Director, Strategic Research Initiatives, Computer

18

Scarf at 1/6th speedJonathan Kaldor, Steve Marschner, Doug James, Cornell University

Page 19: The Profitable Collision of Real and Virtual Worlds · The Profitable Collision of Real and Virtual Worlds. Steve Eglash Executive Director, Strategic Research Initiatives, Computer

Implications for businessIndustry Some potential applicationsEntertainment and gaming Obviously

Virtual reality and augmented reality Additional sources of data for immersive experiences

Textiles, garments, and fashion Digital prototyping and virtual dressing rooms

Retailing and e-commerce Samples, online interactivity

Education and training Convenient and safe experiential training

Robotics Simulation, planning, tele-operation

Healthcare and medicine Training, telemedicine, research

Insurance Automotive collision repair, for example

Disaster scenario response Prediction and next best action

Manufacturing Integrate design, manufacture, and end user

And much much more!

Page 20: The Profitable Collision of Real and Virtual Worlds · The Profitable Collision of Real and Virtual Worlds. Steve Eglash Executive Director, Strategic Research Initiatives, Computer

20