Three JS, Unreal, Unity. VR/AR/MR without 9 circles of hell
Transcript of Three JS, Unreal, Unity. VR/AR/MR without 9 circles of hell
![Page 1: Three JS, Unreal, Unity. VR/AR/MR without 9 circles of hell](https://reader035.fdocuments.in/reader035/viewer/2022062316/58ed28f71a28abf5278b46eb/html5/thumbnails/1.jpg)
Three JS, Unreal, Unity.VR/AR without 9 circles of hell
by Marina Kolesnichenko, Software Engineer at ElifTech
![Page 2: Three JS, Unreal, Unity. VR/AR/MR without 9 circles of hell](https://reader035.fdocuments.in/reader035/viewer/2022062316/58ed28f71a28abf5278b46eb/html5/thumbnails/2.jpg)
Three.js cookbook
Three.js is a cross-browser JavaScript library/API used to create and display animated 3D computer graphics in a web browser. Three.js uses WebGL.
![Page 3: Three JS, Unreal, Unity. VR/AR/MR without 9 circles of hell](https://reader035.fdocuments.in/reader035/viewer/2022062316/58ed28f71a28abf5278b46eb/html5/thumbnails/3.jpg)
Quick start
http://threejs.org/Click the “Download” link on the left side of your screen. Once the zip has finished downloading, open it up and go to the build folder. Inside, you’ll find a file called three.min.js and, if you’re following along, you should copy this file into your local development directory.
What about NPM?npm install threeBut this library version is 'limited' at moment. So I recommend download library from official site.
![Page 4: Three JS, Unreal, Unity. VR/AR/MR without 9 circles of hell](https://reader035.fdocuments.in/reader035/viewer/2022062316/58ed28f71a28abf5278b46eb/html5/thumbnails/4.jpg)
Browsers
![Page 5: Three JS, Unreal, Unity. VR/AR/MR without 9 circles of hell](https://reader035.fdocuments.in/reader035/viewer/2022062316/58ed28f71a28abf5278b46eb/html5/thumbnails/5.jpg)
Easy 0-20 level
![Page 6: Three JS, Unreal, Unity. VR/AR/MR without 9 circles of hell](https://reader035.fdocuments.in/reader035/viewer/2022062316/58ed28f71a28abf5278b46eb/html5/thumbnails/6.jpg)
Scene
new THREE.Scene();new THREE.PerspectiveCamera(45,
width / height, 1, 10000);
Position Range
![Page 7: Three JS, Unreal, Unity. VR/AR/MR without 9 circles of hell](https://reader035.fdocuments.in/reader035/viewer/2022062316/58ed28f71a28abf5278b46eb/html5/thumbnails/7.jpg)
More power!
OBJLoaderA loader for loading an .obj resource.
![Page 8: Three JS, Unreal, Unity. VR/AR/MR without 9 circles of hell](https://reader035.fdocuments.in/reader035/viewer/2022062316/58ed28f71a28abf5278b46eb/html5/thumbnails/8.jpg)
Basic
var loader = new THREE.OBJLoader();
// load a resourceloader.load(// resource URL'models/monster.obj',
// function when resource is loadedfunction ( object ) {scene.add( object );});
![Page 9: Three JS, Unreal, Unity. VR/AR/MR without 9 circles of hell](https://reader035.fdocuments.in/reader035/viewer/2022062316/58ed28f71a28abf5278b46eb/html5/thumbnails/9.jpg)
With .mtlloadModel(model, options, done) {let name = model.split("/").pop();let path = `${model.substring(0, model.lastIndexOf("/"))}/`;
this.mtl.setPath(path);this.mtl.load(`${name}.mtl`, (materials) => {materials.preload();
for (let key in materials.materials) {let material = materials.materials[key];
material.shading = THREE.SmoothShading;material.side = THREE.DoubleSide;}
this.obj.setMaterials(materials);this.obj.setPath(path);this.obj.load(`${name}.obj`, (object) => {object.name = options.name || "Scene";object.position.fromArray(options.position || [0, 0, 0]);
object.rotation.fromArray(options.rotation || [0, 0, 0]);object.scale.set(options.scale || 1, options.scale || 1, options.scale || 1);
object.traverse((child) => {if (child instanceof THREE.Mesh) {child.current = {};
child.original = {};child.original.material = child.material.clone();
child.receiveShadow = false;child.castShadow = true;}});
done(object, materials.materials);});});}
![Page 10: Three JS, Unreal, Unity. VR/AR/MR without 9 circles of hell](https://reader035.fdocuments.in/reader035/viewer/2022062316/58ed28f71a28abf5278b46eb/html5/thumbnails/10.jpg)
What?Material Library File (.mtl)Material library files contain one or more material definitions, each of which includes the color, texture, and reflection map of individual materials. These are applied to the surfaces and vertices of objects. Material files are stored in ASCII format and have the .mtl extension.
![Page 11: Three JS, Unreal, Unity. VR/AR/MR without 9 circles of hell](https://reader035.fdocuments.in/reader035/viewer/2022062316/58ed28f71a28abf5278b46eb/html5/thumbnails/11.jpg)
Render scene
render() {if (this.mode == 'vr') {this.stereo.render(this.scene, this.camera);} else {this.renderer.render(this.scene, this.camera);}
window.requestAnimationFrame(() => {this.controls.update();this.update(delta);this.render();});}
![Page 12: Three JS, Unreal, Unity. VR/AR/MR without 9 circles of hell](https://reader035.fdocuments.in/reader035/viewer/2022062316/58ed28f71a28abf5278b46eb/html5/thumbnails/12.jpg)
20-65 level
![Page 13: Three JS, Unreal, Unity. VR/AR/MR without 9 circles of hell](https://reader035.fdocuments.in/reader035/viewer/2022062316/58ed28f71a28abf5278b46eb/html5/thumbnails/13.jpg)
Workflow
new THREE.BoxGeometrynew THREE.MeshBasicMaterial( { color: 0xff0000, wireframe: true } )
![Page 14: Three JS, Unreal, Unity. VR/AR/MR without 9 circles of hell](https://reader035.fdocuments.in/reader035/viewer/2022062316/58ed28f71a28abf5278b46eb/html5/thumbnails/14.jpg)
Workflow
![Page 15: Three JS, Unreal, Unity. VR/AR/MR without 9 circles of hell](https://reader035.fdocuments.in/reader035/viewer/2022062316/58ed28f71a28abf5278b46eb/html5/thumbnails/15.jpg)
Workflow
![Page 16: Three JS, Unreal, Unity. VR/AR/MR without 9 circles of hell](https://reader035.fdocuments.in/reader035/viewer/2022062316/58ed28f71a28abf5278b46eb/html5/thumbnails/16.jpg)
Audiothis.camera.listener = new THREE.AudioListener();this.camera.listener.name = 'Audio Listener';this.camera.add(this.camera.listener);
![Page 17: Three JS, Unreal, Unity. VR/AR/MR without 9 circles of hell](https://reader035.fdocuments.in/reader035/viewer/2022062316/58ed28f71a28abf5278b46eb/html5/thumbnails/17.jpg)
Hard 65-80 level
![Page 18: Three JS, Unreal, Unity. VR/AR/MR without 9 circles of hell](https://reader035.fdocuments.in/reader035/viewer/2022062316/58ed28f71a28abf5278b46eb/html5/thumbnails/18.jpg)
Shaders
A Shader is a piece of code that runs directly on the GPU. Most modern devices have powerful GPUs designed to handle graphics effects without taxing the CPU.
Pixels
Vertex
Shaders
or Babylon...
![Page 19: Three JS, Unreal, Unity. VR/AR/MR without 9 circles of hell](https://reader035.fdocuments.in/reader035/viewer/2022062316/58ed28f71a28abf5278b46eb/html5/thumbnails/19.jpg)
Pixels
Pixel Shaders modify or draw the pixels in a scene. They are used to render a 3D scene into pixels, and also typically used to add lighting and other effects to a 3D scene.
![Page 20: Three JS, Unreal, Unity. VR/AR/MR without 9 circles of hell](https://reader035.fdocuments.in/reader035/viewer/2022062316/58ed28f71a28abf5278b46eb/html5/thumbnails/20.jpg)
Pixels
Shaders that draw an image or texture directly. These types of shaders can be loaded into a THREE.ShaderMaterial to give cool textures to 3D objects.
![Page 21: Three JS, Unreal, Unity. VR/AR/MR without 9 circles of hell](https://reader035.fdocuments.in/reader035/viewer/2022062316/58ed28f71a28abf5278b46eb/html5/thumbnails/21.jpg)
Pixels
Shaders that modify another image or texture. These allow you to do post-processing on an existing texture, for example to add a glow or blur to a 3D scene.
![Page 22: Three JS, Unreal, Unity. VR/AR/MR without 9 circles of hell](https://reader035.fdocuments.in/reader035/viewer/2022062316/58ed28f71a28abf5278b46eb/html5/thumbnails/22.jpg)
Vertex
Vertex shaders generate or modify 3D geometry by manipulating its vertices.
![Page 23: Three JS, Unreal, Unity. VR/AR/MR without 9 circles of hell](https://reader035.fdocuments.in/reader035/viewer/2022062316/58ed28f71a28abf5278b46eb/html5/thumbnails/23.jpg)
Vertex
![Page 24: Three JS, Unreal, Unity. VR/AR/MR without 9 circles of hell](https://reader035.fdocuments.in/reader035/viewer/2022062316/58ed28f71a28abf5278b46eb/html5/thumbnails/24.jpg)
Vertex
![Page 25: Three JS, Unreal, Unity. VR/AR/MR without 9 circles of hell](https://reader035.fdocuments.in/reader035/viewer/2022062316/58ed28f71a28abf5278b46eb/html5/thumbnails/25.jpg)
Physics
![Page 26: Three JS, Unreal, Unity. VR/AR/MR without 9 circles of hell](https://reader035.fdocuments.in/reader035/viewer/2022062316/58ed28f71a28abf5278b46eb/html5/thumbnails/26.jpg)
VR
![Page 27: Three JS, Unreal, Unity. VR/AR/MR without 9 circles of hell](https://reader035.fdocuments.in/reader035/viewer/2022062316/58ed28f71a28abf5278b46eb/html5/thumbnails/27.jpg)
VR
this.stereo = new THREE.OculusRiftEffect(this.renderer);let hmd = this.stereo.getHMD();
hmd.hResolution = this.width;hmd.vResolution = this.height;
this.stereo.setHMD(hmd);
this.stereo.setSize(this.width, this.height);
this.stereo.render(this.scene, this.camera);
![Page 28: Three JS, Unreal, Unity. VR/AR/MR without 9 circles of hell](https://reader035.fdocuments.in/reader035/viewer/2022062316/58ed28f71a28abf5278b46eb/html5/thumbnails/28.jpg)
VR
![Page 29: Three JS, Unreal, Unity. VR/AR/MR without 9 circles of hell](https://reader035.fdocuments.in/reader035/viewer/2022062316/58ed28f71a28abf5278b46eb/html5/thumbnails/29.jpg)
VR
![Page 30: Three JS, Unreal, Unity. VR/AR/MR without 9 circles of hell](https://reader035.fdocuments.in/reader035/viewer/2022062316/58ed28f71a28abf5278b46eb/html5/thumbnails/30.jpg)
Raycaster
this.raycaster = new THREE.Raycaster();this.raycaster.setFromCamera(new THREE.Vector2(), this.camera);
let intersects = this.raycaster.intersectObject(object, true);
![Page 31: Three JS, Unreal, Unity. VR/AR/MR without 9 circles of hell](https://reader035.fdocuments.in/reader035/viewer/2022062316/58ed28f71a28abf5278b46eb/html5/thumbnails/31.jpg)
OSGJS
OSGJS is based on OpenSceneGraph API, which itself is based on a few concepts that allow for a solid grasp around the whole library once and for all, and those are mandatory in order to dive into code.
![Page 32: Three JS, Unreal, Unity. VR/AR/MR without 9 circles of hell](https://reader035.fdocuments.in/reader035/viewer/2022062316/58ed28f71a28abf5278b46eb/html5/thumbnails/32.jpg)
Difference
viewer = new osgViewer.Viewer(canvas, {antialias : true, alpha: true });rotate = new osg.MatrixTransform();viewer.setupManipulator(new osgGA.OrbitManipulator());viewer.getManipulator().setDistance(20.0);viewer.run();
![Page 33: Three JS, Unreal, Unity. VR/AR/MR without 9 circles of hell](https://reader035.fdocuments.in/reader035/viewer/2022062316/58ed28f71a28abf5278b46eb/html5/thumbnails/33.jpg)
Why not?
1. FPS and freezes2. Limited scene3. Difficult work with animation and shaders
![Page 34: Three JS, Unreal, Unity. VR/AR/MR without 9 circles of hell](https://reader035.fdocuments.in/reader035/viewer/2022062316/58ed28f71a28abf5278b46eb/html5/thumbnails/34.jpg)
Unity/Unreal
Easy Mobile
Fast
graphics
Best
![Page 35: Three JS, Unreal, Unity. VR/AR/MR without 9 circles of hell](https://reader035.fdocuments.in/reader035/viewer/2022062316/58ed28f71a28abf5278b46eb/html5/thumbnails/35.jpg)
Unreal Engine uses C++ and Unity uses mostly C# or JavaScript. Unreal Engine has Blueprint visual scripting. Technically you don't ever need to write a single line of code.
![Page 36: Three JS, Unreal, Unity. VR/AR/MR without 9 circles of hell](https://reader035.fdocuments.in/reader035/viewer/2022062316/58ed28f71a28abf5278b46eb/html5/thumbnails/36.jpg)
All pointers of objects that are at the level should point to objects at the level and objects outside the level can't have pointers pointing to objects at the level.
![Page 37: Three JS, Unreal, Unity. VR/AR/MR without 9 circles of hell](https://reader035.fdocuments.in/reader035/viewer/2022062316/58ed28f71a28abf5278b46eb/html5/thumbnails/37.jpg)
VR
Virtual reality (VR), which can be referred to as immersive multimedia or computer-simulated reality, replicates an environment that simulates a physical presence in places in the real world or an imagined world, allowing the user to interact in that world.
![Page 38: Three JS, Unreal, Unity. VR/AR/MR without 9 circles of hell](https://reader035.fdocuments.in/reader035/viewer/2022062316/58ed28f71a28abf5278b46eb/html5/thumbnails/38.jpg)
VR
![Page 39: Three JS, Unreal, Unity. VR/AR/MR without 9 circles of hell](https://reader035.fdocuments.in/reader035/viewer/2022062316/58ed28f71a28abf5278b46eb/html5/thumbnails/39.jpg)
VR
![Page 40: Three JS, Unreal, Unity. VR/AR/MR without 9 circles of hell](https://reader035.fdocuments.in/reader035/viewer/2022062316/58ed28f71a28abf5278b46eb/html5/thumbnails/40.jpg)
VR
![Page 41: Three JS, Unreal, Unity. VR/AR/MR without 9 circles of hell](https://reader035.fdocuments.in/reader035/viewer/2022062316/58ed28f71a28abf5278b46eb/html5/thumbnails/41.jpg)
Device?
![Page 42: Three JS, Unreal, Unity. VR/AR/MR without 9 circles of hell](https://reader035.fdocuments.in/reader035/viewer/2022062316/58ed28f71a28abf5278b46eb/html5/thumbnails/42.jpg)
AR
Augmented reality (AR) is a live, direct or indirect view of a physical, real-world environment whose elements are augmented (or supplemented) by computer-generated sensory input such as sound, video, graphics or GPS data
![Page 43: Three JS, Unreal, Unity. VR/AR/MR without 9 circles of hell](https://reader035.fdocuments.in/reader035/viewer/2022062316/58ed28f71a28abf5278b46eb/html5/thumbnails/43.jpg)
VR, ok! AR, ok!MR O_O?Mixed reality - is the merging of real and virtual worlds to produce new environments and visualisations where physical and digital objects co-exist and interact in real time. Mixed reality is an overlay of synthetic content on the real world that is anchored to and interacts with the real world—picture surgeons overlaying virtual ultrasound images on their patient while performing an operation, for example. The key characteristic of MR is that the synthetic content and the real-world content are able to react to each other in real time.
![Page 44: Three JS, Unreal, Unity. VR/AR/MR without 9 circles of hell](https://reader035.fdocuments.in/reader035/viewer/2022062316/58ed28f71a28abf5278b46eb/html5/thumbnails/44.jpg)
![Page 45: Three JS, Unreal, Unity. VR/AR/MR without 9 circles of hell](https://reader035.fdocuments.in/reader035/viewer/2022062316/58ed28f71a28abf5278b46eb/html5/thumbnails/45.jpg)
WOW!
![Page 46: Three JS, Unreal, Unity. VR/AR/MR without 9 circles of hell](https://reader035.fdocuments.in/reader035/viewer/2022062316/58ed28f71a28abf5278b46eb/html5/thumbnails/46.jpg)
Overdraw
Overdraw view allows you to see what objects are drawn on top of another, which is a waste of GPU time. Look at reducing overdraw as much as possible. You can view overdraw in the Scene View by using the Scene View Control Bar.
![Page 47: Three JS, Unreal, Unity. VR/AR/MR without 9 circles of hell](https://reader035.fdocuments.in/reader035/viewer/2022062316/58ed28f71a28abf5278b46eb/html5/thumbnails/47.jpg)
Overdraw
![Page 48: Three JS, Unreal, Unity. VR/AR/MR without 9 circles of hell](https://reader035.fdocuments.in/reader035/viewer/2022062316/58ed28f71a28abf5278b46eb/html5/thumbnails/48.jpg)
Occlusion
Occlusion Culling stops objects from being rendered if they cannot be seen. For example, we don’t want to render another room if a door is closed and it cannot be seen.
![Page 49: Three JS, Unreal, Unity. VR/AR/MR without 9 circles of hell](https://reader035.fdocuments.in/reader035/viewer/2022062316/58ed28f71a28abf5278b46eb/html5/thumbnails/49.jpg)
Occlusion
![Page 50: Three JS, Unreal, Unity. VR/AR/MR without 9 circles of hell](https://reader035.fdocuments.in/reader035/viewer/2022062316/58ed28f71a28abf5278b46eb/html5/thumbnails/50.jpg)
Rendering
Approximating your own distortion solution, even when it “looks about right,” is often discomforting for users.
Any deviation from the optical flow that accompanies real world head movement creates oculomotor issues and bodily discomfort.Consider supersampling and/or anti-aliasing to remedy low apparent resolution, which will appear worst at the center of each eye’s screen.
Tabu
![Page 51: Three JS, Unreal, Unity. VR/AR/MR without 9 circles of hell](https://reader035.fdocuments.in/reader035/viewer/2022062316/58ed28f71a28abf5278b46eb/html5/thumbnails/51.jpg)
Minimizing LatencyYour code should run at a frame rate equal to or greater than the Rift display refresh rate, v-synced and unbuffered. Lag and dropped frames produce judder which is discomforting in VR.Ideally, target 20ms or less motion-to-photon latency (measurable with the Rift’s built-in latency tester). Organize your code to minimize the time from sensor fusion (reading the Rift sensors) to rendering.
Decrease eye-render buffer resolution to save video memory and increase frame rate.
Avoid visuals that upset the user’s sense of stability in their environment. Rotating or moving the horizon line or other large components of the user’s environment in conflict with the user’s real-world self-motion (or lack thereof) can be discomforting.
![Page 52: Three JS, Unreal, Unity. VR/AR/MR without 9 circles of hell](https://reader035.fdocuments.in/reader035/viewer/2022062316/58ed28f71a28abf5278b46eb/html5/thumbnails/52.jpg)
Feel free to ask questions in the
comments