Building Collaborative AR Experiences · Multi-User Experiences in ARKit 2 Recap User1 User2....
Transcript of Building Collaborative AR Experiences · Multi-User Experiences in ARKit 2 Recap User1 User2....
![Page 1: Building Collaborative AR Experiences · Multi-User Experiences in ARKit 2 Recap User1 User2. Multi-User Experiences in ARKit 2 Recap User1 User2. ... First device acts as “host”](https://reader030.fdocuments.in/reader030/viewer/2022011916/5fd690c8154f054c1412d96b/html5/thumbnails/1.jpg)
© 2019 Apple Inc. All rights reserved. Redistribution or public display not permitted without written permission from Apple.
#WWDC19
Kuen-han Lin, ARKit Engineering David Paschich, Tools Foundation
•Building Collaborative AR Experiences
![Page 2: Building Collaborative AR Experiences · Multi-User Experiences in ARKit 2 Recap User1 User2. Multi-User Experiences in ARKit 2 Recap User1 User2. ... First device acts as “host”](https://reader030.fdocuments.in/reader030/viewer/2022011916/5fd690c8154f054c1412d96b/html5/thumbnails/2.jpg)
![Page 3: Building Collaborative AR Experiences · Multi-User Experiences in ARKit 2 Recap User1 User2. Multi-User Experiences in ARKit 2 Recap User1 User2. ... First device acts as “host”](https://reader030.fdocuments.in/reader030/viewer/2022011916/5fd690c8154f054c1412d96b/html5/thumbnails/3.jpg)
![Page 4: Building Collaborative AR Experiences · Multi-User Experiences in ARKit 2 Recap User1 User2. Multi-User Experiences in ARKit 2 Recap User1 User2. ... First device acts as “host”](https://reader030.fdocuments.in/reader030/viewer/2022011916/5fd690c8154f054c1412d96b/html5/thumbnails/4.jpg)
•Collaborative Session •ARAnchor Best Practice for Multi-User AR •SwiftStrike — A New Multiplayer AR Experience
![Page 5: Building Collaborative AR Experiences · Multi-User Experiences in ARKit 2 Recap User1 User2. Multi-User Experiences in ARKit 2 Recap User1 User2. ... First device acts as “host”](https://reader030.fdocuments.in/reader030/viewer/2022011916/5fd690c8154f054c1412d96b/html5/thumbnails/5.jpg)
•Collaborative Session
![Page 6: Building Collaborative AR Experiences · Multi-User Experiences in ARKit 2 Recap User1 User2. Multi-User Experiences in ARKit 2 Recap User1 User2. ... First device acts as “host”](https://reader030.fdocuments.in/reader030/viewer/2022011916/5fd690c8154f054c1412d96b/html5/thumbnails/6.jpg)
Saving and Loading Map in ARKit 2 Recap
ARWorldMap data • Map of sparse 3D landmarks • List of named ARAnchors
![Page 7: Building Collaborative AR Experiences · Multi-User Experiences in ARKit 2 Recap User1 User2. Multi-User Experiences in ARKit 2 Recap User1 User2. ... First device acts as “host”](https://reader030.fdocuments.in/reader030/viewer/2022011916/5fd690c8154f054c1412d96b/html5/thumbnails/7.jpg)
Multi-User Experiences in ARKit 2 Recap
User1 User2
![Page 8: Building Collaborative AR Experiences · Multi-User Experiences in ARKit 2 Recap User1 User2. Multi-User Experiences in ARKit 2 Recap User1 User2. ... First device acts as “host”](https://reader030.fdocuments.in/reader030/viewer/2022011916/5fd690c8154f054c1412d96b/html5/thumbnails/8.jpg)
Multi-User Experiences in ARKit 2 Recap
User1 User2
![Page 9: Building Collaborative AR Experiences · Multi-User Experiences in ARKit 2 Recap User1 User2. Multi-User Experiences in ARKit 2 Recap User1 User2. ... First device acts as “host”](https://reader030.fdocuments.in/reader030/viewer/2022011916/5fd690c8154f054c1412d96b/html5/thumbnails/9.jpg)
Multi-User Experiences in ARKit 2 Recap
User1 User2
![Page 10: Building Collaborative AR Experiences · Multi-User Experiences in ARKit 2 Recap User1 User2. Multi-User Experiences in ARKit 2 Recap User1 User2. ... First device acts as “host”](https://reader030.fdocuments.in/reader030/viewer/2022011916/5fd690c8154f054c1412d96b/html5/thumbnails/10.jpg)
Multi-User Experiences in ARKit 2 Recap
User1 User2
![Page 11: Building Collaborative AR Experiences · Multi-User Experiences in ARKit 2 Recap User1 User2. Multi-User Experiences in ARKit 2 Recap User1 User2. ... First device acts as “host”](https://reader030.fdocuments.in/reader030/viewer/2022011916/5fd690c8154f054c1412d96b/html5/thumbnails/11.jpg)
Collaborative Session
Live multi-user AR experience
Continuously share ARAnchors and map data
No host user
![Page 12: Building Collaborative AR Experiences · Multi-User Experiences in ARKit 2 Recap User1 User2. Multi-User Experiences in ARKit 2 Recap User1 User2. ... First device acts as “host”](https://reader030.fdocuments.in/reader030/viewer/2022011916/5fd690c8154f054c1412d96b/html5/thumbnails/12.jpg)
![Page 13: Building Collaborative AR Experiences · Multi-User Experiences in ARKit 2 Recap User1 User2. Multi-User Experiences in ARKit 2 Recap User1 User2. ... First device acts as “host”](https://reader030.fdocuments.in/reader030/viewer/2022011916/5fd690c8154f054c1412d96b/html5/thumbnails/13.jpg)
![Page 14: Building Collaborative AR Experiences · Multi-User Experiences in ARKit 2 Recap User1 User2. Multi-User Experiences in ARKit 2 Recap User1 User2. ... First device acts as “host”](https://reader030.fdocuments.in/reader030/viewer/2022011916/5fd690c8154f054c1412d96b/html5/thumbnails/14.jpg)
![Page 15: Building Collaborative AR Experiences · Multi-User Experiences in ARKit 2 Recap User1 User2. Multi-User Experiences in ARKit 2 Recap User1 User2. ... First device acts as “host”](https://reader030.fdocuments.in/reader030/viewer/2022011916/5fd690c8154f054c1412d96b/html5/thumbnails/15.jpg)
![Page 16: Building Collaborative AR Experiences · Multi-User Experiences in ARKit 2 Recap User1 User2. Multi-User Experiences in ARKit 2 Recap User1 User2. ... First device acts as “host”](https://reader030.fdocuments.in/reader030/viewer/2022011916/5fd690c8154f054c1412d96b/html5/thumbnails/16.jpg)
![Page 17: Building Collaborative AR Experiences · Multi-User Experiences in ARKit 2 Recap User1 User2. Multi-User Experiences in ARKit 2 Recap User1 User2. ... First device acts as “host”](https://reader030.fdocuments.in/reader030/viewer/2022011916/5fd690c8154f054c1412d96b/html5/thumbnails/17.jpg)
![Page 18: Building Collaborative AR Experiences · Multi-User Experiences in ARKit 2 Recap User1 User2. Multi-User Experiences in ARKit 2 Recap User1 User2. ... First device acts as “host”](https://reader030.fdocuments.in/reader030/viewer/2022011916/5fd690c8154f054c1412d96b/html5/thumbnails/18.jpg)
Different Coordinates in Collaborative Session
![Page 19: Building Collaborative AR Experiences · Multi-User Experiences in ARKit 2 Recap User1 User2. Multi-User Experiences in ARKit 2 Recap User1 User2. ... First device acts as “host”](https://reader030.fdocuments.in/reader030/viewer/2022011916/5fd690c8154f054c1412d96b/html5/thumbnails/19.jpg)
Different Coordinates in Collaborative Session
![Page 20: Building Collaborative AR Experiences · Multi-User Experiences in ARKit 2 Recap User1 User2. Multi-User Experiences in ARKit 2 Recap User1 User2. ... First device acts as “host”](https://reader030.fdocuments.in/reader030/viewer/2022011916/5fd690c8154f054c1412d96b/html5/thumbnails/20.jpg)
Different Coordinates in Collaborative Session
User 2 External Map
User 1 External Map
![Page 21: Building Collaborative AR Experiences · Multi-User Experiences in ARKit 2 Recap User1 User2. Multi-User Experiences in ARKit 2 Recap User1 User2. ... First device acts as “host”](https://reader030.fdocuments.in/reader030/viewer/2022011916/5fd690c8154f054c1412d96b/html5/thumbnails/21.jpg)
Different Coordinates in Collaborative Session
User 2 External Map
User 1 External Map
![Page 22: Building Collaborative AR Experiences · Multi-User Experiences in ARKit 2 Recap User1 User2. Multi-User Experiences in ARKit 2 Recap User1 User2. ... First device acts as “host”](https://reader030.fdocuments.in/reader030/viewer/2022011916/5fd690c8154f054c1412d96b/html5/thumbnails/22.jpg)
Different Coordinates in Collaborative Session
User 2 External Map
User 1 External Map
![Page 23: Building Collaborative AR Experiences · Multi-User Experiences in ARKit 2 Recap User1 User2. Multi-User Experiences in ARKit 2 Recap User1 User2. ... First device acts as “host”](https://reader030.fdocuments.in/reader030/viewer/2022011916/5fd690c8154f054c1412d96b/html5/thumbnails/23.jpg)
Different Coordinates in Collaborative Session
User 2 External Map
User 1 External Map
![Page 24: Building Collaborative AR Experiences · Multi-User Experiences in ARKit 2 Recap User1 User2. Multi-User Experiences in ARKit 2 Recap User1 User2. ... First device acts as “host”](https://reader030.fdocuments.in/reader030/viewer/2022011916/5fd690c8154f054c1412d96b/html5/thumbnails/24.jpg)
Different Coordinates in Collaborative Session
User 2 External Map
User 1 External Map
![Page 25: Building Collaborative AR Experiences · Multi-User Experiences in ARKit 2 Recap User1 User2. Multi-User Experiences in ARKit 2 Recap User1 User2. ... First device acts as “host”](https://reader030.fdocuments.in/reader030/viewer/2022011916/5fd690c8154f054c1412d96b/html5/thumbnails/25.jpg)
Different Coordinates in Collaborative Session
User 2 External Map
User 1 External Map
![Page 26: Building Collaborative AR Experiences · Multi-User Experiences in ARKit 2 Recap User1 User2. Multi-User Experiences in ARKit 2 Recap User1 User2. ... First device acts as “host”](https://reader030.fdocuments.in/reader030/viewer/2022011916/5fd690c8154f054c1412d96b/html5/thumbnails/26.jpg)
Different Coordinates in Collaborative Session
![Page 27: Building Collaborative AR Experiences · Multi-User Experiences in ARKit 2 Recap User1 User2. Multi-User Experiences in ARKit 2 Recap User1 User2. ... First device acts as “host”](https://reader030.fdocuments.in/reader030/viewer/2022011916/5fd690c8154f054c1412d96b/html5/thumbnails/27.jpg)
Collaborative Session Flow Set up communication
User 1 Set up Network
Set up Network
ARSession Starts
Localize User 2 Map
Localize User 1 Map
Receive User 2
ARAnchor
Receive User 1
ARAnchor
Collaboration Data
User 2 ARSession Starts
Collaboration Data
![Page 28: Building Collaborative AR Experiences · Multi-User Experiences in ARKit 2 Recap User1 User2. Multi-User Experiences in ARKit 2 Recap User1 User2. ... First device acts as “host”](https://reader030.fdocuments.in/reader030/viewer/2022011916/5fd690c8154f054c1412d96b/html5/thumbnails/28.jpg)
Collaborative Session Flow Transmit collaboration data
User 1 Set up Network
Set up Network
ARSession Starts
Localize User 2 Map
Localize User 1 Map
Receive User 2
ARAnchor
Receive User 1
ARAnchor
Collaboration Data
User 2 ARSession Starts
Collaboration Data
![Page 29: Building Collaborative AR Experiences · Multi-User Experiences in ARKit 2 Recap User1 User2. Multi-User Experiences in ARKit 2 Recap User1 User2. ... First device acts as “host”](https://reader030.fdocuments.in/reader030/viewer/2022011916/5fd690c8154f054c1412d96b/html5/thumbnails/29.jpg)
// Collaborative Session
// Create world tracking configuration let config = ARWorldTrackingConfiguration()
// Enable collaborative session config.isCollaborationEnabled = true
// Run the configuration session.run(config)
![Page 30: Building Collaborative AR Experiences · Multi-User Experiences in ARKit 2 Recap User1 User2. Multi-User Experiences in ARKit 2 Recap User1 User2. ... First device acts as “host”](https://reader030.fdocuments.in/reader030/viewer/2022011916/5fd690c8154f054c1412d96b/html5/thumbnails/30.jpg)
// Collaborative Session
// Create world tracking configuration let config = ARWorldTrackingConfiguration()
// Enable collaborative session config.isCollaborationEnabled = true
// Run the configuration session.run(config)
![Page 31: Building Collaborative AR Experiences · Multi-User Experiences in ARKit 2 Recap User1 User2. Multi-User Experiences in ARKit 2 Recap User1 User2. ... First device acts as “host”](https://reader030.fdocuments.in/reader030/viewer/2022011916/5fd690c8154f054c1412d96b/html5/thumbnails/31.jpg)
// Collaborative Session
// Create world tracking configuration let config = ARWorldTrackingConfiguration()
// Enable collaborative session config.isCollaborationEnabled = true
// Run the configuration session.run(config)
![Page 32: Building Collaborative AR Experiences · Multi-User Experiences in ARKit 2 Recap User1 User2. Multi-User Experiences in ARKit 2 Recap User1 User2. ... First device acts as “host”](https://reader030.fdocuments.in/reader030/viewer/2022011916/5fd690c8154f054c1412d96b/html5/thumbnails/32.jpg)
// Collaborative Session
// Create world tracking configuration let config = ARWorldTrackingConfiguration()
// Enable collaborative session config.isCollaborationEnabled = true
// Run the configuration session.run(config)
![Page 33: Building Collaborative AR Experiences · Multi-User Experiences in ARKit 2 Recap User1 User2. Multi-User Experiences in ARKit 2 Recap User1 User2. ... First device acts as “host”](https://reader030.fdocuments.in/reader030/viewer/2022011916/5fd690c8154f054c1412d96b/html5/thumbnails/33.jpg)
// ARSession delegate function to output ARCollaborationData func session(_ session: ARSession, didOutputCollaborationData data: ARSession.CollaborationData) { // Transmit Data representation of the data to all other participants using MPC do { try self.mpcSession.send(data.data, toPeers: self.peerIds, with: .reliable) } catch { // Re-transmit the data if failed } }
// MPC delegate function when receiving collaboration data from other users func session(_ session: MCSession, didReceive data: Data, fromPeer peerID: MCPeerID) { // Pass the received data to ARSession self.arSession.update(data: ARSession.CollaborationData(data: data)) }
![Page 34: Building Collaborative AR Experiences · Multi-User Experiences in ARKit 2 Recap User1 User2. Multi-User Experiences in ARKit 2 Recap User1 User2. ... First device acts as “host”](https://reader030.fdocuments.in/reader030/viewer/2022011916/5fd690c8154f054c1412d96b/html5/thumbnails/34.jpg)
// ARSession delegate function to output ARCollaborationData func session(_ session: ARSession, didOutputCollaborationData data: ARSession.CollaborationData) { // Transmit Data representation of the data to all other participants using MPC do { try self.mpcSession.send(data.data, toPeers: self.peerIds, with: .reliable) } catch { // Re-transmit the data if failed } }
// MPC delegate function when receiving collaboration data from other users func session(_ session: MCSession, didReceive data: Data, fromPeer peerID: MCPeerID) { // Pass the received data to ARSession self.arSession.update(data: ARSession.CollaborationData(data: data)) }
![Page 35: Building Collaborative AR Experiences · Multi-User Experiences in ARKit 2 Recap User1 User2. Multi-User Experiences in ARKit 2 Recap User1 User2. ... First device acts as “host”](https://reader030.fdocuments.in/reader030/viewer/2022011916/5fd690c8154f054c1412d96b/html5/thumbnails/35.jpg)
// ARSession delegate function to output ARCollaborationData func session(_ session: ARSession, didOutputCollaborationData data: ARSession.CollaborationData) { // Transmit Data representation of the data to all other participants using MPC do { try self.mpcSession.send(data.data, toPeers: self.peerIds, with: .reliable) } catch { // Re-transmit the data if failed } }
// MPC delegate function when receiving collaboration data from other users func session(_ session: MCSession, didReceive data: Data, fromPeer peerID: MCPeerID) { // Pass the received data to ARSession self.arSession.update(data: ARSession.CollaborationData(data: data)) }
![Page 36: Building Collaborative AR Experiences · Multi-User Experiences in ARKit 2 Recap User1 User2. Multi-User Experiences in ARKit 2 Recap User1 User2. ... First device acts as “host”](https://reader030.fdocuments.in/reader030/viewer/2022011916/5fd690c8154f054c1412d96b/html5/thumbnails/36.jpg)
Collaborative Session Flow Start shared AR experience
User 1 Set up Network
Set up Network
ARSession Starts
Localize User 2 Map
Localize User 1 Map
Receive User 2
ARAnchor
Receive User 1
ARAnchor
Collaboration Data
User 2 ARSession Starts
Collaboration Data
![Page 37: Building Collaborative AR Experiences · Multi-User Experiences in ARKit 2 Recap User1 User2. Multi-User Experiences in ARKit 2 Recap User1 User2. ... First device acts as “host”](https://reader030.fdocuments.in/reader030/viewer/2022011916/5fd690c8154f054c1412d96b/html5/thumbnails/37.jpg)
Collaborative Session Flow Start shared AR experience
User 1 Set up Network
Set up Network
ARSession Starts
Localize User 2 Map
Localize User 1 Map
Receive User 2
ARAnchor
Receive User 1
ARAnchor
Collaboration Data
User 2 ARSession Starts
Collaboration Data
![Page 38: Building Collaborative AR Experiences · Multi-User Experiences in ARKit 2 Recap User1 User2. Multi-User Experiences in ARKit 2 Recap User1 User2. ... First device acts as “host”](https://reader030.fdocuments.in/reader030/viewer/2022011916/5fd690c8154f054c1412d96b/html5/thumbnails/38.jpg)
ARAnchor in Collaborative Session
ARAnchor • Synchronized lifetime • Session identifier • Subclass ARAnchors are not shared
Introducing RealityKit and Reality Composer WWDC 2019
![Page 39: Building Collaborative AR Experiences · Multi-User Experiences in ARKit 2 Recap User1 User2. Multi-User Experiences in ARKit 2 Recap User1 User2. ... First device acts as “host”](https://reader030.fdocuments.in/reader030/viewer/2022011916/5fd690c8154f054c1412d96b/html5/thumbnails/39.jpg)
// ARAnchor in collaborative session
// ARSession delegate function when an ARAnchor is added. func session(_ session: ARSession, didAdd anchors: [ARAnchor]) { for anchor in anchors { // Use session identifier to determine creator of the ARAnchor
if anchor.sessionIdentifier == session.identifier { // Self-placed ARanchor } else { // ARAnchor from another participant } } }
![Page 40: Building Collaborative AR Experiences · Multi-User Experiences in ARKit 2 Recap User1 User2. Multi-User Experiences in ARKit 2 Recap User1 User2. ... First device acts as “host”](https://reader030.fdocuments.in/reader030/viewer/2022011916/5fd690c8154f054c1412d96b/html5/thumbnails/40.jpg)
// ARAnchor in collaborative session
// ARSession delegate function when an ARAnchor is added. func session(_ session: ARSession, didAdd anchors: [ARAnchor]) { for anchor in anchors { // Use session identifier to determine creator of the ARAnchor
if anchor.sessionIdentifier == session.identifier { // Self-placed ARanchor } else { // ARAnchor from another participant } } }
![Page 41: Building Collaborative AR Experiences · Multi-User Experiences in ARKit 2 Recap User1 User2. Multi-User Experiences in ARKit 2 Recap User1 User2. ... First device acts as “host”](https://reader030.fdocuments.in/reader030/viewer/2022011916/5fd690c8154f054c1412d96b/html5/thumbnails/41.jpg)
// ARSession delegate function when an ARAnchor is removed. func session(_ session: ARSession, didRemove anchors: [ARAnchor]) { for anchor in anchors { // Use session identifier to determine creator of the ARAnchor if anchor.sessionIdentifier == session.identifier { // Self-placed ARanchor } else { // ARAnchor from another participant } } }
![Page 42: Building Collaborative AR Experiences · Multi-User Experiences in ARKit 2 Recap User1 User2. Multi-User Experiences in ARKit 2 Recap User1 User2. ... First device acts as “host”](https://reader030.fdocuments.in/reader030/viewer/2022011916/5fd690c8154f054c1412d96b/html5/thumbnails/42.jpg)
// ARSession delegate function when an ARAnchor is removed. func session(_ session: ARSession, didRemove anchors: [ARAnchor]) { for anchor in anchors { // Use session identifier to determine creator of the ARAnchor if anchor.sessionIdentifier == session.identifier { // Self-placed ARanchor } else { // ARAnchor from another participant } } }
![Page 43: Building Collaborative AR Experiences · Multi-User Experiences in ARKit 2 Recap User1 User2. Multi-User Experiences in ARKit 2 Recap User1 User2. ... First device acts as “host”](https://reader030.fdocuments.in/reader030/viewer/2022011916/5fd690c8154f054c1412d96b/html5/thumbnails/43.jpg)
ARParticipantAnchor
ARParticipantAnchor • Position of other user • Update at high frame rate • Created after localizing other user’s map
![Page 44: Building Collaborative AR Experiences · Multi-User Experiences in ARKit 2 Recap User1 User2. Multi-User Experiences in ARKit 2 Recap User1 User2. ... First device acts as “host”](https://reader030.fdocuments.in/reader030/viewer/2022011916/5fd690c8154f054c1412d96b/html5/thumbnails/44.jpg)
Practical Advice for Localizing Other Users
Shared AR experience starts after seeing same area
![Page 45: Building Collaborative AR Experiences · Multi-User Experiences in ARKit 2 Recap User1 User2. Multi-User Experiences in ARKit 2 Recap User1 User2. ... First device acts as “host”](https://reader030.fdocuments.in/reader030/viewer/2022011916/5fd690c8154f054c1412d96b/html5/thumbnails/45.jpg)
Practical Advice for Localizing Other Users
Shared AR experience starts after seeing same area
Approach other user to have same camera perspective
![Page 46: Building Collaborative AR Experiences · Multi-User Experiences in ARKit 2 Recap User1 User2. Multi-User Experiences in ARKit 2 Recap User1 User2. ... First device acts as “host”](https://reader030.fdocuments.in/reader030/viewer/2022011916/5fd690c8154f054c1412d96b/html5/thumbnails/46.jpg)
Practical Advice for Localizing Other Users
Shared AR experience starts after seeing same area
Approach other user to have same camera perspective
Stay in map-tracking status ARFrame.WorldMappingStatus.mapped
![Page 47: Building Collaborative AR Experiences · Multi-User Experiences in ARKit 2 Recap User1 User2. Multi-User Experiences in ARKit 2 Recap User1 User2. ... First device acts as “host”](https://reader030.fdocuments.in/reader030/viewer/2022011916/5fd690c8154f054c1412d96b/html5/thumbnails/47.jpg)
![Page 48: Building Collaborative AR Experiences · Multi-User Experiences in ARKit 2 Recap User1 User2. Multi-User Experiences in ARKit 2 Recap User1 User2. ... First device acts as “host”](https://reader030.fdocuments.in/reader030/viewer/2022011916/5fd690c8154f054c1412d96b/html5/thumbnails/48.jpg)
![Page 49: Building Collaborative AR Experiences · Multi-User Experiences in ARKit 2 Recap User1 User2. Multi-User Experiences in ARKit 2 Recap User1 User2. ... First device acts as “host”](https://reader030.fdocuments.in/reader030/viewer/2022011916/5fd690c8154f054c1412d96b/html5/thumbnails/49.jpg)
![Page 50: Building Collaborative AR Experiences · Multi-User Experiences in ARKit 2 Recap User1 User2. Multi-User Experiences in ARKit 2 Recap User1 User2. ... First device acts as “host”](https://reader030.fdocuments.in/reader030/viewer/2022011916/5fd690c8154f054c1412d96b/html5/thumbnails/50.jpg)
![Page 51: Building Collaborative AR Experiences · Multi-User Experiences in ARKit 2 Recap User1 User2. Multi-User Experiences in ARKit 2 Recap User1 User2. ... First device acts as “host”](https://reader030.fdocuments.in/reader030/viewer/2022011916/5fd690c8154f054c1412d96b/html5/thumbnails/51.jpg)
•ARAnchor Best Practice forMulti-User AR
![Page 52: Building Collaborative AR Experiences · Multi-User Experiences in ARKit 2 Recap User1 User2. Multi-User Experiences in ARKit 2 Recap User1 User2. ... First device acts as “host”](https://reader030.fdocuments.in/reader030/viewer/2022011916/5fd690c8154f054c1412d96b/html5/thumbnails/52.jpg)
Behind the Scenes — ARWorldMap
Internal tracking data • Map of sparse 3D landmarks • Camera poses
![Page 53: Building Collaborative AR Experiences · Multi-User Experiences in ARKit 2 Recap User1 User2. Multi-User Experiences in ARKit 2 Recap User1 User2. ... First device acts as “host”](https://reader030.fdocuments.in/reader030/viewer/2022011916/5fd690c8154f054c1412d96b/html5/thumbnails/53.jpg)
Behind the Scenes — ARWorldMap
Internal tracking data • Map of sparse 3D landmarks • Camera poses
![Page 54: Building Collaborative AR Experiences · Multi-User Experiences in ARKit 2 Recap User1 User2. Multi-User Experiences in ARKit 2 Recap User1 User2. ... First device acts as “host”](https://reader030.fdocuments.in/reader030/viewer/2022011916/5fd690c8154f054c1412d96b/html5/thumbnails/54.jpg)
Behind the Scenes — ARWorldMap
Internal tracking data • Map of sparse 3D landmarks • Camera poses
Landmarks are grouped based on camera view
![Page 55: Building Collaborative AR Experiences · Multi-User Experiences in ARKit 2 Recap User1 User2. Multi-User Experiences in ARKit 2 Recap User1 User2. ... First device acts as “host”](https://reader030.fdocuments.in/reader030/viewer/2022011916/5fd690c8154f054c1412d96b/html5/thumbnails/55.jpg)
Behind the Scenes — ARAnchor
ARAnchor • Global position against world
![Page 56: Building Collaborative AR Experiences · Multi-User Experiences in ARKit 2 Recap User1 User2. Multi-User Experiences in ARKit 2 Recap User1 User2. ... First device acts as “host”](https://reader030.fdocuments.in/reader030/viewer/2022011916/5fd690c8154f054c1412d96b/html5/thumbnails/56.jpg)
Behind the Scenes — ARAnchor
ARAnchor • Global position against world • Relative position against map
![Page 57: Building Collaborative AR Experiences · Multi-User Experiences in ARKit 2 Recap User1 User2. Multi-User Experiences in ARKit 2 Recap User1 User2. ... First device acts as “host”](https://reader030.fdocuments.in/reader030/viewer/2022011916/5fd690c8154f054c1412d96b/html5/thumbnails/57.jpg)
Best Practice for Using ARAnchor
Respond to anchor update delegate
![Page 58: Building Collaborative AR Experiences · Multi-User Experiences in ARKit 2 Recap User1 User2. Multi-User Experiences in ARKit 2 Recap User1 User2. ... First device acts as “host”](https://reader030.fdocuments.in/reader030/viewer/2022011916/5fd690c8154f054c1412d96b/html5/thumbnails/58.jpg)
Best Practice for Using ARAnchor
Respond to anchor update delegate
![Page 59: Building Collaborative AR Experiences · Multi-User Experiences in ARKit 2 Recap User1 User2. Multi-User Experiences in ARKit 2 Recap User1 User2. ... First device acts as “host”](https://reader030.fdocuments.in/reader030/viewer/2022011916/5fd690c8154f054c1412d96b/html5/thumbnails/59.jpg)
Best Practice for Using ARAnchor
Respond to anchor update delegate
Place 3D content near the ARAnchor
![Page 60: Building Collaborative AR Experiences · Multi-User Experiences in ARKit 2 Recap User1 User2. Multi-User Experiences in ARKit 2 Recap User1 User2. ... First device acts as “host”](https://reader030.fdocuments.in/reader030/viewer/2022011916/5fd690c8154f054c1412d96b/html5/thumbnails/60.jpg)
Best Practice for Using ARAnchor
Respond to anchor update delegate
Place 3D content near the ARAnchor
![Page 61: Building Collaborative AR Experiences · Multi-User Experiences in ARKit 2 Recap User1 User2. Multi-User Experiences in ARKit 2 Recap User1 User2. ... First device acts as “host”](https://reader030.fdocuments.in/reader030/viewer/2022011916/5fd690c8154f054c1412d96b/html5/thumbnails/61.jpg)
Best Practice for Using ARAnchor
Respond to anchor update delegate
Place 3D content near the ARAnchor
Multiple anchors for independent 3D content
![Page 62: Building Collaborative AR Experiences · Multi-User Experiences in ARKit 2 Recap User1 User2. Multi-User Experiences in ARKit 2 Recap User1 User2. ... First device acts as “host”](https://reader030.fdocuments.in/reader030/viewer/2022011916/5fd690c8154f054c1412d96b/html5/thumbnails/62.jpg)
Best Practice for Using ARAnchor
Respond to anchor update delegate
Place 3D content near the ARAnchor
Multiple anchors for independent 3D content
![Page 63: Building Collaborative AR Experiences · Multi-User Experiences in ARKit 2 Recap User1 User2. Multi-User Experiences in ARKit 2 Recap User1 User2. ... First device acts as “host”](https://reader030.fdocuments.in/reader030/viewer/2022011916/5fd690c8154f054c1412d96b/html5/thumbnails/63.jpg)
•SwiftStrike — A New Multiplayer AR Experience
David Paschich, Tools Foundation
![Page 64: Building Collaborative AR Experiences · Multi-User Experiences in ARKit 2 Recap User1 User2. Multi-User Experiences in ARKit 2 Recap User1 User2. ... First device acts as “host”](https://reader030.fdocuments.in/reader030/viewer/2022011916/5fd690c8154f054c1412d96b/html5/thumbnails/64.jpg)
Inside SwiftShot: Creating an AR Game WWDC 2018
SwiftStrike — A New Multiplayer AR Experience
Inspired by SwiftShot
All-new experience for 2019
Built with RealityKit and ARKit 3
SwiftStrike Tabletop available now
![Page 65: Building Collaborative AR Experiences · Multi-User Experiences in ARKit 2 Recap User1 User2. Multi-User Experiences in ARKit 2 Recap User1 User2. ... First device acts as “host”](https://reader030.fdocuments.in/reader030/viewer/2022011916/5fd690c8154f054c1412d96b/html5/thumbnails/65.jpg)
•RealityKit networking •Physics simulation •SwiftStrike game design •
![Page 66: Building Collaborative AR Experiences · Multi-User Experiences in ARKit 2 Recap User1 User2. Multi-User Experiences in ARKit 2 Recap User1 User2. ... First device acts as “host”](https://reader030.fdocuments.in/reader030/viewer/2022011916/5fd690c8154f054c1412d96b/html5/thumbnails/66.jpg)
RealityKit Networking
Based on entity-component architecture
All state synchronized, including physics!
Custom components for app/game logic
Uses MultipeerConnectivity as network layer
Create network session, hand to ARView
Building Apps with RealityKit Wednesday, 10:00
![Page 67: Building Collaborative AR Experiences · Multi-User Experiences in ARKit 2 Recap User1 User2. Multi-User Experiences in ARKit 2 Recap User1 User2. ... First device acts as “host”](https://reader030.fdocuments.in/reader030/viewer/2022011916/5fd690c8154f054c1412d96b/html5/thumbnails/67.jpg)
Different Roles
First device acts as “host”
Controls game state, physics simulation
Other devices participate
![Page 68: Building Collaborative AR Experiences · Multi-User Experiences in ARKit 2 Recap User1 User2. Multi-User Experiences in ARKit 2 Recap User1 User2. ... First device acts as “host”](https://reader030.fdocuments.in/reader030/viewer/2022011916/5fd690c8154f054c1412d96b/html5/thumbnails/68.jpg)
Custom Components in RealityKit
Define your own components to store application state
Register components before instantiating ARView
Implement Codable to enable synchronization
![Page 69: Building Collaborative AR Experiences · Multi-User Experiences in ARKit 2 Recap User1 User2. Multi-User Experiences in ARKit 2 Recap User1 User2. ... First device acts as “host”](https://reader030.fdocuments.in/reader030/viewer/2022011916/5fd690c8154f054c1412d96b/html5/thumbnails/69.jpg)
Use Case — Starting the Game
Match object tracks whether enough players are ready to start the game
State maintained on host, synced to clients
Component maintains full log
![Page 70: Building Collaborative AR Experiences · Multi-User Experiences in ARKit 2 Recap User1 User2. Multi-User Experiences in ARKit 2 Recap User1 User2. ... First device acts as “host”](https://reader030.fdocuments.in/reader030/viewer/2022011916/5fd690c8154f054c1412d96b/html5/thumbnails/70.jpg)
![Page 71: Building Collaborative AR Experiences · Multi-User Experiences in ARKit 2 Recap User1 User2. Multi-User Experiences in ARKit 2 Recap User1 User2. ... First device acts as “host”](https://reader030.fdocuments.in/reader030/viewer/2022011916/5fd690c8154f054c1412d96b/html5/thumbnails/71.jpg)
![Page 72: Building Collaborative AR Experiences · Multi-User Experiences in ARKit 2 Recap User1 User2. Multi-User Experiences in ARKit 2 Recap User1 User2. ... First device acts as “host”](https://reader030.fdocuments.in/reader030/viewer/2022011916/5fd690c8154f054c1412d96b/html5/thumbnails/72.jpg)
// Custom component for game start
struct MatchStateComponent: Component, Codable { struct Transition: Codable { var date: Date var state: MatchOutput } var transitions = [Transition]() }
// Registering the component, in application(_:didFinishLaunchingWithOptions:) MatchStateComponent.self.registerComponent()
// On client class MatchObserver { var matchOutputEvents: AnyPublisher<MatchOutput, Never> }
![Page 73: Building Collaborative AR Experiences · Multi-User Experiences in ARKit 2 Recap User1 User2. Multi-User Experiences in ARKit 2 Recap User1 User2. ... First device acts as “host”](https://reader030.fdocuments.in/reader030/viewer/2022011916/5fd690c8154f054c1412d96b/html5/thumbnails/73.jpg)
// Custom component for game start
struct MatchStateComponent: Component, Codable { struct Transition: Codable { var date: Date var state: MatchOutput } var transitions = [Transition]() }
// Registering the component, in application(_:didFinishLaunchingWithOptions:) MatchStateComponent.self.registerComponent()
// On client class MatchObserver { var matchOutputEvents: AnyPublisher<MatchOutput, Never> }
![Page 74: Building Collaborative AR Experiences · Multi-User Experiences in ARKit 2 Recap User1 User2. Multi-User Experiences in ARKit 2 Recap User1 User2. ... First device acts as “host”](https://reader030.fdocuments.in/reader030/viewer/2022011916/5fd690c8154f054c1412d96b/html5/thumbnails/74.jpg)
// Custom component for game start
struct MatchStateComponent: Component, Codable { struct Transition: Codable { var date: Date var state: MatchOutput } var transitions = [Transition]() }
// Registering the component, in application(_:didFinishLaunchingWithOptions:) MatchStateComponent.self.registerComponent()
// On client class MatchObserver { var matchOutputEvents: AnyPublisher<MatchOutput, Never> }
![Page 75: Building Collaborative AR Experiences · Multi-User Experiences in ARKit 2 Recap User1 User2. Multi-User Experiences in ARKit 2 Recap User1 User2. ... First device acts as “host”](https://reader030.fdocuments.in/reader030/viewer/2022011916/5fd690c8154f054c1412d96b/html5/thumbnails/75.jpg)
// Custom component for game start
struct MatchStateComponent: Component, Codable { struct Transition: Codable { var date: Date var state: MatchOutput } var transitions = [Transition]() }
// Registering the component, in application(_:didFinishLaunchingWithOptions:) MatchStateComponent.self.registerComponent()
// On client class MatchObserver { var matchOutputEvents: AnyPublisher<MatchOutput, Never> }
![Page 76: Building Collaborative AR Experiences · Multi-User Experiences in ARKit 2 Recap User1 User2. Multi-User Experiences in ARKit 2 Recap User1 User2. ... First device acts as “host”](https://reader030.fdocuments.in/reader030/viewer/2022011916/5fd690c8154f054c1412d96b/html5/thumbnails/76.jpg)
Ray CameraInfo GameVelocity HitCatapult SlingData GrabInfo LeverMove PhysicsSyncData CollisionSoundData PhysicsSyncSceneData PhysicsSyncSeneDataDelegate
GamePhysicsSmoothComponent SlingshotComponent GameBoardDescription GameBoardLocation LevelConfiguration LevelInfo LeverMove BitStreamEncodable BitStreamDecodable WriteableBitStream ReadableBitStream
![Page 77: Building Collaborative AR Experiences · Multi-User Experiences in ARKit 2 Recap User1 User2. Multi-User Experiences in ARKit 2 Recap User1 User2. ... First device acts as “host”](https://reader030.fdocuments.in/reader030/viewer/2022011916/5fd690c8154f054c1412d96b/html5/thumbnails/77.jpg)
Ray CameraInfo GameVelocity HitCatapult SlingData GrabInfo LeverMove PhysicsSyncData CollisionSoundData PhysicsSyncSceneData PhysicsSyncSeneDataDelegate
GamePhysicsSmoothComponent SlingshotComponent GameBoardDescription GameBoardLocation LevelConfiguration LevelInfo LeverMove BitStreamEncodable BitStreamDecodable WriteableBitStream ReadableBitStream
![Page 78: Building Collaborative AR Experiences · Multi-User Experiences in ARKit 2 Recap User1 User2. Multi-User Experiences in ARKit 2 Recap User1 User2. ... First device acts as “host”](https://reader030.fdocuments.in/reader030/viewer/2022011916/5fd690c8154f054c1412d96b/html5/thumbnails/78.jpg)
Ray CameraInfo GameVelocity HitCatapult SlingData GrabInfo LeverMove PhysicsSyncData CollisionSoundData PhysicsSyncSceneData PhysicsSyncSeneDataDelegate
GamePhysicsSmoothComponent SlingshotComponent GameBoardDescription GameBoardLocation LevelConfiguration LevelInfo LeverMove BitStreamEncodable BitStreamDecodable WriteableBitStream ReadableBitStream
![Page 79: Building Collaborative AR Experiences · Multi-User Experiences in ARKit 2 Recap User1 User2. Multi-User Experiences in ARKit 2 Recap User1 User2. ... First device acts as “host”](https://reader030.fdocuments.in/reader030/viewer/2022011916/5fd690c8154f054c1412d96b/html5/thumbnails/79.jpg)
Physics
Physics state synchronization handled by RealityKit
Configure physics properties via components • Rigid bodies • Collision masks • Mass, friction, restitution (“bounciness”)
Host device owns simulation
Client devices interpolate between updates
![Page 80: Building Collaborative AR Experiences · Multi-User Experiences in ARKit 2 Recap User1 User2. Multi-User Experiences in ARKit 2 Recap User1 User2. ... First device acts as “host”](https://reader030.fdocuments.in/reader030/viewer/2022011916/5fd690c8154f054c1412d96b/html5/thumbnails/80.jpg)
Bowling Pin
![Page 81: Building Collaborative AR Experiences · Multi-User Experiences in ARKit 2 Recap User1 User2. Multi-User Experiences in ARKit 2 Recap User1 User2. ... First device acts as “host”](https://reader030.fdocuments.in/reader030/viewer/2022011916/5fd690c8154f054c1412d96b/html5/thumbnails/81.jpg)
Bowling Pin
![Page 82: Building Collaborative AR Experiences · Multi-User Experiences in ARKit 2 Recap User1 User2. Multi-User Experiences in ARKit 2 Recap User1 User2. ... First device acts as “host”](https://reader030.fdocuments.in/reader030/viewer/2022011916/5fd690c8154f054c1412d96b/html5/thumbnails/82.jpg)
Bowling Pin
![Page 83: Building Collaborative AR Experiences · Multi-User Experiences in ARKit 2 Recap User1 User2. Multi-User Experiences in ARKit 2 Recap User1 User2. ... First device acts as “host”](https://reader030.fdocuments.in/reader030/viewer/2022011916/5fd690c8154f054c1412d96b/html5/thumbnails/83.jpg)
Bowling Pin
![Page 84: Building Collaborative AR Experiences · Multi-User Experiences in ARKit 2 Recap User1 User2. Multi-User Experiences in ARKit 2 Recap User1 User2. ... First device acts as “host”](https://reader030.fdocuments.in/reader030/viewer/2022011916/5fd690c8154f054c1412d96b/html5/thumbnails/84.jpg)
SwiftStrike Game Design
Designing for People Occlusion
On-site experience
Control mechanism
![Page 85: Building Collaborative AR Experiences · Multi-User Experiences in ARKit 2 Recap User1 User2. Multi-User Experiences in ARKit 2 Recap User1 User2. ... First device acts as “host”](https://reader030.fdocuments.in/reader030/viewer/2022011916/5fd690c8154f054c1412d96b/html5/thumbnails/85.jpg)
People Occlusion
SwiftStrike designed with this in mind
Opens up new experiences
![Page 86: Building Collaborative AR Experiences · Multi-User Experiences in ARKit 2 Recap User1 User2. Multi-User Experiences in ARKit 2 Recap User1 User2. ... First device acts as “host”](https://reader030.fdocuments.in/reader030/viewer/2022011916/5fd690c8154f054c1412d96b/html5/thumbnails/86.jpg)
SwiftStrike On-Site Experience
Full scale experience
Floor features designed for quick localization
![Page 87: Building Collaborative AR Experiences · Multi-User Experiences in ARKit 2 Recap User1 User2. Multi-User Experiences in ARKit 2 Recap User1 User2. ... First device acts as “host”](https://reader030.fdocuments.in/reader030/viewer/2022011916/5fd690c8154f054c1412d96b/html5/thumbnails/87.jpg)
SwiftStrike Control Mechanism
Device is your controller
Faster movement means larger push
Physics body in the scene • Ball bounces off people • Pins and ball bounce off each other • Pins don’t bounce off people
![Page 88: Building Collaborative AR Experiences · Multi-User Experiences in ARKit 2 Recap User1 User2. Multi-User Experiences in ARKit 2 Recap User1 User2. ... First device acts as “host”](https://reader030.fdocuments.in/reader030/viewer/2022011916/5fd690c8154f054c1412d96b/html5/thumbnails/88.jpg)
Ownership of Player Paddle
![Page 89: Building Collaborative AR Experiences · Multi-User Experiences in ARKit 2 Recap User1 User2. Multi-User Experiences in ARKit 2 Recap User1 User2. ... First device acts as “host”](https://reader030.fdocuments.in/reader030/viewer/2022011916/5fd690c8154f054c1412d96b/html5/thumbnails/89.jpg)
Ownership of Player Paddle
HostAnchorEntity
![Page 90: Building Collaborative AR Experiences · Multi-User Experiences in ARKit 2 Recap User1 User2. Multi-User Experiences in ARKit 2 Recap User1 User2. ... First device acts as “host”](https://reader030.fdocuments.in/reader030/viewer/2022011916/5fd690c8154f054c1412d96b/html5/thumbnails/90.jpg)
Ownership of Player Paddle
Client PlayerLocationEntity
HostAnchorEntity
![Page 91: Building Collaborative AR Experiences · Multi-User Experiences in ARKit 2 Recap User1 User2. Multi-User Experiences in ARKit 2 Recap User1 User2. ... First device acts as “host”](https://reader030.fdocuments.in/reader030/viewer/2022011916/5fd690c8154f054c1412d96b/html5/thumbnails/91.jpg)
Ownership of Player Paddle
Client PlayerLocationEntity
HostAnchorEntity
![Page 92: Building Collaborative AR Experiences · Multi-User Experiences in ARKit 2 Recap User1 User2. Multi-User Experiences in ARKit 2 Recap User1 User2. ... First device acts as “host”](https://reader030.fdocuments.in/reader030/viewer/2022011916/5fd690c8154f054c1412d96b/html5/thumbnails/92.jpg)
Ownership of Player Paddle
Client PlayerLocationEntity
HostAnchorEntity
HostPaddleEntity
![Page 93: Building Collaborative AR Experiences · Multi-User Experiences in ARKit 2 Recap User1 User2. Multi-User Experiences in ARKit 2 Recap User1 User2. ... First device acts as “host”](https://reader030.fdocuments.in/reader030/viewer/2022011916/5fd690c8154f054c1412d96b/html5/thumbnails/93.jpg)
Ownership of Player Paddle
Client PlayerLocationEntity
HostAnchorEntity
HostPaddleEntity
![Page 94: Building Collaborative AR Experiences · Multi-User Experiences in ARKit 2 Recap User1 User2. Multi-User Experiences in ARKit 2 Recap User1 User2. ... First device acts as “host”](https://reader030.fdocuments.in/reader030/viewer/2022011916/5fd690c8154f054c1412d96b/html5/thumbnails/94.jpg)
![Page 95: Building Collaborative AR Experiences · Multi-User Experiences in ARKit 2 Recap User1 User2. Multi-User Experiences in ARKit 2 Recap User1 User2. ... First device acts as “host”](https://reader030.fdocuments.in/reader030/viewer/2022011916/5fd690c8154f054c1412d96b/html5/thumbnails/95.jpg)
![Page 96: Building Collaborative AR Experiences · Multi-User Experiences in ARKit 2 Recap User1 User2. Multi-User Experiences in ARKit 2 Recap User1 User2. ... First device acts as “host”](https://reader030.fdocuments.in/reader030/viewer/2022011916/5fd690c8154f054c1412d96b/html5/thumbnails/96.jpg)
![Page 97: Building Collaborative AR Experiences · Multi-User Experiences in ARKit 2 Recap User1 User2. Multi-User Experiences in ARKit 2 Recap User1 User2. ... First device acts as “host”](https://reader030.fdocuments.in/reader030/viewer/2022011916/5fd690c8154f054c1412d96b/html5/thumbnails/97.jpg)
![Page 98: Building Collaborative AR Experiences · Multi-User Experiences in ARKit 2 Recap User1 User2. Multi-User Experiences in ARKit 2 Recap User1 User2. ... First device acts as “host”](https://reader030.fdocuments.in/reader030/viewer/2022011916/5fd690c8154f054c1412d96b/html5/thumbnails/98.jpg)
Summary
Collaborative Sessions enable new experiences
Use ARAnchors to attach content to the real world
SwiftStrike Tabletop available now!
Building AR Experiences with Reality Composer WWDC 2019
![Page 99: Building Collaborative AR Experiences · Multi-User Experiences in ARKit 2 Recap User1 User2. Multi-User Experiences in ARKit 2 Recap User1 User2. ... First device acts as “host”](https://reader030.fdocuments.in/reader030/viewer/2022011916/5fd690c8154f054c1412d96b/html5/thumbnails/99.jpg)
More Informationdeveloper.apple.com/wwdc19/610
ARKit Lab Thursday, 3:00
RealityKit and Reality Composer Lab Thursday, 3:00
![Page 100: Building Collaborative AR Experiences · Multi-User Experiences in ARKit 2 Recap User1 User2. Multi-User Experiences in ARKit 2 Recap User1 User2. ... First device acts as “host”](https://reader030.fdocuments.in/reader030/viewer/2022011916/5fd690c8154f054c1412d96b/html5/thumbnails/100.jpg)