Path Tracing - Q2VKPT · •Real-time path tracing is possible (in the near future) •Transition...

46
Christoph Schied @c_schied q2vkpt Quake 2 Vulkan Path Tracer http://brechpunkt.de/q2vkpt

Transcript of Path Tracing - Q2VKPT · •Real-time path tracing is possible (in the near future) •Transition...

Page 1: Path Tracing - Q2VKPT · •Real-time path tracing is possible (in the near future) •Transition difficult ... id Software @c_schied Q2VKPT uses a texture addon collected by Tosher

Christoph Schied

@c_schied

q2vkptQu

ake 2

Vu

lkan

Path Tracer

http://brechpunkt.de/q2vkpt

Page 2: Path Tracing - Q2VKPT · •Real-time path tracing is possible (in the near future) •Transition difficult ... id Software @c_schied Q2VKPT uses a texture addon collected by Tosher

Path Tracing

2

Page 3: Path Tracing - Q2VKPT · •Real-time path tracing is possible (in the near future) •Transition difficult ... id Software @c_schied Q2VKPT uses a texture addon collected by Tosher

Path Tracing

𝐿 ≈1

𝑛

𝑖=1

𝑛

Number of samples, n=1 in q2vkpt

3

Page 4: Path Tracing - Q2VKPT · •Real-time path tracing is possible (in the near future) •Transition difficult ... id Software @c_schied Q2VKPT uses a texture addon collected by Tosher

Main challenges

• Better Sampling → less noise

• Denoising

4

Page 5: Path Tracing - Q2VKPT · •Real-time path tracing is possible (in the near future) •Transition difficult ... id Software @c_schied Q2VKPT uses a texture addon collected by Tosher

Q2VKPT• Research prototype to evaluate current

state of real-time path tracing

• Open sourcehttps://github.com/cschied/q2vkpt

• Entirely raytraced

• Real-time path tracing(one indirect bounce)

• C99, Vulkan, GLSL, RTX5

Page 6: Path Tracing - Q2VKPT · •Real-time path tracing is possible (in the near future) •Transition difficult ... id Software @c_schied Q2VKPT uses a texture addon collected by Tosher

video

https://www.youtube.com/watch?v=vrq1T93uLag

Page 7: Path Tracing - Q2VKPT · •Real-time path tracing is possible (in the near future) •Transition difficult ... id Software @c_schied Q2VKPT uses a texture addon collected by Tosher

2560x1440, RTX2080 Ti

Page 8: Path Tracing - Q2VKPT · •Real-time path tracing is possible (in the near future) •Transition difficult ... id Software @c_schied Q2VKPT uses a texture addon collected by Tosher

Denoising

Page 9: Path Tracing - Q2VKPT · •Real-time path tracing is possible (in the near future) •Transition difficult ... id Software @c_schied Q2VKPT uses a texture addon collected by Tosher
Page 10: Path Tracing - Q2VKPT · •Real-time path tracing is possible (in the near future) •Transition difficult ... id Software @c_schied Q2VKPT uses a texture addon collected by Tosher

Denoised result

Page 11: Path Tracing - Q2VKPT · •Real-time path tracing is possible (in the near future) •Transition difficult ... id Software @c_schied Q2VKPT uses a texture addon collected by Tosher

Path tracer output(1spp)

Page 12: Path Tracing - Q2VKPT · •Real-time path tracing is possible (in the near future) •Transition difficult ... id Software @c_schied Q2VKPT uses a texture addon collected by Tosher

Denoising (A-SVGF)

12

Page 13: Path Tracing - Q2VKPT · •Real-time path tracing is possible (in the near future) •Transition difficult ... id Software @c_schied Q2VKPT uses a texture addon collected by Tosher

Denoising (A-SVGF)

13

Page 14: Path Tracing - Q2VKPT · •Real-time path tracing is possible (in the near future) •Transition difficult ... id Software @c_schied Q2VKPT uses a texture addon collected by Tosher

Main concepts of SVGF

Filter hierarchically, starting small

• Estimate temporal stability aftereach filter iteration

→ Strong blur more likely in early iterations

Analyze input over time

• Temporally unstable → blur more

• Temporally stable → blur less

14

Page 15: Path Tracing - Q2VKPT · •Real-time path tracing is possible (in the near future) •Transition difficult ... id Software @c_schied Q2VKPT uses a texture addon collected by Tosher

SVGF

Path TracerRemove Textures

Denoising Filter

Reapply Textures

Temporal Antialiasing

TemporalAccumulation

VarianceEstimation

À-trous waveletfilter

15

Page 16: Path Tracing - Q2VKPT · •Real-time path tracing is possible (in the near future) •Transition difficult ... id Software @c_schied Q2VKPT uses a texture addon collected by Tosher

Edge-avoiding À-trous Wavelets

In q2vkpt:• 3x3 box kernel• 5 iterations

16

Page 17: Path Tracing - Q2VKPT · •Real-time path tracing is possible (in the near future) •Transition difficult ... id Software @c_schied Q2VKPT uses a texture addon collected by Tosher

17

https://www.youtube.com/watch?v=Fv-jStEsCpE&t=48s

Page 18: Path Tracing - Q2VKPT · •Real-time path tracing is possible (in the near future) •Transition difficult ... id Software @c_schied Q2VKPT uses a texture addon collected by Tosher

SVGF

Path TracerRemove Textures

Reconstruction Filter

Reapply Textures

Temporal Antialiasing

TemporalAccumulation

VarianceEstimation

À-trous waveletfilter

18

Page 19: Path Tracing - Q2VKPT · •Real-time path tracing is possible (in the near future) •Transition difficult ... id Software @c_schied Q2VKPT uses a texture addon collected by Tosher

Denoising (A-SVGF)

19

Page 20: Path Tracing - Q2VKPT · •Real-time path tracing is possible (in the near future) •Transition difficult ... id Software @c_schied Q2VKPT uses a texture addon collected by Tosher

Screen-space Reprojection

Ƹ𝑐𝑖 𝑥 = 𝛼 ⋅ 𝑐𝑖 𝑥 + 1 − 𝛼 ⋅ Ƹ𝑐𝑖−1(𝑥)

Current frame 𝑐𝑖 Previous filtered frame Ƹ𝑐𝑖−1

𝑥

20

𝑥

Page 21: Path Tracing - Q2VKPT · •Real-time path tracing is possible (in the near future) •Transition difficult ... id Software @c_schied Q2VKPT uses a texture addon collected by Tosher

Adaptive Temporal Filtering

• Set 𝛼 according to changes of the shading function• Moving shadows, glossy highlights, flickering light sources, …

• Make 𝛼 per-pixel weight for local adaptivity

• Need information about changes of shading (temporal gradient)

21

Ƹ𝑐𝑖 𝑥 = 𝛼 ⋅ 𝑐𝑖 𝑥 + 1 − 𝛼 ⋅ Ƹ𝑐𝑖−1(𝑥)

Page 22: Path Tracing - Q2VKPT · •Real-time path tracing is possible (in the near future) •Transition difficult ... id Software @c_schied Q2VKPT uses a texture addon collected by Tosher

Path tracer output1 sample per pixel

22

Page 23: Path Tracing - Q2VKPT · •Real-time path tracing is possible (in the near future) •Transition difficult ... id Software @c_schied Q2VKPT uses a texture addon collected by Tosher

23

Difference of luminancegreen positive

red negative

Page 24: Path Tracing - Q2VKPT · •Real-time path tracing is possible (in the near future) •Transition difficult ... id Software @c_schied Q2VKPT uses a texture addon collected by Tosher

Path tracer output1 sample per pixel

24

Page 25: Path Tracing - Q2VKPT · •Real-time path tracing is possible (in the near future) •Transition difficult ... id Software @c_schied Q2VKPT uses a texture addon collected by Tosher

25

Difference of luminancegreen positive

red negative

Page 26: Path Tracing - Q2VKPT · •Real-time path tracing is possible (in the near future) •Transition difficult ... id Software @c_schied Q2VKPT uses a texture addon collected by Tosher

Path tracer output1 sample per pixel

(correlated samples)

26

Page 27: Path Tracing - Q2VKPT · •Real-time path tracing is possible (in the near future) •Transition difficult ... id Software @c_schied Q2VKPT uses a texture addon collected by Tosher

27

Difference of luminance(correlated samples)

green positivered negative

Page 28: Path Tracing - Q2VKPT · •Real-time path tracing is possible (in the near future) •Transition difficult ... id Software @c_schied Q2VKPT uses a texture addon collected by Tosher

28

Adaptive temporal filter weight

• Sample and reconstruct temporal gradient

• Change α according to relative rate of change

Reconstructed temporal gradient

Adaptive filter weight α

0

1

Page 29: Path Tracing - Q2VKPT · •Real-time path tracing is possible (in the near future) •Transition difficult ... id Software @c_schied Q2VKPT uses a texture addon collected by Tosher

Video in quake with gradients?

https://www.youtube.com/watch?v=HP4Xwp0ETfc

Page 30: Path Tracing - Q2VKPT · •Real-time path tracing is possible (in the near future) •Transition difficult ... id Software @c_schied Q2VKPT uses a texture addon collected by Tosher

Path Tracer

Page 31: Path Tracing - Q2VKPT · •Real-time path tracing is possible (in the near future) •Transition difficult ... id Software @c_schied Q2VKPT uses a texture addon collected by Tosher

Treat each triangle of light meshes as individual area light

Page 32: Path Tracing - Q2VKPT · •Real-time path tracing is possible (in the near future) •Transition difficult ... id Software @c_schied Q2VKPT uses a texture addon collected by Tosher

Light selection / sampling

Tried:

• Light hierarchy

Issues:

• Speed

• Inconsistent quality under animation

32

Page 33: Path Tracing - Q2VKPT · •Real-time path tracing is possible (in the near future) •Transition difficult ... id Software @c_schied Q2VKPT uses a texture addon collected by Tosher

Light selection / sampling for static lights

MeshPotentially visible lights

Stochastically selected subsetCDF

Simplified BRDF and projected solid angle

33

Page 34: Path Tracing - Q2VKPT · •Real-time path tracing is possible (in the near future) •Transition difficult ... id Software @c_schied Q2VKPT uses a texture addon collected by Tosher

Light selection / sampling for dynamic lights

All dynamic lights

Stochastically selected subsetCDF

Simplified BRDF and projected solid angle

No culling

34

Page 35: Path Tracing - Q2VKPT · •Real-time path tracing is possible (in the near future) •Transition difficult ... id Software @c_schied Q2VKPT uses a texture addon collected by Tosher

Light selection

Static light Dynamic light

Sample bycontribution

35

Page 36: Path Tracing - Q2VKPT · •Real-time path tracing is possible (in the near future) •Transition difficult ... id Software @c_schied Q2VKPT uses a texture addon collected by Tosher

Path tracer

• One path per pixel• One indirect bounce• Two shadow rays

36

Page 37: Path Tracing - Q2VKPT · •Real-time path tracing is possible (in the near future) •Transition difficult ... id Software @c_schied Q2VKPT uses a texture addon collected by Tosher

water

Mirror reflection• No transmission• Demodulate indirect albedo• Fixed lower mip-level for

texture sampling

No explicit Environment Map sampling• Use indirect bounce• No illumination for indirect

bounce (missing raycast)

Constant Blinn-Phong BRDF for everything

Page 38: Path Tracing - Q2VKPT · •Real-time path tracing is possible (in the near future) •Transition difficult ... id Software @c_schied Q2VKPT uses a texture addon collected by Tosher

Sampling Pattern

Blue noise dither mask

Page 39: Path Tracing - Q2VKPT · •Real-time path tracing is possible (in the near future) •Transition difficult ... id Software @c_schied Q2VKPT uses a texture addon collected by Tosher

Source: http://momentsingraphics.de/?p=127

blur

White noise

Page 40: Path Tracing - Q2VKPT · •Real-time path tracing is possible (in the near future) •Transition difficult ... id Software @c_schied Q2VKPT uses a texture addon collected by Tosher

blur

Blue noise

Source: http://momentsingraphics.de/?p=127

Page 41: Path Tracing - Q2VKPT · •Real-time path tracing is possible (in the near future) •Transition difficult ... id Software @c_schied Q2VKPT uses a texture addon collected by Tosher

Magnitude of Fourier Transform

White noise Blue noise

Source: http://momentsingraphics.de/?p=127

Removed bylowpass filter

Removed bylowpass filter

Residual noise Residual noise

Page 42: Path Tracing - Q2VKPT · •Real-time path tracing is possible (in the near future) •Transition difficult ... id Software @c_schied Q2VKPT uses a texture addon collected by Tosher

Implementation details

Page 43: Path Tracing - Q2VKPT · •Real-time path tracing is possible (in the near future) •Transition difficult ... id Software @c_schied Q2VKPT uses a texture addon collected by Tosher

Acceleration structures

Top-Level

Bottom-LevelStatic geometry

Bottom-LevelDynamic geometry

Built once on map-load Rebuilt from scratch per frame

43

Page 44: Path Tracing - Q2VKPT · •Real-time path tracing is possible (in the near future) •Transition difficult ... id Software @c_schied Q2VKPT uses a texture addon collected by Tosher

Forward / Backward projection

• Required for Adaptive Temporal Filtering

• Visibility buffer for forward projection

• Map instances between frames

Visibility buffer

44

Page 45: Path Tracing - Q2VKPT · •Real-time path tracing is possible (in the near future) •Transition difficult ... id Software @c_schied Q2VKPT uses a texture addon collected by Tosher

Conclusion

• Real-time path tracing is possible (in the near future)

• Transition difficult• Random access to everything

• Tweaking of assets

• Need more research specifically tailored towards real-time rendering• Fast and robust importance sampling

• Denoising

45

Page 46: Path Tracing - Q2VKPT · •Real-time path tracing is possible (in the near future) •Transition difficult ... id Software @c_schied Q2VKPT uses a texture addon collected by Tosher

Thanks!

http://brechpunkt.de/q2vkpthttps://github.com/cschied/q2vkpt

[email protected]

Addis DittebrandtAlisa JungAnton KaplanyanChristoph PetersFlorian ReiboldJohannes HanikaStephan BergmannTobias Zirr

NVIDIAid Software

@c_schied

Q2VKPT uses a texture addoncollected by Tosher includingoriginal work by D Scott Boyce(@scobotech), released underCreative Commons Attribution-NonCommercial-ShareAlike 2.0