09 towardBlurryRasterizerMunkberg BPS2011

Toward a Blurry Rasterizer
Jacob Munkberg
Intel Corporation
Beyond Programmable Shading Course, ACM SIGGRAPH 2011
Goals
Realistic camera model
Finite aperture: Depth of field Slow shutter: Motion blur
Ease of use
Set camera parameters and
per-vertex motion vectors
High quality & low cost

This Talk
Covers:
Stochastic rasterization for motion blur & depth of field
Not covering:
Post-processing techniques [Potmesil81, Max85, Shinya93, Sousa2008, ...] Layered depth image approaches [Lee2009, Lee2010, ...] Blurry ray tracing [Cook84, Walter2006, Wald2007, Hachisuka2008,
Overbeck2009, Hou2010, Grnshlo2011, ...]
Analytical visibility [Korein83, Sung2002, Gribel2010, Gribel2011, ...] Reconstruction & filtering [Egan2009, Soler2009, Parker2010, Lehtinen2011, ...]
Motivation
Depth of Field
Direct viewers attention Match footage with CGI
Motion Blur
Enhance effect of motion Reduce temporal aliasing
Temporal Aliasing
Motion Blur Rendering

Pixel color is an integral over (x,y,t)
y
pixel
y x
t
Depth of Field Rendering

Pixel color is an integral over (x,y,u,v)
object in focus object out of focus
pixel
image plane (x,y)
lens (u,v)
focus plane
Depth of Field and Motion Blur

Pixel color is an integral over (x,y,u,v,t)
object in focus object out of focus
pixel t=0 image plane (x,y) lens (u,v) focus plane t=1
Sampling
Approximating the visibility integral
Approximate Integral
Accumulation buffering
Render scene at many fixed
times (and lens positions)
[Korein83, Haeberli90]
uniform
Stochastic sampling [Cook86]

Sample pixel at non-uniformly
spaced locations in (x,y,t,u,v)
Aliasing is replaced by noise
stochastic
Interleaved Sampling [Keller2001]

N fixed sample times
Position triangle and
sample sparsely in (x,y)
t=0
Improvement over acc.

buffering as (x,y) pos vary from tile to tile
t=0.3
t=0.6
5D: use (ui,vi,ti) tuples

[Fatahalian2009]
t=1.0
Sampling for Motion Blur (3D)

A set of 64 fixed times
is sufficient to avoid banding artifacts Many samples per pixel needed to reduce noise
4 spp
8 spp
16 spp
32 spp
256 spp
Sampling for DOF (4D)

Need many unique lens positions
InterleaveUV (64 lens pos)
InterleaveUV (256 lens pos)
64 random spp
Sampling for MB+DOF (5D)
Need large number of unique (ui,vi,ti) tuples To appear in ACM Harder to find good sample distributions
Images courtesy of Laine et al.[2011] taken from Assassins Creed by Ubisoft
InterleaveUVT 4 spp
Sobol sampling 4 spp
InterleaveUVT 16 spp
Sobol sampling 16 spp
Visibility
The rasterizer stage
Recap: Standard Rasterization
The Graphics Pipeline

Input Assembler Vertex Shader Hull Shader Tessellator Domain Shader Rasterizer Pixel Shader Output Merger
Memory System (buffers, textures)
Rasterization of Static Triangle

Determine which pixels a
triangle covers
1.Project triangle on screen 2.Traverse covered area 3.Test samples
"
! !"#$$%&!'("$
Test Samples
Tiled Traversal
Test tile of pixels for overlap
Reject samples in tiles completely outside triangle Enables coarse occlusion culling, improves coherence
Blurry Rasterization
Blurry Rasterization
Moving triangles
q2 t=0.0 r0 q0 r2 t1=0.37 q1 t=1.0
r1
Triangles out of focus
The Modified Graphics Pipeline

Input Assembler
Memory System (buffers, textures)
Output two vertex positions
Vertex Shader Hull Shader Tessellator Domain Shader
Sample in (x,y,u,v,t)
Rasterizer Pixel Shader Output Merger
Clipping Moving Triangles

Clipping moving primitives is very difficult
Vertices intersect the frustum at different times
t=1 t=1 t=0.8 near plane t=0.6 z=0 t=0 t=0.7
Screen-Space Bounds
Bound full motion and all lens positions Handle moving triangles intersecting z=0 with care
t=0 t=1 t=1
image plane z=0 t=0
DOF BBox
Spatio-Temporal Visibility
Moving triangle may cover large
region in (x,y,t) More samples per pixel needed for low noise levels We need to quickly cull samples in both space and time
" "
! # #
!
Screen Space Convex Hull

Reduce spatial bounds [McGuire2010]
A tile is only visited once, enabling
multisampling & coarse Z culling No temporal culling
! # ! # " "
Interval
Use N time intervals
[Cook90, Fatahalian2009]
" "
Bound triangles movement in

each interval and test all samples in (x,y,t) box
! # ! #
Interleaved Rasterization
Use N fixed sample times
[Keller2001, Fatahalian2009]
"
Position triangle at each time Rasterize coarsely in (x,y) Restricted sampling patterns
! #
Tile-Based Traversal
Tile-test reduces samples to test
Trivially rejects tile or returns
temporal bounds for overlap
[Laine2011, Munkberg2011b]
! # "
Preserves GPU traversal order
!
5D Rasterization
Interval [Cook90, Fatahalian2009]
Many strata in (u,v,t). Find (x,y) bounds for each stratum
Interleaved sampling [Keller2001, Fatahalian2009]

Generate fixed set of (u,v,t) tuples and
rasterize coarsely in (x,y)
Tile-based traversal with (u,v,t) bounds [Laine2011]

Tile-test returns bounds for u,v and t individually
Tile-Overlap Tests
Find time interval when moving triangle overlaps
with the screen space tile [Laine2011, Munkberg2011b]
t=0
t=1
[0,1/4]
[0,1/2]
[1/4,3/4]
[1/2,1]
[3/4, 1]
Tile-Overlap Test for DOF

objects in focus objects out of focus
pixel
image plane
lens
focus plane
objects out of focus
Idea: Only test lens region that sees the object through a certain screen space tile
lens
focus plane
Lens Bounds Per Tile

Separating plane tests determine active lens
region [Akenine-Mller2011, Laine2011]
lens separating lines triangle lens separating lines
tile
triangle
tile
focus plane
focus plane
Shading
Shading Static Primitives

Shading after visibility
Shade per 2x2 pixels in screen space
Shading before visibility [Cook87]

Shade in object space per vertex Decoupled shading and visibility
Shading Moving Primitives

Supersampling
Shade each visibility sample Prohibitively expensive
Extend MSAA [McGuire2010]

Motion blur example
Shade once per covered pixel at a certain time Texture filters take motion into account [Loviscach2005]
MSAA Failure Case

Reverts to supersampling for large motion
Each sample hits a new triangle
t=0 Movement
t=1/4
t=1/2
t=3/4
Decoupled Sampling [Ragan-Kelley2011, Burns2010]

Many-to-one mapping function
Between visibility and shading samples
Low shading rate

Even for large motion and defocus
Needs cache for shaded values

Remapping logic per sample
Decoupled Sampling Example

remap to triangle at t=0 visibility samples snap remapped visibility sample to closest shading sample
t=1
T=1
t=0
T=0
T=0
screen space
barycentric space
shading space
Reconstruction
Reduce noise in stochastic rendering
May drastically reduce required #samples per pixel Store additional information with samples [Lehtinen2011]
Images courtesy of Lehtinen et al.[2011]
Input image (1 spp)
Reconstructed
Reference (256 spp)
View frustum culled
Culling Backface
culled Occlusion culled
View Frustum Culling

Solve for temporal interval when triangle is
inside the view frustum [Laine2011, Munkberg2011b]
t=1 t=1 t=0.6 near plane z=0
t=0
Motion Blur Backface Culling

Backface test is cubic
polynomial in t [Munkberg2011a]
Not conservative to test
facing at t=0 and t=1
p2 p2 p0 p0 p1 p1 t=0 t=0.5 t=1
p1 p0 p2
DOF Backface Culling

BF cull at center of lens
Correct BF cull
Lens
DOF Backface Culling

BF cull at center of lens
Correct BF cull
Lens
Occlusion Culling for Motion Blur

Standard z-culling less efficient Spatio-temporal occlusion culling
[Akenine-Mller2007, Boulos2010]
Illustration adapted from Boulos et al.[2010]
t x
solid occluder valid for t=[0,1]
solid occluder valid for t=ti

Conclusions
Stochastic rasterization does not come for free
Many more sample tests and shading evaluations Higher color, depth & texture bandwidth Less efficient culling and compression
Efficient shading is a requirement Very easy for end-user if implemented in HW

Q&A
Thank You:
The Advanced Rendering Technology team at Intel The Graphics Group at Lund University Mike Doggett, Jonathan Ragan-Kelley, Samuli Laine,
and Timo Aila for fruitful discussions
References
[Akenine-Mller2007] Tomas Akenine-Mller, Jacob Munkberg, and Jon Hasselgren, "Stochastic Rasterization using Time-Continuous Triangles", Graphics Hardware, August 2007. [Akenine-Mller2011] Tomas Akenine-Mller, Robert Toth, Jacob Munkberg, and Jon Hasselgren, "Efcient Depth of Field Rasterization using a Tile Test based on Half-Space Culling", technical report, June 2011 [Boulos2010] Solomon Boulos, Edward Luong, Kayvon Fatahalian, Henry Moreton and Pat Hanrahan Space-Time Hierarchical Occlusion Culling for Micropolygon Rendering with Motion Blur High Performance Graphics 2010 [Burns2010] Burns, C.A., Fatahalian, K., Mark, W.R. A Lazy Object-Space Shading Architecture With Decoupled Sampling. High Performance Graphics, 19-28. 2010 [Cook84] Robert L. Cook, Thomas Porter, Loren Carpenter, Distributed Ray Tracing SIGGRAPH 1984 [Cook86] Robert L. Cook. Stochastic Sampling in Computer Graphics. ACM Transactions on Graphics, Volume 6, Number 1, January 1996. [Cook87] Robert L. Cook, Loren Carpenter, Edwin Catmull. The Reyes Rendering Architecture. 1987 [Cook90] Robert L. Cook, Thomas Porter, Loren Carpenter. Pseudo-random point sampling techniques in computer graphics. United States Patent 4,897,806. 1990. [Egan2009] Kevin Egan , Yu-Ting Tseng , Nicolas Holzschuch , Frdo Durand , Ravi Ramamoorthi, Frequency analysis and sheared reconstruction for rendering motion blur, ACM Transactions on Graphics (TOG), v.28 n.3, August 2009 [Fatahalian2009] Kayvon Fatahalian, Edward Luong, Solomon Boulos, Kurt Akeley, William R. Mark and Pat Hanrahan. Data-Parallel Rasterization of Micropolygons with Defocus and Motion Blur High Performance Graphics 2009 [Fatahalian2011] Kayvon Fatahalian Evolving the Real-Time Graphics Pipeline for Micropolygon Rendering. Ph.D. Dissertation, Stanford University, 2011 [Grunschlo2011] Leonhard Grunschlo, Martin Stich, Sehera Nawaz, Alexander Keller. MSBVH: An Efficient Acceleration Data Structure for Ray Traced Motion Blur High Performance Graphics 2011
References cont.
[Gribel2010] Carl Johan Gribel, Michael Doggett, and Tomas Akenine-Mller, "Analytical Motion Blur Rasterization using Compression", High Performance Graphics, pp. 163-172, June 2010 [Gribel2011] Carl Johan Gribel, Rasmus Barringer, and Tomas Akenine-Mller, "High-Quality Spatio-Temporal Rendering using Semi-Analytical Visibility", SIGGRAPH 2011. [Hachisuka2008] Toshiya Hachisuka, Wojciech Jarosz, Richard Peter Weistroffer, Kevin Dale, Greg Humphreys, Matthias Zwicker, Henrik Wann Jensen Multidimensional adaptive sampling and reconstruction for ray tracing SIGGRAPH 2008 [Hou2010] Qiming Hou, Hao Qin, Wenyao Li, Baining Guo, Kun Zhou. Micropolygon ray tracing with defocus and motion blur. SIGGRAPH 2010 [Haeberli90] Paul Haeberli , Kurt Akeley, The accumulation buffer: hardware support for high-quality rendering, Proceedings of the 17th annual conference on Computer graphics and interactive techniques, p.309-318, 1990 [Korein83] Jonathan Korein , Norman Badler,Temporal anti-aliasing in computer generated animation, Computer Graphics, vol 17, no .3, p.377-388, 1983 [Keller2001] Alexander Keller , Wolfgang Heidrich, Interleaved Sampling, Eurographics Workshop on Rendering Techniques, p.269-276, 2001 [Laine2011] Samuli Laine, Timo Aila, Tero Karras and Jaakko Lehtinen. Clipless Dual-Space Bounds for Faster Stochastic Rasterization. SIGGRAPH 2011 [Lee2009] Sungkill Lee, Elmar Eisemann and Hans-Peter Seidel. Depth-of-field rendering with multiview synthesis. SIGGRAPH Asia 2009 [Lee2010] Sungkill Lee, Elmar Eisemann and Hans-Peter Seidel. Real-time lens blur effects and focus control. SIGGRAPH 2010 [Lehtinen2011] Jaakko Lehtinen, Timo Aila, Jiawen Chen, Samuli Laine and Frdo Durand. Temporal Light Field Reconstruction for Rendering Distribution Effects. SIGGRAPH 2011 [Loviscach2005] Jrn Loviscach, Motion Blur for Textures by Means of Anisotropic Filtering, Eurographics Symposium on Rendering, 2005. pp. 105-110 [Max85] Max N. L., Lerner D. M.: A two-and-a-half-d motion-blur algorithm. SIGGRAPH '85 [McGuire2010] McGuire, Enderton, Shirley, and Luebke, Real-Time Stochastic Rasterization on Conventional GPU Architectures, High Performance Graphics, June, 2010.
References cont.
[Munkberg2011a] Jacob Munkberg and Tomas Akenine-Mller, "Backface Culing for Motion Blur and Depth of Field", journal of gpu, graphics and game tools, 2011. [Munkberg2011b] Jacob Munkberg, Petrik Clarberg, Jon Hasselgren, Robert Toth, Masamichi Sugihara, and Tomas Akenine-Mller, Hierarchical Stochastic Motion Blur Rasterization, High Performance Graphics 2011. [Navarro2011] Fernando Navarro, Francisco J. Sern and Diego Guitierrez. Motion Blur Rendering: State of the Art. Computer Graphics Forum Volume 30 (2011), number 1 , pp. 3-26 [Overbeck2009] Ryan S. Overbeck , Craig Donner , Ravi Ramamoorthi, Adaptive Wavelet Rendering, ACM Transactions on Graphics (TOG), v.28 n.5, December 2009 [Potmesil81] Potmesil M. Chakravarty I: A lens and aperture camera model for synthetic image generation. SIGGRAPH 81 pp. 389-399 [Ragan-Kelley2011] Jonathan Ragan-Kelley, Jaakko Lehtinen, Jiawen Chen, Michael Doggett, Frdo Durand. Decoupled Sampling for Graphics Pipelines. SIGGRAPH 2011 [Shinya93] Shinya M.: Spatial anti-aliasing for animation sequences with spatio-temporal filtering. SIGGRAPH 93 pp. 289-296 [Shirley2011] Shirley, Aila, Cohen, Eric Enderton, Laine, Luebke, McGuire, A Local Image Reconstruction Algorithm for Stochastic Rendering, ACM Symposium on Interactive 3D Graphics and Games (I3D 2011 proceedings), February 2011 [Soler2009] Cyril Soler, Kartic Subr, Frdo Durand, Nicolas Holzschuch, Franois Sillion. Fourier depth of field. ACM Transactions on Graphics (TOG) , Volume 28 Issue 2, 2009 [Sousa2008] Sousa T.: Crysis next-gen effects. In Proceedings of the Game Developers Conference 2008. [Sung2002] K. Sung , A. Pearce , C. Wang, Spatial-Temporal Antialiasing, IEEE Transactions on Visualization and Computer Graphics, v.8 n.2, p.144-153, April 2002 [Wald2007] Wald I., Mark. W. R., Gnther J., Boulos S., Ize T., Hunt W., Parker S.G.,Shirley P.: State of the art in ray tracing animated scenes. Eurographics 2007 State of the Art Reports [Walter2006] Bruce Walter, Adam Arbree, Kavita Bala, Donald P. Greenberg. Multidimensional Lightcuts. SIGGRAPH 2006

09 towardBlurryRasterizerMunkberg BPS2011

Hochgeladen von

Dokumentinformationen

Originaltitel

Copyright

Verfügbare Formate

Dieses Dokument teilen

Dokument teilen oder einbetten

Freigabeoptionen

Stufen Sie dieses Dokument als nützlich ein?

Sind diese Inhalte unangemessen?

Copyright:

Verfügbare Formate

09 towardBlurryRasterizerMunkberg BPS2011

Hochgeladen von

Copyright:

Verfügbare Formate

Toward a Blurry Rasterizer

Beyond Programmable Shading Course, ACM SIGGRAPH 2011

High quality & low cost

Beyond Programmable Shading Course, ACM SIGGRAPH 2011

Beyond Programmable Shading Course, ACM SIGGRAPH 2011

Motion Blur Rendering

Depth of Field Rendering

image plane (x,y)

Beyond Programmable Shading Course, ACM SIGGRAPH 2011

Depth of Field and Motion Blur

Beyond Programmable Shading Course, ACM SIGGRAPH 2011

Stochastic sampling [Cook86]

Aliasing is replaced by noise

Beyond Programmable Shading Course, ACM SIGGRAPH 2011

Interleaved Sampling [Keller2001]

Improvement over acc.

5D: use (ui,vi,ti) tuples

Beyond Programmable Shading Course, ACM SIGGRAPH 2011

Sampling for Motion Blur (3D)

Beyond Programmable Shading Course, ACM SIGGRAPH 2011

Sampling for DOF (4D)

InterleaveUV (64 lens pos)

InterleaveUV (256 lens pos)

Beyond Programmable Shading Course, ACM SIGGRAPH 2011

Sampling for MB+DOF (5D)

Sobol sampling 4 spp

Beyond Programmable Shading Course, ACM SIGGRAPH 2011

Sobol sampling 16 spp

Beyond Programmable Shading Course, ACM SIGGRAPH 2011

Recap: Standard Rasterization

Beyond Programmable Shading Course, ACM SIGGRAPH 2011

The Graphics Pipeline

Memory System (buffers, textures)

Rasterization of Static Triangle

Beyond Programmable Shading Course, ACM SIGGRAPH 2011

Beyond Programmable Shading Course, ACM SIGGRAPH 2011

Beyond Programmable Shading Course, ACM SIGGRAPH 2011

Beyond Programmable Shading Course, ACM SIGGRAPH 2011

Triangles out of focus

Beyond Programmable Shading Course, ACM SIGGRAPH 2011

The Modified Graphics Pipeline

Output two vertex positions

Vertex Shader Hull Shader Tessellator Domain Shader

Rasterizer Pixel Shader Output Merger

Beyond Programmable Shading Course, ACM SIGGRAPH 2011

Clipping Moving Triangles

Beyond Programmable Shading Course, ACM SIGGRAPH 2011

image plane z=0 t=0

Screen Space Convex Hull

Beyond Programmable Shading Course, ACM SIGGRAPH 2011

Bound triangles movement in

Beyond Programmable Shading Course, ACM SIGGRAPH 2011

Beyond Programmable Shading Course, ACM SIGGRAPH 2011

Preserves GPU traversal order

Interleaved sampling [Keller2001, Fatahalian2009]

Tile-based traversal with (u,v,t) bounds [Laine2011]

Beyond Programmable Shading Course, ACM SIGGRAPH 2011

Tile-Overlap Test for DOF

Lens Bounds Per Tile

Beyond Programmable Shading Course, ACM SIGGRAPH 2011

Shading Static Primitives

Shading before visibility [Cook87]

Shading Moving Primitives