Direct Volume Rendering

(DVR): Ray-casting
Jian Huang

This set of slides references slides used by Prof.
Torsten Moeller (Simon Fraser), Prof. Han-Wei
Shen (Ohio State).
Papers
Tuy and Tuy, 1984, IEEE CG & A (one of the
earliest volume rendering techniques)
Levoy, 1988 IEEE CG&A, and later
improvements
Drebin, Carpenter, Hanrahan, 1988, SIGGRAPH

Direct: No conversion to surface geometry
Basic Idea
Based on the idea of ray tracing
Trace from eat each pixel
as a ray into object space
Compute color value along
the ray
Assign the value to the
pixel

Data Representation
3D volume data are represented by a finite number of cross
sectional slices (hence a 3D raster)
On each volume element (voxel), stores a data value (if it uses
only a single bit, then it is a binary data set. Normally, we see a
gray value of 8 to 16 bits on each voxel.)
N x 2D arraies = 3D array
Data Representation (2)
What is a Voxel? Two definitions
A voxel is a cubic cell, which
has a single value cover
the entire cubic region
A voxel is a data point
at a corner of the cubic cell
The value of a point inside the
cell is determined by interpolation
Viewing
Ray Casting

Where to position the volume and image plane
What is a ray
How to march a ray
Viewing (1)
1. Position the volume
Assuming the volume dimensions is w x w x w
We position the center of the volume at the world origin
y
(0,0,0) x
z
Volume center = [w/2,w/2,w/2]
(local space)
Translate T(-w/2,-w/2,-w/2)
(data to world matrix?
world to data matrix )
Viewing (2)
2. Position the image plane
Assuming the distance between the image plane and
the volume center is D, and initially the center of the
image plane is (0,0,-D)
y
(0,0,0) x
z
Image
plane
Viewing (3)
3. Rotate the image plane
A new position of the image plane can be defined in terms
of three rotation angle o,|, with respect to x,y,z axes

Assuming the original view vector is [0,0,1], then the new
view vector g becomes:

cos| 0 sin| 1 0 0 cos sin 0
g = [0,0,1] 0 1 0 0 coso sino sin cos 0
sin| 0 cos| 0 sino coso 0 0 1
Viewing (4)
y
(0,0,0) x
z
u
v
E
S
E0 u0
v0
+
S0
B
B = [0,0,0]
S0 = [0,0,-D]
u0 = [1,0,0]
v0 = [0,1,0]
Now,
R: the rotation matrix
S = B D x g
U = [1,0,0] x R
V = [0,1,0] x R
Viewing (5)
R: the rotation matrix
S = B D x g
U = [1,0,0] x R
V = [0,1,0] x R
+
S
Image Plane: L x L pixels
E
Then

E = S L/2 x u L/2 x v

So
Each pixel (i,j) has coordinates

P = E + i x u + j x v
u
v
We enumerate the pixels by changing
i and j (0..L-1)
Viewing (6)
4. Cast rays
Remember for each pixel on the image plane
P = E + i x u + j x v
and
the view vector g = [0,0,1] x R
So the ray has the equation:

Q = P + k (d x g) d: the sampling distance at each step
x
x
d
x
x
p
Q
K = 0,1,2,
Early Methods
Before 1988
Did not consider transparency
did not consider sophisticated light
transportation theory
were concerned with quick solutions
hence more or less applied to binary data

non-binary data -
require sophisticated classification/compositing
methods!
Ray Tracing -> Ray Casting
another typical method from traditional
graphics
Typically we only deal with primary rays -
hence: ray-casting
a natural image-order technique
as opposed to surface graphics - how do we
calculate the ray/surface intersection???
Since we have no surfaces - we need to carefully
step through the volume
Ray Casting
Stepping through the volume: a ray is cast into
the volume, sampling the volume at certain
intervals
The sampling intervals are usually equi-distant,
but dont have to be (e.g. importance sampling)
At each sampling location, a sample is
interpolated / reconstructed from the grid
voxels
popular filters are: nearest neighbor (box),
trilinear (tent), Gaussian, cubic spline
Along the ray - what are we looking for?
Example: Using the nearest
neighbor kernel
In tuys paper
Q = P + K x V (v=dxg)

At each step k, Q is rounded
off to the nearest voxel
(like the DDA algorithm)

Check if the voxel is on the
boundary or not (compare
against a threshold)

If yes, perform shading


Basic Idea of Ray-casting
Pipeline
- Data are defined at the corners
of each cell (voxel)

- The data value inside the
voxel is determined using
interpolation (e.g. tri-linear)

- Composite colors and opacities
along the ray path


- Can use other ray-traversal schemes as well

c1
c2
c3
Ray Traversal Schemes
Depth
Intensity
Max
Average
Accumulate
First
Ray Traversal - First
Depth
Intensity
First
First: extracts iso-surfaces (again!)
done by Tuy&Tuy 84
Ray Traversal - Average
Depth
Intensity
Average
Average: produces basically an X-ray picture
Ray Traversal - MIP
Depth
Intensity
Max
Max: Maximum Intensity Projection
used for Magnetic Resonance Angiogram
Ray Traversal - Accumulate
Depth
Intensity
Accumulate
Accumulate opacity while compositing colors:
make transparent layers visible!
Levoy 88
Raycasting
color
opacity
1.0
volumetric compositing
object (color, opacity)
Raycasting
color
opacity
Interpolation
kernel
1.0
object (color, opacity)
volumetric compositing
Raycasting
color c = c
s
o
s
(1 - o) + c
opacity o

= o
s
(1 - o) + o
1.0
object (color, opacity)
volumetric compositing
Interpolation
kernel
Raycasting
color
opacity
1.0
object (color, opacity)
volumetric compositing
Raycasting
color
opacity
1.0
object (color, opacity)
volumetric compositing
Raycasting
color
opacity
1.0
object (color, opacity)
volumetric compositing
Raycasting
color
opacity
1.0
object (color, opacity)
volumetric compositing
Raycasting
color
opacity
object (color, opacity)
volumetric compositing
Volume Rendering Pipeline
Acquired values
Data preparation
Prepared values
classification shading
Voxel colors
Ray-tracing / resampling Ray-tracing / resampling
Sample colors
compositing
Voxel opacities
Sample opacities
Image Pixels
DCH DVR Pipeline
DCH - Pipeline
Original data
Material percentage volumes
Color volume Opacity volume Density volume
Gradient Shaded volume
Transformed volume
Final image
Classification
Normals
Shading
shears
compositing
Common Components of General
Pipeline
Interpolation/reconstruction
Classification or transfer function
Gradient/normal estimation for shading
Question: are normals also interpolated?
Levoy - Interpolation
eye
image pixel
viewing ray
voxel
sample point
trilinear
interpolation
Levoy - Interpolation (2)
binary
smooth
Closest value Weighted average
Levoy Gradient/Normals
Central difference
per voxel
2
2
2
1 , , 1 , ,
, 1 , , 1 ,
, , 1 , , 1
+
+
+

=
k j i k j i
z
k j i k j i
y
k j i k j i
x
v v
G
v v
G
v v
G
X+1
y-1,
Y+1
z-1
Levoy - Compositing
Image order
back-to-front
using the over operator
o + o o = o
o + o =
) 1 (
) 1 (
in out
in out
C C C
Levoy - Shading
Phong Shading + Depth Cueing
( )
( )
( ) ( ) ) ) ( ) ( (
2 1
n
s d
p
a p
H x N k L x N k
x d k k
C
k C x C +
+
+ =
C
p
= color of parallel light source
k
a
/ k
d
/ k
s
= ambient / diffuse / specular light coefficient
k
1
, k
2
= fall-off constants
d(x) = distance to picture plane
L = normalized vector to light
H = normalized vector for maximum highlight
N(x
i
) = surface normal at voxel x
i
Classification
Classification: Mapping from data to opacities

Region of interest: high opaicity (more opaque)
Rest: translucent or transparent

The opacity function is typically specified by the user

Levoy came up with two formula to compute opacity
1. Isosurface
2. Region boundary (e.g. between bone and fresh)
Classification/Transfer
Function
Maps raw voxel value into presentable entities:
color, intensity, opacity, etc.
Raw-data material (R, G, B, o, K
a
, K
d
, K
s
, ...)

May require probabilistic methods (Drebin).
Derive material volume from input. Estimate % of
each material in all voxels. Pre-computed. AKA
segmentation.

Often use look-up tables (LUT) to store the
transfer function that are discovered

Levoy - Classification
Usually not only interested in a particular iso-
surface but also in regions of change
Feature extraction - High value of opacity exists
in regions of change
Transfer function (Levoy) - Saliency
Surface strength
Levoy - Classification
Chemistry Data
only iso-value - loose information of layers
iso-range - could be too narrow or too wide
thickness of region should be constant
hence linear fall off of opacity
wider fall off for larger gradient
Medical Data
assume at most two tissues meet
linear transition between opacities of neighboring
tissues
reflects linear combination of tissues within one voxel
Opacity function (1)
Goal: visualize voxels that have a selected threshold
value fv

- No intermediate geometry is extracted
- The idea is to assign voxels that have value fv the
maximum opacity (say o)
- And then create a smooth transition for the surrounding
area from 1 to 0
-Levoy wants to maintain a constant thickness for the
transition area.
Opacity function (2)
Maintain a constant isosurface thickness
opacity = o
opacity = 0
Can we assign opacity based
on function value instead of
distance? (local operation:
we dont know where
the isosurface is)

Yes we can based on the
value distance f fv
but we need to take into
account the local gradient
Opacity function (3)
Assign opacity based on value difference (f-fv) and
local gradient

gradient: the value fall-off rate grad = Af/As
Assuming a region has a constant gradient and the isosurface
transition has a thickness R


opacity = o
F = fv
opacity = 0
F = fv grad * R
thickness = R
F = f(x)
Then we interpolate the opacity

opacity = o o * ( fv-f(x))/ (grad * R)
Levoy - Classification A
Opacity
o
(x
i
)
Acquired value f (x
i
)
Gradient magnitude |
V
f (x
i
)|
Opacity
o
(x
i
)
Acquired value f (x
i
)
Gradient magnitude |
V
f (x
i
)|
o
v
f
v
( )
( )
( )
|
|
.
|

\
|

o = o
i
i v
v i
x f
x f f
r
x
'
1
1
Levoy - Classification B
Opacity
o
(x
i
)
Acquired value f (x
i
)
Gradient magnitude |
V
f (x
i
)|
o
vc
o
vb
o
va
f
va
f
vb
f
vc
( ) ( )
( ) ( )
|
|
.
|

\
|
|
|
.
|

\
|

o +
|
|
.
|

\
|

o = o
+
+
+
+
n n
n
n
n n
n
n
v v
i v
v
v v
v i
v i i
f f
x f f
f f
f x f
x f x
1
1
1
1
'
DCH - Material Percentage V.
Probabilistic classifier
probability that a voxel has intensity I:


p
i
- percentage of material
P
i
(I) - prob. that material i has value I
P
i
(I) given through statistics/physics
p
i
then given by:


( ) ( )

=
=
n
i
i i
I P p I P
1
( )
( )
( )

=
=
n
j
j
i
i
I P
I P
I p
1
DCH - Classification
Like Levoy - assumes only two materials per voxel
that will lead to material percentage volumes

from them we conclude color/opacity:



where C
i
=(o
i
R
i
, o
i
G
i
, o
i
B
i
, o
i
)

=
=
n
i
i i
C p C
1
DCH- Classification
Air
Fat Tissue
Bone
CT Number
Air
Fat Tissue
Bone
histogram
Constituents Distributions
Material Assignment %
CT
Levoy - Improvements
Levoy 1990
front-to-back with early ray termination
o = 0.95
hierarchical oct-tree data structure
skip empty cells efficiently
Volumetric Ray Integration
color
opacity
object (color, opacity)
1.0