You are on page 1of 25

# 

Monte Carlo
Sampling Methods

Jasmina L. Vujic
Nuclear Engineering Department
University of California, Berkeley
Email: vujic@nuc.berkeley.edu
phone: (510) 643-8085
fax: (510) 643-9685

UCBNE, J. Vujic


Monte Carlo
Monte Carlo is a computational technique based on
constructing a random process for a problem and
carrying out a NUMERICAL EXPERIMENT by N-fold
sampling from a random sequence of numbers with a
PRESCRIBED probability distribution.

x - random variable
N

∑x
1
x̂ = ---- i
N
i=1

## X̂ - the estimated or sample mean of x

x - the expectation or true mean value of x

## If a problem can be given a PROBABILISTIC

interpretation, then it can be modeled using RANDOM
NUMBERS.
UCBNE, J. Vujic


Monte Carlo
• Monte Carlo techniques came from the complicated diffusion
problems that were encountered in the early work on atomic energy.
• 1772 Compte de Bufon - earliest documented use of random
sampling to solve a mathematical problem.
• 1786 Laplace suggested that π could be evaluated by random
sampling.
• Lord Kelvin used random sampling to aid in evaluating time
integrals associated with the kinetic theory of gases.
• Enrico Fermi was among the first to apply random sampling
methods to study neutron moderation in Rome.
• 1947 Fermi, John von Neuman, Stan Frankel, Nicholas Metropolis,
Stan Ulam and others developed computer-oriented Monte Carlo
methods at Los Alamos to trace neutrons through fissionable
materials

UCBNE, J. Vujic
Monte Carlo


## a) The problems that are stochastic (probabilistic) by

nature:
- particle transport,
- telephone and other communication systems,
- population studies based on the statistics of
survival and reproduction.

## b) The problems that are deterministic by nature:

- the evaluation of integrals,
- solving the systems of algebraic equations,
- solving partial differential equations.

UCBNE, J. Vujic
Monte Carlo


## a) ANALOG, where the natural laws are PRESERVED

- the game played is the analog of the physical
problem of interest
(i.e., the history of each particle is simulated
exactly),

## b) NON-ANALOG, where in order to reduce required

computational time the strict analog simulation of
particle histories is abounded (i.e., we CHEAT!)
Variance-reduction techniques:
- Absorption suppression
- History termination and Russian Roulette
- Splitting and Russian Roulette
- Forced Collisions
- Source Biasing
UCBNE, J. Vujic
Example 1: Evaluation of Integrals


fmax
f(x)
y1 • reject

• accept
a x1 b
b
I =
∫ f( x )dx - area uder the function f(x),R = ( b – a )f
a
max - area of rectangle

I
P = ---- - is a probability that a random point lies under f(x), thus I = RP
R

## Step 3: Repeat this process N times, Ni - the number of accepted points

Ni Ni
Step 4: Determine P = ------ and the value of integral I = R ------
N N

UCBNE, J. Vujic
Major Components of Monte Carlo


## Major Components of a Monte Carlo Algorithm

• Probability distribution functions (pdf’s) - the physical (or
mathematical) system must be described by a set of pdf’s.
• Random number generator - a source of random numbers uniformly
distributed on the unit interval must be available.
• Sampling rule - a prescription for sampling from the specified pdf,
assuming the availability of random numbers on the unit interval.
• Scoring (or tallying) - the outcomes must be accumulated into
overall tallies or scores for the quantities of interest.
• Error estimation - an estimate of the statistical error (variance) as a
function of the number of trials and other quantities must be
determined.
• Variance reduction techniques - methods for reducing the varinace
in the estimated solution to reduce the computational time for Monte
Carlo simulation.
• Parallelization and vectorization - efficient use of advanced
computer architectures.

UCBNE, J. Vujic
Probability Distribution Functions


## Random Variable, x, - a variable that takes on particular val-

ues with a frequency that is determined by some underlying
probability distribution.

P{a ≤ x ≤ b}

## Discrete Probability Distribution

P { x = xi } = pi

UCBNE, J. Vujic
Probability Distribution Functions


## Probability Density Function (PDF) - continuous

• f(x), f ( x )dx = P { x ≤ x' ≤ x + dx }

• 0 ≤ f(x) , ∫ f ( x )dx = 1
–∞
b f(x)
• Probability{ a ≤ x ≤ b } =
∫ f( x )dx
a
x→
Cumulative Distribution Function (CDF) - continuous
x

• F(x) =
∫ f( x' )dx' = P{ x' ≤ x } 1
–∞

• 0 ≤ F(x) ≤ 1 F(x)
d
• 0≤ F(x) = f(x)
dx 0
b x→

∫ f( x' )dx' = P{ a ≤ x ≤ b } = F( b ) – F( a )
a
UCBNE, J. Vujic
Probability Distribution Functions


## Probability Density Function (PDF) - discrete

• f(xi), f ( x i ) = p i δ ( x – x i )

• 0 ≤ f ( xi ) p3

• ∑ f( x )( ∆x ) = 1 or ∑ p
i
i i
i
i = 1 f(x)
p1
p2

x1 x2 x3
Cumulative Distribution Function (CDF) - discrete
F(x)
• F(x) = ∑ p = ∑ f( x )∆x
xi < x
i
xi < x
i i p1+p2+p3 1

p1+p2
• 0 ≤ F(x) ≤ 1 p1

x1 x2 x3

UCBNE, J. Vujic
Monte Carlo & Random Sampling


## Sampling from a given discrete distribution

Given f ( x i ) = p i and ∑p
i
i = 1 , i = 1, 2, ..., N

## 0 p1 p1+p2 p1+p2+p3 p1+p2+...+pN=1

and 0 ≤ ξ ≤ 1 , then P ( x = x k ) = p k = P ( ξ ∈ d k ) or

k–1 k

∑p ≤ ξ < ∑p
i i
i=1 i=1

UCBNE, J. Vujic
Monte Carlo & Random Sampling


## Sampling from a given continuous distribution

If f(x) and F(x) represent PDF and CDF od a random variable x, and if ξ is a random
number distributed uniformly on [0,1] with PDF g( ξ )=1, and if x is such that

F(x) = ξ

than for each ξ there is a corresponding x, and the variable x is distribute according
to the probability density function f(x).

Proof:

For each ξ in ( ξ , ξ +∆ ξ ), there is x in (x,x+∆x). Assume that PDF for x is q(x). Show
that q(x) = f(x):

## Thus, if x = F-1( ξ ), then x is distributed according to f(x).

UCBNE, J. Vujic
Monte Carlo & Random Sampling


## Categories of Random Sampling

• Random number generator uniform PDF on [0,1]
• Sampling from analytic PDF’s normal, exponential, Maxwellian, .....
• Sampling from tabulated PDF’s angular PDF’s, spectrum, cross sect

## For Monte Carlo Codes...

• Random numbers, ξ , are produced by the R.N. generator on [0,1]
• Non-uniform random variates are produced from the ξ ’s by
— direct inversion of CDFs
— rejection methods
— transformations
— composition (mixtures)
— sums, products, ratios, .....
— table lookup + interpolation
— lots (!) of other tricks .....
• < 10% of total cpu time (typical)

UCBNE, J. Vujic
Random Sampling Methods


## Random Number Generator

Pseudo-Random Numbers
• Not strictly "random", but good enough 1
— pass statistical tests f(x)
— reproducible sequence
0 1
• Uniform PDF on [0,1]
• Must be easy to compute, must have a large period 1

## Multiplicative congruential method F(x)

• Algorithm
S0 = initial seed, odd integer, < M 0 1

ξ k = Sk / M

## • Typical (vim, mcnp):

Sk = 519 • Sk-1 mod 248
ξ k = Sk / 248
period = 246 ≈ 7.0 x 1013
UCBNE, J. Vujic
Random Sampling Methods


## Direct Sampling (Direct Inversion of CDFs)

Direct Solution of
–1
x̂ ← F ( ξ )

Sampling Procedure: 1
• Generate ξ ξ
• Determine x̂ such that F ( x̂ ) = ξ F(x)

0
x→ x̂
Advantages
• Straightforward mathematics & coding
• "High-level" approach

Disadvantages
• Often involves complicated functions
• In some cases, F(x) cannot be inverted (e.g., Klein-Nishina)

UCBNE, J. Vujic
Random Sampling Methods


Rejection Sampling
Used when the inverse of CDF is costly ot impossible to find.
Select a bounding function, g(x), such that
• c ⋅ g ( x ) ≥ f ( x ) for all x
• g(x) is an easy-to-sample PDF

Sampling Procedure:
–1
• sample x̂ from g(x): x̂ ← g ( ξ 1 )
cg ( x )
• test: ξ 2 ⋅ cg ( x̂ ) ≤ f ( x̂ )

## if true accept x̂ , done • reject

if false reject x̂ , try again f(x)
6

• accept
Advantages
x→
• Simple computer operations
Disadvantages
• "Low-level" approach, sometimes hard to understand

UCBNE, J. Vujic
Random Sampling Methods


Table Look-Up
Used when f(x) is given in a form of a histogram

f1 fi
f(x) f2

x1 x2 xi-1 xi
F(x)
ξ

x1 x2 xi-1 x xi

## Then by linear interpolation

( x – x i – 1 )F i + ( x i – x )F i – 1 [ ( x i – x i – 1 )ξ – x i F i – 1 + x i – 1 F i ]
F ( x ) = --------------------------------------------------------------------- , x = ----------------------------------------------------------------------------------
xi – xi – 1 Fi – Fi – 1

UCBNE, J. Vujic
Random Sampling Methods


## Sampling Multidimensional Random Variables

If the random quantity is a function of two or more random variables that are
independent, the joint PDF and CDF can be written as

f ( x, y ) = f 1 ( x )f 2 ( y )

F ( x, y ) = F 1 ( x )F 2 ( y )

## EXAMPLE: Sampling the direction of isotropically scattered particle in 3D

Ω = Ω ( θ, ϕ ) = Ω x i + Ω y j + Ω z k = v + w + u ,
dΩ sin θdθdϕ – d ( cos θ )dϕ – dµdϕ
-------- = -------------------------- = -------------------------------- = ------------------
4π 4π 4π 4π

1 1
f ( Ω ) = f 1 ( µ )f 2 ( ϕ ) = --- ------
2 2π
µ

1
F1 ( µ ) = f 1 ( µ' ) dµ' = --- ( µ + 1 ) = ξ 1 or µ = 2ξ 1 – 1
–1 2
ϕ ϕ
F1 ( ϕ ) =
0 ∫f 2 ( ϕ' ) dϕ' = ------ = ξ 2 , or ϕ = 2πξ 2

UCBNE, J. Vujic
Random Sampling Methods


## Probability Density Function Direct Sampling Method

Linear: f(x) = 2x , 0<x<1 x← ξ
(L1, L2)
–x
Exponential: f(x) = e , 0<x x ← – log ξ
(E)
2D Isotropic: 1 , u ← cos 2πξ 1
f ( ρ ) = ----- ρ = ( u, v )
(C) 2π v ← sin 2πξ 1

3D Isotropic: u ← 2 ξ1 – 1
1
(I1, I2) f ( Ω ) = ------ , Ω = ( u, v, w ) 2
4π v ← 1 – u cos 2πξ 2
2
w ← 1 – u sin 2πξ 2

Maxwellian: x π
f ( x ) = ----------- --- e –x / T, 0<x x ← T ( – log ξ 1 – log ξ 2 cos2 --ξ 3)
2
(M1, M2, M3)
T π T
2

## Watt – a b ⁄ 4 –x ⁄ a w ← a ( – log ξ 1 – log ξ 2 cos2 π-- ξ 3)

f ( x ) = ---------------- 0<x
2e 2
Spectrum: e sinh bx ,
(W1, W2, W3) πa b3
a b
2
2
x ← w + --------- + ( 2 ξ 4 – 1 ) a bw
4

Normal: 1  x – µ
2
– - -----------
- x ← µ + σ – 2 log ξ 1 cos 2πξ 2
1
------------- e  σ 
(N1, N2)
f(x) =
2

σ 2π
UCBNE, J. Vujic

← ξ
Random Sampling Examples


Example — 2D Isotropic
1 ,
f ( ρ ) = ----- ρ = ( u, v )

## Rejection (old vim)

SUBROUTINE AZIRN_VIM( S, C )
IMPLICIT DOUBLE PRECISION (A-H, O-Z)
100 R1=2.*RANF() - 1.
R1SQ=R1*R1
R2=RANF()
R2SQ=R2*R2
RSQ=R1SQ+R2SQ s
IF(1.-RSQ)100,105,105
105 S=2.*R1*R2/RSQ
C=(R2SQ-R1SQ)/RSQ
RETURN c
END

## Direct (racer, new vim)

subroutine azirn_new( s, c )
implicit double precision (a-h,o-z)
parameter ( twopi = 2.*3.14159265 )
phi = twopi*ranf()
c = cos(phi)
s = sin(phi)
return
end
UCBNE, J. Vujic
Random Sampling Examples


## Example — Watt Spectrum

– a b ⁄ 4 –x ⁄ a
2e
( x ) = ---------------- e sinh bx , 0<x
πa b 3

Rejection (mcnp)
• Based on Algorithm R12 from 3rd Monte Carlo Sampler, Everett & Cashwell

• Define K = 1 + ab ⁄ 8 , L = a { K + ( K 2 – 1 )1 / 2 } , M = L ⁄ a – 1

## • If { y – M ( x + 1 ) } 2 ≤ bLx , accept: return (Lx)

otherwise, reject

## Direct (new vim)

• Sample from Maxwellian in C-of-M, transform to lab
w ← a ( – log ξ 1 – log ξ 2 cos2 π-- ξ 3)
2
2
a b 2 (assume isotropic emission from fission fragment
x ← w + --------- + ( 2ξ 4 – 1 ) a bw
4 moving with constant velocity in C-of-M)
• Unpublished sampling scheme, based on original Watt spectrum derivation
UCBNE, J. Vujic
Random Sampling Examples


## Example — Linear PDF

f(x) = 2x , 0≤x≤1

Rejection
(strictly — this is not "rejection", but has the same flavor)

if ξ 1 ≥ ξ 2 , then x̂ ← ξ 1
else x̂ ← ξ 2

or
x̂ ← max ( ξ 1, ξ 2 )

or
x̂ ← ξ 1 – ξ 2

Direct Inversion

F( x ) = x2 , 0≤x≤1

← ξ

UCBNE, J. Vujic
Random Sampling Examples


## Assume (for photon interactions):

µ tot = µ cs + µ fe + µ pp

Define
µ cs µ fe µ pp
p 1 = --------
- , p = --------- , and p = ---------
µ tot 2 µ tot 3 µ tot

3
with
∑p i = 1.
i=1

Then
0 p1 p1+p2 p1+p2+p3=1

p1 < ξ < p1 + p2

UCBNE, J. Vujic


## a) Need to know the physics of the system

b) Need to derive equations describing physical
processes
c) Need to generate material specific data bases
(cross sections for particle interactions, kerma-
factors, Q-factors)
d) Need to “translate” equations into a computer
program (code)
e) Need to "describe" geometrical configuration of
the system to computer
f) Need an adequate computer system

## g) Need a physicist smart enough to do the job and

dumb enough to want to do it.

UCBNE, J. Vujic
Transport Equation


## Time-Dependent Particle Transport Equation

(Boltzmann Transport Equation):

1∂
--- ψ ( r, E, Ω, t ) + Ω • ∇ψ ( r, E, Ω, t ) + Σ t ( E )ψ ( r, E, Ω, t ) =
v∂t

∞ 

0 4π
s ∫
 dE' Σ ( E' → E, Ω' → Ω )Ψ ( r, E', Ω', t ) dΩ' +
 

∞ 
χ(E) 
4π ∫ 0

------------  dE' υΣ f ( E' )Ψ ( r, E', Ω', t ) dΩ' +
4π 

1
------Q ( r, E, t )

UCBNE, J. Vujic