The Lighthouse Problem Revisited

Alison Crocker March 14, 2006 In the lighthouse problem, a lighthouse is situated a perpendicular distance x from an infinite straight shore and a distance along the shore (see Figure 1 !The lighthouse sends out pulses of light at random angles uniforml" distributed from #$% to &#$%, 'here ( ) * is the line perpendicular to the shore! Let + be a random variable that gives the distance along the shore measured from the ,ero-point of the shore! Then + has a .auch" probabilit" distribution given b"/ fY ( y =
1 x % x +(y

(see 0ppendix for derivation !

The .auch" distribution is interesting because it has significant tails1 extreme " values are much more li2el" than in other distributions! In fact, the variance of the .auch" distribution is infinite!

345R6

"

Figure 1. The lighthouse problem.

0s a 70TL08 home'or2 problem, 'e considered the one-dimensional lighthouse problem in 'hich is a parameter, but x is held constant! For this pro9ect, I consider the t'o-dimensional problem in 'hich both and x are parameters! The first thing I did 'as to investigate ho' changing x changes the shape of the .auch" pdf! To do

this, I made three graphs consisting of a histogram of the location of 1*** light pulses from a lighthouses at x ) !:, 1 and %! For these graphs, I let ) *, since changing simpl" shifts the 'hole distribution!

Figure 2. Sample histograms (blue) and Cauchy pdf (red) for x = . ! 1 and 2.

0s the graphs in Figure % sho', the further a'a" from shore the lighthouse is, the more spread out the distribution of light pulses on shore! ;ext, I 'anted to loo2 at the li2elihood surface for and x! To do this, I too2 samples from the ) *, x ) 1 .auch" distribution and found the li2elihood generated b" the samples for different and x values (see code in glh!m for details ! I then plotted the li2elihood values as a surface! 0 li2elihood surface for n ) 1** samples is sho'n belo' in Figure <! The surface clearl" has a pea2 some'here near ) *, x ) 1! In order to see ho' the li2elihood function changed 'ith sample si,e, I used contour plots and a variet" of n values! .ontour plots for n ) 1*, <% (=1*1!: , and 1** are sho'n belo' in Figure >!

Figure ". # "$ li%elihood function for the & = '! x = 1 Cauchy (ith 1'' samples.

The contour plots are ver" interesting! The clearest trend is the concentration of the li2elihood in one steep pea2 'ith increasing n! The n ) 1* li2elihood contour plot is ver" spread out 'hile the n ) 1** li2elihood is much more concentrated! 0s n increases, the shape of the li2elihood function becomes more and more regular! I expect this increase in regularit" comes from the decreasing importance of a fe' extreme "i values! 0n interesting extension of these contour plots 'ould be to ta2e a random sample from the .auch" distribution and add an artificial "n?1 'ith an extreme positive or negative value and see ho' much it changed the shape of the li2elihood function!

Figure ). *i%elihood contour plots for n = 1'! "2! 1'' (cloc%(ise from top left).

The contour plots also sho' that it is more difficult to determine x than 'hen n is small! This difficult" ma2es sense intuitivel"! Imagine n ) : 'ith three "i clustered around *, one extreme positive and one extreme negative value! In this situation, is li2el" to be near ,ero, but 'hat about x@ Perhaps x is small and the t'o extreme values are 9ust that, extreme! 5r perhaps x is larger and the fact that three "i are more concentrated 'as 9ust random (see Figure : !

Figure . T(o possible x +alues for a sample of fi+e.

;ext, I loo2ed at the maximum li2elihood (7L estimates for and x! To find the 7L estimates, I found the indices of maximum li2elihood value in m" li2elihood matrix b" appl"ing the 70TL08 function AmaxB t'ice! 4ere is the code/
[C,I]=max(L); [D,J]=max(C); MLmu = x(I(1,J)) MLx = z(J).

L is m" li2elihood matrix, 'hen max is applied to L it gives a ro' vector . consisting of the maximum value in each column and a ro' vector I specif"ing the ro' at 'hich the maximum value for each column 'as found! Chen max is applied to ., it gives the maximum value D, 'hich does not actuall" matter, and the index E of D! Thus E specifies the column in 'hich the matrix maximum occurs and I(1,E specifies the column! I then translate these indices into and x values using m" previousl" defined x vector for and , vector for x! I found that the 7L estimator for does approach as n increases! In Figure F, the f(x value of the graph is the 7L estimate after the first x of the 1** samples have been used to compute the 7L estimate! This graphing procedure 'as performed on four different sample sets as sho'n in the figure! 0ll four sample sets seemed to be converging to the correct ) *, but I 'anted to loo2 at the behavior for even larger n!

Figure ,. -* estimates of & as a function of sample si.e.

To do this, I too2 a sample of 1*** from the ) *, x ) 1 .auch" pdf and evaluated the 7L estimate at n ) <, 1*, <%, 1**, <1F and 1*** to get the blue line in Figure G! 0nother, seemingl" reasonable 'a" to estimate 'ould be b" using the sample mean! I used the same sample of 1*** and evaluated the sample mean at the same n values to get the dashed pin2 line! .learl" the 7L estimate is a much better estimator than the mean! The same samples 'ere used to calculate both 'hich indicates that the 7L estimate is able to avoid placing too much importance on the extreme values that ma2e the mean so erratic!

Figure /. -* estimate and sample mean as estimators of & as a function of sample si.e.

The 7L estimate for x also seems to converge to the correct x ) 1 as seen belo' in Figure H! I used three different sample sets of 1***, plotting the 7L estimate of the first n samples at n ) <, 1*, <%, 1**, <1F and 1***!

Figure 0. -* estimates for x as a function of sample si.e.

;ext, I 'ent bac2 to the one dimensional li2elihood functions to see 'hat shape the" too2 and ho' the" changed 'ith increasing n! The li2elihood functions for all loo2 ver" Iaussian, 'hile the li2elihood function for x reJuires higher n before it loo2s

normal (Figure K !

Figure 1. *i%elihood functions of & (left) and x (right) (ith n = 1' (blue)! "' (green) and 1'' (pin%).

The Iaussian appearance of the li2elihood function can be understood if one Ta"lor expands the natural log of the li2elihood function around the maximum li2elihood value! The first term in the expansion 'ill be a constant, the second 'ill be ,ero because 'e are

expanding around the 7L value and the third 'ill be Juadratic! To return an approximation to the li2elihood function, raise e to this Ta"lor expansion! This procedure
d % ln( L 'ill a Iaussian 'ith mean eJual to the 7L estimate and = d
1 $ %

evaluated

at the 7L estimate (for further details, see 3ivia pages %%-%> ! Lnfortunatel", in this problem even the 7L estimate is ver" hard to find anal"ticall" and evaluating M 'ould be even more difficult! 3o I cannot anal"ticall" demonstrate 'h" the Iaussian is immediatel" a good approximation to 'hile it ta2es higher n for it to be a good approximation to x, but it must be due to the as"mmetr" of L(x reJuiring further terms in the Ta"lor expansion! 0s n gets larger, these higher order terms must decrease more Juic2l" than the third term! It is evident from both li2elihood functions in Figure K that M is decreasing 'ith increasing n! To investigate ho' it 'as decreasing, I graphed the standard deviation of the li2elihood using the first n samples for four different sample sets of 1** (Figure 1* ! The Juanti,ation of the graph comes from m" initial choice of resolution for ! Cith a hint from 3ivia, page <>, I plotted a graph of
a 'here a 'as chosen to roughl" agree n

'ith the data at n )1**! The line does seem to be about the same shape as the data, ma2ing it believable that
1 ! n

Figure 1'. Standard de+iation of & as a function of n. The blac% line gi+es the expected 12s3rt(n) dependence.

I have investigated in detail both the <D and %D li2elihood functions for the lighthouse problem! 3ince I normali,ed m" li2elihood functions, the" are actuall" 8a"esian posteriors assuming a uniform prior! 3o I could easil" 'rite a bit more code to evaluate the probabilit" that or x is in some range of values given a certain set of data! I 'ill not continue 'ith this though, because I thin2 all of this enough is enough of a revisit to the lighthouse problemN 4eferences5 Iull, 3tephen! O8a"esian Inductive Inference and 7aximum 6ntrop"!O In MaximumEn ro!y and "ayesian Me hods in #cience and En$ineerin$ %&ol' ()' 6d! I! E! 6ric2son and .! R! 3mith, Plu'er 0cademic Publishers, 1KHH! :<-G>! 3ivia, D! 3! *a a Analysis+ A "ayesian ,u orial' 5xford/ .larendon Press, 1KKF!

#ppendix5 $eri+ation of Cauchy distribution Let (x, be the location of the lighthouse, ( be the angle at 'hich a light pulse is emitted and " be the distance along shore 'here the light pulse hits! 8" the geometr" of the problem, y i = x tan i ! Find the cdf for +/
-( y i < y = -( x tan i < y = -(tan i < y y = -( < tan 1 ( x x

!
1

;ote that ( is randoml" distributed from &#$% to #$%, so f ( =

1 1 y 1 y Thus .Y ( y = - < tan x = tan x ! The derivative of the cdf gives the pdf/
f Y ( y1 , x = 1 y 1 tan 1 = x x 1
%

1 1 x = % y x x + (y 1+ x