DIGITAL HOLOGRAPHY AND DIGITAL IMAGE PROCESSING

University of Alicante, Valencia, Spain, Dec. 21, 2004

Digital holography and image processing: twins born by the computer era Digital holography: - computer synthesis, analysis and simulation of wave fields Digital image processing: - digital image formation; - image perfection; - image enhancement for visual analysis; - image measurements and parameter estimation; - image storage & transmission; - image visualization

Digital reconstruction of of electronically recorded optical holograms Target location and tracking Multi-component image restoration Image re-sampling with discrete sincinterpolation Nonlinear (rank) filters for image restoration, enhancement and segmentation 3-D visualization

DIGITAL HOLOGRAPHY: numerical reconstruction of electronically recorded optical holograms

Digital Holographic/Interferometric Microscopy

One of the main drawbacks of microscopy: the higher is the spatial resolution, the lower is depth of focus. This problem can be resolved by holography. Holography is capable of recording 3-D information. Optical reconstruction is then possible with visual 3-D observation. Drawbacks of optical holography: -Intermediate step (photographic development of holograms) is needed. -Quantitative 3-D analysis requires bringing in additional facilities Radical solution: optical holography with hologram recording by electron means (digital photographic cameras) and digital reconstruction of holograms. This is the principle of digital holographic microscopy.
Beam spatial filter Collimator Lens Digital Photographic camera

Laser Object table

Microscope

Computer

Digital Reconstruction of Holograms (Equivalent optical setup)

Hologram

First focal plane

Fourier Plane

Second focal plane

Digital Holography: Digital Reconstruction of Holograms

M.A. Kronrod, N.S. Merzlyakov, L.P. Yaroslavsky, Reconstruction of a Hologram with a Computer, Soviet Physics-Technical Physics, v. 17, no. 2, 1972, p. 419 - 420

Hologram

Hologram sensor

Computer Analog-todigital conversion Preprocessing of digitized hologram Image reconstruction (DFT/DFrT) Image processing Output image

PSF, resolving power and speckle phenomena in reconstruction of electronically recorded optical holograms Problems: How the point spread function of the reconstruction process depends on parameters of the optical set-up and recording camera Resolving power of the reconstruction process Potential accuracy in measuring phase component of the object wavefront Statistical properties of the reconstruction speckle noise

Discrete representation of optical transforms: Discrete Fourier Transforms

Continuos signal
a(x) u

L.P. Yaroslavsky, Shifted Discrete Fourier Transforms, In: Digital Signal Processing, Ed. by V. Cappellini, and A. G. Constantinides, Avademic Press, London, 1980, p. 69- 74.

Sampled signal
x

a( x ) =

N 1 k =0

ak sign _ reconstr ( x (k + u )x )
Sampled signal spectrum

Continuous signal spectrum

( f )

v
f

( f ) =
f

N 1 k =0

r spn _ reconstr ( f (r + v )f )
Shifted DFT (canonic form)
N 1 k =0

Fourier integral

Signal and spectrum sampling

( f ) = a ( x )exp(i 2fx )dx

u ,v r

1 = N

ak exp i 2

(k + u )(r + v ) N N = 1 / x f

Direct and inverse Shifted DFTs (reduced form)

ru ,v =
1

(k + u )r kv a k exp i 2 N exp i 2 N N k =0
N 1

ru ,v =

1 N

N 1 k =0

ru k (r + v ) exp i 2 exp i 2 N N

Discrete representation of optical transforms: Discrete Fresnel Transforms

Integral Fresnel Transform:

( x f )2 dx ( f ) = a ( x )exp i D

frinc ( N ; q; r ) =
1 0.9

2005
1 N
N 1 k =0

kr exp(iqk )exp i 2 N
2

{ k ( x ) = sinc[ ( x (k + u)x ) / x ]}

Signal and its transform discretization with shift basis functions

q=0 0.8 0.7 0.6 0.5 q=0.0001

{ r ( f ) = sinc[ ( f (r + v )f ) / f ]}
Discrete Fresnel Transforms

r , w =
ak ,w =

1 N 1 N

N 1 k =0

a
N 1 k =0

exp i (k r / + w ) / N
2 2

0.4 0.3 0.2 0.1 0 q=0.0002 q=0.0004 q=0.0008 q=0.0016

exp{i (k r / + w )
r

/N

N = 1 / xfD 2

= (f / x )

1/ 2

w is the overall shift w=u-v/

50 100 200 A 0pictorial representation of the 150 discrete frinc-function 250 (DFT of the chirp-function). Amplitude is shown in green, phase values of function frinc ; ; r for of the Absolutein red. Horizontal coordinateN qargument N=256 and function; vertical coordinate focusing parameter. Binary different focusing parameter q random noise is added just for fun.

Point spread function of numerical reconstruction of electronically recorded optical holograms

Digital reconstruction of samples of the object wave front amplitude from samples of its hologram is treated as a process of sampling the object wave front. Signal sampling is a linear transformation that is fully specified by its point spread function:

ak = a ( x )PSF ( x , k )dx
X

Wave front samples

Objects wave front

Sampling device Point Spread Function

According to the sampling theorem, ideal sampling PSF is sinc-function

PSF (k , x ) = sinc[ ( x kx ) x ] sin[ ( x kx ) x ] = [ ( x kx ) x ]

PSF of reconstruction of holograms recorded in far diffraction zone spread functions of digital reconstruction of digitally recorded holograms, L. Yaroslavsky, F. Zhang, I. Yamaguchi, Point
In: Photonics Asia conference on Information Optics and Photonics Technology, 8-12 November 2004, Beijing, China

For hologram recording in far diffraction zone, wave propagation kernel WP(x,f) is :
WP ( x , f ) = exp( i 2 xf ) Z

Assume that, for hologram reconstruction, shifted and scaled DFT is used with the reconstruction kernel:
DR( k , r ) = 1 k (r + vT ) exp i 2 N N

where vT and are shift and scale parameters With this reconstruction kernel, point spread function of the reconstruction process PSFFZ(x,k) is
x x N 1 k N 1 x ) exp i 2 v r + sincd N , x k vT + x Z 2 2 x xf x ) df is frequency response of the hologram sampling device and where d ( ) = d ( f ) exp ( i 2 Z Z sin( x ) x = Z S H = Z Nf sincd( N , x ) = N sin( x N ) PSF FZ ( x , k ) = d (

Define hologram discretization and reconstruction device coordinate system through the object coordinate system by choosing vr=vT=(N-1)/2. Then
PSF FZ ( x , k ) = d (
x x ) sincd N r , x k x Z

PSF of reconstruction of holograms recorded in far diffraction zone (ctnd)

As one can see from the equation,
PSF FZ ( x , k ) = d ( x ) sincd[N r , ( x kx ) x ] Z

The point spread function is a periodical function of k:

PSF FZ (k + gN r ) = ( 1)
g ( N r 1 )

PSF FZ (k );

(g is integer). It generates N samples of object wavefront masked by the frequency response of the hologram recording and sampling device, the samples being taken with discretization interval x/ = Z/ SH =Z/ Nf within the object size So= Z/ f. The case =1 corresponds to a cardinal reconstructed object wavefront sampled with discretization interval x= Z/ SH =Z/ Nf . When >1 , reconstructed discrete wavefront is -times over-sampled, or -times zoomed-in. One can show that in this case the reconstructed object wavefront is a discrete sinc-interpolated version of the cardinal one.

Discrete sinc-function

sin(x x ) sincd( N , x x ) = N sin( x Nx )

Discrete sinc-function is a discrete analog of the continuous sampling sinc-function, which is a point spread function of the ideal low-pass filter. As distinct from the sinc-function, discrete sinc-function is a periodical function with period Nx or 2Nx depending on whether N is an odd or an even number and its Fourier spectrum is a sampled version of the frequency response of the ideal low pass filter
N is an odd number N is an even number

Nx

2Nx

Reconstructed object area

Continuous (red dots) and discrete (blue line) sinc-functions for odd and even number of samples N Frequency response of the ideal low pass filter (red) and Fourier transform of the discrete sinc-function (blue)

Reconstructed object area

Statistical characterization of speckle noise in coherent imaging

L. Yaroslavs ky, A. Shefler, Statistical characterization of speckle noise in coherent imaging systems, in: Optical Measurement Systems for Industrial Inspection III, SPIEs Int. Symposium on Optical Metrology, 23-25 June 2003, Munich, Germany, W. Osten, K. Creath, M. Kujawinska, Eds., SPIE v. 5144, pp. 175-182
a) b)

Generating 2-D array of pseudo-random numbers that specify the phase component of the object wave front

Computing objects wave front

Simulating wave front propagation (DFT, DFrT) Introducing signal distortions: -Array size limitation -Dynamic range limitation -Quantization

2-D array that specifies amplitude component of the object wave front

c)

d)

Yes

Continue iterations ? No

Comparing reconstructed and initial wave fronts; computing and accumulation of noise statistical parameters Output data

Simulating wave front reconstruction (IDFT, IDFrT)

Computer model

Illustrative examples of simulated images: a) - original image; b) - image reconstructed in far diffraction zone from 0.9 of area of the wave front; c) - image reconstructed in far diffraction zone from 0.5 of area of the wave front; d) image reconstructed in far diffraction zone after limitation of the wave front orthogonal components in the range.

TARGET LOCATION AND TRACKING

Target localization in clutter in multi-component images

L. Yaroslavsky, Optimal target location in color and multi component images, Asian Journal of Physics, Vol. 8, No 3 (1999) 355-369

Object tracking in video sequencies: examples

Leonid P. Yaroslavsky, Ben-Zion Shaick Transform Oriented Image Processing Technology for Quantitative Analysis of Fetal Movements in Ultrasound Image Sequences. In: Signal Processing IX. Theories and Applications, Proceedings of Eusipco-98, Rhodes, Greece, 8-11 Sept., 1998, ed. By S. Theodorisdis, I. Pitas, A. Stouraitis, N. Kalouptsidis, Typorama Editions, 1998, p. 1745-1748

For details see http://www.eng.tau.ac.il/~yaro

Tracking fetus movements in Ultrasound movie

Face detection in complex images

Ben-Zion Shaick, L. Yaroslavsky, Object Localization Using Linear Adaptive Filters, 6th Fall Workshop, Vision, Modeling And Visualization 2001 (Vmv01), November 21-23, 2001, Stuttgart, GermanyStuttgart, Germany, November 21-23, 2001, pp. 11-17 http://www.eng.tau.ac.il/~yaro/Zion/ZionPhdSeminar4Web.pdf

The developed algorithm is capable of detecting, with high reliability, faces of varying size from minimum size of 12 pixels width and 15 pixels height to the maximum size of the input image. The face detection capability of the developed system was experimentally examined on two test databases of images of high and low quality. The detection rates 96% and 84% were achieved for these databases, respectively.

Face detection: two-stage algorithm

1st stage

Face

Non-face

Maybe face 2-nd stage

Facelikenonface

Nonface likeface

The non-face detection algorithm was proved to have non-face rejecting rate of ~99% and false alarm rate of 1.3% (faces wrongly rejected), thus leaving only 1% of the image area for subsequent thorough analysis by the multitemplate classification algorithm. The algorithm is fast and requires approximately 200 flops per pixel in an input image of 640480 pixels size.

Face detection: Face-like-non-face and non-face-like face data bases

Multi-template classification algorithms use a very large set of templates prepared for different target shapes and varying illumination conditions. The developed algorithms were trained using a specially created training database obtained by extending four face databases to 32,000 images and one non-face database to one million images by means of scaling and rotating database images. In particular, face, nonface, faces like clutter and clutter like faces templates were generated from these training databases.

MULTI COMPONENT IMAGE RESTORATION

Spatial/temporal adaptive linear filters

L. Yaroslavsky, Image Restoration, enhancement and target location with local adaptive filters, in:International Trends in Optics and Photonics, ICOIV, ed. by T.Asakura, Springer Verlag, 1999, pp. 111-127

3-D Local adaptive spatial-temporal filtering: denoising and deblurring thermal video (ctnd)

3-D Local adaptive spatial-temporal filtering: denoising and deblurring thermal video

L. Yaroslavsky, A. Stainman, B. Fishbain, Sh. Gepstein, Processing and Fusion of Thermal and Video Sequences for Terrestrial Long Distance Observation Systems Processing and Fusion of Thermal and Video Sequences for Terrestrial Long Distance Observation Systems, ISIF, Seventh International Conference on Information Fusion (FUSION 2004), Stockholm, Sweden, 28 June - 1 July 2004 .

Before

After
(5x5x5 DCT domain filtering)

IMAGE RESAMPLING: discrete sinc-interpolation

Image resampling and geometrical transformations: efficient discrete sinc-interpolation algorithms

Spline interpolation and discrete sinc-interpolation

Spectrum of DFT-error (green) and bicubic-error (red) Test random image

S pe c tra o f DFT-e rro r (le ft) and Beta11-e rro r (rig ht)

Spectrum of DFT-error (red) and Beta11 error (green)

Discrete sinc-interpolation for image resizing

Image (a) is reduced with reduction factor 77/256 and then magnified with magnification factor 256/77. b) bilinear interpolation; c) bicubic interpolation; d) nearest neighbor interpolation; e) sincd-interpolation 11x11; f) sincd interpolation 31x31

Discrete sinc-interpolation in DCT domain

L. Yaroslavsky, Boundary effect free and adaptive discrete sinc-interpolatioon, Applied Optics, 10 July 2003, v. 42, No. 20, p.4166-4175

Initial signal

(ak )

DCT
rDCT

DFT of zero-pad sincinterpolation kernel

re r

{
{ }
DCT r

( p )}

{r ( p )}

im r

( p )}

IDCT
+

IDST
-

p-shifted sinc-interpolated signal {ak + p }

Zooming an image fragment (left) by sinc-interpolation in DFT domain (right upper image) and in DCT domain (right bottom image). Oscillations due to boundary effects that are clearly seen in DFT-interpolated image completely disappear in DCT-interpolated image.

Sliding window sinc-interpolation DCT domain: Simultaneous image

resampling and restoration/enhancement

Input signal

Inverse DCT/DST Output signal Computing sliding window DCT Introducing p-shift Modification of the spectrum for restoration/ enhancement

Sliding window sinc-interpolation in DCT domain: signal resampling and denoising

a)

b)
Noisy image (a) and a result of the rotation and denoising with sliding window DCT sinc-interpolation and denoising (b).

Sliding window sinc-interpolation DCT domain: local adaptive interpolation

Discrete DCT domain sinc-interpolator

Input signal

Computing sliding window DCT

Analysis of local spectrum

Mixer

Output signal

Nearest neighbor interpolator

Adaptive versus non adaptive signal interpolation

Signal (upper plot) shift by non-adaptive (middle plot) and adaptive (bottom plot) sliding window DCT sinc-interpolation. One can notice disappearance of oscillations at the edges of rectangle impulses when interpolation is adaptive.

Comparison of nearest neighbor, linear, bicubic spline and adaptive sliding window sinc interpolation methods for zooming a digital signal (From left to right, from top to bottom: Continuous signal; initial sampled signal; nearest neighbor -interpolated signal ; linearly interpolated signal; cubic spline -interpolated signal; sliding window sinc-interpolated signal ).

Image rotation with adaptive and non-adaptive discrete sinc interpolation

Direct Fourier method for inverse Radon transform: polar-to- Cartesian coordinate spectrum conversion with discrete sinc-interpolation
Object Projections Projection spectra Interpolated 2-D object spectrum

Reconstructed image

Fourier method for inverse Radon transform: polar-to- Cartesian coordinate spectrum conversion by means of zooming with variable zooming factor
(L. P. Yaroslavsky, Y. Chernobrodov, Sinc-interpolation methods for Direct Fourier Tomographic Reconstruction, 3-d Int. Symposium, Image and Signal Processing and Analysis, Sept. 18-20, 2003, Rome, Italy)

radius
Initial Polar Grid

angle

1-D spectra of projections

Target Cartesian Grid

Polar Grid after 2 time zooming

Initial Polar Grid

Target Cartesian Grid

a
r1
r2

b
??

Stabilization and restoration of atmospheric turbulent video

S. Gepshtein, A. Shtainman, B. Fishbain, and L.P. Yaroslavsky, Restoration of atmospheric turbulent video containing real motion using rank filtering and elastic image registration, Eusipco2004, Vienna, Austria, Sept. 2004

Restored stabilized video with moving objects unaffected

Moving objects

Initial video

Turbulent atmosphere video

Stabilized turbulent atmosphere video

NONLINEAR (RANK) FILTERS for image restoration, enhancement and segmentation

Image restoration and enhancement: nonlinear filters

L. Yaroslavsky, Nonlinear Filters for Image Processing in Neuromorphic Parallel Networks, Optical Memory and Neural Networks, v. 12, No. 1, 2003 B. Hirshl, L. Yaroslavsky, FPGA implementation of sorters for non-linear filters, EUSPCO 2004, Vienna, Austria, Sept. 2004

Noisy image, stdev = 20, Pn=0.15

Iterative SCSigma-filter . Wind. 5x5, Evpl=Evmn=15; 5 iterations

Nonlinear filters: Image enhancement

Initial image

SIZE(Evnbh(Wnbh5x5,2,2))-filter

HIST(W-nbh)-filter

Nonlinear Filters: Image Enhancement

Local histogram equalization: Wnbh, EV-nbh and KNV-nbh

Initial image

RANK(Wnbh15x15)

RANK(KNV (Wnbh15x15;113))

RANK(EV (Wnbh15x15;10,10))

Local P-histogram equalization: color images (blind calibration of CCD-camera images)

3-D VISUALIZATION: improved anaglyph method and 3-D video from 2-D video

Computer synthesis and display of stereoscopic images

I. Ideses, L. Yaroslavsky, Efficient Compression and Synthesis of Stereoscopic Video, 2nd IASTED Internat. Conference, Visualization, Imaging and Image Processing (VIIP 2002), Sept. 9-12, 2002, Malaga, Spain, Ed. J.J. Villanueva, Acta Press, Anaheim, Calgary, Zurich, 2002, pp. 191-194 (ISBN: 0-88986-354-3; ISSN: 1482-7921) http://www.eng.tau.ac.il/~yaro/Ideses/malca1.html

Bahai garden, Haifa, Israel

Roman Querry, Tarragona, Catalonia

Computer generated stereo from 2-D video

I. Ideses, L. Yaroslavsky, A Method for Generating 3D Video from a Single Video Stream, in: G. Greiner, H. Niemann, T. Ertl, B. Girod, H.-P. Seidel (Eds.), Vision Modeling and Visualization 2002, Proceedings, Nov. 20-22, 2002, Erlangen, Germany, Academishe Verlagsgesellschaft Aka GmbH, Berlin, 2002, ISBN 3-89838-034-3, ISBN 158603-302-6, p. 435-439

Current projects:
Multi-component image processing:
Restoration and fusion of atmospheric turbulent, thermal and visible range video Super-resolution from video sequencies 3-D color display and artificial 3-D from video Moving object detection and tracking

Digital holography:
Point spread functions and resolving power of digital reconstruction of near and far zone holograms Theory and new algorithms for fast transforms

L. Yaroslavsky,
Ph.D., Dr. Sc. Phys&Math, Professor Dept. of Interdisciplinary Studies, Faculty of Engineering, Tel Aviv University, Tel Aviv, Israel www.eng.tau.ac.il/~yaro