The field af multirate and wavelet signal processing &ads applications in
speech and image compression, the digital audio and digital video indus- tries, adaptive signal processing, and in many other applications. The utilization of muleirate techniquw is becoming an. ind-irspensable tool of the electrical engineering profession, This point can be illustrated in three ways, First, if a performace specification is controlling the design of a particular system, that is, the performance specification exewds the eur- rent state-of-art , then by converting the syskm to a multirate system, the overall system specification can be met with slower camponents. Secondly; if the dol l ~r cost specification is controllng the design of a particular sys- tern, that is, the design of a competitive commercial system where bottom line cost is most important;, then by converding the system to a multirate system, the overall system cost will be reduced through the utilization of slower, cheaper devices. Thirdly, if power czansumption, is eoxlCrofling the design of a particular system, that is, the design of a hand-held system powered by a couple AA batteries, or possibly a satellite system, then by converting the system to a muleirate system will reduce power consump- tion through the utilization of devices with slower switching speed, and zls a result, lower power dissipation. Wavelet transforms are closely related to fitter banks. As such, a back- ground in filter banks will make it eaier for the reader to understand, design, and implement wavelet transforms. Many of the most important applications, such as video compression, and many challenging research problems are in the area of multidimensional multirate. As such, multidimensional multirate is integrated throughout the book, The focus of this book is to present a sound theoretical foundation by emphasizing the general principfes of multirate. This book is setf-contained for readers who haw some prior exposure to linear algebra (at the level of Horn and Johnson's Matrix Analysjs) and multidimensional signal process- ing (at the level of Lirn's Two-Dimensional Signal and Image Processing or Dudgeon md Mersereau's MuItr"dimmsiond Digitd Sign& Processing). xi i Preface Moreover, this text will bring the reader to a point where helshe can read, understand, and apprecii%tie the vast muftirate literature. The organization of this book is as fallows. The first two chapters are de- voted t o basic multirde ideas including decirnators, expanders, polyphae notation, etc. This presentation is first given for one-dimensional signals in Chapter 1 and then generalized to multidimensional signals in Chapter 2. The next two chapters deal with filter banks, Chapter 3 presents the theory of Pilter ba nk for both one-dimensional and multidimensional sig- nals. Chapter 4 deals with lattice structures, an, eEeiextt inzpEementation strategy for filter banks. Chapter 5 highlights an important applieaeion of mu1tir;tt;e -- the impEernentation of wavelets. I would also like to talre this opportunity to thank Professor Charles Chui for his enthusiasm about this projwt and for ineluding this text in his distinguished wavelet series. The hlXowing people have provided very useful feedback during the writing of this book. They include: Bill Cowan, Tom Folttz, Jerry Gerace, Ying Huang, You Jang, Matt Kabrisky, Mark Qxiey, b b e r d Parks, Juan. Vmquez, and Dan Zahirniak. Fairborn, Ohio February 9, 1997 Bsuee VV. Suter Chapter 1 Mult irat e Signal Processing Thics chapter provides the bai c concepts used in the study of multirate and wavelet signal processing. Some of the earliest contributions to the study of the fundamentals of multirate were due to Schafer and Rabiner[40], Meyer and Burrus[32], Oetken et a1.[37], and Crochiere and Rabiner[lO]. The idea of polyphme representation ie a key concept throughout the development of this book. This nontrivial idea war first articulated by Bellanger et a1.[3]. Much more recently, Evangalistaflq carefully examined another important idea - digitd comb filters, Many of the concepts developed in. this chapter are also discussed in the other multirate texts by Crochiere and Rabiner(ll1, Fliege[lS], Strang and Nguyen [46] and Vaid yanat han [49]. Section 1 2 presents a framework b r mtlltirate and it introduces two important representations for discrete signah. Section f -3 introduces the basic: building blocks. Section 1.4: provides ways to interchange the basic building blocks. Section 1.5 presents a filter bank example. 3.2 Foundations of md$ira;l;e First we will. examine some samplixlg considerations and then present some basic transforms for andyzing signah, haultirate is the study of time-varying systems. As such, the sampling rate will change at various points in time in an impIementation. This will require us t o wr y the gain (magnitude) of filters in series with the time- varying building blocks so that the raulting gain is consistent with what one would expect if the sampling internal after the time-varying block had 2 Chapter 1 Muleirate Signal Processing been the original sampled frequexlcy. Towards this end, let us axlalyze a train of impulses, jftnmt Theorem 1.2.1.1. xzD=_m6(t- kT) = $ x z = - me ~ p ( ). Proof: Let us expand Ego=_, 6( t - kT) in a Fourier series. So, 00 c G( t - k T ) = 5 a(m) exp where, or equivalently, Let T ==: t - kT, Then, We recognize this as a sum of integrals with adjoining limits artd simplify t a 1 Hence, m Let F denote the Fourier transform. So that if z( t ) is a signal, then Let us examine the Fourier trarlvform of an impulse train. rn Theorem 1.2.1.2. 7 [ ~ ~ m : _ o o 6( t - k T ) = + Cz==_, a(f - .). Praofi Rom the previous theorem, 00 6 ( t - k T ) = l I' 2 exp Therefore, + [C;"=-, 6 ( t - kT)] 1 c@ = FC,=-,6(f- m Now, performing the Fourier transform of a sum of the product of the input z(t ) and Dirac delta functions, which can be expressed as the convo- lution of the corresponding functions, produces = J-",x(f -f')+C:"=-,s = +CZ"=-,J-",x(f - f ' ) 6 = +XE"=_, X At this point, we will interpret these results for linear time-varying systems, If the sampling intewal is incremed by an; integer fwtor of M, where M > 1, then the magnitude of the Fourier transform will need t o be deerewed by a factor of & to reconstruct the original system, that is Similarly; if %he sampling interval is de c r e ~e d by an integer factor of E, where i; > I, then the magnitude of the Fourier transform will need to be increllsed by a factor of .L t s reconstruct the original sign&, that is For completeness, the definitian, of z-transfarms and the discrete Etsurier transforms will, be pwented. Then, we wilt present two sampled signal representations: the modulation representation and the polyphase repre- sentation. The theory of mutt irate and wavelet signal processing utilizes both af these representations Chapter 1 Muitirate Sign& Pracessing Befinition 1.2.2,1, The z-tf?amfown of a sequence z(n) is defined by An important property of x- transforms is the following scaling theorem. Theorem 1.2.2.1. Xlf the x-transform ofz eksts aad a is a sedar, then Proof: By defix~iticsn, or equivalently, QO n=-oa Hence, Xf the z-transform converges far aU. z of the form z = exgljw) for real w, then the z-transform can be represented as the sum af harmonicdly related which ics sometimes edled the discrete-time Fourier draasform. 1.2.2.2 Discrete Fomfer tramform Defini-Lian 3L.2.2.2. The di~ewrte F~uGer t r ~ n s f o m of a periodic sequence s(n) sf length .lV b given by. IV-l X( k ) = C z(n) exp gt=@ and the correapondiag inver~e discrete F0aTiier tmnsfim is @ven by f .2 Foundations of mrxltirate 5 Let = exp (q), the (principal) Mt h root of unity. Then, the discrete Fourier transform matrk, denoted WN, is an N x N mat rk, defined by [WNIk,% = WE = exp 1.2.23 ModuXa&iom representation DefiniGion 1.2.2.3. Given a, sequence s(n) and a positive integer M, then the components of the modulation representation of the r- transform of z( n) are defined X(ZW$), k = 0,1,. . . , M - 1. If M == 2, then the eomponen%a of the modulation representation arts X(z) and X(--z). The t e m modulation representation can be most eilvily visualized in the time domain. Using the scaling theorem of x-transfarms, we obtain ~ - ~ [ x ( z w ~ ) ] = f t is interesting to note that the components of the modulation represen- tation can be combined pairwise t o form a r ed signal, that is Definition 1.2.2.4. Given a sequence z(n) a d a positive integer M, then the Type-f polgphase contponents of as(%) are defined a- sk(n) = z( Nn+k) , k = O 9 l ? ' . ' , M- I . If M ;;: 2, then so(n) would be the even-numbered ~ampl es and zl (n) would be the odd-numbered sarriples, Now, let us investigate the z-transform of the Type-$ poXypXlwe components, that is or equivalently, then X( z ) becomes The success af this decomposition rests on its usage of the well-known Division Theorem for Integers, whidl says any integer p can be represented it5 p ; z =Mn+k wfiere, the quotient n and the remainder It;: are unique integers. Definition 1.2.2.6. Give11 a sequence z(n) and a positive integer M, its Type-lsr polgphase componenfs of z ( n) are given by a&(%) = z ( Mn + M - 1 - k ) , k==0, ..., M - 1" The ttransforrn of z(n) written in the Type-I1 polyphme representation is given by or equivalently, Let then It is also important t o natc that Type-Xf pdyphase components are iden- tical t o Type-X polyphme components - only the indexing i s performed in reverse arder. 1.3 Bmie building Mock8 Figure 1.1. L-fold apmdler. This section defines and analyzes the two types of bwic building blocks used in multirate signal processing. One type of building block deals with changes in ampl ling rate of the input sign& and the other type of building block deds with changes in filter length, We will first discuss two types sf building blocks which change the sampling rate of the input signal -- decilna~tors t o reduce it and ezpanders to increme it. Secondly, another type of building block is discussed which changes the length of a given filter - comb filters to increme its length. In this way, a comb filter version of a given filter is andogoas to m expander acting on an input signd. Definition 1.3.1.1, Let L be a yoYifive hteger, An L-fold ezpander (&a known as an upsampler) applied to an input signal z(n) inserts L - 1 zeros between adjacent ssrupfes of the i npd sign&. For a samp2ed sign& a(%), the outgue of dhe expander is givm by An L-fold expander is depicted pictoridly in Figure 1.1. Let us consider the following example to gain some intuition into the behavior of expanders. If L == 2, then the inputs and outputts of the expander me given by: The operation. of expansion is invertible, or in other wards, it is possible to recover i ( n) from samples of yB(n). We will now analyze the behavior of the &-fold =pander by taking a %transform of it, that i~ 8 Chapter 1 Mal tirage Sigad Proce~sing By the definition of the expander, gE(n) = 0 if n is nat ia multiple of L, Therefore, YB( Z) = ~E( %) z - * . Let n == KL. Then, CIQ By the definitian of the L-fold expander, zlk) - yls(kl;). Hence, aie equivalently, YE(.) = x(st). In the Iiterature, the ~ot at i on is oecasiondly used* If the region of convergence of YEf z) includes the points of the form z = exp(jccr), where w is real? then WE: can replace z with. e x pl j w) to yield ra?(expfjw)> = X(exp(PLw)). By suppressing the exponentid and, a such, changing the notation so that &( w) denotes &(exp(jw)), then vpe will write Therefore, YE( w) = X(Lw) means that a given frequency wo in X(w) is transformed to a new frequency in YE(@). A8 a r e~ul t af this transfor- mation L - 1 unmnted image spectra of the input signd s pe~t r a appeared after each of the rzrigiad spectra at the output of the expander, This suggests the use of a lowpass filter with cutoff frequency f immediately following the expmder ZM illustrated in Figure 1.2. This Iowparrss filter i9 called an anti-imaging filter. RecaU from the previous section, if the sampling intervztl. i s decrewed by a factor of L, then the product, of the pi ne of the expander and the anti-im+ag filter must equal L. Sinee the expander has unity gain, the anti-imaging filter must Chapter 1 Mul tira t e Signal Processing Figure f .4. Spectrum of @waded signai. Figure 1.5. Fitter to recover input signal, 1.3 Basic building block8 Figure 1.6, M-fold decirn8tor. The operation of decimation is not invertible, or in other wordpl, it is not possible to recover z(n) from yo(n). We will now analyze the behavior of the M-fold decimator by taking a z-transform af it, that i~ or equivalently, At this point we can not make the substitution k: = Mn and proceed with a frequency domain expression for YD(z). Why? Because s(n) is not zero for noninteger multiples of Mn. So define an intermediate sequence that is zero for noninteger multiples of Mn, that is a;(%), where n is a multipfe of M XI(%) -- 0, otherwise. Therefore, at the decimated sample values yo(%) =: %( Ma) = XI f Mn). Hence, DD Y'(z) = zl(Mn)z-". lt=-X3 Led b = Mn. Then, Now, we need to express X1 (1) in terms of X(z). By the definition of sl(n), we can write 21 (a) = CM (a)%(%) 12 Chapter 1 Mu1f;ir;st;e Sign& Proces~ing where, clle(nj is a sampling fuwtion and is defined by 1, whenever n is a multiple of M C M ( ~ ) = Q, otherwise. So, for example, if 1M = 3 then cM(n) becomes Mathematically, vve say that the sampling function cM(n) i s a sequence of period M, which can be repres-ented by the Fourier serim expansion N-' cM (n) = - C C(k) exp M I @=@ where, the Fourier series coemcient C( k) is defined by on the interval (0, M) with samples at n = 0, ..., M - 1. a(%) is defined by 1, wh e n n = 0 a( n) = 0, otherwise. So, CP) = 1 far all k. Thus, Since zl (n) = cM(n)z(n), then the z-transform of zt ( n) is given by Substituting the equation for eM( a) into the laad equation yields 11.3 Basic building I;rIor=ks or equivalently, But Yo(z) = XI (z'/~). Hence, or equivalently, M- 1 Y,(,) = - C X M k=O where WM = exp(+) is the (principle) Mth root of unity. The &th power af z means that the original spectrum is stretched by a factor of M. In the literature, the notation i s occasionally used. If the region of convergence of YD(z) includes the points of the form a = expf j w) where w i s real, then we can replace z with exp( j u) to yield By s~ppressing the expanenti& and, as such, cha~ghg the natation so that YD(w) denoteg YD[exp(j w)J, then we will write Hence, a frequency wa in X(w) i s transformed to M new frequencies (wo - 2nlc)M, ic = 0, ..., M - 1, in YD(w). Now, consider the response y (R) of B decimator to an input sequence z(n), where every other sample is zero. This is simply an example af an intermediate qzxence with decimation by 2. So, YD(Z) = x(z=j2). But using the analysis of the decimator for an arbitrary input r(n), we can say that Chapter 1 Muldirade Signal Processing Figure 1.7. Spectrum of input ~i gnd. In the first cac, eveq other input sample wzs zero valued. Hence, the support of X( w) is limited t o -: 5 w < 5. Under these conditions, decimatio~l by 2 will not create aliasing, that is x( - z' / ~) = X ( Z ' / ~ ) - Therefore, the analysis is consistent for a signal bandlimited to -f < w < f. For this example, if the spectrum of the bandlimited input signal is given by Figure 1.7 then the corresponding spectrum of the output, of the decirnator is given by Figure 1.8, This bandlimited signal could be realized by the use of a lowpass filter with cut-off frequency & immediately proceeding the decimator as illustrated in Figure 1.9. This lowpass filter is known as an anti-aliosing filter. Recall from Sec- tion 1.2, if the sampfixlg interval is increased by a factor of M, then the product of the gains of the decirnator and the anti-aliwing filter m s t equal &. Since the decimator has a gain of &, the anti-aliasing filter must have unity gain. It will be shown later in this chapter that these operators, M-fold deei- mators and &-fold expanders, are conlntutative proGded that L and M arc relatively prime. Deftmitian 1.3.3.1. Given the impulse respunse h(n) of a filter, one ca~l build a comb filter g( n) by inserting L - 1 zeros be tween the coeEcien ts of Figure 1.8. Spectrum of decimated signat. Figure 1.9. Decimator with mt i - di ~i ng filter. 1.3 1S2ilsr"e building blockg 17 and tldditions required. The following example demonstrates the lower costs achievable with an lFXR filter implementation. Example 1.3.3.1. Suppose we wish Is design a linear phme FIR filter with specifications: peak passband ripple: = 0.001 peak stopband ripple: 62 = 0.001 passband edge: wp = 0. 01~ stopband edge: c ~ , -- 0. 02~. The normalized transition bandwidth, A f , is given by: Then, we can estimate the filter order by: So. The number sf multiplications required is @yen by and the number of additions required is given by Let us first peak passband ripple: = % = 0.0005 peak stopband ripple: b2, = b2 = 0.001 passband edge: up, = (0.01n)15 = 0 . 1 5 ~ stopband edge: w,, = (8, 02~)15 == 0 . 3 ~. Again consider an equiripple design. The normalized transition bandwidth, (A f ) , , is given by: 28 Chapter 1 Multirate Signal Processing Then, we can estimate the filter order by: The number of mubiplications required far G( z) is given by and the number of additions required is gima by (# adds)G = NG = 46. Let us now consider the filter I(s). Then, its specifications become peak passband ripple: 6%, = 2 = 0.0005 peak stopband ripple: b2, r= Sz = 0.001 passbaztd edge: wp, =" 0.01 T stopband edge: w,, = - 0 . 0 2 ~ = 0. 113~. Considering ark equiripple design, the rrormafized transition bandwidth, (A f ), , is given by: - w,, - wP, - 0.103?r [ Af l l - - =. 0.0515. 2% 2n Then, we em estimate the Alter order for I ( z ) by: Z'he number of multiplications required far I ( z ) is given by Figure f . If. Conversion from studio work to CD mwtering. and the number of aidditions required is given by Hence, (# multiplies)lFIR = (# r n~l t i pl i es) ~ + (# m~l t i pl i es) ~ = 113 (# = (# adds)G + (# adds)l = 58. So, using the IFIR design technique, cskn s i ei kant l y deerease the com- putatianal complexity over canventional equiripfle filter design. 2.4 Interchanging bslsic building blacks This section. first presents tihe conditions for interchanging decirnators a ~l d exprtnders, Secondly, we consider the noble identities, an appraxh for interchanging filters with. the basic building blocks for decimation, and ex- pansion. 31.4.1 f nterchangiurg deelmat ors and expanderrs Sampling rate conversion is important h r many signal processing appli- cations. For examphe, conversion between the three standards for digital audio, which are 48KHz for studio work, 44.1KHz for CD mwtering (both digital tape and cornpnct discs), and 32 KHz for broadcasting digital audio. Ta convert from tbe studio work frequency to the CD mastering frequency, one could usc: the arrangement in Fip;ure 1.11. Thus, sampling rake conver- sion will require the cmcade of an &-fold expander and an M-fold decirna- tor, separated by a filter. Thus, by emcading an &-fold expander and an M-fold decirnator, one obtains a ra$ionat sampling rate change with rate &/Me Under what conditions can the decinaator and the expander be inter- changed? First consider the configuration in which the expander proceeds the decimator as depicted in Figure L12, Writing the wsaciated elemental equations yields Figure 1.12. L-fold expader proceeding M-fold decimakor, Figure 1.13. M-fold deeimator proceeding L-fold expmder, Hence, Second, consider the configuration in which the decimator proceeds the expander as depicted in Figure 1.13. Writing the aysociat;ed elemental equations yields and Therefore, an L-fold expander can be interchanged with an M-fold dec- intdor if and only if Yl(x) = G( x) . By inspection, this vvill only occur ), k = 0, ..., M - 1, generates the Mt h roots of unity. The following theorem provides the coxrditions on l; and M for this t a occur. Theorem 1.4.1.1. The set (exp(- )Ik: =t 0, ..-,A4 - 1) equiblfs the set of the M th roots of unit& if and only. if l; and M itre rda tivejy prime, Ghat i s gcd(L, M) += 1. Proof: Case 1: Assume exp(- ), k = 0, ..., M - I, generatea dl the MLh roots of unity. Assume, for contradiction purposes, that gcd(k, M) =: d > I. Then, there e;x;ist relatively prime chi that L = dL1 and M = dMl . Hence, exp(- . But, ) wiU generat;e at most IM; mots of unity, Since A/I; < M, a eoatradictlon occurs. Catse 2: Assume 2; and M are refatiwly prime. Assume, for cor~tradic- tion purposes, that expf - ), k = 0, ..., M - 1, generates MI < M roots of unity, where M = %MI for some relatively prime positive integers el and J1. Then, exp(- )), Since k can take on aUI values from 0 t o M - I an 1, then el [ L. Since ell L and el M, then gcd(L, M ) = el. Therefore, a contradiction occurs. I He~~ce, an &-fold expander and an M-fald decimalor commute provided I/ and M are relatively prime, Let us illustrate this result with a couple 2 and M =. 3, then gcd(2,3) = 1. Thus, exp(- ) , So, Eor k -- @,I, 2 we have t be following t e ms exp(-j0) =: I; exp ) becomes exp(- ). So, for k = 0,1,2 vve have the following terms Therefore, as expected, the two expressions are equivalent. Now, as a, counterexample suppose f; = 2 and M = 4, then gcd(2,4) = ) becomes exp(- j xk) , So, for Ic .- 0,1,2,3 we have the following terms ) becomes exp(-?). So, for k = 0,1,2,3 we have the following terms exp(- j0) = 1; exp = - j ; exp(- jn) = -1; exp Chapter 1 Muidirate Sign& Processing Figure 1.14. A more complicated example, Figure 1.16. Equivdent version of Figure 1.14, Thus, as expected, these two expressions are not equivalent. Now, let us consider a skghtly more complicated example, which is depicted in Figure 1.14, Find an expression for y (lz) in terms of %(a). Applying Theorem 1,4.1,1, we can interchange the first two blocks, since the expansion ratio of 5 is relatively prime to the decimation ratio of 2. This will yield Figure 1.15, Now, consider the two middle blocks afs depicted in Figare 1.16, Then, or equivalently, * 4 T( z ) = C S (t) h=O So, we can eliminate the two middle blocks without effecting the output;, Hence, Figure 1. 17 shows the simplified version of Figure 1.14. Figure 1.16. Middle Blocks. Figure 1.17. Simplified version of Figure 1.24. Figure 1.18. Expwder falowed by inherpolation filter. By inspection, w(n> = ~ ( 2 %) (f) for n. even for n odd, Therefore, ~ ( n ) for tn even O for n odd. In, mast applications involving interpolation, an interpolation filter follows an expander as in Figure 1.18, The fitter must handle data at an increwed sampling rate of I;. We would like t a find ways to reduce the speed at which the filter oper&es. One way to do this is to interchange the expander and the filter in an equivalent system ars in Figure 1.19. Figure 1.19. Filter proceeding expander, Figure: 1.20. Decimation filter followed by decimat~r. Figure 1.21. Decimator proceeding filter. Let us di~cuss the conditions under which, they me equivalent, Similarly, in. many applications involving decimators, a decimitt;ion 6lter precedes the deeimator sls in Figure 1.20. Analogously, we mufd like to allovv W( z ) t a operrtte at the slower rate of the decimator. 1s there any equivalent system to allow this intercl~antge? What must G( z) be for the system depicted in Figure f .21? If H( z ) is a comb filter, then can the buading blocks be interchanged? . Interchanging Alters and expanders Consider the con6guration depicted in Figure 1.22, Writing the elemental, equations, we obtain V( z ) = x(zh) and Y( z ) = G( z h ) ~ ( z ) . Wence, Y( z) = G ( ~ ' ) X ( Z ~ ) * But, this equation could also be interpreted zw Figure 1.23. These two equivalent block diagrams in Figure 1.24 present iz systeraakic approach for Figure 1X.Z2. Expander followed by comb filter, Figure 1.23.. Filter proceeding L-faId e vade r . Figure 1.24. Noble identity for expaders. interchanging filters with expanders a d tagether they will be referred to zw the noble identity for expanders. Please note that the noble identiey far expanders does mot, in general, apply if G( z ) is irrational. 1.4.2.2 Inter ging filters and deeimators Conisider the configuration depicted in Figure 1.25. Writing the elemental equations, V( L) = G ( z M ) ~ ( z ) and Hence, Chapter I MuIthtte Signal Processr'ng Figuire 1.26, DecimaBoir pmoeeding filter. Figure 1.27. Noble idmtity far deciaaatsrs. But this equation could be interpreted aa Figure 1.26. These two equivalent block diqrams, depicted in Figure 1.27, present a, systematic approach for ixzterchanging filters with decirnatars and together they will be referred to m the noble ideneiGy for decirnatczrs. Piewe note that the noble identity for decinrators does not, in general, apply if G( z ) is irrational, 1.5 A filtar bank: example Consider the following example depicted in Figure 1.28 to gain sorne intu- ition into the importance of polyphase and decimators and expanders for multirate filter banks. As part of t hk example, let us examine the inputs, autputs, and inter- mediate results for this 8im;pXe filter bank. For this exaraple, let us assume that the first nonzero input sample to the filter bank will be designaed s(O). Then, 1.6 Problems Figure 1.28, A simple filter Bank, Thus, the upper declrnator pases only even-numbered samples, while the lower decimalor passes only odd-numbered samples. Therefore, the outputs of the decimators correspond t o the polyphwe components of the filter bank, 1.6 Problems 1. Let an input signal z(n) be periodic with period N, t hat is, z(n) = rc(n + MI. Let; y (n) be an M-fold decimated version af $(a), t hat is, y (n) =. z ( Mn) , Shovv that y(n) is periodic with some period 8, t hat is, ~ ( n ) =r ~ ( n + P). What is the smallest value of P in terms af M &nd N? 2, In the systems depicted ixr Figures 1-12 and 1.13, derive the sequence domairk equations for their outputs in terms of their input, t hat is, yl (n) and yz(n) in t e ms of re(n). Then, prove that these two ex- pressions yield the same result if and only if .L and M are relatively prime, 3. Prove the following: 4. Let the input 4%) be defined by s(n) =; &,,. Apply this input t o a decirnator and show that the decbat or could be interpreted as a periodic time-invariant systerr~, 5. Some authors have proposed a Type-Iff polyphme representation us- ing the fdlouring definition: Given a sequence z(n), then its Qpe-IXI polyphase components are defined ;zs Chapter d Maltirate Sign& Procesahg How are the Type-f and Type-I1 polyphme components related Z;a Type-111 polyphase companents"! 6. fn example 1.3.3.1, we compared the complekly of a conventional equiripph filter design with an XFXR desip with a stretching h t o r of 15, Repeat the analysis far a stretching f a~t or of 40. Chapter 2 Multidimensional Mult irat e Signal Processing 2,1 Introduction This chapter extends the basic concepts of rnultirate signal proce~sing to multidimensional multirate signal processing. Examples of multidimen- sional signals include images in two dimensions and video in three dimen- sions, Good reference texts for bwkground materid on. multidimensional signal promwing are Dudgeon and Mermresu[lfjj and Lim[26]. The im- portant concept of sampling is related to the mathematics of lattices; see for example Cassels[41. The engineering analysis of sampling begari in the classic paper by Petersen and Middleton[38] and later extended to include decimation and expansion considerations by Mersereau and Speake[31] and Dubois[ltij. The multidimensional z-trclrrsform is carefully described by Viscito and Allebrzeh[55) The idea of the Smith form was first articulated by Smithpq, Many recent papers have dealt with applications of the Smith form to the multidimensional DFT (Guessoum and MersereaupO]), to the multidimensional DCT (Giindgzhan et a1.[21]), and for the development of multirate CAD toolo (Evansf28I). Some of the concept8 dewloped in this chapter are also discussed in the text by Vaidyanathan[49]. Section 2.2 presents a framework for the study of multidimensional ml - tira;te signd processing and it introduces two important representations far multidimensional signals. Section. 2.3 presents the multibirnensional build- ing bloehs. Section 2.4 provides ways to interchange the mltidimensiorral building blocks. A multidimnsional signal is a signal of more than one variable. This sec- tion systematically presents concepts that act as s kamework for our study 363 Chapter 2 Muftidimensionaf Muftirate Signal Processing of the application of multirate s p t e m~ to multidimensional signals, These canceF>ts include sampling of a multidimensional signal, which involvw tbe mathematical concept of a sampling lattice, and introduce multidimen- sional sampled signals by way of multidimensional ;r-tramforms and mul- tidirnensiclnal Fourier transforms.. Now, we will discuss the mathematics needed t o describe the s~mgling considerations of multidimensianal multi- rate. 2.3; .I Sampling lattices Sampling a multidirnemional signal is more complicated than a one-dimen- siorral signd because of the many ways t o choose the sampling geometry* Sampling points could be arranged on a rectangular grid in a straight- forward manner, but many times there are more efZicient ways to sample mu1 tidimensional signals, Some applications are mare suitalble for nonrectangular sampling than rectangular samplng. F"or example, the testing of a phared-array antenna rquires the measurement of the electric field on a plane in the near field of the antenna. Since phased-array antennm are typically designed with an hexagand arrangement of efennends, the resulting processing is mare wcurato if hexagonal sampling is used for the memurements. One reman to consider nonrectangular sampling is Lo minimize the number of points rtwded to charwterbe an nil-dimensional hypervolume. Moreover, if the functions of interest are bandlimited aver a circular region, then Figure 2.1 shows that significant savings are possible if hexagonal sampling is used instead of rwtangular sampling. In order to precisely describe both rectangular and nsnrectangulw sampling, we nwd a convenient way to describe an arbitrary sampling geometry. To do this activity9 we must appeal to the laxlguage af linear akgebra in order t o present the mathematical theory of sampling lattices. 2.2.1.1 Linear independence and sampling lattices The idea of a vector in an integer-valued k-dimensional space is a general- ization of vectors in a plane by ixl6eger-tralrred Cartesian coordinate. This leads to the foliowing definition. Definition 2.2.1 .I. The set of a11 le-dirnensio~a1 integer vectors wjfl be cdIed the fgndamental lattice and it will be denoted N. That is, T N = {r = [rl , rz, . . . ,rkj rI is an integer). The set of all k-dimensional real vectors will be denoted 8. Mow, let us review a few definitions related to sums of vectors. 2.2 Multidim czngianal frame work Signal Gtim-snsionaliQ Figure 2.1. Percent swings: hexagon& versus rectangulw sampling. Deftnition 2.2.1.2. A linear combination of N vectors {rl, . . . rN) E hi is an expression of the form N where %, 1: = 1, . . . , N, are integer8 and arc called cocfticients. The set of vectors {rl,. . . r ~ ) L4 aid t o be linearly independent if Cni r i = * ni = 0 for all ii. i= 2 I f the set of vectors {rl , . . . rN), is linearly independent, then the totality of vectors of the f orr~ is called a N-dimensional lattice and it will be denoted by R. IB other words, the space R spaaned by t he set of vectors {rl , . . . ~ J J ) is the space consisting of all h e a r combinations of the vectors, that is, The set; 08 vectors (rl, . . . rN) Is a hsi s for R if dhey are linearly indepers- dent and the g p x e is spanned by {rl, . . . rlv) is egud to R. Then, we say that R has dimension N. To better understand the definition of the lattice, consider the following illustration in two dimensions, Let TI 1 T12 a1 r1 == , r2 " and n, == T21 c22 7% Then, or equivalently., nlrll t nzr12 72.1r1 + 7L212 = nl Tzl + rb2T22 This can be rewritten as a matrk-vector product, i.e, where the matrix R i s cdallfed the sampl i ~g matris, In general, let ri be the ith column of the matrix IR, that is then the sampling miaLrix R is said to generate the lattice R. As such, the lattice R is dso given by R=LAT(R) = { ~ E N I m=Rn for n EN 1. If R is the identity matrix, then each ri is a unit vector painting in the ith direction, and the resulting lattice, 'R, is the fundamental lattice N. Let us present some examples of sampling lattices usirlg bf zk dots to represent the lattice points and white circles to represent points in N that are not in LAT(R). Example 2.2.1.1. Consider rectangular sampling defined by the sampling 2 0 rnatraX R = . It i s depicted in Figure 22. 0 3 Figure 2.2. Lat tiee structure using rectangular sampling matrix. Figure 2.3. Lattice structure using hexagand ;tlmpling matrix, Chapter 2 Multi&rnensr"o~a1 Mulgirate Signal Processing Figure 2.4, Latt;ice structure with quincunx smgfing matrix. Example 2.2.1.2. ... Consider .. hexagonal sampling defined by the sampling 1 1 matrjac R .=: . It is depicted in Figure 2.3, 2 -2 Example 2,2.i ,3. Consider quincunx sampling defined by t he sampliag 1 1 matrix R -- . It is depicted in Figure 2.4, -1 1 Far a given sampling matrix R, the corresponding Fourier domain sam- pling matrix is ~ T R - ~ and the lattice it generates is called the reciprocal lattice. The mat r k that generates a, la~ttice is not unique, AS we wiH gee later in this subsection, the follawing matrices generate the same lattice. The theory underlying the nonuniqueness af these sampling litLtiGes is based on unimadular matrices. Thus, in order to dlkcus8 the nonuniqueness of sampling lattices, we must first briegy discuss unimoduiar matrices. 2.2 Multidimensional framew~rk 35 DeBxritian 2.2.1.3. An integer-valued matrix A is a unimortulizr matrix if The following theorems provide properties of these mizlriees. Theorem 2.2.1.1. If A is an integer-valued unimoduiar matrix, then A-' eksts and is an integer- vafued unimodular matrix. Praof: Let A be an integer-valued unimodular matrix. Moreover, since det A f 0, then A-' e f i ~t s and AA-' = I. Since det A det A"-' = 1, then = 1. Since A is unirnodular, then where the (i , j ) elernelit of the adjugate(A1 is equd to the cofactor of the ( j , i ) element of (A). Since A irs uriirnodufar, then A-' = f adjugate (A). Since A is an integer-wlued matrix, then ad,jugatefG) ia an integer-valued matrix. Therefore, A-I is an integer-valued unimodular matrix. II With this background on integer-vaIued unimodular matrices, we are ready to discuss the isaue of nonurriqueness of sampling 1;jtetices. Theorem 2.2.1.2. Given a nonsingular maLrix M and an iateger-vdued unimodular matrix V, then LATWV) -- LATW), Proof: If x E LAT(M), then there exists an n E N, such that x = Mn = MVV-'n = MVm, where m = V-'n . Since V is an integer-valued uni- modular matrix, then V-I is also an integer-valued unimodular matrix by Theorem 2.2.1.1. Since V-' is integer-valued, then m = V-'n i s integer- valued. Therefore, x E LAT(MV). Therefore, LAT(M) E LAT(MV). Obviously; LAT(MV) C LACP(M). Since N and MV generate the sarare la.ttice, then ZAT(MV) =: LAT(M). 1 FOF notation pufposes, let 3(M) =f det; M the absolute value of the determinant of the sampling matrix: ha. Theoxem 2.2.1.3. Given a nonsinguliar matrix R?1; and an integer-dued unirnodular matrix Y. Then, JfM) = J f MV) = d(VM), 36 Chap ttsr 2 Mul ti&mensiond MuItirate Signal Procesging Prooh Sirice J(M) =I det M 1, then J(MV) = J(M) J(V) = J(V)J (M) = J (VM). Since V is an intqer-vrtlued unimodulax: matrix, then J(V) = 1. Herlce, a Therefore, J(M) is unique and independent af the chaiee of basis vee- tors. Moreover, J(M) can be interpreted geomet r i dy a the k-dimensiond volume of the parallelepiped defined by M. Sometimes J(M) is called the sampling density. Now, consider the falltowing sampling matsices 1 0 1 1 R = and S = 2 -4 2 --2 where J(R) = J(S) = 4. Sirice RE = S for unimodular matrix then R and S generate the same sampling lattice. In this case, the sampling lattice is known aa an hexagonal saxnpEing laktice, 2.2,1,3 Unit c e b and fundamental p~dlelteplpeds Definition 2,3.1.4. Given an integer-vrtlued matrix R, a unit cell i ~c f ade s one lattice point from &AT@) and J@) - l adjacent points in N t hat are not in &AT@). If these unit cells are periodicdly replicated 0x1 LAT(R), then the en- tire spwe is tiled witfi no overlap. Thus, the unit cell is a footprint that characterizes tire samplirkg lattice. Definition 2.2.1,6, Given an irmteger-mlued matrix R, the fundawkentrzl parallelqipetf of 1Iattice LATO, denobd FPD( a) , is the unit cell fiat ineludes the origin mid i~ bounded by d Iat tice poini;s one positive unit away. Formdy, the f u a d a me ~ t d pard1dep;tped is givm by where E is the set crf aJ f k-dimension& real wetors. 2.2 Multidimensional framework 31 Consider the fallowing example as an illustration of the concept of fun- 2 0 damental parallelepipeds. Assume the sampling matrix R = 0 3 Then, fundamental ppardlelepiped is given in Figure 2.5. By inspection, there are J(R) = 6 points in the fundomental parallelepiped. Theorem 2.2.3 -5%. Given an i nt egef i vd~ed unimod~lar matrix U and an integer-vdued diagonitl rzlaitrk A, t f i e ~ Prosh By the definition of the fundamental psrauelepiped, FPD(UA) = {y E E y = UAx for all x E Let s = Ax. Then, x = A-'g. Hence, Since iJ i s an integer-valued unirnodular matrix, then Uz is an integer- valued vector if and only if a k an integer vector, Therehre, FPD(UA) = U{y E 6' y = z for all A-'z E [ O , ~ ) ~ ) . 38 Chapter 2 Multidimc?nsionai Muitirate Sigad Processing But 2; = AX. Eenee, FPD(f5A) = W(y E @ Y = Ax for all x E [O, I ) ~ ) or equivalently, FPD(UA) = U FPDCA). 1 Sometimes im the literature, authors will refer to the Symmetric para- llelepiped, denoted SPD(R). It is defined by SPD(R) = {y E E y = Rx for all x E [-I., I.)~). DefiniCisn 2.2.1.6. Let V1 and Vz be Ic x Ic integer matrices. LAT("V2) iu cdled ia sabladbice of LAT(ITl) if LATrJ2) C LATF1), that is, every point of LAT(V2) is dga id p~j nt of LATp l f . Let V1 and Vz be integer matrices, where LAT(V2) C LAT(VI). Then, for every m E H, there exist n E such that or equivalently, n = V ; ' V ~ ~ . Since n and m are integer vectors and since Vl n = Vzm, then v;'v~ must be an integer-valued matrix. Let L = V;'V~. Then, VIL = Vz. Since det Y rL = (det Vr) (det L) , then det V2 = (det Vt ) (det L) so that, J(Vz) = J(V1) (det L) . 4 0 Hence, J(Vz) is an integer multiple of 3(V1). For example, if Vz = 0 4 2 0 md V1 = , then Vz is a sublattice of V1. AS such, J(Y2) = I6 0 2 and J(V1) = 4, then J(V2) iu, as expected, an integer multiple of J(V1), i.e. J(Vz) = 4J(V1). An important special case results when the lat- tice LAT(V1) coincides with the fundamental lattice N, i.e., LAT(V1) = LAT(I). In addition, p = represent8 the number of cells of FPD(V1) which can fit into FPD(V2). The lattice point in each one of these cells can 2.2 IMirltidI'mensr'ur~af framework 39 be thought of as a shift vector a, which if added t o each vector of LAT(V2) will generate an equivalence class of points called a coset. The union of all cosets is LAT(V1). Thus, the concept of a coset will provide a natural way t o partition LAT(V1) into subsets, which is a necessary step for the generation of mul- tidimensional polyghttse components. Befinition, 2.2.1.T. Let Vl and V2 be integer-vdued nzatrr'cm such that Let a E LAT(Vl) fl FPD(V2). Define the coset C(V1, Vz , a) to be If LAT(V1) = hi, then by convention, V1 is not explicitly identified, t hat is, the coset is simply Given integer-valued matrices V1 and Vz such that LAT(V2)CLAT(V1), then denote the set of shifit vectors by Similar t o the convention for cosets, the convention for shift vectors when LAT(V1) = ..A/' is not to explicitly identify V1, t hat is, the set of shift vectors is simpXy 4 0 Returning t o the example above, if TJZ = and V1 = 0 4 then the cosets are uniquely defined by the following set of shift -vectors Let us briefly review some elexnentary operatioas that be applied to integer matrices, These oper;;tlioxrs will be essential when we subse- quently discuss the Smith form decomposition, a method for diagonalizing the sampling matrh. 40 Clxapeer 2 Multidimensional Muitirate Sign& Procrtsging Etementary ruur (or column) operatima on integer matrices are important because they permit the patterning of integer matrices i&o simpler forms, such as triangular and diagonal forms. DeBnitian 2.1&.1,8. Any elementary row operatioxl on m integer-vdued mat r k P i s dde ~e d to be any of the fo&owing: Type-1 : In terchaagtt two rows. Type-2: Multiply a row by a nonzerointeger constant c, Type-3: Add an i ~ t e p r multiple of a row to anather row, These aperatiotons can be represented by gremultiplying E) with an ap- propriaLLe square matrix called an. elementary m&rix, To illurJtrate these elementary operations, consider. the followhg examples, (By convellt;ion, the rows and columns are numbered starting with zero rather than one.) The first emmgle is a Type-l elemenlazy mat r k that interchanges row 0 and raw 3, which h a the form The second example is a Type-2 elementary matrix that multiplies elements in row 1 by c $ if:? which has the form The third example is a Type-3 elementary matrix that replaces row 3 with row 3 + fa * r w Q), which has the farm Afl three types of elementary polynomial matrices w e integer-valued uni- modular matrices. 2.2 Muf t;idimensioxl al framework 41 2.lt.X.6 S d t h form decomposidfon The Smith Form Decomposition p r o ~d e s a method for dbgonalizing the sampling matrix, When matrices are diagonal, most one-dimensional re- sults esn be edended automatically by performing operations in each di- mension separrately. However, in the nondiagond cwe, these extensions are nontrivid and require complicated notations and mat r k operations, Theorem 2.2.1.9, Every infeger-valued matrix R ewl be expre~sed in its corre~p on d i ~ g Smith form deeompo~i t i o~ aa where U and V are iateger-dued unimodular matn"ces md the Smitfr farm A is given by where r is the rank of R and A;[ Xi+1, i = 0,. . . ,T - 2. Proof: Assume the zeroth column of R contsbins a nonzero element, which may be broughL to the fQ,O) position by elementary operdions. This de- ment iar the ged of the zersth column. If the new (0,O) element does not divide all the elements in the zeroth row, then it may be replxed by the gcd of the elements of the zeroth rmv (the effect wilt be that it will contain fewer prime factors than belare). This process is repe&cd until ran elerneat in. the (0,O) position is obtained which divides every element of the zeroth row arid column, By elementary row and column operations, all the ele- ments of the zeroth row and column, other than the (0, Of element, may be made zero, Denote the new submatrix formed by deleting the zeroth row and zeroth column by C. Suppose that the submatrix G contains an element C i j which is not di- visible by ma. Add col u~l n j to column 0, Column O then consists of the elernents coo, el, j , . . . , e,-1, j . Repeating the process, we replace by a proper divisor of itself using elementary operations. Then, we must finally reach the stage where the element in the (0,O) position divides every ele- ment of the matrix, and all other elements of the zeroth row and column are zero. The entire process is repeated with the submatrix obtained by deleting the zeroth row and column. Eventually a stage is reached when the mat r k has the form 42 Chapter 2 Multidimensional MulGirate Signal Processling where D = diag (A@, . . . , and &[&+li,i -- 0,. . . ,r - 2, But E must be the zero mat rk, since otfierwiiie R woufd have rank larger than r. m Note that although the two unimodular matrices U and '\r are not unique, the diagonal matrix II is uniquely determined by 33,. Example 2,2.1.4. To illustritte the Smith form decomposition, consider the which corresponds to hexa,~~onal sampling. If we divide the (1,0) element, 2, by the (0,O) element, 1, ure obtain 2== 2 ( I ) + 0 * quotient remainder Therefore, if we apply a Type-3 row operation, which is defined by to R, we will reduce the (1,O) element to zero. Therefore, Dansform the (@,I) element t o zero by a Type-3 column operation, which 1, -1 is defined by . Then, we obtain 0 1 Finally, the (1,l) element is forced t o be positive by a Type-2 row operation, 1 Q which is defined by . The~l, we obtain Q -1 Thus, 2.2 Multidjmerr~ionai framework Le% E be the product of efementary raw operations, i.e. Let E" be the product of elementary column operations, i.e, since only one elementary column operation was performed. Therefore, Then, the Smith form decomposition is given. by where, and Theorem 2.2.1.6, Let and TC" be unimodular matrices and let A be a &agoad matrix. H the Srnith form decomposition of smpfing matrix R is given b;y R == UAVl then ProoE Since LAT(AV) = LAT(II) by Theorem 2.2.1.2, then FPD(AV) = FPD(A). Therefore, FPD(UAV) = FPD(UA). Since FPD(UA) = U FPD(A) by Theorem 2.2.1.4, then FPD(R) = U FPD(A). rn 2.2.2 Mul$idimensionill sampled sign& Unfortunately, some important sampling structures can not be represented as a, lat;lice, Fbr example, consider an important sampling structure h r High Definition Television(HDTV) called line quincunz, where two sam- ples are placed one vertically above the other in place of every sample in 44 Chapter 2 MuItidimerlsiorrd Multirate Sig~aX Praces~ing the sampling grid, But, line quincunx can be represented as the union af twa shieed lattices using the rnuftidimensiond ~t r an~~f or mi , VVe will firat define some underlying vector mathematics and, then we wiu present the multidimensional ;=-transform and the multidimensianal discrete 1Faurit;r transform, Then we will present two nnuftidimen~iod sign.& representa- tions - the modulation representation. ~ n d the pollyphmt: representation. The theory of multirate and vavelet signal processing i s considerably sim- plified by the use of these representations. 2,2.2, J Vector In order to gencrdize the definitions tihat we have grown accustomed to seeing in one-dimension, we wilX provide the definition of a vector raised t o a vector power and, subsequently, the definition of a vector ra&ed ta it mat r k power. T Definition 2.2.2.1. Given eomplttx-vdued vectorr = . . . , 7 ~ - 1 ] and integer-valued vector s = [ so, .=. , s ~ - 1 T . Then, the vector r r&sed to the vector s power is a sealtar and it is deaned to be Then, building on this definition, we will define a vector raised to a matrix power. T Definition 2.2.2.2. Given a complex-valued vectorr = [ ro, . . . , ~j v- 1 ] a d an i~teger-vdued ma&rix &= I d0, . ..* LN" 4 ,where hi i s the i tlx column of L, the^, the vector r r&~ed to the matrix L pawer i s a row ve e t ~r and if; is defined to be Definition 2.2.2.3. The b-dimensional z- transform of z(no, . . . , lab - 1 ) is defined by 2.2 Mtlltidimeasiand framework where, z = [zO, ..., z ~ - ~ ] ~ i~ a complex-valued vector, n = (no, ..., n k - l l T is aa integer-vdued vecCor, and Let L be an integer-dued noasingufar m;ttrix, &en by Elefiaitjon 2.2.22, zL i~ given by where, Li irs $he ith colrrrnn of L, that is, Theorem 2.2.2.1. Let L he an integer-valued matrix where Zi, z' = 0,. . . k- 1, are the columns of Z1, Then, Proof: Using the Definition 2.2.2.2, vve can write where, Li is the ith column of L. Substituting the definition of a vector raised to a vector power yields where Lmli is the .nth comporlent of Lie By definition, a vector raised to a wctor power gives a scalar. Hence, k- l / k - X ', 46 Chapter 2 Muitidimensiond Multirate Si p& Processing Sirrce the product of terms with the same base equals the base to the sum of the exponents, the last equatkn becomes I If the z-transform converges for all a;, of the form z, =; exp(jwm), rn = OO?. . . k - f then the n-transform car1 be represented as the sum of harmonicdly refated sinusoids, i.e. which is the multidimensional generalization of the discrete-lime Fourier transform, In order to quickly distinguish vectors in Fourier space, the vector Fourier variitble 2 i s denoted by an underlined omega rather than a Bold omega. Theorem 13.2.2.2, Let I; be an i~lteger-valued matrix. Then, exp ( j wj L = exp ( j ~ ~ g ) . Proof: Using the Definitiozl 2.2.2.2, we find that where, Li is the ith column af L. Since the exponents of each term are simply inner products between g and a coXumn of L, their order can be interchmged, that is, The multidimensiond discrete hur-ier transform is an e x x t Fourier regre- sentation for periodicasjly sampled arrays, Therefore, it takes the form of a periodically sampled Fourier trax~sftzlrm, As irz the one-dimensionai case, the nnultidimensiond discrete Fourier transform can be interpreted ES a Fourier series representation for one period af a periodic sequence. 111 this formulatim, we wiH have to address t m types of geriodicities --- orre due to the sampling lat;tice and one due to the s i p & (tbat is, defined on lattice points) to be Fourier trazuformed. Let V denote the sampling mat^, Lee, hexagonal, quincunx, rectangular, etc. Let N denote the peri- odicity matrix, which characterizes the periodicity of the lattice points on which the signd to be Rurier trmsformed is defined. Assume LAT(M) is a sublattice of LAT(V). Then, we define equivalence eIwses betvveen periodic replicw of the data by [n] = { m E LAT(V) n - me LAT(N) ). Therefore, if parailelogribms are drawn between the elements of LAT(PJ), then any tm vectors that occupy the same relati= pasition are in the same equivalence cfms, Many properties of the periodicity matrix, N, follow by a n d 0 0 from the corresponding facts far sampling matrices. For example, the density of the periodiciQ matrix is uniquely defined by , denoted J(N); but for a given periodic sequence the periodicity matrix N is not uxlique, since it, can be multiplied by any unirnodular miztriur and still describe the same periodic signd. In additioa, the columns of fiJ indicate the vectors along which it is periodically replicated, RefOniLion 2.2.2.4. A multidhcn.siond sequence a(n) is pesio&c FviGh pe- n'od N, that i ~ ? for all n? r E JI/, z(n) = ~ ( n + Nr). Let XM represent one period of z(n). Tllea, where V defines the unddying sarrrpling lattice, Moreover, since z(m) is periodic with period N, X(g) crzn also be written as X(w) = x z (n) exp - j g ~ n ] exp[- j w T v ~ r ] . nEZr-3 But, Therefore, exp[- j w T v ~ r ] = 1, which is equivalent to the condition, where m is a vector of integers. Upon further examination of 2, we observe t bat uT = 2?rmT(VN)-I, " n. . or equivalently, Therefore, w = 2 n ( ~ ~ ) - ~ m . - The matrix ~ ? I ( v N) - ~ serves as a Fourier domain sampling matrix. Sub- stituting this equation inta the equation for Xfg) yields or equivalently, Let us further examine the inner product which occurs in the argument of the exponeatiaf, Suppose the multidimenaiand sequence X(mf i s periodic with period P, that is, X(m) = X(m + Pq) for m, q E N. Also, let IF reprevent one pe- riod of X(m). Then, by analogy with the one-dimensional discrete Fourier transform, assume that ~(a) has the follovvlng form for some constarlt a, Invoking the periodicity of X(m), that is, X(m) = X(m + Pq), will cause z(n) t o become ar equivalently, But, 2 z(n) = - X(m) exp[jnT ( 2?r~- ~) m] . a mEZp Therefore, e ~ ~ [ j n ~ ( 2 ? r ~ - ~ ) ~ q ] = 1 for all q E N. Since xl and q are integer-valued vectors, then or equivalently, P = bIT. Therefore, X(m) is periodic with period NT, that is, X(m) = X(m + NTq). Hence, + Now let us determine the constant er: by substituting the equation for z(n) into the equation for X(m), Hence, which is as expected, since J(N) =;I det N I is the number of samples in one period for LATIN), Therefare, the multidimensiond discrete Fourier transform pair are given by: It should be noted that these equations reduce to the usuaf discrete Fourier trantzform. pair in the one-dimensional cme and t o the familitzr rectangular muldidimensional discrete Fourier transform when N is a diagonal matrk. As an illustration of this theoretical development, sometimes it is of interest to input data from an arbitrary lattice and output it an iz rect- angular lattice, so that it could be conveniently displayed an a eompzoter display. Assume that V is degned by For hexagonal input: a = 2, b = 1, c = 2, Moreover, for quincunx input: a == 2, b = 1, c = 1. In addition, for rectangular input: b == 0, Select ht periodicity madrk so that VN is a diagonal mat r k so that the resulting Fourier analysis will1 be on a rectangular grid, Now, let ur~ pick N to be then this N mat r k is a good choice for a periodicity mittrk. Therefore, the DFT becomes where, 2.2 Muf tidimensional framework 51 This suggests the following algorithm: (I) Compute &HI-point FFTs, one for each row in the 7&1 direction. (2) Apply. a. phauc shift t o each paint of the resulting data. Since we are mrki ng with a sampling grid with samples at integer-valued locations, it is important that we perform the phase shift for integer multi- ples of R. But :nzml is real-valued. Therefore, we will need t o quantize :nzml t o integer values through the use of the round function. ( 3) Compute &&-point FFTs, one fbr each column in the 7 ~ 2 direction. 2.2.2.4 The S d t b foran and the DFT First;, let us begin with the mul$idirnensionaI discrete Fourier trrtrrsfornn, that is, F(k) = C f (n) exp D E ~ M Replace M with its Smith form. decornpositian, that is, M = TJAV. Then, using Theorem 2.2.1.2, LA'I"(UAV) =: LAT(UA). Novv, using elementary linear dgebraic operations, let us simplify the exyorrent of the exponentid = exp [- j 2 ? r ( ( ~h~) - ' n) ~k = exp [-j2nkT(UhV)-In Then, the mzrltidirnensiond DF'T becomes where m = U-'n and pT= kTv- ' . 52 Chapter 2 Mult;idimensi'onal Muf tirat e Signal Proeegsing Therefore, the Smith form permits the use of a rectangulm DFT, when the initial data lies on. a nonrectangular sampling grid. The initial data must have pardlelepiped spatial support which becomes rectangular after being mapped by U-I. Moreover, the larger the values of the elements of U-I , the more the spatial suppart will be skewed. Thus, the aill;ar&hrrr for the Smith form tversion of the multidimensiollal DFT is given by the following: (1) Shufee the input data samples by U-I. (2) Perform a septtrabk ~kultidimensional DFT with len@'fis equal to the diagonal elements of A, (3) Shufee the output data samples by VT. Let us examine more carefu;tfy the mapping between the spwe of the initial data samples and the space defined by A. Assume that the data is defined on a quincunx grid with the hllowing SmitEl form decomposition. Then, II characterizes the intermediate space for the Snrrith Farm decom- pasition and the mapping from M to A can be visualized by Figures 2.6 and 2,7. Si~iilarly, the inverse DFT is given by where, det M I=I det U Defiinitrion 2.3.2.15. Let M be the sampfing matrix, Then, given a, multidi- mtm~ional sequence ~ ( n) , the eompoxreats of the muZdidimensionaI modula- t;ion representalion of the multidimensional n-transform of z f n) are defined To more easily interpret this equation, assume z takes on the miue of .xp(jg). Then, Figure 2.6, Ilnput data. Figure 2.7. After shamng input data. But z, = exp(jo,) for m, .- 0, . . . , k - 1. Therefore, Example 2.2.2.3. This example illustrates the idea of the multidimen- sional modulation representation of X( z ) with respect t o a rectangular 2 0 sampling matrix; M = . In this case, I PVI ~ will contain four va.1- 0 2 X(zo, zl ) for ha= 2' X(z0, -21) for hl = [ 0 1 T X~?' (Z) = T X(-zo,zl) for hz= X( - al - xr ) far h3 = 2.2.2.G Polyphase represenkat ion Definition 2.2.2.8. Given a multidimensional sequence s(n) and a nonsin- gular matrix R, then its Tgpe-I multidimensional polyphase components are defined as z.(n) = z (Rn + a), where n E and a E N(R). Now, let us investigate the multidimensional z-transform of the Type-I muEtidimexlsioxla1 polypfime components, that is, or equivalently, Far notatioriaf purposes, let then, X( zj becomes 2.2 MuXtidim ension& framework 55 where zea corresponds to shifts of the multidimensional signal by a vector amount a, Novv consider the Eollawing example, which illustrates the idea of a Type-I multidimensional polyphase decomposition of the multidimen- sional filter Xfs) . Example 2.2.2.2. What is the Type-I multidimensional polyphase decom- position of the multidimensiorkal filter X (1;) with respect to a rectangular 2 0 sampling mabtrk R = ? In this case, there are J(R) = ldet R I = a 2 4 values of the shift vectors are given by [0, OI T , [I, OI T , (0,1 T Let s = [za, rl] and a = [(a), , (a),lT then, zma = z~"'~z;(a'z. ThusI the quantity ia given by ( z,' z;' for a = 10, 0jT r,'z;' for a = [ l , OI T = zcaz;' for a = 10, llT z for a = (I, ljT , I ' for a = [o, o ] ~ zo = Cl Since for arbitrary R = , zR is given by zR = ( ~, 2 ~z Y, z:' z? ) . Yo o1 Hence, the quantity x i R ) (sR) is defined by So, the polypkitse decomposition of Xfz;) is giiven by X( z ) = xiR) (zi, 2:) + z;lxLR) (Zi, zf ) -1 (R) 2 2 + z1 x, ( Z ~ , L ~ ) + Z ~ = Z ; ~ X ~ ~ ~ ( Z ~ Z , I ~ ) . Definition 2.2.2.7. Given a multidimensional vector z(n)and a nonsingu- 1ar matrix Et, its Type-11 nzultidimewionad palyphme components are given by z,(n) = g(Rn - a), where a E hi(R) and n E N. 56 Chapter 2 A/lulf idirneusioud Mul tirate Signal Processi~g Now, led us inwstigate the multidinrensionat ~~transfornr of the Type-I1 multidimensionalt polyphasse components, that is, then where za advances the multidimensional signal by a vector amount a. T~us , the set of dl integer xctora can be partitioned into J(R) equivalence classes using either Type-I or Type-I1 muttidimensiona! polypbitse decomposition. Moreover, the success of these decompositions rests on the following theo- rem. Theorem 2.2.2.3. (Division Theorem. for Integer Vectors) Let R be a k x k nonsingular integer matrix, let; p and n he m integer veetors, and let a be s sbr'ft vector of R, the^, w can uniquely express p w Praof: Write g +EM P' PP+P~; where, pp and pr, are unique vectors with Since R i s an integer matr2, then pr; ia an integer vector, Moreover, since p and E)L are integer vectors, then p~ is an integer vector. Moreover, since p~ E FPD(R), it follows that p~ E M(R). By letting p~ = a and p~ = Rn, we obtain p=Rn+a. I The remainder, a, can be expressed %s a ZE p mod R or simply ((a)) =: 2-3 Multidimensional building blocks Figure 2.8. Mult idimensiond ezcpantler, 2.3 Mdtfdimemional bui l di ng blocks Thits section defines and analyzes two types of multidimensional building blocks used in multidimensional rnultirate signal processing. One type of building block deals with changes ia the sampling rate of the input signal, and the other type of building; block deals with changes in filter length, Analogous t o the one-dimensional cae, there are two types of multidirnensionaf building blocks, which change the sampling rate of the input, signal ---- decimators to reduce it and expanders t s increwe it. But in the multidimensional cme, deeirnators iznd expanders aRect not only the sampling rate but also the geometry of the sampling lattice, Then, a multidimensional building block is discussed which cha~rges the length of a given iilter - multidimeni~ional comb filters to increase its Imgtfr. fn this way, a multidimensional comb filter version, of a given filter is analogous to ~nul t i di men~i o~al expander acting on an input signal. 2.3.1 Multidlmemiond expanders Definition 2.3.1.1. Let x(n) be a k-di mmsi ond signd md let L be an integer- vdued matrix. Then, $file muf tidimensional procesg of L-fold ex- pruldillg mtqps a signat on I\( to another signal that; is nonzero only rct points on $he subIattice LATP), The output of the R-dimensional expander is related to the input ~ignd z(n) by where XI is a nonsingular k x k integer matrix. Since LAT(L) denotes all vectors of the form Lm, where m E N9 then the condition L-'n E N is equivalent to n 6 LAT(L). The matrix L is knowxk the expansion matrix. The multidimensional expander is de- picted pictorially in Figure 2.8, Now, led us anitlyze the multidimensianal expander using the definition of the multidimensional 2-transform, Chapter 2 MuliCidimensional Muidirate Signal Pr oc e s ~i ~g Figure 2.9, Cascade of multidimensional expmders. Then, applying the definition of the nrultidimension.Ell expander will require that Y( z ) is zero for the lattice points defined by L-'n $! N. So, let n. = Lm. Then, Y(z) = y(~m)e-Lm. By the definition of the rnultidimensiond expander, z(m) = y(L@ for d l m E hl* . Henee, Y(z) = ~ ( r n ) z - ~ ~ , EN or equivalently, using Theorem 2.2.2.1, the equation becomes Hence, Y ( s ) = x(zL). In order to investigate the behavior of the multidimensionizl expander in the Fourier domdn, replace rt with exp(jg) to yield Utilizing Theorem 2.2.2.2 this equation becomes By suppressing the exponential and, as such, changing the notation so that Y (2) denotes Y (exp(jo)), then Now consider the carrcade of two expanders fcrl and L2, which can be de- picted grapkicdly by Figure 2.9. Substituting ][I F= LILz into the equation for Y (x) yields Y(o) = x( ( L, L~) ~s) , or simply, Y( o ) = x ( ~ T L T ~ ) , 2.3 Multidimensional building Mocks Figure 2.11. Smith form cascade of expanders. which can be represented by Figure 2.10. This cwcade of tvvo multidimensional expanders can be verified by re- dizing that ~ ( o ) = ~( LTo) and S( 0) = x(L:*). Therefore, Vu) = X( Lf ( LTw) ) = X( ( LI L~) ~A) = X(LTr??). So, for example, if L is repfaced by itis Smith form decomposition, that is, L -L=: ULnlLVLt then we can graphically depict it in Figure 2.11.. Definition 2.3.2.1. Let z(n) bc a k-dimensional signal and let M be a k x k integer-vdued matrix. Tlxe multidimensianal M-fold decimalor sitmples input z (n) by mapping points on the sublat tice LAT(M) to .A/' according $0 yln) = s(Mn) and dises~rding samples ofaln) not on LATfM). M irj. ca,Ifed the decimatioa matrix. The multidimensionaf decimator is depicted pictoridly in Figure 2.12. Let us analyze the muXticiimenaiorra1 decirnator using the definition of the Figure 2.12. Multidlixeewianal deeinn8tor. mul tidimensional z- transform, i. e. Substituting the definition of the muXtidimensiona1 decimator., we obtain Note that zfn) is not zero for noninteger multiples of Mn, So, define an intermediate mapping of points that i s zero f'or aoninteger multiples of Mn, that is, z(n), where M-'n EN ZI(~) = 0, otherwise . So, y (xr) = 4Mn) = zl (Mn). Hence, Let m = Mn. Then, by the definition of zl (m), if M-'n $N, then sl(m) = 0. Therefare, Y( z) = C al(rn) a-M-'my mN ar equivalently, Y(.) = C zl(m) (8 M-' )--me =EN Next, we need t o express XI (z) in terms of X(e). By the definition of sl(n), we can write the foliowing: 2.3 Multidimensional building blocks 61 where CM(n) is a scalar-valued sampling function associated with the sam- pling matrix M, t hat is, 1, whenever M-'n E G M( ~ ) = 0, otherwise . Since CM(n) is periodic in LAT(M) with spatial variable n, i . e. CM( ~ ) = C M ( ~ + Mm), it can be expressed as a complex Fourier series Applying the definition of the multidimensional rtransform t o zl(n) yields Substituting the equation for GM(n) yields or equivalently, Then, performing a rnultidimensiond z-transform yields Since Y ( B ) = XI (zM-I ), then Y (z) becomes If z-" equals exy( - j gn) , then sM-' i s equivalent t o e ~ ~ ( - j ~ - ~ ~ ) . Therehre, Chapter 2 Multl"dimensiondt1 Multirate Sign& Proces~ing Figure 2.13. Cwcade af mtlltidimensiond dedmstars. Figure 2.14. Interpreting the cmcade of multidimertsiond decinzators. Utilizing Theorem 2.2.2.2, Y(exrpba]) becomes By suppressing the exponential and, za such, changing the notation so that Y (o) denotes Y (expb*]), then Now consider the emcade of two decirnators MI and M2, which can be depicted grqhically in Figure 2.13. Substituting M = M5M2 i ~ t o the definition of the multidimensional ddecimator yields which can be repre~ented in Figure 2.14. This cascade of multidimensional deeimators can be veriGeb by realizing that Therefore, y(n) = z(M1M2n). So, for examplie, i f M is r epked by its Smith hrrn decomposition, that is, M = UMAMVnn, then we can graphicdly depict it as Figure 2.15. Let ucr brieAy examine unimodular deeimators with decimation matrix Y. Consider multidimensional decimator, 2.3 Multidimensio11d building block8 Figure 2.15, Smith form cascade of decimatms. Figure 2.16, Unimodular decimator, Since V is a, unirnodular, then J(V) = 1. Therefore, Therefore, there is no diasing and, as such, unimodvtar decimation can be viewed as just a rearrangement of samples. This inte~pretation of unirnod- ular decimation is depicted in Figure 2.16. Definition 2.3.3.1. Given the impulse response h(n) of a mull-idimensioaal filter, one can build a muttidimensional comb filter g(n) by mapping the filter h(n) on to another one that is nonzero only at points on the subfat tice LATCL), i. e. As mentioned earlier, &-In E N is equivalent t o n E LAT(L). The im- pulse response of a multidimensional comb filter can be represented by where 6,,L, is the multidimensional Kronecker delta function. Taking the mul t i hensi onal x-transform of gfai), we obtain 64 Chapter 2 Malt;idimensiond Maltirate Signal Proce~~i ng or equivalently, If LAT(L) is not rectangular, that is, L is not a diagonal malrix, then the components of ~ ( z ' ) are no longer separable. To illustrate F I 0 %I this point, let L = , then ~ ( a l ) = H(z,'~r?, z,Z1zp) and 90 YI H(z) = H(ro,zl). For example, we will consider both rectangular Sam- pling and hexagonal sampling, For rectangular sampf ng, let then ~ ( s ~ ) = ~ ( r i z y , z:z:) = H(zi z:) and H( z ) = B(ro, zl). This indi- cates the separability of components of ~ ( z ~ ) for a rectangular sampling. Far hexagond sampling, let then ~ ( z ' ) = x ( z ~ z ~ , z ~ z ; ~ ) = H(z~z:,z~z;~) and H(z) = H( z&, zl). Thk indiea-tes the lack of separability of components of the m-tlllidimen- siortal comb filter H(zL) for non-rectangular sampling. 2.4 Interchanging fsdlding blocks This section defines wq s to interchange multidimensional Building blockis. First, the conditions for irtterchanging muldidimensiond dceimators and expanders Tor sampling rate conversion are presented. Secondly, we con- sider the multidimensional noble identities, an approach for interchanging multidimensional building blocks with multidimensiond decimation and interpolatiox~ filters. 2,4.1. Irrderchangimg deciurralors and expanders Multidimensional sarnplirxg rate conversion is irnportaxxt for many signal processing applications, because many times it is necessary to interfact: irn- agc or video data between systems which use different sampling lai;t;ices. Examples include the conversion Between European and American televi- sion systems and the conversion between high definition television (HDTV) 2.4 I~Cercha~ging building blocks Figure 2.17. Decimatoz preceding expander. Figure 2.18. Substituting Smith form dewmposi trioxls. signals and conventional tefevision. signals, Thus, sampling rate conver~ion will require the cascade of a multidimensional I;-foid exprlnder and a multi- dimensional. M-fold decirnator, separated by a filler. Under what conditions can the decimdor and the expander be interchanged? We will see that we will need to as s um that (a) M and L commute, that is, L M = ML, and (b) M and L are coprime. We wiH see that through the utilization sf the Smith hr m that coprimeness in muftidimensions can be xhieved in each dimension independently: Conr~ider the configuration in whicEl the decinzator precedes the ex- yader , which is depicted in Figure 2.17, Let us assume the following Smith farm decompositions for M and L: M == iJMAMVlyr and XI = fJLALVL where UM, VM, UL, VL are unimodular matrices and AM, AL are diagonal matrices. Therefore, the resulting configuraLion is depicted in Figure 2J.8, or equivalently in Figure 2.19, Without loss of generdity, we can ilssurvze that Hence, we obtain the configuration depicted in Figure 2.20, Since AM and Figure 2.19. Substituting cmcade definitions. Chapter 2 Multidimensional Multirate S i g ~a l Procesgiag Figure 2.20. Simplifying the cascade. Figure 2.2 1, Interchan[;;ing lambda matrices, AL are diagonal matrices, then interchanging AM and .AL can be? achiemd in each dimension individually, provided that the associated decimation and expansion ratios in each dimension are relatively prime. In this way, the Smith form decomposition. has llelped us transform a multidimensional problcrn into a series of one-dimensional problems. Therehre, we obtain the conliguration depicted in Figure 2.21, Since unirnodular deeimators me equivalent to inverse uxxirnodrrlsr expanders (as depicted in Figure 2-22), then for a ~i y unirnariulw mat rh T, Figure 2.23 results. Now choose a unimadulatr mat r k UA such t hat or equivalently, UA = &UMh;'. Choose a matrix VB such. that or equivalently, Figure 2.22. Unimodular deeimatars and unimodular expanders, 2.4 In terehangirjlg building block8 Figure 2.23, Rmulting figure. Substituting Ui' into the last equation yields Since AM and AL are relatively prime in each. dimension, we can inter- change A 2 and AL. Therefore, vB = U ~ A ~ A ~ ~ U ~ L - ~ M . Since VM = VL, then VLvkl ;: 1. Therefore, VB = u~A~v~v&' A~' uG' L- ~M, or equivalently, vB = LM- I L- ~M. If L and M are assumed to commute, then L-'M = ML-'. Therefore, VB = 1. Hence, we ltave shown that the nrultidimensionaf L-fold expar~der and the multidimensional M-fold decimator can commute provided (a) 3L ztr~d M commute and (b) L artd M are coprime, Exampla 2.4,li.T. Let us consider the following example. Assume a mul- tidimensional decirnator is defined by the decimation matrix D is given and a multidimensional expander is defined by the expan~iorl mat r k E is Clan these multidimensiond Building blocbs be inderchlzngedl First, we will find the Smith form decomposition of rnaLtrices D a ~l d E, that is, Chapter 2 MaltidimensionaJ Malgirate Sign& Proee~sing Figure 2.24. Multidimeasionial; expander followed by after, Figure 2-25. Multidimensionaf dedmator preeded by filter. Since the diagonal entries in Ax> and are relatively prime, then r f r , and ljlE me coprime, which in turn implies that 33 and E are coprime. Secondly; we need to check to see whether D and E commute, that is, and Since DE = ED, then I) md E commute, Since D and E are coprime and since D and E commute, then the multidimensional decimator (defined by the decimation matrix D) and the multidimensional expander (defined by the expansion matrix E) can be interchanged. In moat izppkcations involving multidimensional interpolation, an inter- polation filter foUows an expander as in Figure 2.24. Similarly, in many app1icatiorra involving multidimensional becimators, a decimation filter pre- cedes the decimator as in Figure 2.25. If H( z ) and K( B) are multidimen- sional comb filters, then can the multidimensional building blocks be inter- changed? Figure 2.26. MuILidimensional, expmder with comb filter. Figure 2.27. Filter preceding mnltidimensiond expmder . Consider the configuration depicted in Figure 2.26. Writing the elerrlentd equ;ztions, vve obtain V( z ) = x(sL) But, this equation could dm be interpreted as Figure 2.21, where and T ( z ) =: G( z ) X( e) . These two equivalent block diagrams present a systematic approwh for in- terchanging multidimensional filters with multidimensional expanders, and together they will be referred to cw the Nable identity for multidimensional expanders, Consider the configuration depicted in Figure 2.28, Writing the! elemental equations, V( e ) = c(sM) X(Z) Chapter 2 Mull;r.dimensiond Muf tirate Sign& Proces~ing Figure 2.28. Comb fitter preceding malt idirnensional decimator. Figure 2.29. Mufdidimensiond decimator preceding filter. Hence, using the multidimensional z-transform identities msociated with x raised to a matrix power, we obtain the following But this equatioxt. could be interpreted a r ~ Figure 2.29, where Y (z) =: G(z-)T(z) These two equivalent block diagrams present, a systematic approxh for interchanging multidimensianal filters with multidimensional decimatorts, and together they will be referred to as the Noble identity far multidimen- sional decimadors, 2.5 Problem 1, Consider the following sampling matrix: Sketch the lattice LAT(T). Clearly indicate the fundmental paral- lelepiped FPD(T) and highlight the J(T) points in N which belong to FPDIT). fat n 72 Ghapter 2 Multidimensio~~d Multirate Si p & Processing 7, Let us generalhe the notion cvf multidimensional expanders and dleci- matarg. Let N be a nonsingular mat^. In the definition of the mul- tidimensional expander, replace JV' with LAT(N), where LAT(L) C LATIN), Similarly, in the definition of ttte mullidirnensional decima- tor, replace A/' with LAT (N), where LAT(M) C LAT(N). Interpret the resulting operators and provide examples to illustrate the use of eaeh of them, Chapter 3 Mult irat e Filter Banks This chapter introduces the notion of maltirate filter banks. Using eon- ventiond filter design techniques, a high-order filter is required to obtain a faat roll-off capability, Filter bank8 are bwed on an alternative approach to reaiizing a high-order filter consistjng of tire cascade of lower-order (analy- sis) filters, that are designed with aliwing, and (synthesis) filters, that ape designed to cancel the alim-componerrts of the analysis filters. Working independently, Smith and BarnwellL44j and Mintzer[35j re- ported the e ~s t e nc e of a two-channel filter bank, which permitted perfect reconstruction of the input signal. Subsequently, Smith and Barnwell[45] developed the aliars-component mat r k formulation for analyzing M-channel filter banks. Meanwhile, Vetterlil53) and Vaidyanathan(481 independently discovered the polyphwe formulation of analyzing M-channel filter banks. The theoretical underpinnings of filter banks were carefully examined by Vaidyanathan and Mitra[5q, who showed the connection between pseu- docirculant matrices and aliars-free filter banks, and by Vaidyanatthan[48j, who showed a connection between paraunitary matficetl and perfect recon- fstruction. These developments in multirate signal processing were recently complemented by research involving the joint consideration of quantization effects and filter bank design(KovaEevie(25l; Westerink et a1.[56]). Subse- quently, Koilpillai and Vaidyanat han [24] developed perfect reconstructing cosine-modu1at;cd filter banks, whicb have the advantage that d l filter8 are derived from a single prototype filter. Xn an effort to eliminate block- ing effects in low-bit rate image compression, Malvar[29,30] independently developed cosine modulated filter banks and he called them lapped arthog- onal t mnsf oms (LOTS). Subsequently, using the very general conditions for windows, which were developed by Coifman and Meyer(S], Suter and Oxley[47], and independently Auucher, Weiss, and Wickerhauser[l] devel- oped the corttinuous generalization of LBTs to permit the conytruction o_f 74 Chapter 3 Muftirate Filter Banks diEerent l ac4 b ~ e s in diRerent time intervalis. Yves Meyer cdled the eon- tinuous generalization of LOTS by the name Malvar wavelet[33), since they complement elwsical wavelet theory. Recently; Xia and Suter have gerreral- ized the theory of Malvar wavelets to include two-dimensional nonseparable Malvar wavelets [58] and Malvar wavelets on hexagons[59]. Vetterli[52] was the first person to write a paper on multidimensional multirate filter banks. His paper dealt with two-channel filter banks with quincunx decimation, Using an analogy with one dimensiand systems, Viscito and Allebach[55] formalized the theory of multidimensional mul- tirate operations for arbitrary multidimensional lattices. Karlsson and Vetterli[23] and Chen and Vaidyanathan[G] formulated polyphase repre- sentations of mullidimensionril signals. Some of the concepts developed in this chapter are also discussed in the texts by Fliege[lS], by Strang and Nguyen[46], and by Vaidyanathan[49]. Section. 3.2 introduces quadrature mirror filter banks, Then, Section 3.3 presents the theoretical foundations; of multirate filter banks, Section 3.4 presents filter banks for spectrd analysis. Section 3.5 introduces mul- tidimensional quadrature mirror filter banks. 3.2 Qua&ature mirror Alter bads This section define8 and andyzea quadrature mirror filter f&MF) banks, The role of $ME" banks in source coding is examined first, This is fal- lowed by a, discussion of two important QMF bank formulations -- al i a- component and polyphase. Then, a multirate source coding design example is presented that illustrates many of the QMF bank concepts of this sec- tion. Finally, a brief discussion of quantization and their egects on filter banks is presented. 3.2.1 Sawee coding and QMF b a d s Consider the two-channel QMF bank that is depicted in Figure 3.1, The ana'fysis bank together wiLh the decirnaLors decompose the input signal z(n) into tvvo subband signals, y@(n) and yl(n). This is follwed by expanders and the synthesis bank, which produce an output sigxlal that reconstructs the input signal, For the QMF bank, the number of samples per unit time for the sum of both subband signals equals the number of samples per unit time for the input signal, But, the power of the subband signals is usually much lower than. the origin4 signitf.. After decoding, the input signal is reconstructed, Before we can design a system bitsed on these ideas, we will need t o become acquainted with the approaches that are used to analyze filter banks. 3.2 Quadrature mirror filter banks Figure 3.1, Two-chansel gllter bmk, 3.2.2 Filter b a d formulations We will discuss two difFercnt approaches t o the analysis of filter banks: (li) diw-component formulation, and (2) polyphwe formulation. Then, we will show that the two formulations are equivalent, 3.2.2.1 rlliaa-campanent farmulat ion of ftXter ba&s Consider two-channel filter banks. Let &f w) and El(w) represent the frequency response sf &( z) and L;l; ( z) , respectively. For a QMF bank, the magnitude responsest 1 Ho(w) ] and I HI(@) are mirror-images of each other with respect t o frequency $,which is one quarter of the sampling frequency 27r. For M-channels ( M > 21, t kh structure should not be called a QMF bank, because the traditional tw~-cftanncl meaning does not hold, However, QMF hm been used by niany ather authors, so we will retain the same nomenclature. An M-channel QMF hank, t hat is depicted in Figure 3.2, partitions the signd spectra, into M barrds of equal bandwidth and later recornbitres these frequency bands, The input is denoted z(n) and the output is denoted y(n). It consists of M decinlators (each with a decimation ratio of M) ; M expanders (eadl with an expansion ratio of M); M a~lalysis filters (denoted by Hk(z)? k = 0,. . . , M - 1); and M synthesis filters (denoted FL(z), k = 09" . , M - 1). The basic philoaoyhy behind the design of QMF banks is t o permit a l i ~ i n g in the filters of the andgsis bank and then. choose the filters of the synthesis bank so that the alia-components in the filters of the analysis bank are cancelled, Now, we will proceed with the andysis af the quadrature mirror filter bank, by exaxnining ihtage-by-stage. First, we will consider the avralysis t ank, which is depicted in Figure 3.3. Chapter 3 Muitirate Filter Banks Figure 3.2. M-Gfiwnel filter bank. Figure 3.3, Andyais bank. 3.2 Quadrature mirror filter banks Figure 3.4. Bank of decinaators, The elemental equation for this stage is given by for each k = 0, I,. . . M -- 1. Then, we will consider the bank of decirrtators, which is depicted in Figure 3.4. The elernentat equation, for this stage is given by The following stage? consists of a bank of expanders, which is depicted ixr Figure 3.5. The elemenitaf equation for this stage is given by Uk ( z ) = v k( z M) . Lwtly, we will consider, the synthesis bank, which is depicted in Figure 3.6, The elernentd equation for this stage is @ven by Combining the equation for Xk ( z ) with the equation for Vk(z) yields Chapter 3 Multirate F2ter Barrks Figure 3.5. Bank of expanders, pthests bank Figure 3.6. Synthesis bank, 3.2 Quadrature mirror filter banks Conlbining this equation with the equation for Uk( z ) yields Finally, combining this equatioxl with the equation for Y( z) yields Y( z ) = desired terms terrrrs due t o ajliming A few observations can be made. First, the desired term can be interpreted as the input signal weighted by the mean of the product of the analysis and synthesis lifters, Secondly, if the coefficients of the aliasing term can be set to zero then a linear time invariant (LTI) system can be constructed out of linear time varying (LTV) components - decirnators and expanders. In this ease, when aliasing is calcelled the distortion function is given: by Unless T( z) is an allpass filter? that is, ) T(w) =cj f Of or a~~w, wes . yY( x) suffers from amplitude distortion. Similarly, unless T( I ) has linear phase, that is, aarp;I"(u) = a + bw for constcants a and B, wc say Y( x) suEers from phase di~torbios~. This leads us to the definition af a perfect reconstruction system. Definition 3.2.2.2, Let z(n) and y(n) be the input and ouCgud, rmpec- t i v e l ~ to the filter bank. Then, a perfect reconstruction system is a gygtem free from &wing; ampfitude distortion, and phase distortion, As such, y (n) iu a scaled and delayed vervion of z(n), that is, Let us recxamiac the output of the filter bank Y(x), which wa63 given or equivalent ly, 80 where Chapter 3 Multirate Filter Banks mi t i ng the s pt em of equations A,( z ) , n = By . . . , M - f ( in rniitrijc form yields I A(z) = --H(r)f ( z ) , M where, = [Aa(z), (z), . . . , AM- 1 (z)IT9 the synthesis bank is given by and the so-cdrled alias-cornpaned (AC) matrix H(x) is given by In this formulation, aXiasing can be eliminated if and only i f the gain for each of tbe aliasing terms equitfs zero, t hat it^, .Alc (2) = 0 for JG == 1, . . . , M - 1, Moreover, t o illsure peqect recolastaction Ao(z) must be a delay9 %hat is, AO(x) = z - ~ @ . The solution of this system of equations for yynthesis filters f(z), may have prxt i cal diaeulties. It requires the inversion of the alias-component matrix H( r ) . Even if successful, that is, if H(z) is nonsingular, there is no guarantee that the resulting filters f(r) are stable, t hat is, the poles of f (2) are inside the unit circle. The approach given in the next sectiolz presents a. difirent technique in which dl of the above CtiBculties are remowd. Let us provide an example t o illustrate the use of the alias-component m;tt;rix. Assume the analysis bank is cftnract;erized by the folowing filters: &(+) = l and H1( z ) =+- I . Find the analysis filters Fa(+) and F~(z) that satisfy the alias cancella- tian property, So, the alim-componmt forrnul&ion of the filter bank is characterized by 3.2 Quadrature mirror filter banh Writing this mat r k equation in terrns of its terms yields To insure alias cancellation, Aa(z) = a( r) and Al ( r ) = 0. Hence, Let us construct the alias-component matrix H( x ) , i . e. , Solving for H-' ( z ) yields Substituting H-'(z) and A(z) into this equation yields Solving for f ( r ) yields For perfect reconstruction, let a(r) = Z - ~ O , where rno 2 1, then 8 ?
- 4 2
a , .
&
8
u
7
. k
Z
z
8
3
u
c r r
! 2 w
! = ,
- s
- g
. a
- 5 Q
2
E
$ 4
-4 .a
= J
s
2
9
! i j
a ,
5 -
5
2i 4
B
4
g z
* %
2
*
" 3
c ; l
r c c
D
8
&
4 3
g
i t ' 4 - (
c d
2 3
2 . 9
= J ,
3
3.2 Quadrature mirror filter banks or equimlently in mat r k notation, where the synthesis bank is given by and the anal ysi ~ bank is given by Wrltiug Hk( z ) , k =z 0,. . . M - 1, in terms of Tme-I polyphase yields M-X f f k ( z ) = z - " ~ k , , ( r ~ ) for k = 0,. . . M - 1. m=@ This systenl of equations can be rewritten in mat r k notation as where the delay vector (called a delay chain) e~ ( z ) is given by eM ( r ) = [I, z-' , . . . , z - ( ~ - ' ) ] * and the polyphwe-component m&rk for the analysis b n k E(z) i s given Then, writing Fk(r), k = 0,. . . M - 1 in terms of Type-I1 polyphase yields M-1 Fk(z) = z - ( ~ - ' - ~ ) & ~ ~ ( z ~ ) for k = O? . . . M - 1. m=O This system of equations can be written in mzttrh notation as where the paraconjugation of e~ (t), denoted ZM ( z) ? is given by 84 Chapter 3 Maltirate Filter Bank8 and the polyphase-component matrix for the synthesis bank R(r) is given Substituting these equations for h(r) and fT(r) irlto the equation for Y( z ) yields Let P(x> = R(x)lE(z). Then, ur equivalently, V(z) = T(z)X(z) where the distortian function T(z) is given by These equations suggest the filter bank in Figure 3.7. 3.2.2.3 Refatioraship between formulatiolns Aa seen by the previous subsections, the study of filter banks can be ger- formed using either afiw-component or polyphme matrices. Now let us examine the relationship between these twa formulations. The AC mat r k XX(z) is given by 3.2 Quadrature mirror filter bank Figure 3.7, Poliyphase-component filter bank, RecaU that h(z) is given t u Hence, H~(Z) = [h(z), h(zWM), . . . , ~( zw$- ' ) ]. ReeaU that h(z) b related to the palypkae-component matrix: E(z) by Therefore, HT( z ) can be written as E ( z ~ ) ~ ~ ( z) , . . . , E ( Z ~ ( W; - ~ ) ~ ) ~ ~ ( Z W; - ' ) 86 But by definition ~ M( Z W; ) = 11, ,-I w ~ L , , . , Z - ( ~ - ~ ) w - ( ~ - l ~ i k : l ~ M 9 or equidently, where A(+) = diag[l, z-', . . . , 2- f M- ' ) I. Note that is just a single column of the W s matrix, where WM is the M x M discrete Fourier transform matrix. So, Since ~ f i = (Wg)T, then With this equation, any results obtained in terms of the polyphase formu- lation can be applied t o the alias-component formulation and vice versa. 3.2.3 A mdtirate sornrce-coding desi e example VV;e will now apply the results of the 1st subsection to a nrultirate souree- coding design problem. Let z(n) be an arbitrary sequence and let zl(n) be its first divided difference, i.e, Consider two sequences yo(n) and yl (n) that are defined by ~ o ( n ) = ~ ( 2 %) and yl(n) = z1(2n). 3.2 Q u h t ure mirror fif ter banb Figure 3.8. Filter bank equivdent circuit, Can we recover z( n) from yo(%) and yl(n)? It is important to note that the even-numbered samples are already available. Can the odd-numbered samples Be recovered? To salve this problem, let us first recast it as a two-channel QMF problem, wbem the aniltysiv filters ares Bo(x) = I. and HI ( z ) = 1 - z- I , the %-transform domain expression for the first divided difirence, So the filter bank is depicted in Figure 3.8, where the goal is to determine Fo(r) and F1( z) S U C ~ that we have perfect reconstruction. Let us first detemine the polyphase matrix E(z). From the last subseclioa, vve found that H(Z) = W?A( Z) E~( Z~) . Recall that w2Wf = 21z and h ( z ) i ( z ) = 12. So, mlving for ET(z2) we obtain 1 E ~ ( Z ~ ) = X( Z) ( ~WZ) H( L) and substituting the appropride errtries into the matrices results in or simply E(x) = Now, choose R( r ) = E-'(z) to ins reconstruction, i . e, y(n) = a(% - 1). Then, 1 0 R(z) = f -I, 88 Chapter 3 MuI1;I'sate Fdter Banks Since the entries of R(z) are not functions of t, then R(zZ) = R(z). There- fore, Compute the synGhesis filters using and substituting entries into %he matrices results in or equivafently, Hence, Fo(z) = I + Z- I . and Fl(r) = -1. Assume z(n) is a slowly varying sequence, that is, adjacent samples differ by a smdl amount. If each sample of s(n) requires 16 bits for its repre- sentation, then the first divided difirence, beirtg very small, requires only 8 bits Tor its representation. Now instead of storing (or trzlnsmitting) all samples af z(a), we dore for transmit) yo(n) using 16 bits per sample a ~ d yr(n) using 8 bits per sample. Thus, we have reduced the bat s rate from 16 bits per sample do an average of 12 Bits per sample, Nate, bowever, ttzat if the polypbwe matrices are utilized directly, that is, then the filter bank can be operated at half the rate. The regulling filter bank is depicted in Figure 3.9. The regular structure of this realization foreshadows the discussion of lattice structures in the next chapter. If the fllter bank is used flor compression, then there will be information loss between the analysis bank and the synthesis bank. This lass is due t a coding errors that age the result of quantization effects, This subsection will show that if the quantizer xloise is correlated with the input signal as in a Lloyd-Mw quantizer, then a sfiight modification to a filter bank can result in the cancellation of all signal-dependent errors, For completeness, we will briefly review Zeibnitz's rule and conLinuous randon1 variables, since they wit1 be used in the derivation of the Lloyd- M w q-ztantizer, 3.2 Quadrature mirror Alter Banks Figure 3.9. Eseient realization of dmign prabbm. Zleibnitzk rule describes haw we can diRerentiate a function which is af the form ! : ( , ; I g(z, t)dz. Theorem 3.2.4.X. Let where vl md vz may depend orr the parameter t and a 5 t 5 b. Then, for a 5 t < b, if g( s, t ) and $f are continuous in both z and t in some region of the st; plane induding y < a < vz, a < t < b and if y and vz are continuous and have continuoulr;. derivatives far a < t < b. Proof: Let pzl t ) P( t ) == g(z, t)&* Let t E (a, b) and At $ O such that t -+ At; E (a, b) , then = ~ ( t + at) - p(t) Chapter 3 Multirade Filter Banks or equivalently, How, grouping the second and fourth terms yields NOW scale this equatiorl by At yields Applying the mean-value theorem far integrals Lo the last two terms yields where 52 is between vz(d) and v ~ ( t -t- At ) a d tI is hetvvee~~ vz(t) and v,(t + At). Then, taking the limit as At --.* 0, the equation becomes 3,2.4.2 Continuaus raindam variables Suppose X is a continuous random variable or1 the interval with a proba- bility density function, f. Then, the probability t hat a 5 X < b is given P( a < X< b ) = where, I( is edled the probabilie density function. The ezpeeted value of X on the interval [a, b], denoted E[ X] , is given by 3.2 Quadrature mirror filter banh where a < X < b. Moreover, for axry positive integer k, where a 5 X < b. The variance of X, denoted Q; , is giverl by Substituting the definition of the expected value yields or equivalently, Simplifying this equation using the definition of E[ X] and E[ X2) yields or simply, G$ = i!z[x2] - [ E[ x ] ] ~ . Fbr completeness, the following descriptioxl of Lloyd-Max quarktizers [Max, 19601 is included. These quantizers minimize the expected value of the distortion (or error between the input and the output of the quantizer) of a signal by a quantizer when the number of levels of the quantizer is fixed, Let u be a randonr variable with a csntinuous probability density func- tion p ( ~ ) . It i~ desired to find the set of increaving transition or decision levels {t,J rn = 1,. . . L) with tl and t~ a9 the mi~kimum and maximum val- ues of ?I, respectively. If u lies in the interval [t,, t,+%), then it is mapped t o rm, the mt h reconstruction Level. The Lloyd-Ma quantizer minimizes the: mean square error for a given number of quantization levels. Therefore, for an L-level quantizer, the meark square error is t o be minimizd. The necessary conditions for this minimization are obtained by differentiating E with respect to t, and r, and setting the r a uk equal to zero. Now let us consider E. It yields By inspmtion, this partial derivative is only nonzero for k = rn. &nee, Setting = 0 yields Thus, the recsmtruction; levels of the Lloyd-Ma quantizer tie at the cen.t;er of mms of the pmbability densiw betwertn trmsition levels, Using the defi- nition of expected value of a, continuous randons vz~riabfe, we can reinterpret where IP i s a rmdom variable and t, 5 EP < t,+l. Now let us consider g. It yields 3.2 Qrxdrafure mirror filter bank8 93 By inspection, this partial derivative is only Elonzero for k = m - 1 and k -- rn. Hence, Then, using Leibnitz's rule, we obtain Setting rm 4- 0 requires us to set to zero the coefficient of p(t,). Thus, or equivalently, Hence, 2t,(rrn - ~ m- I ) = F& - ~ % - ~ r or equivalently, T m 4- Tm-t C, rr= 2 ' Thus, for the Lloyd-Max quantizer, the transition levels lie halfway between the reconstruction levels, We can reinterpret 1, as where X is a uniformly distributed random variable and T,-1 X < rm* As we just observed, the Lloyd-Max quantizer is an example of a quantizer, whose quantization noise i s correlated with the input signal. The most obvious model for such a quantizer is given by Figure 3.10, or equivalently, where the input signal t(n) and the additive noise q ( r ~ ) are correlated. We will show that we could equivale~ltly use the gain plas additive noise model, wltich can be expressed as Chapter 3 Mu1 tirate Filter Bank8 Figure 3-10. First qumtiaer model. Figure 3.1 1 .. Second quantizer model, or equivalently in. block diagram form in Figure 3.11, where a is "cbe gain. filctor (0 < a 5 I) and r ( n) Is the d i t i v e noise term. By making the proper choice of the gain a, we will be able to agsurne that r ( n) is additive uncorrelated noise. For these quantizers, assume t hat the expected value of the quantization noise q(n) is zero, t hat is, Let us briefly examine soxae interrelationships between the two models, Ram the first model, or equivalently, Els(n)l = E[t(n)l+ Eiq(n)l* Since E[ Q( ~ L ) ~ = 0? then E[s(n)l = E[t(n)l* from the secorid model, or equivalently, E[.(")] = .Bit(.)] + E[.(n)]. Equati~lg the two equations for E[s(n)] yields " - " - ,
, - " %
e
w
k
d @ - - .
F z
u
C S
& - - a
P s S
+
-
/ - - .
f2
u
I - .
Q
e. - - - - - ,
, + - .
r ?
v
k
r * 3 *
& 2
u
U
-
E z r
4 -
-
@ " " - .
F t
u
I - .
-
r ; 3
-
r " . " 4 1
r
w
* S
-
Q
I
N u
b
P -. T 4
I
u
i l
N
C r "
b
i
t ; ;
c l l
$ ; ,
!z
- - 'I
I? I . 4
s f
i ?
8
3.2 Quadrature mirror filter b a ~ k ~ Figure 3.12. M-channel filter bank with quantizeas. Figure 3.13. Additive: noiw eqnident circuit, 3.2A.5 Filter b with qum$izers The filter bank with quantizers will consist of the insertion of the quantizer made1 in each channel between the analysis bank and the synthesis bank, Thia ia depicted by the block diagram in Figure 3.12* The basic philosophy behind the design of QMF banks with quantiliser~ is to permit aliwing in the filters of the an&Iysis bank and coding gain in the quantizers, and then to choose the filters of the synthesis bank so that the dias-components in the filters of the andysb bank and the coding gdn of the quantizers are cancelled. The andysis of the QMF bank can be performed easily. First, realize that the eRect of ak is simply a scale factor for h t h the desired terms and the terms due Lo a-liming, Second, tke egect of the additive noise source .rk(n) can be described by the equivalent circuit per cllannel as depicted in Figure 3.13. Thus, the filter bank with quantizers yields Now choose new synthesis filters Gk( z ) to be defined iry 98 Chapter 3 Maltirate Filter Banks Then, the equation for V(x) becontea -.- desired terms I terms due to aliasing terms due to quantization Therefore, the errors due t o quantization are random signal-independent errors, which could be eliminated by post processi~zg the output of the filter bank with a filter that is designed to remove additive uneorrelsted rioise. In addition, the analysis that was presented in. the previous subsections is still applicable for all signal-dependent terms, In the previous section, we showed that the polyphme matrix P( 4 charac- terizes the behavior of a QMF bank. Wfiat are the properties of PCx) that correspond to alias-free ar perfect reconstruction filter banks? This section will address these questions. 3.3.1 Alias-free Alter bmka First, a background section on pnreudocirculant matrices is presented. This is foXlawed by the theory of diw-free filter banks. 3 3. X. I. Pseudocirculant matrices Definition 3.3.1.1. A matrix P(z) is sa,id to be pseudociwulant if it is of the following form Two important special eases of the pseudocircufant property are given below. f 1) If z = 1, then P(z) is circu1ant, (2) If z =. - 1, then P( z) is skew circulant. For example, a pseudocirculant matrix P(s) of dimension three is given by 3.3.1.2 Elimina&ing alias distortion Consider the filter bank depicted in Figure 3.14. Now, we will proceed with the a~nalysis af the filter bank, by examining it stage-by-stage, First, we will con~ider, the bank of decimators, which is depicted in Figure 3. f 5. The elemental equatiorl for this stage is given by Next, we will consider the P(z) block, which is depicted in Figure 3.16. The elemental equation for this stage is given by Lastly, we consider the expander block as giwn in Figure 3.17. The elemental equation for this stage is given by Ghapde-r 3 Muf tirate Filter B m k ~ Figure 3.14, Polyphwe reprmentatiorz of M-cfiiiwlnel%lter bank. Combining the equation for C,(r) and B.(r) yields Combining this equation with the one for Y( z ) yields or equivalently; where M- l V,(z) = C z- "z- ( ~- ' - ' ) P #?a (z"). Since x( ~w&) , k $0, represents the alias terms, then the resulting equa- tion for Y ( z ) is free from aliasing if and only if the coefficients of x(zw&), k $ 0, are zero, that is, M- 2 w&'"v,(z) = 0 for k jk 0, R=O 3.3 Foundations of filter banks Figure 3.1 3. B a k of dmirrza-k-or. Fi ere 3.16. P(x) Mock. Figure 3-17, Bank of expanders. ar equivalently in matsix notation, where * is a nonzero term. Premultiply both sides of this equation by WM. Then, using the fact that WM Wg = MI1 this equation becomes Since all the entries in the first colu~nn of WM are equal to one, then &( z ) = &(.) = V,-,(r). Therefore, let 3.3 Foundations of fillter banks and the alias ttems become M-I M-I j2~rFn I-exp(j2nk) C W;'" = C e ~ p ( ~ ) = = O for k $0. n==O n=O Thus, U(.) =: V(z)X(x) and the M-channel filter bank is free from sliming. Now, let us examine & ( z ) and & ( z) more carefully. By definition, ( z ) is given by Perform a change of variables by letting q = M - 1 -- s, Then, Similarly, by definition &( z ) is given by or equivalently, Perfarm a change of variables by letlbg q = M - s. Then, Because of the requirement that h ( z ) = Vl(z), the polyphase-components should be the same. So, Vo(z) is simply a pseudocirculant shift of &( z) . Similarly, the nth column is obtained from the (n + 1)st. Thus, P( r ) must be pseudocircuEant for the filter bank to be alias-free, Example 3.3.1.1. Consider the following example t hat illustrates these re- sults. Consider t he following P(r) matrix (4 .'&@I (2) pi2 ( z ) pie(z) f'i~(r) &(z) &a(%) P2l(Z) Pz2(r) Thus, Since then Bence, we earr write Let P2(2) = Poz(r), Pl(r) = q2(+), and Po(z) = q2(z). Then, P(z) becomes s ( ~) 4(+) S(+) z-'&(z) Po(z) PI(%) r-'Pl(r) z-'P,(z) PO(z) 3.3 Foundations of filtc3r ;banks 105 First, a background section on paraunitary matrices is presented. This is followed by the theory of perfect reconstruction filter banks. Definitian 3.3.2.1. A matrix P(z) is stkid ta be parraupzitary ifibnd only if where c is a pa~itive canstant. If c = I, thee PCs) knom as a namai i zed puraunitraq matrix. Two important speeid eases of the paraunitary property are given belaw when P ( z ) is a constant matrix. (1) Xf VV is: a matrix of complex-valued constants, then W is unitary if and only if WaW = I. (2) If U is a mat^ of ~al -val ued constants, then U is odhogonal if and only if UTU = I. Let us consider the following t hearem, which provides a fundamental result for products of paraconjugated matrices. Thearenn 3.3.2.1. Let A(x),B(x), and C(x) be maGrices suelr that then ~ ( t ) = ~ ( z ) ~ ( z ) . Proof: Let A(z) be I x n, C(L) Be Ix rn and Bj z ) be m x n so thal C( z ) B( r ) is defined. Then, e ( z ) is m x I and g(z) -- is n x m SO g ( z ) c ( z ) is also defined and it has the same dimension as ~ ( z ) . Let [A( z) l i j = ai j ( z ) , [c(J)lit = rili(~), a d [B(z)]kj = Bhj ( +) * Then, the ( i , j ) element of A(%) is given by = a;i (l / z*). Then, i j 3.3 Foundations of filter bank 107 First, consider the distortion function, which corresponds t o an alias- free filter bank, i.t?,, (see Secti~rr 3.2.2) or equivalently, For simplicity, represent P(z) in terms of its zeroth row, t hat is, PO,,-k (z), for k = O7 . . . 1~ &,a(.) = Z - ' ~ @ , ~ - ~ + ~ ( Z ) , for k = n + I.,. . . , M - I.. Perform a change of variables in the first summation by letting k = n - s and in the second summation by letting k = n- s+M. Then, the distortion f~~nedion beearnes or equivalently, Thus, the M-channel filter bank is free from a l i ~ i n g if a d only if the P ( x) is pseudocirculant, Moreover, the distortion function, corresponding t o this alias-free filter bank, is given by where S( z) is the polyphase representation of the zeroth row of P(z), that or equivalently, where B ~ ( z ) , the delay chain, is given by eM( r ) = [ I , z-', . . . , z - ( ~- ' ) ] * . Then, the kth rovv fk > 0) is a riglzt circularly shifted version of S(xf, or, The corresponding system of equations ia given by Performing a change of variables by replacing z with ZW;' yields the fallowing equation or equivalently, where e M( z w i k ) = ( z ~ & ~ ) - ' ~ . . . , (2 . This set of equa- tions can be described by 3.3 Foundations of filler banks where A(z) = diag[l, r -I, . . . , z- ( ~- ' ) ] . Since t h i ~ equation holds for all values of k for k = 0,. . . , M - I, we can write it as A( . ) ~ MQ( . ) = P ( ~ ~ ) ~ ( . ) w M where WM is the M x M DFT matrix and -(M-l l )]* Q( z ) = diag [S(z), . . . , S( z W, Thus, any prseudocirculant matrix can be diagonalized, 3.3.2.3 AmpUtude distort $an-free Alter bank Since A(r) and WM are paraunitary matrices, then Q(a) is a paraunitary matrix if and only if the peeudocirculank P(xf is paraunitary Moreover, Q( z ) is a diagonal matrix with element8 ~ ( r ~ i ' ) . Therefore, Q( t ) is a paraunitary mat;rix if and only if S( z ) is allpws, that is the magnitude of S(z) is constant, or equivalently, Q(z) is a paraunitary matrix if a ~ d only if T( z ) is allpass, since T(z) is simply a delayed version of S(a). Note that T(x) is allpass if and only if the filter bank is amplitude dislorliun- free with m o n o t o n i a l - r i g phase, hence, TCx) is dlpms if and only if the pseudocirculant P(z) k paraunitmy or equivnlently, the filter bank is amplitude distortion-&ee with monotonically-varying phme if and only if the pseudocirculant mat r k P(x) is paraunitary, 3'3.2.4 Perfect recomtrmcdioa. mt er bad8 Since P(.~)A(z)wM = &( ~) WM$( Z) then det(p(zM)) = det($(z)). However, M- l Therefore, M-1 det(p(zM)) = a s( zw~"$ I%=@ OP eqtuivalexttly in. the Fourier domain, Chapter 3 Muldiratc? Filter Banks h s u me S( z) is allpw8, then, for some nonzero d and a monotonically varying phave rp M- l det(P(exp ijwMj)) = dM n exp k=O or equivalently; Assume S ( I ) hau linear phase, then, ' p(w) = gw + f fer constants g and f . Hence, Recall t hat cE~=;' k = M( 7 - g . Therefore, or equivalently, where a = Mg and b = Mf - z g( M - 1). Hence, det(P(exp i j wM])) = dM exp [ j ( aw + b)] or simply, det(P(exp bwM] ) ) = c exp(- j wm) 3.3 Fouadstions of filter banks 111 where c = dM exp(jb) and rn = -a. In the z-transform domain, this equa- tion becomes det(p(zM)) = C Z - ~ . This equation implies that PoSr(z) must have a single norizero term. Let po(t) be a vector of the polyphase terms in the zeroth row of P(z), that is? Since each po(z) can have only one noIiaero component, then successive circular shifts of p@(z) will result in M different P( r ) matrices, each of which corresponds to a different perfect reconstruction system. Now, we (k) wilt enumerate them using the notation p, (z) to denote the fact that k circular shifts have been applied to po (t). Now, aYvume the initial unshifted fa) vector po (2) is given by which represents tlre yseudocilieulaxlt matrix Apply one circular shift to pr' (z) to yield pt' ( z ) , that is, The corresponding pseudocirculant mat r k Therefore, after k circular shifts of pr' ( z ) , we obtain which represents To illustrate this result, a two-channel filter bank has perfect reeoxrstructiori property if and orrly if 112 Chapter 3 Maltirate Filter Banks 3.3.3 Filter bank formulations in retrospect Xn this chapter, we introduced two fitter bank formulaliana: alia-component (AC) and polyphme formulations. The st;rategy of the AC formulation is to farce the ~oefticient~ of tlie Aimed terms to zero by imposing conditions on Ak(x), k =z 0 . . . , M - 1, the components of the gairk vector. Attributes of Filter Bank Characterizadian of A(z) Perfect Reconstructioxl AEI(z) = zWmo and AL(x) = 0; k f O As we saw in Section 3.2.2, this approwh can lead to dificulties. Xt requires the inversion of the alias-cornyonerrt xxratrk and, even if this is successful, there is no guarantee that the resulting analysis filters are stable. The strategy of the polyplime formulation is to =same the desired result and then to impose conditlarks oxr the polyphase ma-lrix P( z) in order to obtain t he correct behavior of the filter bank. Attributes of Filter Bank Properties of Pix) Alia-Ree, Anrplitude Distortion-fiee, Paraunitary, Pseudoeircufank and Monotonically-Varying Ph a c Perkct Reconstruction Paraunitstry, Pseudocireulant Linear P hme We will exarmine two types of filter bar~ks far spectral analysis, rtamelly the DFT filter bank and the cosine-modulated filter baxik. Bath of the filter bariks dkeussed in this sectiorh have the advantage that all their filters are derived from a single prototype filter. In addition, we will see th& the cosixie-modulated filter bank car1 achieve perfect reconstruction. 3,4,1 DFT Alter bank Traditionally, the OFT haa been used to analyze the spectrum of a given Bignal and an eacient, imglernentat;ian using FFT makes DFT a popular choice. Let h(n) be a towpass FIR prototype filter of length N" Then, the bmk of andysis filtenl, Hk( x) , k = 0,. . . M - 1, can be defined as Assume that the filter length N = mM, where M is the number of channels in the filter bmb md m is an arbitrary integer constant. Using the Integer Division Tbeorenn, let n = Mp + q, then Hk(a) becomes Since w$~'+') = W ~ ~ P W ~ = wZ, then the polyphase representation of Ek(z) is given by or equivalently, M-I H~(z) = )3 Gs(zM) W$ 2- 9 q=O where, Now expressing the analysis filter bank tlk(t), k = 0, ..., M - 1, in matrix form yields h(z) = w M where WM is a M x M DFT mat rh, Let 21' where the delay chain eM (2) = 1, z , , . . , ] . Since the polyphwe decomposition of h( z) is defined t o be then, by inspection, E ( z ~ ) = W M ~ ( Z ~ ) - If-$ Chapter 3 Mulkirate Filter Banks Assume that, the polyphme-comporrent matrix of the synthesis baizk R(xf = E( z) . Then, P(z) = R(z)E(z) = %(z)E(z)* or equivalently, p(4 = ~( ~) wEwM~( . ) . Since WgWM = MI M, then or equivalenll~r, Pfz) is a diagonal mat r k whose elements are given by Since P( x) is a diagoxlal matrix, it can becomc psendocirculant if all the diagonal elenlents are equal, that is, Orrce P(z) is pseudocireulant, then, by definition, the filter bank is alias- free, But, if all the diagonal elerrierlts are equal, that; is, then this corresponds to a square window for %he DFT corngutillion, which will cause artifact8 in Lire form of large main and side lobes. To reduce these artifwts we could use overlapping windows, but dltk would force us t o give up the aliw-free property. A way around this difficulty is to use cosine-rrtodulated filter bax~ks- This cstudy of eosine-alodctlated filter banks will be broken into two parts. First, we will define reversal matrices and then we will, examine properties of cosine-xnodulated matrices. Then, we will analyze cosine-modulated filter banks, 3.4.2.1 Reversal matrices Definition 3.4.2.1. Reversal nlatrices are matrices with zeroes in all ea tries except on the anticIiagonal. Reversal matrices of dimensio~i m are denoted ISy Jm* 3,4 Filter banlrrs for spectral ma l y ~ i ~ For example, J3 is given by: 3.4.2.2 Gosinc?-moddated matrices In order to andyze cosine-modulated filter banks, we will need to provide a couple of theorems. Theorem 3.4.2.1. For each k , q = O ,... , N - 1, l et c kv q = 2cos[(2k + l)&i7(~ - mM + 3) + (-I)"]* Then, C L , ( ~ M ~ + ~ ) = Ck,g. Proot: By definition C ~ , ( Z M ~ + ~ ) is given by Recall that for two angles a and P, cos(a + P) = cosa cos @ - sin a sin P. Hence, C L , ( ~ M ~ + ~ ) = 2 C O S [ ( ~ ~ + l ) p r ] COS [ ( ~ ~ + l)&;i(;i(4 - mM f $1 +(-1)"] + 2 sin[(2k + l)p?r] x sinl(2k + l ) &( q - mM + $) + (-I)"]. Since sinL(2k + l)plr] = O and cos[(2k + l ) p] = (- 1)P, then Theorem 3.4.2.2. ctTef = 2 ~ 1 ~ ~ + 2 ~ ( - I)("-') JM is an M x M reversal matrix and for every ~ k , ~ by. T - 4
; r i
I
C 4
5
3
+
- 5
w
c ' "
8
* g
I
I I
4
-
4 2
a
b
3
Q
, ,
4 - z
t :
E d
* -
t i !
g i h B a
. E l
2
&
4 .a
a
*
3
8
' 2
1
F
-
k t *
3 . .
1 1 . 1
I
u
h m ." " d
E :
r f i
E n
-
@ -%
& I N
+
@ r ,
u
k t $
/- %
4
+
IS I?
C I J
& .&
& ," " .,a
a
r e *
w 3
C J
I
Since co~[(-l)~$] = $ and sin[(-I)"] = (-I)'+$ therl (Ao]~,~ and IAZIL,~ become sr equivalently, Recall that cos(a-p) = cos a cos p+sin a si n /3 and sin(&-p) = sin a cos P- cos a sin p. Then, ( AQ] ~ . ~ and became Since cos[(2k + I)$] = O and sin[(Zk + I):] = (-I)', then [A@]k5p [Al]h,p become and = &c..[(zk + l)&(p + ?)I -(-l)kJZsin[(2k + 1)&(g + f ) ] . Let G and S denote Type-IV discrete cosine transforms (DCT) and Qpe- I V discrete sine tran~forms (DST) matrices, respectively. Then, the elements of C and S are given. by and %= = G s i n [(2X + 11- t21M Let A be an M x M xnadrk defined by [A]Lp = ( - 1 ) ~ 6 k * ~ ; k, q = 09 * a * M -- 1. Then, l a s l k , = (-Ilk [SIkq. Therefore, [ Ao ] ~ , ~ and [ AI ] ~, ~ can be written as and [-&I,,, = Jii;i [C - ASI,, * Consider the equations for AFA~, A ~ A ~ , ATA~, and ATA~ . A ~ A I = M (C + ~ $ 1 ~ (C - AS) cTc - C ~ A S + S ~ A ~ C - S ~ A ~ A S ) , and aTa, = M (C - AS)^ (C - AS) k,k = (-llk (-1)' = 1. Therefore, A ~ A = Inr In addition, since the Type-IV DCT and DST matrices are orthogonal, then CTC = IM and ST$= IM. IIence, 3.4 Fijter banks for spec trd andysis 119 Let J M be the M x M reversal matrix. Then, ASJM = C. Moreover, since 52, = IM, then CTAS = J M and STnTC = Jnn Therefore, which proves the special cave of m = 1. We must now extend our results for arbitrary integer > 1, by expressing the interrelationships between {A@> Al ) and {A0,Al). For m even, A; = ( - 1) T~x A; = (-1)Y-lAo Utilizing these relationships gives the follawing: For rn even, A ~ A ; = ~ M( I M - JM) nbTa; = o A;TA~ = o n;Ta; = ~ M ( I ~ + JM). For rn odd, A ~ ~ A ; = 2M(Ins + JM) AiTA; = O a;Tab = o A; ~A; = 2M (IM - JM) . Combining the odd and even results yield8 120 Chapter 3 Muidirate Filter BanErs 3.4.2.3 Cosine-mad&Led filter banks Let h(n) be a low-pass FIR prototype filter of length N. Then, the bank of analysis filters, Hk(z), k = 0,. . . M - 1, can be defined as where and h(n) is a linear phase, low-pass FIR prototype filter. Assume t hat t he glter length PJ = m2M, where M is the number of channels in the lilter bank and m is an arbitrary integer constant. Using the Integer Division Theorem, let n. = ( 2 M) p + q, then Hk (z) becomes Since G , ( ~ M ~ + ~ ) = (- ~ ) P c ~ , ~ , then the polyphase representation of Hk ( r ) is given by where, n-l Now, expressing the analysis filter bank Hk( z ) , k = 0, ..., M - 1, in matrix h r m yields G ~ ( - z ~ ~ ) Z-'G~(-Z~~) h(r) = C' 9 z-(zM-l)~zM-l ( - z ZM) where C' is a M x 2M matrix defined by Let go(-.) = diag [Go(-z), GI ( - z) , . . . , GM-I (-z)] I" where the delay chain eM ( z ) = 1, I-', . . . , z - ( ~ - ' ) ] . Since the polyphase decomposition of h(x) i s defined lo be then, by inspection, Atraume that the polyphase-component matrix of the synthesis bank R(z) = @( I ) . Then, P( z ) = g ( z ) ~ ( z ) JM 0 Since c' ~c' = 2M12M + 2 ~ ( - l ) ( ~ - I ) , then 0 -JM 122 Chapter 3 Mul6l"rat.e Filter Bank8 Assume H(r) has linear phase. Hence, the polyphase-components of H(z) are related by GL( z ) = z-'m-1'G2M-1-k(%). Since and 2 g, ( - ~2) = diag [ ~ M- l ( - r ~ ) , GM-2(-z ), . . . r GL)(-z') %hen r"C go(-+') = z-~("-') JM diag and "C 2 GI(-z2) = z - ~ ( ~ - ~ ) JM diag [ GQ( - Z~) , G~( - ZZ) , . . . ,GM-l(-z ) or equivalently, go ( - z2) = ~ - ' ( ~ - l ) J Mlpl (-Z2) and - gl (-zZ) = J ~ ~ ~ ( - z ~ ) . Substituting these equations for go(-z2) and g1 (-2') into the equation for P (z) yields Since go (-rZ) and g1 (-r2) are both diagonal matrices, then ( - z2) com- mutes with (-zZ), d.e. Therefore, the second term of the equation for P( r ) equals zero. Hence, or equivalently, P( z) is a diagonal matrix whose elements are given by Since P(z) is. a diagonal matrix, it can became pseudocirculat If all the diagonal elements art: equal, i, e., IP(z)lO,O = [p(z)ll,l = * = p(z)lM-l* M- 1 * Once Pf a) is pseudocirculant, then, by definition, the filter bank is di m- free. Moreover, if all the diagonal elements of pseudocirculant P( z) are equal t a a constant timers a deEw then the cosine-modulated filter bank satisfies the perfect reeoxlstruetion property. 3
R "
I
.+ -'%
h S ,
p
G 2
w
% a .
, - - ,
F
+ -
- ' *
t i .
t z l
F a Q M
,5 = J
Y
Y
k - '
" 5 ' Proof: Obserw that z~i , i i : h a compact support and by construcGion satisfies ~ ~ j , k l l & . ( ~ ) < m for each j E % and k E N. Therefore, { ajVk[ j E Z, E NJ c L~( R) . Next we demonstrate that { 1 j E Z, k E N) is an orthonormal set. By the nonoverlapping prop rty of the positive support intervals, it i s clear that (U j , h, %i , l ) = 0 if li - > 2 f or d l b,l E N, "f"herefore, we need to consider two cases: (I) i =: j and (11) Case 1: We will show (uj,k, u j , l ) = 6 ~ , ~ . To see this, observe that P From the definitions of even and odd extensions one sees that Note that C = 6 k t I by choice. For A, let a: = aj -0 and for B, let z = aj +a to yield BY Property (c) , wj(z - C) = wj - l ( s + 0) and so property ( d) implies hence, A + B = 0. Similarly, in integral D, let a: = aj+l - CF and in E, let z == aj+l 4.o t a get Properties (c) and (dl ) imply D -1- E: .- 0. Consequently, Case II: We will ~ h o w ( u ~ ~ ~ ~ z L ~ ~ ~ ) = O for i = j - l for every j E Z* k, rT E N then the ewe i = j + 3. wi11 follow ewily. 3.4 Filter banlrs for spectral analysis 125 By construction, %j-l,k and uj,i are possibly nonzero only on (aj - E j , aj + ~ j ) , hence From the definitions of the extensions In the first integral, let s = ai -cr and in the second integral, let s = aj +o, t hen Proper@ fc) inrpller~ Therefore, ( ~j - ~, k, uj J) = 0. Cases I and II demonstrate that { ujtk[ j E Z, k E N) is an orthonormal set. Lastly, we prove that .( %j,k ( j E 2, k E N) is a basis for L2 (R) . To do this, we will show that given s E L2(R) there exists a set of scalars .( aj,kl j E 21, k E N) such that in the LZ(R) sense. Let s E L2(R) and define sj(z) = s( z) ~j ( z) for each j E 2. Since sj has positive support on ( aj W E j , aj+l +ej+l) We fold sj(z) on (aj - Ej7aj) and (aj+I,aj+l + Ej+l) into the interval [aj,aj+l] by defining kj(~) Now hj is supported on [ aj , aj+l] . Consequently, there exi st real numbers aj,k for k E N such that 226 Chapter 3 Mtlltl'rate Filter Banks where convergence and equality is in the L2( I j ) sense, and ajsr, is given by the i~lner product rule Applying the furretion rules at both exldpoirrts yields n where hj h a an odd extension about z = aj and an evctn ext;ensicsn about )c z = aj+l. That is, the use of the extension rules for fjtk(z) applied t o hj ( z ) yields the following Multiplying ^h j (z) by wi (z) and summing over j produces To coxnplete the proof, we wiU show If z is a point where s(z) is defined and z E [aJ + E J , a ~ + i + E $ + ~ ] for some fixed J E 25 then If 2: E (aJ - E J , ~ J + E J ) , then + [ ( n : - f ~ z ) f ' g - ( a ) f s j ( z ) f m + [ ( z - r ~ ) g ) x - r ~ + ( Z ) I - T S ] ( z ) I - f m = ( r ) g y ( z ) ~ m z 3 [ 3 w Chapter 3 Maltirate cFilter Banks Figure 3.19. M-channel muldidimensionaf. Glter bank, and polyphase, Then, in a manner andogous t o the one-dimensional case, vve will closely examine the polyphme matrix, whkh characterizes the be- havior of the multidimen~ional QMF banks. For simplicity irr Clrapter 2, we were able to refer to shift vectors as a without having to msume an ordering of them. In multidimen~iorlal filter banks, we will. need ta asociate an ordering t o the shift vectors, so they T vvill be denoted ai, where a0 = [Q, 0,. . . , O] by convention. 3 .5,1 Mufrtidimensional Alter b a d formulationis We wilt discuss two digerent approaches t o Lhe analysis of multidimensional. filter banks: the abas-component formulation and the polyphwe formula- tion. Then, we vvill show that the two formulations me equivalent. The input is denoted s(n) lund the output is denoted y(n). It consists of J(M) multidimensional decimators; J(M) multidimensional expanders; the J(M) multidimensional analysis filters denoted Hk(s), k = 0, . . . , J(M) - 1; and the J(M) multidimensional synthevis filters denoted Fk( a) , k = 0, . . . , J f M) - 1, Figure 3.19 depi&s an M-channel multidimensional filter bank. The basic philosophy behind the design of MD-QMF banks is t o per- mit aliasing in the multidimensiond filters of the analysis bank and then choose the multidimensional filters of the ayn"cfiesis bank so that the alias- Figure 3.20. Multidimensionill mafysis bank. compments in the mullidimensionaI, filters of the analyois bank are can- celled. Now, we wilt proceed with the analysis of the nrultidimensiorld yuadra- tare mirror filter bank, by examining it stage-by-stage. First, we will con- sider the analysis bank, which is depicted by Figure 3.20. The elemental equation for this stage is given by Then, we will consider the bank of deeimatom, which is depicted t?y Figure 3.21. The elctmentrtl equation for this stage is given by The following stage consists of a bank of expanders, which is depicted by Figure 3.22. The elemental equation for this s t q e is given by Lastly, we will consider the synthesis bank, which is depicted by Figure 3.23. The elemental equation for this stage is given by Chapter 3 Multirate Filter Bank8 Figure 3.2 1. Multidimensiond bmk af decimators. Figure 3.22. Multidimeasional; b m k of expmder;~. Figure 3.23. Multidimensional; synthwis bmk. Combining the equations for Xk( z ) , V k (z), and Uk (I) yields Finally, combining t hi8 equation with the equation for Y (z) yields desired terms terms due to aliasing where, arld fies at the origin. A few observations cm be made. First, the de- sired term can be interpreted as the multidimensional input signal weighted by the mean of the product of the multidimensional analysis and multidi- measion& synthesis filters. Secondly, if the coefficients of the aliasing term can be set to zero then a linear time invariant (LTf) system can be con- structed out of linear time varying (LTV) canrponents - multidimensional decirnators and multidirnensionat expanders. In this cae, when diwing is cancelled the rnultidimnsianal distortion function is given by Unless: T( s) is allpass, i,e. = e $i 0 for all g, we say V( z ) suEers from amplitude disto ly, unless T( z ) hw linear pphase, i . e. #(g) = arg g(T(exp (jg)) = a f bg for constant a and b, we say Y( z ) suffers kern phme distortion. If the s y~t em is free from dirtsing, amplitude distortion, and phase distortion, then T(z) is a pure delay, i. e. T(z) = CZ-"? c + 0. In such a system, yfn) is a sealed and delayed version of z(n), i. e. y (n) = cz(n - no), and the resulting system is called a perfect reconstr2~clion systern, Let us reexamine the output of the filter bank Y(z), which was given BY JIM)-l 1 Y( z ) = C r n A i ( e ) X z exp [- j(2?r~-~)a~ i =O where Writing the system of equations A~( z) , i = 0,. . . , J(M) - 1, in matrix form yields 1. A(z) = -H(z)f(z), J(M) where the vector of gain terms A(z) is defined by [A(%)], = A k ( g ) , the synthesis bank f (z) icr defined by and the alias-component matrix H(z) is defined by In this formuladion, aliasing can be eliminated if and only i f the gain for eaeh of the aliasing terms equals zero, t hat is, A*(%) = 0 for i = 1, . . . , J(M) - 1, Mor emr , t o insure perfect recuwfwction Aofz) must be a delay, i . e. A@(%) = z-rnO for some integer rector m. The solution of this system of equ&ions for f ( ~ ) may have practical difficulties. It requires the inversioll of the alias-component matrix H(z). Even if successful, that is, Hfz) is nonsingular, there is no guara~ltee that the resulting filters f(z) are stable. The approach given in $he next section presents a, CfiRerenL technique in which all of the above diEculties vanislli, Now, we consider the polyphme representation formu1a;tion of multidimen- sional filter banks. Towards this end, we will expand the desired result in terms of pofyphanse, Then, we wifl determine the conditions t o be placed on this result su aij t o achieve perfect reconstruction or simply slim can- ceUation, Now, recalf t hat the desired result is given by or equivderrtly in matrix notation 1 Y( z) = -fT(g)h(z)~(z), JIM) where the synthesis bank is defined by [f(z)lr: = Fk(z), and the analysis bank is defined try [h(e)l, = HL(E)* Writing Hk (s), k = 0, . . . , J (M) - 1, in terms of Type-I polyphase yields for aj E N(M) and k = 0,. . . , JfM) - 1. This system of equations can be rewritten in matrix notation at4 h(z) = E ( ~ ) (zM )eM (z), I34 Chapter 3 Multirate Filter Banks where the delay ehain eM(z) is defined by and the J(Mf x JIM) polyphase-component nlatrix for the multidimerl- sional analysis bank E(~)(z~) is defined by aj E N(M) and le = 0,. . . , J(M) - 1 . Then, writing Fk(a), k = 0,. . . , 3(Mf - I in terms of Type-II polyphwe yields for aj E N(M) and k = 0, . . . , J(M) - 1. This systenl of equations can be written in miltr-irtr notation as where the paraconjugation of eM (z), denoted gM (z), is given by and the J(M) x J(M) polyphase-component matrix for the synthesis bank R ( ~ ) ( z ~ ) is defined by aj E hr(M) and k = 0,. . . ,J(M) - 1. Substituting these equations for h(z) and fT (z) into the equation for Y (z) yields Let ~ ( ~ 1 (aM) = R ( ~ ) ( ~ ~ ) E ( ~ ) (zM), Then, or equivalently, Yl zf =. T(x)X(z), where the multidimensional distortion function T( z ) is given by These equations suggest the filter bank depicted in Figure 3.24. Figure 3.24. Multidimensional filter bank, 3.5.1.3 Retalions;lftip between formdaklions As seen by the previous subaeetions, the study of filter banks can be per- formed using either alim-component or polyphae matrices. Now let US examine the relationship betwen these two formulations. The AG m & ~ x H(z) is defined by so, Recsll t hat h(z) is defined by [h(z)l, = Hm(z)* Let the ith row of HT(z) be designated by Recall that h(a) is related to the polyphase-component matrix E( a) by Therefore, HT(a) can be defined by Since [eM (%)Ik = then 136 Chapter 3 Mu1 tirate Filter Bank8 Let A(z) = diag[l, z-a' , . . . , z- ~J( M) - ' ] . Then, H~(z) = E(~)(z~)A(z)w& With this equation, any results obtained in terms of the polgphwe formu- lation can be ayplied to the alias-compoaent formulation and vice versa. First, a background section on generalized pseudocirculant matrices is pre- sented. This is followed by the theory of multidimexrsionstf aEiw-free filter banks. Definition 3.6.2.1. Gives a sampling matrix M a ~ d a specific ordering of shift vectors ai, i = 0,. . . , J(M) - 1, in N(M). Then, a generdized pseudocirculant matrix P(z) is defined by where a* + aj - at(i ,j )) and f (i , j ) is the integer defined by ((ai + = af(i,j). Algorithmically speaking, P(z) can be determined by the following se- quence af operzations: 1. Evatuate ai + aj; i=13. ". , J(M)-1, j =O,. . . ,J[M)-I* 2. Define f (i, j ) = m, where ((& + aj))M = and 3. Solve for gf i, j ) using 4. Evaluate $he folowing equation Therefore, given a sampling matrix M with k cosets, then all the relation- ships that define the generalized pseudocirculant matrix are of the form z~~'~~~P~+~(z) = Pfci,jl*i(z) ; where i , j , f ( i , j ) E { O, 1, , k - Thus, P(z) is a k x k matrix, since dim(N(M)) = k. For example, let us determine the generalized pseudocirculant matrix with cosets given by In addition, note that M-'=!j 1 1 Case I (aa + R) : 0 1 - 1 (I) Evaluate a~ + a1 = 3- - = ax. 0 0 0 (2) Since a0 + a1 E N(M), then ((a0 + = ao + al = ale Thus, f ( 1, O) = 1. 0 (3) Since a0 + a1 = af(lpo) = al, then Mg(1,O) = 0 0 Hence, g(l , 0) = 0 (4) Therefore, .oO$~o,o(~) = 4,1(.), or equivalently, P0,0(~) = Pl,l(~). Case I1 (% + az) : 0 1 (1) Evaluate a@ + a2 = -4- = 832. 8 1 (2) Since ao + N(M), then ((ao + = + = a2. 138 Chapter 3 XMulLirade Filter Banks 0 (3) Since a 0 + a2 = af(2.0) = az, then Mg(2,O) = 0 0 Hence, g(2,O) = 0 (4) Therefore, . : . ! PO. @( . ) = Pz,z(.), or equivalently, Po,o(s) = Pz, 2(z). (2) Since a1 + a 1 # N(M), then Thus, f (I, 1) = 2. (3) Solve for gf 1,1) : (4) Therefore, . : ~! ~t , o( ~) = Pz91 (z), or equivalently, ~ 0 ~ 1 , 0 ( ~ ) = &,I(Z). Case IV (al + az) : 1 1 (I) Evaluate al + % =: 4- .-,. - 2 0 1 1 (2) Since a1 + a2 # N(M), then Figure 3.25. Multidirrteasianaf filter bank, "m". 0 Hence, g(2,2) = 4 - 1 (4) Therefore, .oO.:pz,o(.) = Pt, a(g), or equivalently, aPZ,o(g)' 211,2(%). With the conditions that were obtained by considering the above cases, the follawing generdized pseudocirculant mat r k i s obtained, Consider the follovving multidimensional filter bank depicted in Figure 3.25. Xn the polyphae representation of one-dimensional filter bank, we utilize delay elements at both ends of the filter bank, since there is no need to make a naneausal filter bank because of delays. On the other hand, in a muldidirnensionl filter baak, we will ~hi f e the inputs and shift them back at the outputs, We observe that Y ( z ) =: where Since X( z e ~~[ - j ( 2 ? r M- ~) k ~] ) , I $0, represents the alias terms, then the resulting equatiun for Y (z;) is free from Aiming if and only if J(M)-t C ~ X ~ [ - ~ ( ~ E M - ~ ) ~ ~ ] - ' ~ V ~ ( Z I ) = O for I # 0, or equivalently JfM)-x e ~ ~ l j 2 ? r a T ~ - ' ~ ] ~ , ( z ) = O for I # 0. m=O But exy lj2akT~-'k,] i s simply the (1, m)th element of the complex con- jugate transposed of the generalized discrete Fourier transform matrix, w&'. Hence, Pre~nultiply both sides of this equation by w$'. Then, using the fact that H L= J(IJlIr)I) this equation becomes Since all the entries in the first column of wZ' :)re equal to one, then %(E) = %(Z) = a s * = VJ(M)-l(~) Therefore, let V(z) = V,(z) for m = 0,. . . ,J(M) - 1. 142 Hence, Chapter 3 Multz'rilte Filter Banks for d l i = 0,. . . , SCM) - 1, or equivalently, for all i = 0,. . . , J(M) - 1. Using the Multidimensional Division Theorem, we can express ai $- a* as where g( i , j ) E N and at($,) E N(M). Then, for all i = 0, . . . , J(M) - 1. This leads t a for all i, j = 0,. . . , J(M) - 1. Polyr~omial matrices satisfying this relati011 are called gerleralized pseudocirculaxll matrkcs with respect t o M for a specific ordering of ais in N(M). 3.5.2.3 Perfect reeomtructioa QMF bank Tlie multidimensional filter bank xhieves perfect reeonstruetian if and only if P(z) i s a generalized pseudocirculant matrix and all the elements in the first column are zero except the one P3, 0( ~) that is equal t o a delay, t hat is, if j == j0 is the index of the nonzero entry, then czM if j = jO P,%@(Z) = 0 otherwise. 3.6 Problems I, Consider a two-channel aliw-free filter bank. Using the alia-cornpone& formuhtion, solve for the most general form of synthesis filters. If we assume perfect reconutructio~, then how do these equations simplify? 3.6 Problems 143 2. Let z(n) be an arbitrary sequence and let sl(n) be the first divided diEerencc, that is, and let zz fn) be the sum of the laat t m samples, that is, z2(n) =I z(n) + z(lt - I). Consider the two sequences yl(n) ctnd yz(n) tllstt are defined by yl (n) = z1 (2%) and yz (n) = zZ (2n). Can we recover z f n) from yl (n) and (a)? 3. Consider a special case of the Lloyd-Ma quarrtizer, where the prob- ability density function is a constant over the &-transition XeveXs of the quantizes. Wfi;tfi is the probability density functicm? What is the corresponding mcta~i square error? 4. Let A(z) and Bfx) be k x k pseudocirculztnt matrices. Prove that A( z) commutes with B( x) , tha'c is, 5. Construct the generalized pseudocirculant matrix using the sampling matrix and with eosets defined by a0 = 0 and a1 r= 0 Chapter 4 Lattice Structures This chapter introduces the notion: of lattice structures for the redization of filter banks, The origin of lattice structures for continuous lossless systems was a classicrtl theory of LC circuit networks, since they do not generate or dissipate energy (see, for example, Belevitch(21). Results on discrete time systems and their factorization can be found in Vaidyanathan et aI.[51] and Doganata and Vaidyanat han [14]. Sonle of the concepts developed in this chapter are ds o discussed in the texts by Fliege[lSI1 by Strang axkd Nguyen [46], and by Vaidyar~at ha11 [49]. Section 4.2 introduces multi-input multi-output (MZMO) linear system theory: Section 4.3 presents lattice structures for lassless systems. 4.2 Ramework for latit ice structwes This section systematieallly presctlts concepts that a t as. a framework for our study of lattice structure^. These concepts include an introduction t o multi-input multi-output systcrris, the Smith-MeMil1an.n form, and the McMillan degree of a system. 4.2.1 Multi-input mufti-output systems Befinition 4,2.1.1. Let u(n) be an input vector of length r , that is, and let Y(R) be an output vector of length p, that is, 146 Cliapter 4 La t tice Structures men, the multi-input multi-ou tpu t system is described by where, and [H(z)lk, denotes the transfer function from the m th input to the k th out;put* The matrix H(z) i~ called the transfer matrLv of the system. We will use the term r-iaplut p-ozdQut s ~ s t e m and p x r sgstem, iater&imgeably. 4.2.1.3, Lossliess system An irnpsrtartt elms of MfMO systems are lossless systems. De6nit;ion 4.2.1.2. Let H(z) be a p x r 8ys"tem. H(z) is said to be lossless if (a) each entry [H(z)jkm is stable and (b) H(z) is unitary on the unit circle, that is, H" (exp(jw))~(exp(jw)) = cI, for all w E [O, 2%) and some real constant c. I f c = 1, then H(z) is known as a nomal i zed lossteas ~y~t t j na, Since all FIR systems are stable, refere~lees to FIR systems tend t o use the words pa~azlnitary and lsssless interchangeably. 4.2'1.2 Imp&@ response matrix Let hkm(n) denote the impulse response of the transfer function Hkm( z ) . In addition, let h(n) reprevent the p x r matrix of impulse reuponses, where hrm(n) = [h(n))L, Then, where H(z) and h(n) are p x 7 matrices. The matrix sequence h(n) is called the impulse response of the system. To illustrate this idea, let 4.2 fimework for lattice sdrtrcturcs Tken, this can be writterr as The following theorem provides a11 important property of the impulse re- sponse matrices of laasless systems. Theorem 4.2.1.1. Given P( r ) = c,=, p(n) r-". I f P( r ) is paraunitary, then p B ( ~ ) p ( ~ ) = pB( ~) p( 0) = 0. Proof: Apply the definiltion of paraunitarinegs to P(z) to yield or equivalently, Since P( L) is yaraunitary, then $( z) ~( r ) = cI. Therefore, Since @( r ) ~ ( z ) will be nonzero only for coefficients of r', then pH ( N ) ~ ( o ) = pH(N)p(o) = a* It Matrk polynomials play an important role in MXMQ systems, r matrix whose ent ri e~ are poIynomids in z. The matrix earl be expregsed If p(k) is not the zero matrix, then k is called the order of the poIynomial matriz. For example, a causal FIR system is a polynomial matrix in I-' k t h order k, $fiat is, k 148 Chapter 4 Lattice Structures Definition 4.2.1.4. A unimodular polynamid matrix U(E) in variable x is a square polynomial matrix in s vvith a coastan t noxlzero determinant, To illustrate unimodular polynomial matrices, consider the hlluvving examples: is unimodular, because det (Ul (2)) = 1. and 1+z" u2(z) - is unirnodular, because det (Uz f x ) ) = 2. 22 2 The following theorems provide properties of these matrices, Theorem 4.2.1.2. EA is a u n h d u l a r polynomial matrix, then A-lexists and i s a u~liartodular po!ynorr;iial matrix. Proof: Let A be a unimodular polynomial matrix, Since det .A $ 0, then A-I exists a11d AA-' = I. Since det A det Am' = 1 and I det A1 =c , then I det A-' 1 = $. Since det AB = (det A) (det B), then (det A)(de = 1. Since A is uaimodular, then ldet A1 = c. Hence, ldet A-I Therefore, A- 5 s a unimodular polynomial matrix, II Tbeorevn 4.2.2.3, If A and B are unimobufar polynomial matrices, then A13 is a uriim~dltlar polyn~micitf matrix. Since A and B are unimodular matrices, = d. Since det AB =I (det A)(det B), then fore, AB is a urtimoduEar polynomial matrix, 4.2.1.5 m& of a poliynomiaf matrix: Definition 4.2.1.5. The raak of a poIynomial matrix is defi ~ed as the di- merrsiort of the su bmatrix that; corregpond~ to the large86 Banzero de termi- nanGal minor. CIearIfi if f (L) i s a p x r polynomial matrix, then rank (P(z)) 5 minip, T }. 4.2 framework for lattice structures 149 The theory of the Smith-McMlllm form is developed f-or causal Linear Time Invariant(LT1) systems. The theory of the Smith form for polynomial matrices is prevented first in order to provide the necessary background for the Smith-McMillan form. The analysis of the Smith form, that ipl developed in this ehapter for polynomid matrices, is analogous to the theory developed for integer ma- trices in Section 2.2. This is true because the set of polynomid ma t ~c e s and the set of integer matrices belong to a common algebraic structure called the p ~ i ~ e i p a l ideat damain, 4.2.2.1 Elementary operations Elementary raw (or column) operations an polynomial nratrites are im- portant because they permit the patternixlg of polynomial matrices into simpler forms, such z s triztngular and diagond forms. Definition 4.2.2.1. AR elementary row operation on a polynomial matrix P(z) is dearled to be any of the follawing: Type- 1 : Interchange two rows. Type-2: Multiply s row Izy a nonzero eonstant c. Type-3: Add a polynomial multiple af a row $0 a ~ot he r row, These operations can be represented by premultiplying P(z) with an appropriate square mat r k , called an elementary matrix, To illustrate these elementary operaions, consider the Eallowing examples. (By convention, the rows and columns me numbered starting with zero rather than one.) The first example is a Type-l elementary matrix: that interchanges row 0 and row 3, which hm the form The second example irs a Type-2 elementary mat r k th& multiplies elements in raw f by e f 0, which Itas the form 1 0 8 0 o c o o O O f Q 0 0 0 1 150 Chapter 4 Lattice Structures The third example is a Type-3 elernexltary mat r k that repfaces row 3 with raw 3 -+ (a(zf * rowQ), which has tbe form A11 three types of elementary polynomial ~katriees are unimodular polyrro- mial matrices. Elerneatary column operations are defined in a similar w;zv by post multiplying P ( x ) with the qprapriate square matrix, These elementary operations calk be used to diagondize a, polynomial matrk. The key theorem whiclr enables us to obtain this diagondization is the Division Theorem for Polynomialu. It states that if N( t ) and D(r) are polynomials in x, where the order of N( x) >_ order of Dfx), then there exists unique polynorniala Q( z ) and R(z) such that where the order of R( z) < order of D( z ) . 4.2.2.2 Smith form &eeo;mposit;ion Theorem 4.2.2,1, Ewr ~l poIy11omid matrix A(z) can be exprcissed in its eorrespondi~g Smith form decomposition as where TT(a), V(z) are uniraoduk polynomial matrices and the Smith form S(s) is given by $3 f z ) =: diag (so ( z ) , . . . 8,- (z), 0, Q1 . . . ,0) where r is the r a ~ ~ k ofA(z) &ad si ( z) s~+I(x), i = O,. . . ,T - 2- Proof: Assume that the zeroth column of A(z) contains a narkzero element, which may be brought to the (0,O) position by elementary operationu. This element is the gcd of the zeroth column. If the new (0,O) element does not divide all the ele~lenttr in t he zeroth row, then it may be replaced by the gcd of the elements of the zeroth row (the eRect will[ be that it will coxttajn fewer prime kctors than before). The process is repeated until an element in the fO,O) position is abt;tined which divides every element of the zlflrottl rovv anid eolurrtn. By elementary row and column operations, all the elenients 4.2 Framework for lattice structures 1 51 in the zeroth rovv artd column, other than the (0,0) element, rnily be made zero. Denote this new submatrh formed by deleting the zeroth row and zeroth column by C(z). Suppose that the submatrix of C(z) contains an element s,j (z) which is no% divkible by coo(z), Add column j to colurnn 0. Columxr O then consists of element8 coo, cl j, . . . , c,-1, j , Repeating the previous process, we replace coo by a proper divisor of itself using elerrlentary operations. Then, we must finally rellch the stage where the element in the (Q, O) position divides every element of the nnatrk,;md all other elements of the zersttz. row and column are zero, The entire process is repeat;ed with the submatrk obtrtined by deleting the zeroth row and column, Eventually a stage is rexhed when the matrix has the form 0 El z ) where D(z)=diag (so ( z ) , . . . , ( z ) ) and si(z) isi+% (z), i = 0, . . . , I. - 2. But E(s) must be the zero matrix, since otherwise A(x) would have a. rank larger than. r . rn By cenverltion the polyrlomids s i b ) , i = 0,. . . , r - I, are rnonie poIy- nomials, that is the highest power of the yolyrromial hm a coeBeient af unity. Note that although the two unirnodular polynomial matrices Ufz) and YCz) are, in generd, not unique, tlre diagonal matrix S(z) is uniquely determined by A(z). Example 4.2.2.1, Ta illustrate the Smith form decomposition, consider the fallowing example, Let z + l X Afz) =.; 2z2 + 3 2(2 + lla If we divide the (1,0) element, 2z2+3, by the (0,O) element, z+ 1, we obtaill 2z" 3 = quotient remainder Tlxereforc, if vve apply a Type-3 row operatiorr, whit-rlt is defined by to A(z), we will reduce the (1,0) element t o the constmt value of 5. There- fore, 152 Chapter 4 Lattice Structures Reduce the (0,0) element t o a cor~stant with a Type-3 row operation, which 1 - 2 is defined by 5 . Then, vve obtain a I Transform the (0,1) element t o zero by a Type-3 column ope ratio^^, which $ %( I - 2 t ) 1 -- is defined by 5 . Then, we obtain 0 1 Finally, the (1,0) element is forced t o zero by a Type-3 row operation, which 1 . 0 is defined by . Then, we obtain -5 X Thus, Let E( r ) be the product of elementary row operations, i . e. Let F( r ) be the product of elementary column operationu, i . e. since only one elementary column operation was performed. Therefore, 4.2 &&mework for lattice structures Then, the Smith farm decomposition is ghea by where, U( z) = E-I (z) = 9 1 - V(Z) = F-' (z) = 0 Let H( z) be a p x T transfer matrix of rational functions representing a causal Linear Time Invariant system. Assume each eelement [H(z)l km has been expressed as [H(z))~, = where the polynoaaid d( x) i s a lemt common multiple of the denominators of polynomials of H( r ) . Define a p x T miltrk P(z) wiLh elements fi,(x) and let P (z) = w (z)]t"" (z)V(z) be its Smith form decomposition, where VV(z), V(z) are unimodular poly- nomial matrices and the Smith form r f z ) is given by q-4 = diag ( 7 @ ( ~ ) 7 * . * 97F-l(z),o?aY * . * 90) and % must sattlsfy %(z)I%+l(z), i = 0, . . . , T- 2, Then, the Smith-McMillan decomposition is given by where the Smith-McMilliln form is given toy A(z) = diag (Ao ( z ) , . . . , A, -' (z), 0,0,. . . ,0) and Cancelling common factors between yi ( z) and d( z) yields 154 chapter 4 Lattice Structures where CV;(Z) and &(z) are relatively prime polynomials. In view of the divisibiliw property of the Smith form of P(z), since r;(~)l%~(z)~ then %+I (z) = c(z)%(z), and, as a result, . In addition, the ai+1 (2) and ( z ) O , . * * , T -2. 4.2.3 MeMlibn degree of a system DeAnition 4,2.3.l,. The McMillan degree, p, of a p x r causd sry;stem H(z) is the minimum number of delay units (z -' elements) required to implement it, that is P = det? (H(z)). If the system is noncausal, then the degree is undefined. If W( x ) =.. z-IR, where R is im M x N matrix with rank p, then where "IP is M x p and S is p x N, Therefore, Hence, we can implement the system vvith p deltays. So, the system hw a McMillan degree < p. As an example, consider We can rewrite N(z) i b ~ Thus, the system can, be imglennerrted vvith a single del;ty, illustrating that the MeMiUan degree of the system is unit;y. The Smith-McMillan decomposition prwides insight; into the determi- nation of the McMillan degree af an M x M fossless system, The following result is cent rd to the design of lattice structures. Theorem 4,2.3,1. Let H(z) be an M x M causal Iogsless system. Then, det W( x) is stable aIlpss and deg H(z) = deg (det H(z) 1. 4.2 Ramework for lattice structures 155 Proof: The stable allparcs property of dct H( z ) fallows from the lossless property of H(z), that is a ( z ) ~ ( r ) = c21. From the Smith-McMillan de- corn posit ion, M-L ali ( z ) det H( z ) = a n - *=, Pi(.) ' Since wi(z) and &(+) are relatively prime for every i and j , there are no urnaneelled h t o r s in the equation, Becwse of the causality of H(x), the degree of p j ( z ) (as a polynomial in z) is at least as large as the degree of a ( r ) (as a polynomial in z). Thus, the degree of the det H( z ) is equal to zEd,i1deg Pi ( z) , which is the degree of H( r ) . II Example 4,2,3,%. The following example glustrates this important result, Consider the following system: What i s the degree of the system? W will use tvwa metbods to determine this degree. Met hod 1 (DefilIlit;liola of Smit h-MeMil[larr, Form) The Smith-MeMiXlan decomposition of R(z) is given by 50, the Smith-MeMillan form of H(z) is given by or equivalently, so, ~ o ( z ) = a&) = 1, Po(. ) = r, and &( z ) = 1. Therefore, pa = deg h f z ) = 1 and p1 =. deg &( z ) =s 0. Hence, by the definition of the Smidh- McMillm form, the degree of H(x) = + pl = 1. 156 Chapter 4 Lattice Structures Method 2 (Theorem an McMfllan Degree of a Lossless System) First, we n e ~ d do determine if H(x) is paraunitary so, H[(z) is paraunitary and Theorem 4.2.3.1 applies. So compute the determinant of EIfz), that is det H(z) = (1 +- z - ' ) ~ - (1 - z - ' ) ~ = 4z-I. So, deg [ det H(1;) f =; I. Therelare, by the theorem on the McMillsan degree of a, lossfess system, deg H(z) .= I. 43 Latkice structures for lossless systems This section syslematicdfy presents lattice structures for lossless systems, Topics included are a fundamental degree-oxre building block and structures for losvless syster~s. Bouseholder factorizations are a;n important technique for the factorization of unitary matrices. They play an i~nportant role in the theory and imple- mentation. af lossless systerns, because lossless faetasrizations always involve a urlitary rnaittrix, Definition 4.3.1.1. Let x be an M-dime~sionai complex-.valued vector, such that the ith component d x is given by where Bi is real-valued and j = Assume x + 0 and define where ea is a unit vector in the direction of the zeroth cornponeat of x, (Pt, is important to note that i f x is red-valued, then the definr'gion tafu becomes. u = x 4- sign(z0) l2 eo.) The Householder transformation is defined by 4.3 Sdrucl; ares for ~ o s ~ ~ e s s syg terns Figure 4.1. Impfernentat ion af Householder transformation. The irnple~~entation, of a HousehoXder tranfi3fosmation is depicted ixz Fig- ure 4.1. It is impostant to note that Moreover, Renee, H is a unitary tranrsforrwratisn, In prwtiec, H never has to be exylieitlg cdculated, This is because Hx earl be predicted a pri n~i , But first vve need to compute uWx and uWu, or equivalently, 158 Chapter 4 Lattice Structures Since xHeo = I z a I exp(-j Oo), eFx = zo I exp(j eo), and ef ea = 1, then 1; = xHx. By combining results, we observe that = 1, Hence, or equivalently, Hx = x- ( x + exp( j 6) Therefore, Hx = - e ~ p t j e ~ ) 4.3.2 sl. hndamental, degree-one building black As we saw in tbe previous section, Housebolder transformations are given Definition 4.3.2.1. Let f f z) he a function and Ict v, be a wclor. Thee, the fundamentd d e p e - me structure for Io~sless systems, denoted V, ( x ) , is dearzed aa an extension of the Householder transformation by Let us apply v, to V,fz), that is, So v, is an eigenvector with eigenvalue - f (2). Let u be a vector orthogonal to v,, that is, vEu = 0, then 4.3 Structures for Io~sl e~s wrf~terna. 159 So, u is an eigenvector with eigenvafue one. Let uk, k = 0,. . . , M - 1, be rz set of M mutually orghogonal vectors, where u~. - . 1 = vm. Defi~re an M x M matrix U = (uDI.. . , U ~ - ~ I ~ , then we have Take the deternlinant of both sides of the equation deb (Vm f x)U) =; det But the determinant of the product of matrices i s equal to the product of the determinants of each individual miZt;rkUE. Hence, det V, f z ) det U == det U det Therefore, det V,(z) = -. f ( z ) . Since V,(z) is lossless, then deg Vm() = deg (det V, f a)) == degf-f(.)2. Since V,(z) charwterizes a degree-one building block, then dee; V, (t) Z= 1, Tlrerefarc, deg f - f ( x ) ) = 1. Let us examine V,(z) a little closer, Let Then, 160 Chapter 4 Lattice Structure8 and Q ~ = ( I - P ) ~ = I - ~ P + P ~ = I - P = Q . Thus, the projection operators P and $ satisfy the idempotent property, that is, P" P and Q2 = Q. In addition, Thus, P and Q satisfy the Hermitian symmetric property. Since P arld Q satisfy both the idempotent axkd symmetric properties, then P and Q are orthogonetl projections. III addition, Consider the fuxzction For Vm(z) to be lossless, then or equivalently, f f (zIf*(,) = 1' This implies that poles (and zeros) of f ( r ) are cancelled by zeros (and poles) of fs(+)* Con~equently, f ( z ) f *($) = 1 implies that poles (and zeros) must have zeros (and polea) in conjugate-reciprocd locations, This suggests the following form for f ( z ) , that is, Figure 4.2. Structure for lossIess system. where a, is s constant, Therefore, f (2) has a pole at a, and a zero at 2. Hence, the fundamental degree-one solution for IIR lossless systems is given by h r stability, all, the poles will be located ixlside the unit circle so When V,(z) is quantized, it remains unilary, since every occurrence of v,vg is normalized by vEv,. An interesting special case occurs when a, .= 0, Vnder thew conditions, V,(z) becomes a fundamental building black for FIR lossless systems, that; is, Note that the pure delay term z-l is also aXlpws, since it can be interpreted as a pole at zero and a zero at infinity, This section presents lattice structures for lossless systems. Let WN f z) be an causal losslesa system with deg HN(xf = N. Tk~err, we can fxtorize it (as HN( z ) = VN( ~) VN- I( Z) . * . VI (z)Ho where Ho is unitary and The structure realizing the factorization of HN(x) is given in Figure 4.2. Sometirncs, it, is desirable to replace Ho with its Householder factorization, 162 Chapter 4 La6l;ic.e Structures Example 4,3.3,1, Factorizakisn of M x 1 FIR Lorssfess System Given 8,023 - 0.5772-I Hi(z) = -0.016 - 0 . 8 2 9 ~ ~ ~ . Find the M x I factorizatioxl of HI (zf , Hl(x) can be eqtaidently written a Since h(llHh(0) =O, choose vl = h(1). Then, q1 (Z) = I - h(l)h(llH + h(lfh(l)H h(l ) "h(I) h(l)Hh(l) ' Then, = [I- + 2 h(0) + h(l)z-I = h(O) + h(l)z-' - h(l)z-I + h(1) == h(0) -t- fifl). Hence, P;br constant vector, perform a, Householder transformation, that is, where Therefore, HI Cz) - Vi (z)Uo, As such, the filter HI (2) can be realized by two stages Vt ( r ) and Ua. This filter is depicted in Figure 4.3. 4.3 Scructtlres for I os~i ea~ sy-s terns Figure 4.3. Two-stage filter. Example 4.3.3.2. Factorization of M x M Lossless Sysdem Given some positive ixlkger k, then the system Hk(z) is given by Find the M x M ktoriziltion of Hk(z). By inspection, Hk(z) is FIR, Ifi) Hk(z) paraunitary? Hence, Hk(z) is stable and paraunitary Therefore, deg He f z ) = deg (det Hk(z)). $0, deg HI:(z) = deg [(I + z-k)2 - (1 - r-')'I = deg [ 4 ~ - ~ ] = k. The impulse response matrix of Hk(z) is given by 1 Choose vk such that vfh(0) = 0. So, let vr: = . Then, -1 Chapter 4 Lattice Structures Figure 4.4, K-stage filter. Hence, Then, V(z) = V, ( I ) = * * - = V l i ( z ) will. be the degree-one building black. Then, Therefore, the filter can be realized by k stages V ( r ) and a constant scaling factor of 2. This filter is depicted in Figure 4-4, 4.4 Problems 1. Let G( z ) be given by the foUming: Express G( r) as a sum of impulse response matrices. Is G( z ) uni- modular? 2, Let B be given by the following: where a is a real-vdued constant, Use a Householder transformation to make B an upper triangular matrix, 3. Let H(z) be a system transfer function that is given by the follovving: where e is a real-valued eonatant, Is BIZ) paraunitary? Draw the cascaded structure to implement H( z f , 4. Let E( z ) be given by the following Is H( z ) lossless? What is the degree of H( r) ? Obtai~k a cascaded structure to implement H( xf , 5. Find the Smith-McMillan decomposition of the H( r ) matrix, which wau defixled in the Iafst problem. Then, uaing this result, determine the degree of the system. Chapter 5 Wavelet Signal Processing 5.1 Introduction This chapter introduces the notion of fundamentals of wavelet signal pro- cessing. Morlet et a1.[36] was the first paper t o discuvs the idea of a wavelet, although at this paint in time it WM a rather empirical idea. Later, Daubechieu[l2] showed that it was possible t o develop a theory and an as- sociated algorithm for the generatio11 of compactly supported orthonorrnal wavelets, Meanwhile, MallatftZq 24 developed the notions af muleiresolu- tion analysis and, recently, Shapirok42j has developed an eEcient scherne for encoding them. Results on biorthogonal wavelets can be found in Cohen e t a1.[8]. Many of the corlceptrs presented In this chapter are also discussed in other multirate and wavelet texts, including the engineering-oriented texts by Chan[5], Fliege[lSj, Strang and Nguyen[46], Vaidyanathan[49], Vetterli and KovaEevii.(54], Wornell[5?1 and the mathematicdly-oriented texts by Chui[7], Daubechiesll31, Holschncider[22], and Meyer[33, 341. Good ref- erence texts for background material an vector spaces include Riesz and Sz.-Nagy[39] and Schilling and Lee[.ll]. Section 5,2 introduces the wavelet transform, Section 5.3 presents mul- tiresolution analysis for both orthogonal and Ibiorlhogonal wavelets. 5.2 Wavelet tramform The name wavelet comes from the require~lexkt titat a fiunction should inte- grate t o zero, ul avi ~g above and below the axis. The diminutive eonnota- tion of wavelet suggests the function hw t o be well localized. At this point, some people might ask, why not use t rdi t i onal Fourier methods? Fourier bwis functions are localized in frequency, but not in time, So, Fourier aniiLfysi8 is the ideal tool for the efickexrt representation of very smooth, 168 Chaptw 5 Wavelet; Si g~d Procesging stationary signah, Wavelet bwis furrctions are localized in time and fre- quency. So, wavelet analysis is an ideal tool for representing signa1a that contain discontinuities (in the signal or its derivatives) or for signals that are not stationary. Definition 9.2.1.l. The mvelet tranufarm of g ( ~ ) with regpect to wavejet $(t ) is defined by where, a # 0 a~id b are called the scale and traas1al;ion parameters, regpee- titrely firtherrnore, the Fourier $randorm of m d e t Ift(l), denoted 9( f ) , must sati~fy the followi~g admissibillity condition: which &om that $(C) has to oscillate md decay, The inver~e of the eo~tinuotrs wavelet tran~form is given by It is often useful to think of functions and their wavelet transforms rn occupying two donlains, Then, the falowing properties show the corre- spondence between operations perfornred in one domain with operations in the other. 5.2* ?L. 1, Linearity property Theorem 5.2.1,11. If the wvelet transforms ofg and h exist &ad a and P are scafar~, then WT {ag + ph; a, b) = aWT {g; a, b) + PWT {h; a, b). Proof: By definition, By the linearity of irttegraGion, 170 Chitpter 5 Wavelet Sigaal h c e s s i n g Proof: By definition, t - tr WT (f (e - a); a, b ) = f ( t - a) +*(- ) dt. a Let zt = t - a. Then, or equivalently, WT{f(* - a);a, b) = WT{f;a, b - a). 5.2.1.4 DiRerentiiatioai theorem Theorem 9.2.1.4, I f the wavelet tra~gform of f exidsts and iff "exists, then the wa d e t traasform of f "is given by Proof: By definition, t - b WT( f '; a, b ) = ---- fr(t) $* (y-) fit* But for every red t , f(t + e ) - f ft) f ' (t ) = lim --+a E Then, using Lebesque's DominiitGed Convergence Theorem, we obtain Using the Linearity Thwrem, we obtain, WT{ f '; a, b) = lim,,~ ! 1" -cr0 f(t+e)$*(V)dt -& J?= f(tl$*(y Using the Shift Theorem, we obtain WT{ f';a, b) = lim WT{ f a, b + E ) - WT{f; a, 6) +a E Therefore, d WT{ f '; a, b ) = - WT( f a, b) . d b 5e2.311. 5 Gonvalatian theorem Theorem 5.2.1.6. If the wavelet t r an~f om of g exists and if f rl. g exi8ts1 then the wavelet. Grmsform of J .*. g i s given by WT { f * g; a, b ) = f WT { g ; a, b) , b where * de~ot es eonvdu tion with respect to the b wriablc. Proof: By definition, or equivalently, Using the Shift Theorem we obtain, or equivalently, WT{ f * $;a, b ) = f -k WT{g; a, b). D The following subsection ilkustrates the role that wavelets nliLLurijllly play in radar signal processing. 5,222 -dm signal processing itnd wav~li~ats The problem consists of estimating the location and velocity of some target in a radar application. The estimation procedure can be described by the following. Suppose z(2) is a knovvn emitting signal. In the presence of a target, this signal s(t ) will retur~l to the source ay the received signal h(t ) with a delay T, due to the target" location and a Doppler effect distar- tion, due to the tmget's velocity, If z( t ) is a na mw- ba d signal, then the Doppler effect amounts ta a single frequency shift foe The characteristics af the target can be determined by maimizing the cross-correlation function, This estimator is cdled tbe Ramow-Band Anzbiguily Fanetdon, that is, ~ ( t ) h(t - T ) exp (-j 2?rf ot ) dt 172 Chapter 5 Wavelet Signal Processing whi& is s b p l y the Short Time Fourier Transform (STFT) with shift T &out frequency f a. However, for wide-bmd signals, the Doppler frequency shift varies the signal spectrum, causing a stretching or comprcssian of the emitted signal, The resulting estimstar is c d e d the Wde-Baad Ambiguity Fanctiort, that which is continuaus wavelet transfornt, (CW'P) about a point T at a ~ c d e given by a. Thus, the wavelet transform is an operation that de8emines the similarity or cross correlation between the emitted sign& z(t) axld the received signal, the wavelet h y) at scale a and shift T. 5.3 Mdtiresolutirrrn analysis A8 we have observed, the wavelet transform maps ai fanetian of one di- mension into a two-dimensionaj picture. Using this idea, a mathernia.ticaf microstcope krrown as musltimsslutzlan anulg~is will permit us to look at the details that are added m we go from one scale to another. In this wag, md- tiresolution andysis provides a mathematical frearnevrrork t o conceptualize problems linked to the wavelet decomposition of signals.. Before proceeding with the definition of multiresoliution andysis, we will first need to discuss four tapic8 of great importance far the study of mal- tiresolution analysis: inner products, biarthagandity, Riesa basis, and di- rect sums. 5*3.1.1 Imer products Refinition li.3,I.l. The inner product on a vector space V is a mappiag that assigns a scdm t;o every pair of elements s, t E V. The inner produet I"s denoted (s, t;) and it hias the following prsperliea: A vector space V with an inner product is e 4 e d an inner product space. The operation of performing the inner product is defined by the fallowing theorem. Theorem 5,3.1.1. Let 7J be a finite-dirrtendd i'nner product slpace, Sup- pose {al,. . . ,u,) and {xl,. . . ,x,) are two basis sets for V. Let vectors s and t be given in terms of the basis set {uf,. . . ,u,) and {xf, . . . ,x,), respectively. That is, Let the weighting matrix W be a defined by Thexl, the inner product is @ven by Proof: Consider the ixlx~er product af s and t by first substituting their respective series rnpansions, t hat is, Utilizing the Xinearitg property of the inner product yields By definition, [W]kj = ( U ~ ~ X ~ ) . Hence, or equivalently, rt (s, t) = C t; ( WS) ~ * j= f Therefore, (s, t) = tH ws. IT4 Chilp t er 5 Wavelet Signal Processi~g Definition 5.3.1.2. Two lineark independent sets of vectors S = {ul, . . . , u,) and T = {xl, . . . , x,) form biorthogonal sets if (uj9 x ~ ) = O f i r all j and k where j # k, and ( uj , xj ) f O for dl j , or equivdently; W is a diagond matrk. firtherrnore, S and T are called bio~thonornzal sets if ar ey uivden t l ~ VV is an iden tidy matrix. Moreover, any set of biorthogo- naI vectors c a ~ be made hiorthonormal. I f S = T, then biorthagond sets become orthog~najf sets md biorthonormajl sets become orthonormal sets, The following theorem deals wit Er transformations that preserve all the informalion af the origirirtf signal. Theorem 5.3.1.2. Let { x r ) and {ak) be biorthonormd bases of an inner product space V. Then, y E V can be expressed as Proaf; Since y E V and since ( x k ) and (uk) are complete biorthoaormaf sets of nonzero vectors in art irrner product s pxe V, then y. caan be expressed 1% Taking the inner product of both sides of this equation with x,, yields h Since U L ~ and xm are biorthonammal vectors, then 5.3 Maltiresola Lion an dygis Therefore, the right-hand side is nonzero only if k = m. Hence, The following theorem shows the ixlterrelationship between paraunitary matrices and biarthogonality. Theorem 5.3.1.3. The filter H( r ) satisfies the normalized paraunitaryprop- if and only if its individual filters saGisfy the bforthonormal cmditiurr. where, HL(. L), HL(zWM), . . . , H ~ ( Z W ~ - ' ) Proof; (Case 1: Assume ~ ( z ) H( z ) = JM.) Then, the ( j , k)th entry in the matrix I ?( z) ~( z) i s given by Since ~ ( L ) H ( z ) = IM, thexi (Case 2: Assume ( = bj, k. ) If t h i ~ equation is interpreted as the (j, k)th entry of a matrix, the11 this equation can be rewritten as Then, the corresponding matrix equation is giver1 by 116 Chapter 5 Wavelet Signd Processing 5,3,1,3 Riesz basis DeBnieion 5.3.1.3. Let V be an inner product gpace. Let norm induced by the inner product, that is., The set ifk) c V h called a Riesz Basis i f every element s E V of the space can be written as s = c c k f k k for some chuice ofsealam { e k) and i f positive consta~lts A and B exist such that Riesz basis are also known w a stable bwii3 or u ~ ~ c ~ ~ k d i t i o ~ a ~ bmis. I f the Reisz basii~ is an orthogonal basis, then A = B = I. Consider two biortl~onormal sets S = {al,. . . , u,) and T = {x,, . . . , x,), t hat is, (uj, XL) = 6j . k. Then, S and T are biorthonsrmd (Riesz) basis if there exist positi.ve con- starlts A, B, @, L), such. t hat for all vectors y Now, we will define the concept of a direct sunk ZLI I ~ , subsequentlyl we will grove the projection tlreurern. RefiniLrisn 5.3,f ,4, Let S be a subspaice and let U and V be subspaces of' S, Tflen, S can be decomposed into s dirwct: sum of U and V, that is i f the following conditions are satided: 2, S is generated by U U V. We are now in a position to decompose a vector into its projectiorrs onto subspaces. To introduce this concept, we begin by identifying suitable subspaces, Deftrrridioxz 5.3.1.5. The orthogonal compbemead- of a subset V of an inner product space V i s denoted UI and is defined by Thus, UL will always be a subspace of V and that subspace will be closed. Moreover, if U is a closed subupace, then ULi = U. Theorem 5.3.1.4. (Projection Theorem) Let U be a closed subspace ofa complete inner produet space V. Tjie~l, V can be decomposed in to a direct sum of U and UA, that is, Proof: Let Y = {yl,. . . , y,) and S = {sl,. . . , s, ) be an biorthonormal basis for U. Let x E V. Then, x~ and XI are defined by and XI = x -- Xg. Clearly, x -. xo-+xl, firtherrnore, xo E U since rro is a linear combination of the baais elements Y, Next, we must demonstrate that xA E U' and tllat the decampasitiot.1 of x: is unique, Let us consider the inrler product between xl and yli, E Y, that is, or equivalently, (XI,YL) = ( x , Y~ ) - (xo~Y~). Now replasing xo wit11 its series expansio~t yields or equivalently, 178 Chapter 5 Wavelet SigaLZ-l Processing Since Y and S form a biorthonormal set, (sj, y k) = 6j , k. Hence, Thus, x1 is orthogonal to ewh basis vector in Y. Since each wctor in V is a linear combination of the bmis vectors, it fallows that xl is orthogonal to every vector in U. Her~ce, xl E UL. To show the decomposition of x i~ unique, let us stssurne that x is rxot unique and show that this results in a contradiction. Let's msurne x = xo+xl and x = z~+r;~. Then, or equivalently, Il (xo+x1) - (EO+ZX) 1 1 2 = 0. After rearranging terms this equation becomes, Then, using the Pythagorean Theorem, Therehre, == xo and = X I . Hexlce, the decsrnposition is unique. r Dsfinilion $.3.1,8, Multire~olu ti011 a ndy~i s (MRA) can be viemd as a sequence of approimations of a given Eunction f (t) at &Rerent resolutions, The approximation of f ( t ) at a resolution 2j is defined as an orthogonal projection off ( t ) on a su bspace V j . Now, we will provide a list of properties that tbrtese subspaces fh"i11 need to satisfy. They are: 2. vi C Vj+l; 2. njVj = {0) and clasure of UjVj = L2; 3. f E Vj+l o f(2*) E Vj; 4. f E Vj f (* - k) E Vj for all integer k; 5. { f (* - k)} is a Reisz basis in Vo. A level of a muftiresolution analysis is one of the Vi subspwes arid one level is coarser (finer) with respect to another whenever the index of the corresponding subspace is smaller (bigger). By properties 3, 4? and 5 there e ~ s t s a Rekx bwis cp in Vj of the form wl~ere, c iv a constant. Assume that the energy wsociated with i p( t ) equals the energy associated with ~ j , ~ ( t ) , that is, or equivalently; For the left-hand side, perform a change of variables by letting u = 2 j t - k. The11, Therefore, c 2j/2* Hexlee, the scaling function for spaee VJ hm the form In an analogous fauhion, the wavelet function for the subsyace Wj, which i u l fte complement space of Vj, has the form Since information about revolution 2j+' is given by the approsirnation in subepace Vj and the details in subspsee Wj , Lhc arthogand prajectiurr on the orthogorlal corrlplerme~lt of space Vj in space Vj+1, t hat is where, fj E Vj and (f3+1 - fj) E Wj . Since V1 = Vo @ Wo and V2 = Vt @ Wl then, Vz = Vo @ [Wo @ Wlj. So, irr gexreral, or simply; Vj+l = v 0 @0 5 ~ 5 j Wk where, Va is the subvpace corresponding t o the coarsest scale. Since 9 E Vo and E QOCk< - - jWkY then? an ixiput signal f (s) can be expressed as 180 Gzlp ter 5 Wavelet Sign& Proces8ing where, Let us consider the following example to illustrate many of the ideas of multtresolutio~l analysis. Assume the input signal f (s) is defined by 16, O < s < f 1 1 -8, ~ S Z < $ f b ) = 3 16, $ < Z < Z -20, a sz < 1 0, otherwise . For simplicity of this example, we will use the Haw wwelet. Its wavelet, $(XI, and scaling function, p(z), are defined by I, @ < % < I cp(.I = 0, otherwise and 1, O < Z < + -1, f Sz<l 0, otherwise . Now, let us compute the caeBcienCs. Hence, Thus, - - where, = [s, do, o, dl.@, dl,llT and y is a vector of input signal values, and W4 is the discrete Hmr wavelet t ransfor~l , The discrete wavelet transllarm matrix ia analogous t o the discrete Fourier transform rnat rk - the principal difference being tliat Fourier caeacients come from vdues at rr points, while H a r wavelet coeReients come from n subintervals, According do the definition af the muXLiresoiluLian analysis, tp E Vo C V1. As such, the scal i ~~g funetiozt must satisfy or equivalently, This equation is known by several digereat narnes: the refinement equa- tion, the dilation equation, and the two-scale difference equation. The generalization of the refinement equatiorr is given by The following theorem provides an important property of the sequence of filter (or mask) coefficients {ha). Theorem 5.3.1.5. Ck ho(k) = 4. 182 Chapter 5 Wavelet Sigr~al Processing Proof: IntegraLing both sides of the refinerneat equation yields or equivalently, For the integrd on tlte right-11snd-side, perform s change of variables by letting u ;== 2t -- k. Then, Hence, The scaling function is uniquely defined by the refinement equation and Sirkc, Vl = V, @ Wo, then $I E V1. As such, the wavelet must satisfy $(t ) = fi C hi (k)p(2t - k), k Ttle generdb~at;ion of this equation is giver1 by The wavelet is uniquely dart ed by this equation ai d by Gfic normalization The f ul hi ng theorem provides an importarkt property of the sequence of coefficients { hl }. Theorem 5.3.1.6. Ck hl (k) = 0. Proof: Since the wavelet + must satisfy then by integrating both sides of the equation yields or equivalently, For tfre integral an the right-halid side, perforal a change of variables by letting u = 2t - k. Then, Since J-z +(t )dt = 0 and S : m p(u)dv, = 1, then C hl ( k ) = 0. 5.3.1.8 Filter-ba& implement at ion The projectio~l of a signal f ( t ) E Vj+l is given by where, aj4-1 (n) = (f 9 vj+19n) * In addition, f ( t ) E Vj @ Wi , so f ( t ) can also be written EM where Chapter 5 Wavelet Signal Processing Figure 5.1. An analysis stage of pyramidal filter bank, Let us examine the coefficients a little closer, and These equrttions suggest a pyramidal filter bank implementation, where the analysis bank is defined in terms of stages, where aj+l (m) = ( f , qj+l,m) is decompo~ed into aj(k) = (f, ~ j , ~ ) and b j ( k ) = ( f , $j,k) . This result can be depicted pictoriiaXly in, Figure 3.1, Thus, we have defined the andysis side of the fitter bank. Now, let us examine the synthesis side, Since Vj+l = V j @ Wj9 we can write any vector in Vj+l ar a linear cornbiglation of the basis vectors in Vj and Wj . Since Vj C Vj +l , a shift of n in Vj+l corresponds to a shift of 2 in V j and Wj. That is, Let (2k) = qo (k) and 91 (21) = po ( l ) . Then, 5.3 Muf tiregala tion analysis Figure 5.2. A. synthesis stage of pyrmi dd filter bank, Let ma = k + 5 and ml = I + f. Then, Then, the projection of signal f (t ) on the basis elements ~ ~ + ~ * , ( t ) is given by This equation suggests a wranridal filter-bank implementation, where the synthesis bar~k i u defined in terms of stages, where aj (ma) = (f, Pj *,,) and bj (ml) = ( f , @j,m,) are merged to form aj+l (a) = ( f , P~+~,,) . This result is depicted pictoridly in Figure 5.2. 5.3.J.41 Initial pr0jecl;ion coemcients The accurxy. of the multiresolutian analysis depends on the computation of aN( k ) , i , e, , anr (k) = ( f , rpnr,k) = 2"/2 f ( t ) p( zNt - k) dt. Let us perform a change of variables by letting z~ = 2" t . Then, 186 Chapter 5 Mravdet Si p& Processing Since p ( ~ ) hm compitct support, thia integral can be interpreted to have finite limits. In addition, f ( 2- Nu) may place constraints on the sampling of f (u), dependirzg of course on the smoothness of (u). 5.3.1 .lO ReBtections on the wavelet tramform By examining muftiresolutian andysis, w are ready to draw some inter- esting canclusians about wavelets. 1. Concept, of Scale. The discrete wavelet transform i s usefixl for repre- senting the finer variations irr the signal f ( t ) at various scales. More- over, the function f (t ) can be represented as a linear combination of functions that represer~t the variations at different wales. 2. Localized Basis. If a signal has enerl;y at a particular gcale concen- trated in an interval in the time domain, then the corresponding coeficient has a large value. Therefore, the wavelet baits provides localization irrformation in both the time dom&bin a~ well wnj the scale donrairr. 5.3.2 Image corrzpressihn and wavelets In multiresolution analysis, there are wme dependencies between coeffi- cients at different scales. In. addition, for naturd images, it is unlikely to find significant high-frequency energy if there is little low-frequency energy in the same spatial location. These obsewations motivated a wavelet-based image coding method, that is based on a data structure called a zera-tree. A, zero-tree provides a compact representation to indicate the positians of significant wavelet coescients. A wavelet coefficient c is imignzfieant with respect; to a threshold T, if el < T. The zero-tree is bwed on the hypothesis that if a wavelet coeficient at a give& scale is insignificmt with respect t o a given threshold T, %lien, in most eases, all wavelet coefficients at the same orientation in the same spatial location, at finer sedes are ixtsipificant with respect to 12". An exigineering trade-off can be made be- t we n the choice of the threshold 2" and the number af bits af precision for untllresholded caeEcients. Thus, the zero-tree approach can isolate interesting nonzero details by eliminating large insignificant regions from consideration; arid, therefore, it make8 more bit8 available to encode the significant coeBcients. 53. 3 A generalisation: biorthogoeal wavelets Except for the Eaar vvavetet, all other orthogonal. wavelets are not sym- metric. This is a, problem in applicrttians, such as imal;.e compression, since thcsc unsymmetrical wavelets generitte notieeabfe phave distarbions, So can orthogonal wavelets be generdized to elirni~l&c this problem? Yes, using biort hogonat wavelets, VVc? will develop this idea of biorthogond wavelets by first imagining two intertvPining multiresolution analyaes, that is, Vj+l = V j $ Wj and Tj +l = vj ~3 Gj where, - C_*__ Vj n Wj = Oa l t d Vj n Wj =0. Iastecld sf ssuming the spaces V j and Wj are perpendicular, we will w- - sume that Vj is perpendicular to j . Similarly, we will wvume that the spaces qi and Siij are not perpendicular, but rather that qj is perpen- dicular to Wj . Then, the wavelets lirj,&(t) = 2jj2$(2jt - k) and &i , m(t) = 2' i2&(2' t - m) are biorthogonal wavelet bases. That is, the wavelet bases satisfy the followirlg biort honurniditg condition: In addition, the scali~lg functions v j , ~( t ) = 2 j l z p ( 2 j t - k ) and Fj,,(t) = 2j i 25( 2j t - rn) satisfy the following orthonormdity condition: 5.3.4 Case study: wavelets in data cammu~catlarzs Although this cwe study is focused on data communications, many of the i dea expressed are also appropriate for other applications. Now, let us eon- sider a modulated signal, derrotcd sft), satisfying the dyadic self-similarity property is given by z ( t ) = 2- rnH2( 2rnt ) for some real constant If and d l integers m, This is the elass of signals of interest far fiactul modulatdoln applications. These waveforms have the property that an arbitrary short duration time segment is su%eient to re- cover the exttire waveform and hence tlte embeded information. Similarly; a low bandwidth approximation to the waveform is suflicient t o recaver the undistorted waveforn:~ and also the embedded inhrmation. Now, conlsidcr the wavelet representation of z jt), that is, 188 where, Chapter 5 Wavelet Sign& Processing Let us exa~ni ~l e further z, , , . Substituting the defirlition of z( t ) into z , , , Perform a, change of wi abl es by letting u = Zmt . Then, where /3 = 22H+1. Let us denote z, , , by q(n). Then, TNe wish ta let q(n) correspond to a finite Iengkh message of length L t hat wc wish t o sextd, but the right-hand side of this equation is an infinite sum. Let's use periodic extexisions of ( P( "IL) 1115(t), that is, Moreaver, since p = 22W41, the p a r a ~ r x e H controls the relative power distribution among the freque~rey bands and hence the overall transmitted power, As8un~e that ~ ( t ) is transmitted llicross the chmnel and received ;rs a wdvefarm denoted ~ ( 2 ) . Then, the receiver extrwts the wavelet coescients rm,, of ~ ( t ) usirig the discrete wavelet transform. These coefficients take the form T ~ , ~ = p- m/ 2y( n mod L) + zm,?% where x,?, are the wavelet co~fXiciexits of the channel noise. The duration-bandwidth characteristics of the cf-lax~ne"l@ect; which ob- servation coeficients r,,, xxtay be access&. If the chax~riel is bandlimited t a 2 Ma Hz for some integer Mu, then the receiver call not access coefficients at scdes nz > MU. Similarly, the duration conatraint of the channel resufts in a minimum decoding rate of 2 M ~ symbols/second for some integer MA, whicfi precludes access t o scales m < ML. Therefore, the coeEcients available at the receiver correspond to the indices fib = M&.M& 4- I, ... , Mu where L is the length of the message q(n). Therefore, the number of noisy measurements of the message at, the receivcr is given by Perform a change of rariables by letting p = m - ML yields Thus, the receiver can select the rate/bandwidth ratio dynamically since the transmitter is fixed. 5.4 Problem 1. Let where qk ( w) is a set of orthonormal functions in the range a 5 w < b, that is, Suppose we wish to approximate Fl u) with the finite mmmatiorl where a .(: w < b. We wish to make the approximation best in the least square sense, that is b (F(w) - F, ( w) lzdw must be minimhed. Show that the choice of f i k ) = f ( k ) , k = -my.. . , nz, achieves the desired result. 190 Chapter 5 Witvded Sig~al Procegsir~g 2. Let us examine the Mexiearr hat fu~ietion, which irs defirled by Is h(t ) a wavelet? Justify your altsuver. 3. Given n-dimexlsiortal vectors s and t and arr n x n matrix W. Show that t " ~ s satisfies the properties of an inner product, i . e. (s, t) = tHws. 4. Prove the following: Let yj,k(t) = 2j/2y(2jt - k). I f g( t ) has unit norm, then gj9t ( t ) also has unit norm. Bibliography [I] P, Auscher, G. Weiss, and M, V. Wickerhauser, Local sine and cosine bmis of Coifmar1 and Meyer artd the construction af smooth wavelets, Wavelets: A Tutorial in Theory and Applications (C. K. Chui, ed.), Acaderxlic Press, Bostoxx, 1992, pp, 237-256. [2] V. Belevitch, Clwsical Network Tfieory, Holden DayS hc. , sari ban- cisco, 1968. [3f M. Bcllmger, G. Bannerat, and M. Condreuac, Digital filtering by polyphase rretwork: Appliicatian t o sampling rate dtera;tion and filter baxtks, IEEE Trans. 011 Acotlst., Speech, and Signal Processing 24 (1976), 109-1 14, [4) J. W. S. Cwsels, An Introduction to the Geometry of Numbers, Springer-Verlag, Berfta, Germany, 1959. [5] Y. T. Chan, An ir~troduction to Wavelets, Kluwer Academic Publish- ers, Boston, 1992. (61 T. Chen and P, P. VaidyanaL)~an, ReeerlC developments in rnultidirnen- sional nlultirate systems, IEEE rX"rans. on Circuif;s and Syut, for Video T"crJr. 3 (1993), 116-137. [7j C. K. Chui, An Introduction to Wavelets, Academic Preas, Boston, 1992, [8] A. Cohen, I. Daubechieu, and J. C. Feauveau, Biorthogonal base8 of compactly supported wavelets, Gomm, Pure Appl. &the 45 (1992), 485-560, pf R. Carifmm arld V. Meyer, Remarques sur Ifanalyse de fourier B fengtre, h p t e s Re ~d u s Acad* Sci. Paris Se"rie 1 312 (1 W1 ), 25%261. R. E, CrocE~iere and L. R. Rabiner, Interpolation and decimation of digitd signals: A tutorial review, Proc. of IEEE 69 (1981), 300-331. [I l j , Multirate Digital Signal Proceuuing, Prentice Hall, Englewood Cliffs, NJ, 1983, f12] I. Baubechies, Orthonormal bwes of campatty supy orted wavelets, Comm. Pure Appl. Math. 41 (1988), 909-996, [I31 , Ten Lectures on Wavelets, SIAM, Philadelphia, 1992. [I41 Z. Doganata and P. P. Vaidyanathan, Minimal structures for the imple- mentation of digitd ration4 lossless systems, IEEE naris. on Acous- t i c ~ ~ Speech, and Sign& Processing 38 (19901, 2058-2074, [15] E. Dubois, The sampling and reconstruction of time-varying imagery with applicatiuns in video systems, Proc. of IEEE 73 (1985)) 502-522. [16] D. E. Dudgeon and R. M. Mersereau, Multidimensional Digital Signal Processing, Prentice-Hall, Englewood-CliEs, N J , 1984. 1171 G. Eva~igelista, Comb and multiplexed wavelet transfsmzs and their applicitLiorl to signai processing, E E E Rang. on Si p & Processing 42 (1994), 292-303. [18] B. L. Evans, R. H. Bamberger, and J. H. McClellan, Rules for mul- tidimensional muitirate ~truetures, IEEE Bans. on Sigrid Procegsing 42 (19941, 762--711, [IS] N. J. Fliege, Muitirate Digital Signal Processing, John Wiley, Chich- ester, Errglsmd, 1994, [20] A. Guevsoum and R. M. Mersereau, Fast algorithms for the multi- dimensional discrete Fourier transform, IEEE naris, on Acoustics., Speech, md Signal Processing 34 (1986),931--943. 1211 E. Giindiizhan, A. E. Cetin, and A. M. Tekalp, DCT coding of nonrect- a~kgularly sampled images, IEEE Signal Processing Letters 9 (19941, 131-133. 1221 M. Hofsehneider, Wavelets: An AllraJysk Tbd, Oxford Science Publi- caLions, Ciarendon Press, Oxford, England, 1995, 1231 G. Karlsson and M. Vetterli, Theory of two-dimensional multirate filter banb, IEEE naris. orr Acoustics, Speech, and Si g~al Processing 38 (1990), 925-937, f24] R. D. Koiipillai and P. P, Vaidyaxtatfran, Cosine-modulated FIR filter banks satisfying perfect reconstruction, IEEE fians. oa Signal Pro- cessing 40 (1992)) 770-783. (251 J. KovaEevik, Subband coding systems incorporating quantizer models, IEEE naris. on Image Processing 4 (1995),543-553. Bibliography 193 [26j J . S. Lim, Two-Dimensional Signal and Image Processing, Prentice- Hall, Englewaod Cliffs, NJ, 1990, 1271 S, Mallat, Multiresolution apgroxmations and wavelet ortkoonormd bases, naris. afilirztcrl.. iMatQ.. SOG, X1(1989), 64-88, 1281 , A theory of multiresolutiou signal decompositiart: The wavelet representation, IEEE naris. on Pat tern Andysia and Machine InteHige~ce 11 (1989), 674-693, 1291 H. S. Malvar, Theory of two-dimensional multirate filter banks, IEEE Rans. an Acoustics, Speech, and Signd Processing 38 (1990), 969- 978, (301 , Signal Processing with Lapped Pansforms, Artech House, Norwood, MA, 2992. [31] R. M. Mersereau and T. C. Speake, The processing of periodically sampled multidimensios1it1 signals, IEEE naris, on acoustic^, Speech, and S i p d Processing 31. (1983), 188-194, (321 R. A. Meyer and C. S. Burrus, A unified analysis of multirate and periodically time-varyixig digital filters, IEEE Bans. on Circuits and Sysd. 22 (1975),162-168. [33j Y. Meyer, Waveletu: Algorithms and Applications, SIAM, Philadel- phis, 1993. fJ4] , WaveJets md Operatam, Caxrkbridge Uniwrsity Press, G w - bridge, England, 1993. 1351 F, Mintzer, Filters for distortion-free two-band rnultirate filter bcznb, IE;EE Trar~s. an Acoustics, Speech, arrd Signal Proce~sing 33 (1985), 626-630, [36] J. Morlet, G. Arens, I. Fourgeau, and D. Giard, Wave propagation sntl sampling thcory., Geophysics 47 (1982), 203-236. [37] G. Oetken, T. W. Parks, and W. Schiissler, New results on the design of digital interpolators, IEEE naris. orr Acousdjcs, Speed, ilnd Srignd Processing 23 (1975), 301-309. [38] D. P. Petersen and D. Middleton, Sampling and reconstruction of wave-number-limited functions in N-dimensional euclidean space, In- formation arrd Control 5 (1962), 279-323. 1391 F. Riesz and B. Sz.-Nagy, Functional Anabi s, Rederick Ungar Pub- lishing Co., New Yor-k, 2955. $
F
$ 9
4 3
X
0
5 C
& -
P "
c *
$
0
s
g
" a "
2
!?
r ,
0 "
2 $
m
! i !
c n
* " " -
* *
r " L I .
w
0 -
3 5 :
. $ ? E
g .
t Q
3
E
g ; R
8
g .
m
&
5
p 3
$ 3
* C
[53] M Vetterli, Filter ba~kks allowing for perfect reconstruction, Signal Proce~sing 10 ( 19861, 2 19-244. [54] M. Vetterli and J. KovGevit, Wavelets and Su bband Coding, Prentice- Hall, Englewood CEEs, NJ, 1995. [55] E. Viscito and J. P. Allebach, The analysis and design of multidimen- sional. FIR perkct reconstruetion filter banks, IEEE B a ~ s . on eircuih and Sysd. 38 (1991), 2"5--41. [56] P. H. Westerink, 3. Biemond, and D. E. Boekee, Scalar quantization error analysis for image subband coding using QMFs, IEEE Rang. on Signal Processing 40 (1992), 421-428. (571 G. W. Wornell, Signal Processing with Fractals: A Wavelet-Based Appraaeltr, Prermtice Hall, Upper SttddIe Riwr, NJ, 1996. f581 X, - 6, Xia and E3. VV. Suter, A family of two-dimension& noaacparable malvar wavelets, Applied and Comgu~;ationLtl Harmonic Andpis. 2 (1995),243-256. 1591 , Construction of malvar wavelets on hexagons, Applied and h p u t at i o~al Harmonic Andysis 3 (1996), 65-11, Index A Alim free filter bank, 99 Alias-component QMF, 75 Ambiguity function narrow band, 171 wide-band, 172 Ant i -al i ~i ng filter, 14 Anti-imaging filter, 8 B Bmis local sine, f 27 Riesz, 176 Biort hogonal vectors, 174 Building block degree-one, 158 C Comb filter definition, 114 multidimensional, 63 Coset, 39 D Deeirnstor definition, 9 multidimensional, 59 noble identityt 26 DFT definition, 3 multidimensional, 46 Division theorern for integer vectors, 56 for integers, 6 for polynomials, 150 Downsampler, 9 Expander definition, 7 multidimensionat, 57 noble identity, 25 Filter anti-aliasing, 14 anti-imaging, 8 comb, 14 Interpolated FIR, I6 Filter bank aliizs free, 99 amplitude distortion-free, 109 cosine modulated, 1 14 DFT, 112 multidimensional, 128 perfect reconstruction, 105 hndarnental lattice, 30 hrrdamental parallelepiped, 36 H H a p wavelet, 181 Householder transform, 156 multidimensional, 69 Impulse response matrix, f 46 Inner product, 172 Interpolated FIR filter, 16 L Leibnitsk rule, 89 Linear space orthogonal complement, 177 Lossless system, 146 Malvar wavelets, 123 Matrix circulanl, 99 elementary operations, 149 generalized pseudocirculant, 136 impulse response, 146 normalized paraunilar~t, 105 orthogonal, f 05 paraconjugated, 105 paraunitary; 105 polynomial, 147 pseudacirculaxlt, 98 reversd, 114 skew circulant, 99 urlimodular integer-valued, 35 unitary, 105 MeMiflan degree, 154 Multidimensional comb filter, 63 Multidimensiond decirnator, 59 Multidimensional DFT, 46 Multidimensional expander, 57 MuItiresolution andysirs, 178 N Noble identity fbr decimators, 26 for expanders, 25 Paracanjugation of matrix, 82 of vector, 82 Parzetlelepiped fundamental, 36 symmetric, 38 Polynomial matrix definition, 147 elementary operations on, 149 rank of, 148 unimodutar, 148 Polyphase multidimensional, 54 Type-I representatioxl, 5 Type-I1 representation, 6 Palypbase QMF, 82 Q QMF bank, 74 QMF formulations diw-component, 75 polyphiwe, 82 Sampling laLtice coset, 39 definition, 31 fundamental, 30 hexagonal tf 34 rionuniqucxlesr~, 34 quincunx , 34 reciprocal, 34 rectangular , 32 Index sublattice , 38 unit cell , 36 Sampling matrk, 32 Scaling function, 181 Signal representation modulation, 5 polyphase, 5 Smith form decomposition for integer-valued matrices, 41 for polynorniaf matrices, 158 Smith-McMillan decomposition, 153 Sublattice, 38 Symmetric paralldepiped, 38 Sysstem theory lossless, 146 MeMillm degree, 154 multi-input multi-output, 146 normalized lossless, 146 Unirnodular matrix integer-valued, 35 polgnomiat, 148 Unitmy matrices Householder transform, 156 Upsampler, 7 kct or s bior t hogonat, 174 linear combination, 35, linear independence, 31 to mat r k power, 44 to vector power, 44 w Wavelet trftnsfbrm adnniseibilie condition, 168 biortkagond wavelet, 187 convolu tion theorem, 271 definition, 168 differentiation theorem, 1 TO Bmr wavelet, 181 implementation, 1 83 initial eoeBciertt, 185 linearity property, 168 Malatar, 123 scaling function, 281 shi& theorem, 169 similarity theorem, 169 z transform definition, 3 nrultidimensionaf, 44 Vect;ar space basis, 32 span, 31.