P.P. VAIDYANATHAN AGA UVES SYSTEMS AND OAM wey W Te) —t2}-] Giz) tree-atructured ftir bank a ———_#} paraunitary OMF lattes Generating orthonesmal bass i g-\\ baal Ue a eee ale ee ee aad «2 ALAN V OPPENHEIM. SERIES EXITORMultirate Systems and Filter Banks PP. Vaidyanathan Department of Electrical Engineering California Institute of Technology. Pasadena PRENTICE HALL P TR. Englewood Cliffs, New Jersey 07632Library of Congress Cataloging-in-Publication Data Vaidyanathan, P-P. Mulbrate systems and filter banks / P.P. Vaidyanathan. Pp. cm. Includes bibliographical references (p. ) and index. ISBN 0-13-605718-7 1. Signal processiong—-Digital techniques. |. Tige. TKS1025.V24 1989 92-6547 621.3822—0¢20 CIP Edvorial production supervision. Brenden M, Stewart Prepress buyer: Man: MeCoriner Manufacturing buyer: Susan Brunke Acquisitions editor: Karen Getaman © 1993 by Prentice Hall P TR Prentice-Hall, Inc Englewood Cliffs, New Jersey 07632 The publisher offers discounts on this book when ordered in bulk quantities. For more information, write: Special Sales Professional Marketing. Prentice-Hall. Professional & Technical Reference Division, Englewood Cliffs, NJ 07632 All nghis reserved. No part of this book may be reproduced. in any form or by any means, without permission in writing from the publisher. Pointed in the United States of America 0 9 8B ISBN C-13-£057148-? Prentice-Hall International (1 1B) Lamited, Landes Prentice-Hall of Australia Pry. Lemued, Syilrre ¥ Prentice-Hall of Canada Inc. Foronto Prentice-Hall Hispancamericana.S. A. Afexice Prentice-Hall of India Private Lamited, New Defi Prentice-Hia8 of Sapan. Inc., fokye Pearson Education Asa Pte Lad. Singupore Editors Prentice-Hall do Brasil. Laide. fie de Janeireily achieve perfect reconstruction in these systems, by further imposing the paraunitary property. We first describe cosine modulated systems with ap- proximate reconstruction properties (pseudo QMF banks, Sec. 8.1-8.3), and then show how these can be modified Lo obtain perfect reconstruction (Sec. 84, 8.5). The cosine modulated perfect reconstruction system (Sec. 8.4, 8.5) can. however, be studied independently, with Sec. §.1-8.3 used only as a reference. Chapters 9 to 12: Special Topics on Multirate Systems Chapter 9 studies the effects of finite precision in the implementations of multirate filter banks. This includes roundoff noise analysis and coefficient quantization analvsis. The effect of quantization of subband signals is dealt with in Appendix C. In Chap. LO we study the connection between filter banks and a num- ber of “peripheral” topics such as periodically time varying systems, block filtering, and unconventional sampling theorems. Chapter 11 deals with a special type of time-frequency representation called the short-teme Fourier transformation, and extends this to develop wavelet transforms. Wavelet transforms, in particular, have drawn consid- erable attention in recent years from a wide scientific community, including pliysicists, mathematicians, and signal processors. Many researchers in the signal processing community have taken the view that wavelet transforms are closcly related to filter banks (see Chap. 1] for references). In Chap. 1], we will take this viewpoint; this makes it easier to understand, design, and implement wavelet transforms. In Chap. 12 we study the multidimensional versions of many of the fundamental multirate concepts introduced in earlier chapters. These find applications in image and vidco signal processing. Chapters 13 and 14: Multivariable and Paraunitary Systems Many of the multirate (time-varying) systems discussed in the text can be represented in terms of multi-input multi-output (MIMO) linear time in- variant (LI1) systems. This will be evident when we analyze filter banks using the polyphase approach. It turns out, therefore, that a deeper under- standing of MIMO LT'I systems is very useful in the study of filter banks. Chapter 13 is meant to serve this purpose. Even though the results of this chapter are not explicitly used in earlier ones, they are required to establish come of the deeper properties of paraunitary systems discussed in Chap. 14. Chapter 14 is a complete treatment of paraunitary and lossless trans- fer matrices. These systems find application in perfect reconstruction filter banks (Chap. 6 and 8) as well as in wavelet transform theory (Chap. 11). As mentioned previously, some of the results in Chap. 14 are in turn stated and used in some of the earlier chapters. The detailed discussions in Chap. la 14 ensure completeness of presentation, and also serve as research aids. 10° Chap. 1. IntroductionContents Preface xi PART 1 INTRODUCTORY CHAPTERS 1 Introduction t ll 12 Major Developments 3 Scope andOutlne 8 2 Review of Discrete-Time Systems 12 Introduction 12 Discrete-Time Signals 12 Multi Input Multi-Output Systems 24 Notations 28 Discrete-Time Filters (Digital Filters) 31 Problems 39 3 Review of Digital Filters a2 3.0 31 32 Introduction 42 Filter Design Specifications 42 FIR Filter Design 45 IR Filter Design 60 Allpass Filters 71 Special Types of Fitters 83 IIR Filters Bosed on Two Allpass Filters 64 Concluding Remarks 91 Problems 93 4 Fundamentals of Multirate Systems 100 40 4. 42 43 4A 45 As 47 Introduction 100 Basic Multirate Operations 100 Interconnection of Building Blocks HB ‘The Polyphase Representation 120 Multistage Implementations 134 Scane Applications of Multirate Systems 143 Special Filters and Filter Banks 15) Multigrid Methods 168 Problems 178 Contenta villPART 2 MULTIRATE FILTER BANKS § Maximally Decimated Filter Banks 188 50 3.] 5.2 5.3 54 25 5.6 + 58 59 5.10 Introduction LBR Errors Created in the QM Bank 191 A Simple Alias-Free QMF System ‘196 Power Symmetric QMF Banks 204 Af-channel Filter Banks 223 Polyphase Representation 230 Perfect Reconstruction (PR) Systems 234 Alias-Free Filter Banks 245 Tree Structured Filter Banks 254 Transmultiplexers 239 Summary and Tables 266 Problems 272 & Paraunitary Perfect Reconstruction (PRA) Filter Banks 266 6.0 6.1 6.2 6.3 6.4 6.5 6.6 6.7 Introduction 286 Lossless Transfer Matrices 288 Filter Bank Propertics Induced by Paraunitariness 294 Two Channel FIR. Paraumtary QMF Banks 298 The Two Channel Paraunitary QMF Lattice 302 M-channel FIR Paraunitary Filter Banks 314 Transform Coding and the “LOT” 322 Summary, Comparisons, and Tables 326 Problems 333 7 Linear Phase Perfect Reconstruction QMF Banks 337 7.0 71 72 7.3 Introduction 337 Some Necessary Conditions 337 Lattice Structures for Linear Phase FIR PR QMF Banks 339 Formal Synthesis of Linear Phase FIR PR QMF Lattice 347 Problems 351 & Cosine Modulated Filter Banks 353 Introduction 353 The Pseudo QMF Bank 354 Design of the Pseudo QMF Bank 363 Efficient Polyphase Structures 370 Deeper Properties of Cosine Matrices 373 Cosine Modulated Perfect Reconstruction Systems 377 Problems 392 PART 3 SPECIAL TOPICS S$ Quantization Effects 394 9.0 9.1 9.2 Introduction 394 Types of Quantization Effects 394 Review of Standard Techniques 397 vill ContontaNoise Transmission in Multir: stems Noise in Filter Banks a a Filter Bank Output Noise 412 Limit Cycles 416 Coefficient Quantization 418 Problems 424 eeece Tobe 10 Multirate Filter Bank Theory and Related Topics 427 10.0 Introduction 427 101 Block Filters, LPTV Systems and Multirate Filter Banks 427 10.2 Unconventional Sampling Theorems 436 Problems 454 " 11 The Wavelet Transform and its Refation to Multirate Filter Banks 47 11.0 Introduction 457 11 Background and Outline 458 11.2 ‘The Short-Time Fourier Transform 463 11.3 ‘The Wavelet Transform 481 11.4 Discrete-Time Orthonormal Wavelets 500 115 Continuous-Time Orthonormal Wavelet Basis 510 116 Conciuding Remarks 536 Problems 539 12 Multidimensional Multirate Systems 545 12.0 Introduction 545 12.1 Multidimensional Signals 546 122 Sampling a Multidimensional Signal 555 12.3 Minimum Sarpling Density 568 124 Multirate Fundamentals 572 125 — Alias-Free Decimation _ 597 126 Cascade Connections 603 127 — Multizate Filter Design _ 608 12.8 Special Filters and Filter Banks 623 129 Maximally Decimated Filter Banks 627 12.10 Concluding Remarks G41 Problems 650 PART 4 MULTIVARIABLE AND LOSSLESS SYSTEMS 13 Review of Discrete-Time Muitl-Input Multl-Output LT! Systems 660 13.0 Introduction 660 13.1 Mnlti-Enput Multi-Output Systems 661 13.2 Matrix Polynomials G61 13.3 Matrix Fraction Deseriphions — 665 13.4 State Space Descriptions 609 13.5 The Smith-McMillan Form 687 13.6 Poles of ‘Transfer Matrices 699 13.7 Zoros of Transfer Matrices 703 13.8 Degree of a ‘Transfer Matrix 707 Comecnts ix13.9 13.10 FIR Transfer Matrices 708 Causal Inverses of Causal Systema 711 Problems 715 14 Paraunitary and Lossiess Systems 722 14.0 14.1 14.2 14.3 14.4 14.5 14.6 14.7 14.8 14.9 14.10 14.11 14.12 APPENDICES Introduction 722 A Brief History 723 Fundamentals of Lossless Systema 724 Lossless Systems with Two Outputs 727 Structures for Af x M and M x1 FIR Lossless Systems 731 State Space Manifestation of Lossless Property 740 Factorization of Unitary Matrices 745 Smith-McMillan Form and Pole-Zero Pattern 754 The Modulus Property 758 Structures for IIR Lossless Systems 759 Modified Lossleag Structures 763 Preserving Lossless Property Under Quantization 768 Summary and Tables 771 Problems 775 A Review of Matrices 782 Ad Al A2 A.3 A4 AS A6 A.7 AS AD Introduction 782 Definitions and Examples 782 Basic Operations 783 Determinants 786 Linear Independence, Rank, and Related Issues 787 Eigenvalues and Eigenvectors 790 Special Types cf Matrices 793 Unitary Triangularizetion 798 Maximization and Minimization 798 Properties Preserved in matrix products 799 Problems 800 8 Review of Random Processes 803 B.0 B.1 B.2 B.3 BA BS Introduction 803 Real Random Variables 803 Real Random Processes 806 Passage Through LTI Systems 810 The Complex Case 811 The Vector Case 812 C Quantization cf Subband Signals 816 CH C.l C2 C3 C4 xX Contents Introduction 816 Quantizer Noise Variance 816 The Ideal Subband Coder 818 The Orthogonal Transform Coder 826 Similarities and Differences 833cs Relation to Other Methods 839 Problems B45, D Spectral Factorization Techniques e49 DO Introduction 849. DL ‘The Complex Cepstrum 849 D2 A Cepstral Inversion Algorithm — 853 B33 A Spectral Factorization Algorithm 854 Problems 858 E Mason's Gain Formula 859 Glossary of Symbols = 856 List of important Summaries (Tables) 867 List of important Summaries (Figures) 868 Bibliography 869 Alphabetical List of References Cited inthe Book 869 Some References by Topic 8B8 Index aot Contents xiPreface Multirate digital signal processing techniques have been practiced sinecrs for more than a decade and a half. This discipline finds sc etcnlibad Gy Hee and image compression, the digital audio industry, statistical and adaptive signal pro- cessing, numerical solution of differential equations, and in many other fields. It also fits naturally with certain special classes of time-frequency representations such as the short-time Fourier transform and the wavelet transform, which are useful in analyzing the time-varying nature of signal spectra. Over the last decade, there has been a tremendous growth of activity m the arca of multirate signal processing, perhaps triggered by the first book im this ficld (Crochiere and Rabiner, 1983]. Particularly impressive is the amount of new Titera- ture in digital filter banks, multidimensional multirate systems, and wavelet repre- sentations. The theoretical work in multirate filter banks appears to have reached a level of matuntty which justifies a thorough, unified, and in-depth treatment of these topics. This book is intended to scrve that purpose, and it presents the above men- tioned topics under one cover. Research in the areas of multidimensional systems. and wavelet transforms is still proceeding et a rapid rate. We have dedicated one chapter to each of these, in order to bring the reader up to a point where research can be begun 1 have always believed that it is important to appreciate the generality of principles and to obtain a solid theoretical foundation, and my presentation here reflects this philosophy. Several applications are discussed throughout the book, but the general principles are presented without bias towards specific application- oriented detail. ‘The writing style here ie very much in the form of a fext_ Whenever possible I have included examples to demonstrate new priveiples. Many design examples and complete design rules for filter banks have been included. Each chapter includes a fairly oxtensive sct of homework problems (totaling over 300). The solutions to these are availakle to instructors, from the publisher. ‘Tables and summaries are inserted at many places to enable the reader to locate important results conveniently 1 have also tricd to simplify the reader's task by assigning separate chapters tor niore advanced material. For example, Chap. 11 is dedicated to wavelet transforms, and Chap. 14 contains detailed developments: of many results on paraunitary systems Whenever a result from an advanced chapter (for example. Chap. 1) is used in an earlier chapter, this result is first stated clearly within tha contest of use, and the reador is referred lo the appropriate chapter for proof. The text is self-contained for readers who have some prior exposure to digi- tal signal processing A one-terns course which deals with sampling, discrete-time Fourier transforms, z-transforns, and digital filtering, is sufficient. In Chap. 2 and Preface Kit3 a brief review of this material is provided. A thorough exposition can be found in a number of references, for example, [Oppenheim and Schafer, 1989}. Chapter 3 also contains some new material, for example, eigenfillers, and detailed discussions on allpass filters, which are very useful in multirate system design. A detailed description of the text can be found in Chap. 1. Chapters 2 and 3 provide a brief review of signals, systems, and digital filtering Chapter 4, which is the first one on multirate systems, covers the fundamentals of multirate building blocks and filter banks, and describes many applications. Chapter 5 introduces multirate filter banks. laying the theoretical foundation for alias cancelation, and elimination of other errors. The first two sections in Chap. 4 and 5 contain material overlapping with [Crochiere and Rabiner, 1983). Most of the remaining material in these chapters, and in the majority of the chapters that follow, have not appeared in this form in text books. Chapters 6 to 8 provide a deeper study of multirate filter banks, and present several design techniques, including those based on the so-called paraunitary matri- ces. (These matrices play a role in the design of many multirate systems, and are treated in full depth in Chap. 14.) Chapters 9 to 12 cover special topics in multirate signal processing. These include roundoff noise effects (Chap. 9), block filtering, periodically time varying systems and sampling theorems (Chap. 10), wavelet trans- forms (Chap. 11) and multsdtmenstonal multirate systems (Chap. 12). Chapters 13 and 14 give an in-depth coverage of multivariable linear systems and lossless (or paraunitary) systems, which are required for a deeper understanding of multirate filter banks and wavelet transforms. There are five appendices which serve as references as well as supplementary reading. Three of these are review-material (matrix theory, random processes, and Mason's gain formula). Two of the appendices contain results directly related to filter bank systems. One of these is a technique for spectral factorization; the other one analyzes the effects of quantization of subband signals. Many of these chapters have been taught at Caltech over the last three years. ‘This text can be used for teaching a one, two, or three term (quarter or semester) course on one of many possible topics, for example, multirate fundamentals, mul- tirate filter banks, wavelet representation, and so on. There are many homework problems. The instructor has a great deal of flexibility in choosing the topics, but I prefer not to bias him or her by providing specific course outlines here. In summary, I have endeavored to produce a text which is useful for the class- room, as well as for self-study. It is also hoped that it will bring the rearicr toa point where he/she can start pursuing research in a vast range of multirate areas. Finally, I believe that the text can be comfortably used by the practicing engineer because of the mclusion of several design procedures, examples, tables, and summaries. ACKNOWLEDGMENTS The generous support and pleasant environment. offered by the California Institute of Technology has been most crucial in the successful completion of this project. The funding provided by the National Science Foundation for our research in multirate signal processing has been very helpful in developing many of the results which are included in this text. I also take this opportunity to thank Prof. Alan Oppenheim for his enthusiasm about this project, and for including this text in his distinguished signal processing series. I am indebted to Brendan Stewart, Prentice Hall, for KLV Peefanecarefully supervising tle production of the final book. K Ital, wan Fer) peel dowtnp all stages ef che ee pee at me! Several professional colleagues and students have played a major role in the evo- lution and completion of this project Professor Maurice Bellanger (TRE, France), and Drs. Ron Crochucre, N.S. Jayant, and Larry Rabiner of the AT&T Bell Lab_ oratories, provided the valuable encouragernrnt which 4 needed during the mitial stages. Tam decply thankful to Dr Rabiner for his criticism of the first draft, and for providing many valuable suggestions on the style of presentation His feedback has resulted in significant improvement of the presentation here. My heartfelt gratitude also goes to Prof. Martin Vetterh (Columbia University, New York) for his great enthusinsm, interaction, friendship, and valuable feedback over the years, and to Professors Mark Smith and Tom Barnwell (Georgia Institude of Technology), for their constan* support Prof Smith had studied the first draft of the manuscript very carefully, and provided valuable suggestions. Tam imlebted to Dr. Ajay Luthra and his colleagues (‘Tektronix laboratories), for their enthusiasm and interest in the Project. Some of the research work related to this book has been done with the support from ‘Tektronix Dr Rashid Ausari (Bellcore), Ptof Roberto Bamberger (Washington State University) and Dr Jelena Kovatevié (A'E& T Bell laboratories) provided vabu- able comments on the chapter on multidimensional multirate systems. Dr. Ingrid Daubechies (AF& T Bell laboratories) provided very useful feedback. which wm- proved parts of Chap. 11 (wavelets} substantially Ramesh Gopmath (Rice Univer- sity) also provided important comments on this chapter. [ elso wish to thank Prof. “Tom Parks (Cornell Umiversity) for his valuable feedback on Chap. I to 6 and 1, Prof. P. kk. Rajan (Tennesse ‘Technological University) for his comments on Chap. 12, and Prof A Sideris (Caltech) for comments on Chap. 13, I appreciate the interest shown by Prof. H. Malvar (University of Brasilia), Prof. George Moschytz (Swiss Federal Institute of Technology), and Prof. ‘Tor Ramstad (Norwegian Institute of Technology). I also sincerely acknowledge the constant moral support and encouragement [ received from Prof. Sanjit Mitra (University of California, Santa Batbara) during various stages. Several graduate students at Caltech have participated in the research that eventually gave rise to this book. In this regard, my interactions ith Tsuhan Chen, Zumus Doganata, Phuong-Quan Hoang, David Koilpillai, Vincent Liu, Truong Nguyen, Vinay Sathe, and Anand Soman have been most enjoyable. David's re- search at Caltech has had a major effect on Chap. 8 (cosine modulated Filter banks) In addition to providing intellectual interactions at the deepest level, Tsuhan Chen, David Koilpillai, Truong Nguyen. and Anand Somen have also generated many of the multirate design examples in this text. They have also rend various chapters of the manuscript and provided useful feedback Tn this connection my ‘special thanks go to Tsuhau Chen, Zinnur Doganata, Ian Galton, David Koilpillai, Ramesh Ra- jaram, Ken Rose and Anand Soman for reading many of the chapter. Touhan ‘ae a very careful reader, and provided many valuable suggestions for Chap 12. Ian Galton provided several conments on ing style as well as technical contents, which I found to be extremely useful, Tan's enthusiasm and friendship are gratefully: acknowledged. Debbie McGougan ond Cynthis Ste various phas#s of the manuscript preparati cart at Caltech were very helpful durang ion. Solutions to all the homework prob- Profuc XVlems have been prepared by Tsuhan Chen. Igor Djokovic, See-May Phoong, and Anand Soman. | appreciate their intercst and patience My parents have been responsible for teaching me many valuable “theorems of life” which I could not find in text books and papers. These certainly were the principles which sustamed me during all phases of this project, My deepest gratitude goes to my wife Usha who showed infinite patience and understanding during my absorption in this long project, and offered the type of moral support and loving encouragement which only she can offer. Countless were the evenings, weekends, and holidays during which she provided me with the seclu- sion and peace of mind needed to pursue this ambitious goal. She is certainly my best blessing, and this book would have remained an idle dream without her support. P,P. Vaidyanathan California Insiitute of Technology XV PrefacePART 1 introductory Chapters Introduction A traditional single rate digital signal processing system can be schemat- ically represented as shown in Fig. 1-1-1, which is an interconnection of computational building blocks such as multipliers, adders, and ‘delay ele- ments’ (which store internal signals). Examples are digital filters, Fourier transformers, modulators, and so on. In a multirate signal processing sys- tem, there are two new building blocks, called the M-fold decimator and the L-fold expander (Fig. 1-1-2). These will be defined and illustrated in Chapter 4. For the purpose of the present discussion, the decimator is a device that reduces the sampling rate by an integer factor of M, whereas the expander is used to increase the rate by Z. Such sampling rate alteration can be introduced at the input and/or output of the system or internal to the system, depending on the application. Multiplier Input Multiplication of 2 signals Figure 1.1-1 Schematic of a (smgle rate) digital exgnal processorMultiraic techniques have been in use for many years. Some of the early references are Schafer and Rabiner [1973], Meyer and Burrus [1975], and Octkon, et al. [1975]. The use of multiple sampling rates offers many advan- tages, such as reduced computational complexity for a given task, reduced transmission rate (i.c., bits per second), and/or reduced storage requirement, depending on the application. M - fold decimator L - fold expander Figure 1.1-2 Mullirate buildmg biocks, One of the earliest applications of multirate processing is in professional digital music [Digital audio, 1985]. Broadly speaking, the idea is as follows. Suppose we wish to digitize an analog signal x,(t). Ifthe signal has significant energy only up to a frequency fay. we can first bandiimit the signal to this range using an analog lowpass filter (antialiasing filter), and then sample and digitize it. The lowpass filter in this case has a sharp transition from passband to stopband. A second technique proceeds in two stages: (a) First use an antialias- ing filter with wider transition bandwidth, say by a factor of two. Then oversample by a factor of two before digitizing. so that aliasing due to the poor bandlimiting filter is avoided. (b) Pass the digitized signal through a linear phase digital filler and decimate by two, so that the sampling rate is reduced to the minimum rate. This two-stage process climinates the need for sharp-cutoff antialiasing analog filters, which not only are expensive, but also introduce severe phase distortion. Details of this technique will be con- sidered in Chap. 4. A second application is in fractional sampling rate alteration, for ex- ample, converting a 48 kHz discrete-time signal to a 44.1 kHz discrete-time signal. Such requirements are common in the digital audio industry, where a nuinber of sampling rates coexist [Bloom, 1985]. For example, the sampling rate for studio work is 48 kllz, whereas that for CD production is 44.1 kllz. These, in turn, are different from the broadcast rate (32 kz}. The obvious way to perform the rate conversion would be to first convert the discrete-time signal into a continuous-time signal and then resample it at the lower rate. This method is expensive and involyes analog components, along with the associated inaccuracies. A direct digital (nuultirate) method is to perform the conversion directly in the discrete-Lime domain. Such fractional decimation (or interpolation) is done by combining integer decimators, expanders and filters appropriately. This is more accurate as well as convenient. Details of this technique will be described in Chap. 4. There are many more applications of multirate processing, and several of them are based on the so-called subband decomposition, to be described next, 2 Chap. 1. Introduction1.1 MAJOR DEVELOPMENTS Ifa sequence a(n) is bandii ited, then it is posible to decimate it cither by an integer or by a fraction, by use of appropriate multirate techniques. The ars tos reduce the sampling rate whenever possible is of course under standable, because it usually reduces the storage as well as the processi requirements. . . = xte) 0 r o Figure 1.1-3 Example of Founer transform of a sequence x(r1) which has most of the energy im the low frequency region. Now suppose 2(n) is not bandlimited, but nevertheless has most of the energy in the low frequency region. Figure 1-1-3 demonstrates the Fourier transform of such a signal. Even though this cannot be decimated without aliasing, it seems only reasonable to expect that some kind of data rate re- duction is still feasible. This is indeed made possible by a technique called subband decomposition, implemented with the so-called quadrat ire mirror filter bank. In this technique, the average number of bits per sample is reduced, even though the average number of samples per unit time is un changed. 4.1.1 The Quadrature Mirror Filter (QMF) Bank The quadrature mirror filter bank is shown in Fig. 11-4. Here a discrete- lime signal z(n) is passed through a pair of digital filters M,(z) called anal- ysis filters, with frequency responses as demonstrated in the figure. The filtered signals x1() (subband signals) are thus approximately bandl ited (lowpass and highpass, respectively). They are then decimated by two, so that the number of samples per unit time [counting vo(n) as well as a(n) is the saine as that for x(n). The decimated subband signals, vx(n), are then quantized and transmitted. At the receiver end, these are recombined by us- sae oxpanders and synthesis filters (2). In this manner, an approximation 2(n) of the signal 2(n) is generated. This system will be studied in Chap. 5. The above system can be regarded as a sophisticated quantizer, Thus, assume that we are allowed to trans: bbits per sample. Iw a direct method, we would quantize each sample of x{#) independently to b bits. In the above filter-bank approach, we quantize the lower rate signals mg(1) and us(1) to Soc. 1-1 Major developments 3(a) (bo) hg bits and 4 hits per sample, so that the average bit rate is b = 0.5(bp +). If the signal is dominantly lowpass, then we can make by > band b, < 6. An extreme case is to assign bp = 2b and b, = 0. Thus, depending on the energy distribution in the frequency domain, we can allocate bits to the subbands appropriately, thereby mcreasing the accuracy of representation of z(n), for a fixed hit rate b- xind | we - Analysis 4 Decimators t filters | filters I Quantizers ! Expanders Synthesis ! Riny | —_— — lransmitter ————————> aa Receiver ————_—_—__> Magnitude response aw Figure 1.1-4 (a) The quadrature mirror filter (QMF) bank, and (b) typical frequency responses | Hy(e?™)]. This scheme is calicd subband coding [Croisicr, et al., 1976], [Crochiere, 1977}, [Esteban and Galand, 1977], (Barnwell, 1982], [Galand and Nuss- baumer, 1984]. This has been found to be very useful in speech coding {Crochiere, et al., 1976], where the perceptual properties of the human ear play a major role while assigning bits to vs,(n). More recently, the effectivencss of subband coding has been demon- strated for music signals. Digitized music normally uses 16 bits per sample {at a sampling rate of about 44 kHz). Using subband coding, it has been demonstrated that a major bit rate reduction can be obtained (compared to the traditional 16 bit repesentation), with little compromise of quality [Veld- huis, et al., 1989]. This has been used in the digital compact casscte (DCC). See also the papers iu ICASSP, 1991, pp. 3597-3620, and [Fettweis, et al., 1990}. At the end of this section, more applications of subband splitting will be mentioned. Reconstruction from subband signals. In many applications, the signals e,() (or, more properly, the quantized versions) are recombined to obtain an approximation i(n) of the original signal z( n). This recombina- tion is done by use of expanders (which restore the sampling ratc) followed by digital filters #,(z) (whose purpose will be explained in Sec. 5.1). Such 4 Chap. 1. Introductionis subject to several errors (apart from the error due to qnan- of these is aliasing, created due to decimation of za(nj. Other will be discussed in due course. One of the major developments in multirate signal processing is the recognition of the fact that all of these errors (except quantization ercor) can be eliminated completely at finite cost. hy proper design of filters. The QMF bank, introduced in the mid seventies, has since been ex- tended to the case of more than two subbands. Thus. a system with MW subbands would have M filters followed by M-fold decimators. The deci- mated (and quantized) signals would then be recombined using a synthesis bank (expanders and digital filters), to obtain an approximation #(n) of the signal 2{n). Such a system is called an M-channel maximally decimated filter bank or simply an M-chanocl QMF bank (even though QMF is a misnomer unless Mf = 2, as explained in Chap. 5). xin) fa) Hy He Fourter twansformer Figure 3-1-5 (4) An M-chaunel analysis bank. and typical frequency Ecspomses (b) An 3f-point Fourier transformer, viewed as filter bank. Figure 1.1-5(a) shows the analysis filters of an M-channol system. Two scts of typical frequency Tespanses are also sketched in the figure. One se has uniform filter bandwidths and spacing, while the other has rm (octave) spacing. ‘The latter is particularly useful in the analysis and coding of speech and music. ' ; ae cotta: bark can be viewed axa sophisticated spcetram anulysee as clab- Sec. 1.1 Major developments 5orated in Chap. 4 and 1]. For the moment, we note that a spectrum analyzer takes a signal z(n) and computes the Fourier transform of short blocks, af ter some preliminary processing (such as windowing), Such a system can be interpreted as a filter bank (Fig. 1.1-5(b)]. The outputs of the ‘filters’ represent the discrete-time Fourier transform coeflicients of the blocks of in- put data. While the details will be presented only in Chap. 4 and 11, the main point here is that the filter bank of Fig, 1.1-5(a) is a generalization of the Fouricr transformer, with greater flexibility on the choice of frequency responses. While the Fourier transformer provides filters with overlapping responses, the generalized system can provide filters with arbitrarily sharp cutoff, better interband isolation and unequal bandwidths. 1.1.2 Polyphase Decomposition One of the reasons why multirate processing became practically attractive is the invention of the polyphase decomposition [Bellanger, ct al., 1976), [Vary, 1979]. This enables the designer to perform all computations at the “lowest rate permissible within the given context,” and reduces the speed requirements on the processors. Polyphase decomposition is useful in vir- tually every application of multirate signal processing, and often results in dramatic computational efficiency. It is valuable in theoretical study, practi- cal design and actual implementation of filter banks. This will be introduced in Chap. 4, and subsequently used throughout the text. 1.1.3 Perfect Reconstruction Systems In a practical filter bank system, the filters H;,(2) are not ideal, and decima- tion of the filter outputs results in aliasing errors. As will be seen in Chap. 5, the z-transform of the output signal 7(n) can be expressed as X(z)= T(z)X(z) + terms due to aliasing. It was shown in Croisicr. et al. [1976] that aliasing can be completely elimi- nated in the two channel QMF bank, by proper choice of the synthesis filters Fo(z) and Fy(2). If 7(z) can be forced to be a delay, that is, T(z) = cz~", then the alias-free system is said to have the perfect reconstruction (PR) property. If this is not the case, then the alias-free system still suffers from residual distortion. If the designer does not impose any specifications on the analysis filters such as large stopband attenuation. sharp cutoff rate, and so on, it is an easy matter to choose the filters If,{z) and F(z) so as to satisfy the perfect reconstruction property. However, this is not very practical because, in order to utilize the benefits of subband coding, it is necessary to impose fairly stringent specifications on the attenuation characteristics of the filters. For the two channel QMF bank a fundamental result was proved in- dependently by Smith and Barnwell [1984 and 1986], and Mintzer [1985]. These papers showed that perfect reconstruction can be achicved even af- ter imposing such practical attenuation requirements. This involves careful design of the four filters, as will be seen in Chap. 5. 6 Chap. 1. Introduction7.1.4 Extension fo M Channels The above results from many authors stimulated further research. result- ing in techniques to generalize the subband splitting meas for the case of M-channel QMF hanks {Nussbammer, 1981], [Rothweiler, 1983], [Ramstad. 198-4b], (Smuth and Barnwell, 1985, 1987], (Masson and Ticel, 1985], (Chu. 1985], [Cox, 1986], (Princen and Bradley, 1936], [Wackersrenther, 1986b] [Vetterli, 1986a], [Vaidyanathan, 1987 a}, [Malvar, 1990h|, and [Akansu and Liu, 1991}. Nussbaumer’s pioneering, work on psendo QMF banks provides approximate alias cancelation, which is sufficient in some applications. Smith and Barnwell as well as Ramstad independently showed how to formulate the porfcet reconstruction conditions in matrix form. It was first recognized by Vetterli, and then independently by Vaidyanathan (in the two references mentioned above) that a polyphase component approach results in consid- erable simplification of the theory. It has since been shown that, by using a class of filter banks called paraunitary filter banks, perfect reconstruction can be achicved quite eas- ily. In these systems, the filter bank is constrained to have a paraunitary pelyphase matrix (to be explained in Chap. 6) The designer can specify arbitrary filter attenuation, and at the saine time obtain perfect reconstruc- tion (Vaidyanathan, 1987a], [Neuyen and Vaidyanathan, 1983], [Vetterli and Le Gall, 1989}. Subsequent to this, a class of systems called the cosine modulated filter banks has been developed by some authors [Malvar, 19906}, [Ramstad. 1991}, [Koilpillai and Vaidyanathan, 1991a, 1992]. These have the advantage that the cost of design as well as implementation is largely determined by the cost of one prototype filter, since all the other filters are derived from it The paraunitary property of filter banks offers many advantages, as elaborated in Chap. 6. Interestingly cnough, paraunitary matrices have their origin in classical electrical network theory (see Sec. 14-1 and references therein). In the past, applications of these matrices have becn confined mostly within the nctwork theory and control theory communities. The use of parannitary matrices in digital signal processing, especially filter bank theory, is relatively recent. Filter bank theory has been extended to the case of nonuniform band- widths and decimation ratios [Hoang and Vatdyanat han. 1989], [Kovacevié and Vetterli, L991], and {Nayebi. Barnwell, and Stith, 199tal. 1.1.5 Other Applications and interrelations The suecess of subband coding encouraged rescarchers to extend the ideas to multidimensional signals. The extensions to two dimensional signals has application in image compression and coding. A systematic study of anulti- dimensional filter hanks was first uadertaken by Vetterti [1984]. This adea has since heen applied for iniage coding by Wools and O'Neil {1986}. Since then there has been major progress in multidimensional multirate systems [Ansari and Lau. 1987], [Viscito and Allebach, 19SNb, 1091}, (Simth and Ed- Sec. 1-1 Major developments 71.2 dins, 1990]. [Chen and Vaidyanathan, 1991,1992]. [Bamberger and Smith, 1992], and [Kovatevie and Vetterli, 1992], Further references will be cited in Chap. 12. Reseasch results in multidimensional multirate systems are emerging at a rapid rate now. Recently it has been observed that multirate/subband techuiques are attractive in adaptive and statistical signal processing [Gilloire, 1987), (Sathe and Vavlyanathan. 199.1991], and [Gilloire and Vettesli. 1992}. Research on these topics is still evolving; an excellent reference on the subject is provided by Shynk [1992]. Further applications in communications have been reported by some authors, for example, transmultiplexing [Vetterli. 198Gb], high speed analog to digital conversion [Pertraglia and Mitra, 1990], and equalization {Ranesh. 1990]. In recent years, it has been recognized that there is a close connection between multirate filter banks and the so-called “wavelet transforms”. This relation was revealed by the fundamental contributions by Daubechies [1988] and Mallat [1989a,b}. (See Sec. 11.0 for further references.) This work has opened up considerable amount of research activity in both the signal processing, and mathematics communities. In Chap. 21 we will present this in considerable depth. It will be seen that wavelet analysis is closely related to the so called octave-band filter banks, introduced in the early seventies for analysis of sound signals. Research in wavelet transforms has grown very rapidly after the mid 1980s (and is still growing). SCOPE AND OUTLINE Contributions by many researchers, as outlined above, have resulted in a mature theory of multirate systems. In particular, the detailed aspects of filter bank theory were developed largely during the last decade, subsequent to (and im many cases triggered by) the publication of Crochiere and Rabiner [1983]. The theory of perfect reconstruction filter banks has now reached a state where such systems can be designed as well as implemented with ease. The underlying theory is somewhat complicated, but as a reward it has immense potential for further research aud applications. For example, the theory can be applied directly to areas such as subband coding, voice privacy, image processing, uaultiresolution, and wavelct analysis. The purpose of this text is to present an in-depth study of multirate sys- tems and filter banks. We have assumed that the reader has some exposure to signal processing (eg.. a one-term course from Oppenheim and Schafer [1989], covering sampling, z-transforms, and digital filtering). Except for thi- Aequirement, the book is sclf-contamed. However, this background material ts reviewed in Chap. 2and 3 Fach chapter is supplemented with several homework problems, making, it suitable for classroom use. At the same time, our ain has also been to piovide a useful reference for researchers ‘This is evidenced by the imetusion of several advanced topics. There are many exaniples, design methods, and tables which will aid the practicing professional as well. The chapters can 8 Chap. 1. Introductionfour parts as elaborated. Chapters 2 to 4: Introductory Material A bricf review of linear system findamentals and digital filtenog, i provided in Chap. 2 and 3, More detailed presentation ae found a number of references indicated in these chapters. Iu Chap. 3. IMR elliptic filters, FER eigenfilters, and allpass filters have been treated in greater detail Decause of their special role in multirate systems. Chapter 4 is a detailed study of multirate building blocks, and their interconnections with other systems (such as digital filters). Thix can be considered to be the ‘foundation chapter” for this text. Some of the ezrly sections overlap with the material covered im Croctecre and Rahincr [4953). AU the expense of this overlap, we have ensured Uhat the chapter is self- contained. A number of special types of digital filters, for example, Nyquist filters, power complementary filters and so on. which are frequently encountered in multirate systems, are also studied in Chap. 4. The polyphase decomposi- tion js introduced, along with special types of filter banks, for example, the form-DFT bank. Many applications of multirate processing are also described in Chap 4. This includes subband coding, digital audio, and transmmultiplexers, to name a few. A cotoplete section of this chapter 1s devoted to “multigrid” tecluiques, which find application in the numerical solution of differential ‘equations. Chapters 5 to 8: Maximally Decimated Filter Banks Chapter 5 is a study of the M-channel maximally decimated filter bank system (shown in Fig. 1.1-4(a) for the A = 2 case). Various distortions will be analyzed, foremost being aliasing caused by decimation. Conditions for alias cancelation znd perfect reconstruction will be established. ‘Transmulti- plexers will also be studied Chapter 6 is dedicated to the design of M-channel QMF banks with the perfect reconstruction property. ‘The method presented is based on a class of matrices called paraunitary or lossless transfer matrices. The presentation will use some of the results ou paraunitary snatrices, which will be proved only in Chap. 14. We have chosen to defer the proofs to Chapter 14 (which is devoted te paraunitary systems) in order to ensure an easy and smooth flow. (The results of Chap 14 will also be stated and used in some other chapters, e.g., Chap. 8 and 11.) Chapter 7 deals with Tincar-phase perfect reconstruction QM banks. In these systems the analysis filters have linear phase, which is a requirersent in some applications. Chapter § describes a particular class of M-channel filter banks, in which ul the analysis filters are derived from a single filter by use of co- ‘tine modulation. As a result, this system is very efficient both ftom the deswn aad implementation points of view. It tums out that one can eas Sec. L2 Scope and outtina 9