Middle East Technical University

Department of Electrical and Electronics Engineering

EE 503
Statistical Signal Processing and Modeling
Fall 2019-2020

Short Description

The course aims to unify the knowledge of linear system theory, digital signal processing and
stochastic processes within the framework of statistical signal processing. The goal of the course is to
establish a firm foundation for signal modeling, random processes, estimation theory (particularly
linear minimum mean square error (LMMSE) estimators), Wiener filtering and linear prediction.
Some relevant topics of interest are spectral factorization, LTI processing of random processes,
whitening filter, stochastic signal models such as AR, MA, ARMA, and Harmonic processes, Yule-
Walker equations, causal – non causal IIR Wiener filters, applications in filtering, prediction, and
smoothing, linear decorrelating transforms, reduced dimension stochastic signal representation and
Karhunen-Loeve transform.

Instructor and Schedule

Instructor Office Class times and locations Office hours
Figen S. Öktem DZ04 Tue. 8:40-9:30 EA306 By appointment via email
Thurs. 8:40-10:30 EA306

Main Textbooks
[1] C.W. Therrien, Discrete random signals and statistical signal processing, Prentice Hall, 1992.
(For signal modeling topics mostly, level: moderate)
[2] M.H. Hayes, Statistical Signal Processing and Modeling, Wiley, 1996.
(For random vectors and processes topics mostly, level: moderate)

Additional References
[3] L.L. Scharf, Statistical Signal Processing, Addison-Wesley Publishing Company, 1991. (Level:
advanced)
[4] A. Papoulis, Probability, Random Variables, and Stochastic Processes, McGraw Hill, 1991. (Level:
important reference book, mostly advanced)
[5] S. M. Ross, Introduction to probability models, 7th ed. Harcourt Academic Press, 2000. (Level:
introductory but complete)
[6] Kurt Bernardo Wolf, Integral Transforms in Science and Engineering Plenum Pub Corp, January
1979. (Level: advanced)
[7] P. Lancaster and M. Tismenetsky. The Theory of Matrices. Academic Press, Boston, 2nd edition,
1985. (Level: complete text, very valuable as a linear algebra reference)
Background Requirements
Digital signal processing (EE 430), undergraduate-level probability and basic stochastic processes
(EE 230 & EE 306), linear algebra (EE 501 highly recommended).

Tentative Grading Policy

Midterm I: 25%, Midterm II: 30%, Final Exam: 40%
Homeworks and computer assignments: 5%
HW submission will be uploaded to ODTUClass. Handwritten solutions can be uploaded as a single PDF.
HW Late Submission Policy: for the first day -->10% off, for each additional day --> 20% off per day. No
late submission will be accepted after the solutions are shared.
Lecture notes of a last year’s student is available in ODTUClass.
Course Outline

Relevant
Week Subjects Details
reading
Introduction,
Review of some linear algebra concepts in
Review of some
relation to vector space signal processing:
linear algebra [2, Ch.2]
1-2 vector spaces, solving linear equations, and
and DSP
projections. Basics of optimization. Review of
concepts
some DSP concepts.
Review of probability theory, Random vectors
[2, Ch.3] and
Random vectors and their characterization, Gaussian vectors,
2-3 [1, Ch.2]
linear transformation on random vectors and
whitening
Decorrelating
Decorrelating transforms, Karhunen-Loeve [1, Ch.4]
3-4 transforms
expansion
[2, Ch.3]
Random Random processes and their descriptions,
and [1, Ch.3]
5-7 processes stationarity, power spectral density, ergodicity,
LTI processing of random processes.
Special types of random processes (AR, MA,
ARMA processes, harmonic processes), Yule-
[2, Ch.3 - 4]
Signal modeling Walker equations, spectral factorization,
8-9 and [1, Ch.9]
deterministic signal modelling, Prony’s
method, all-pole modeling, autocorrelation
estimates.
Random and non-random parameter
Introduction to
estimation, properties of estimators, linear min. [1, Ch.6]
9-11 Estimation
mean square error (LMMSE) estimators,
orthogonality principle
FIR Wiener filtering, Wiener-Hopf equations,
Optimal FIR [1, Ch.7-8]
Levinson-Durbin recursion, ergodicity,
11-12 Filters and [2, Ch.7]
applications of filtering, smoothing, and
prediction
Optimal IIR IIR Wiener filters (non-causal, causal), causal [1, Ch.7-8]
13-14 Filters linear prediction and [2, Ch.7]

Course Objectives

Course Objective 1: Students will be able to comprehend the stochastic signal models and utilize
them in signal processing applications.

Course Objective 2: Students will comprehend the optimal linear-time invariant (LTI) filtering of
stochastic processes (in the sense of minimum mean square error) and utilize it in signal processing
applications such as prediction, smoothing, and denoising.

Course Objective 3: Students will be able to comprehend reduced dimension stochastic signal
representation using Karhunen-Loeve transform and its use for signal compression.