PROBABILITY, STATISTICS AND QUEUEING THEORY V. Sundarapandian PHI Learning (;ivciie Uinaitied New Delhi - 110 001 2009PROBABILITY, STATISTICS AND QUEUEING THEORY VW. Sundarapandian © 2009 by Pil Learning Private Lites, New Del, All rights reserved. No part ef this Book ‘may be reproduced in any form, by mmecgraph or any ather means, witheut permission In vwting from the publisher. 19BN-978-01-203-3044.9 ‘The export rights of this book are vested solely with the publisher. Published by Asoke K. Ghosh, PHI Learning Private Limited, M07, Connaught Circus, New Delh-110001 and Printed ty Mohan Makhijani at Rekha Printers Private Limite, New Dh 10020.Contents Preface if PROBABILITY .... la 12 La RANDOM VARIABLE 21 22 Brief History of Probability 1 Sample Space and Events 2 Classical and Empirical Probability 6 Problem Set 1.1 18 Axiomatic Definition of Probability 19 Problem Set 1.2 38 Conditional Probability 39 Problem Set 3 7 Total Probability 73 Problem Set Lf 80 Bayes’ Theorem 81 Problem Sct 1.5 90 92-192 Definition of a Random Variable 92 Problem Set 2.1 95 Distribution Funetion of a Random Variable 95) Problem Set 2.2 102 Discrete Random Variable 103 Problem Set 2.9 110 Continuous Random Variable 144 Problem Set 24 124 Mathematical Expectation 126 Problem Set 2.5 153 Chebyshev's Inequality 155 Problem Set 2.6 169 Moments of a Random Variable 168 Problem Set 2.7 172STANDARD PROBABILITY DISTRIBUTIONS 30 32 38 39 sao saz ‘TWO-DIMENSIONAL RANDOM VARIABLES. 919 42 43 44 45 46 ar 4s Moment Generating Funetion Problem Set 2.8 189 Characteristic Function Problem Set 2.9 192 184 Dogenerate Distribution 193 Bernoulli Distribution — 194 Binomial Distribution — 196 Problem Set 3.1 245 Poisson Distribution — 216 Problem Set 3.2 299 Geometric Distribution 284 Problem Set 3.3 242 Nogative Binomial Distribution Problem Set 3.4 246 Uniform Distribution — 247 Problem Set 3.5 255 Exponential Distribution 255 Problem Sel 3.6 264 Gamma Distribution 265 Problem Set 3.7 274 Weibull Distribution 271 Problem Set 3.8 276 Normal Distribution — 276 Problem Set 3.9 296 Functions of Random Variable Problem Set 3.10 846 Joint Distribution Functions Problem Set 4.1 941 Marginal Distributions Problem Set £2 367 Conditional Distributions Problem Set .9 397 Expectation 400 Problent Set 4.6 408 a2 Sums of Independent Random Variables Problem Set 4.5 447 Functions of Random Variables Problem Set 4.6 440 Covariance 444 Problem Set 4.7 454 Conditional Expectation Problem Set £.8 462 495 369 CONTENTS 14 sn 198-B18, 242 208 319-520 409 418CONTENTS vi 4.9 Correlation and Regression 469 Problem Set .9 510 4.10 Central Limit Theorem — 542 Problem Set £10 520 5 RANDOM PROCESSES... 521-635, 5.1 Definition and Description of Random Processes 521 5.1.1 Classification of Random Processes 522 5.1.2 Deseription of Random Processes 529 5.1.8 Mean, Correlation and Covariance Functions — 524 Problem Set 5.1 534 5.2 Stationary Random Processes 592 5.21 First-Order Stationary Processes 692 5.2.2 Second-Order Stationary Processes 534 5.2.8 Ordor rand Strict-Sende Stationary Processes — 536 Problem Set 5.2 548 5.8 Autocorrelation and Crose-cortelation Funetions 649 Problem Set 5.3 587 Exgodie Process 558 Problom Set 5.4. 572 573 5.1 Markov Chain 5% 5.5.2 Probability Distribution of a Markov Chain 676 5.53 a z Chapman-Kolmogoroy Theorem 577 Stationary Distribution for a Markov Chain 68 5.5.5 Classfieation of States of a Markov Chain 598 Problem Set 5.5 602 5.6 Binomial, Poisson and Normal Processes 604, Problom Sct 5.6 624 5.7 Sine Wave Proves 625 5.8 Birth and Death Process 626 5.8.1 Pure Birth Process — 627 2 Poisson Process 628 3 Birth and Death Process 692 6. SPECTRAL ANALYSIS OF RANDOM PROCESSES 6.41 Autoconelation and Cresscorrelation Functions 636 Problem Sct 6.1 640 6.2 Power Spectral Density and Cross-spectral Dens Problem Set 6.2 667 6.8 Linear Systems with Random Inputs 668 Problem Set 6.3 689 886-685, ity Functions 644 7. QUEUEING THEORY ene 686-749 71 Basie Characteristies of Queueing Models 686 TLL Transient and Steady-States 687CONTE 7.1.2 Kendall's Notation of » Queueing Sys 688 7.1.8 Transient State Probabilities for Poisson Queue Systems 689 T.14 Steady State Probabilities for Poisson Queue Sysiems 690 72 Model I—(M/M/1): (c2/FIFO) Single Server with Infinite Capacity — 692 7.21 Characteristies of Model 1 692 Problem Set 7.1 707 7.8 Model I-(M/Mjs): (c0/FIFO), Multiple Server with Infinite Capacity 709 TALL Characteristics of Model I 714 Problem Set 722 724 74 Model IN—(M/M/1): (4/PIFO) Single Server with Finite Capacity — 722 TALL Characteristics of Model IT 725 Problem Set 7.3780 7.9 Model IV—(M/M/s): (k/FIFO) Multiple Server with Finite Capacity 792 7.5.1 Characteristies of Model IV 734 Problem Set 7.4 739 7.6 The (M/G/1) Queueing System 740 7.6.1 Pollaczek-Khinchine Formula 742 Problem Set 7.5 148 Bibliography .... 751-752 Answers to Problems 7153-806 807-809 Index.Preface ‘The notions of Probability, Statistics and Queueing Theory are fundamental to Tenrning many advanced concepts in various branches of Engineering, Mathematics, biological sciences, and many others. This book is the result of my teaching of Statisties at B.Tech./M.Se. levels at Indian Institute of Technology Kanpur, Vellore Institute of Technology (VIT), and SRM University at Chennai. All basic theories have been presented to make the students clearly understand the subject The book is organized into chapters. Chapter 1 gives a detailed review of probability (with classical, empizical and axiomatic definitions), conditional probability, total probability, and Bayes’ theorem, Chapter 2 elaborates the concept of distribution function, types, and the moment-generating and characteristic functions of a random variable, Tt also discusses mathomatieal expectation and Chebyshev's inequality Chapter 3 provides a detailed description of the standard probability. distributions Chapter 4 diseusses the important. concepts of tiro-dimensional random variables sueh as joint distribution functions, marginal and conditional density functions, sums of independent random variables, conditional expectation, covariance, correlation, and regression, The chapter conelucles with a study on Central Limit Theorem, whieh is important in engineering and physical applications, Chapter 5 deals with random processes and their types, nutocorrelation and crass correlation functions, and the ergodic, Markov, binomial, Poisson, normal, sine wave, and birth and death processes. Chapter 6 analyzes the spectral properties of random processes such fas power spectral density and erossespectral density functions of random processes, and their applications to linear 3 Chapter 7 discusses the basics of queueing theory with a detailed study on the five important queneing: models, viz, (M/M/1}(e0/FIFO), (M/M/s):(ce/FIFO), M/M,/1:(k/FIFO). M/M/si(k/FIFO), and (M/G/1), Besides, the book contains numerous definitions, example, theorems, and proofs to help the students understand the concepts well This book is mainly intended as a text for the undergraduate courses in engineering (B.Tech. /BE) and postgraduate courses in science (M.Sc.) offered by engineering institutes Tike IITs and by varions Indian universities. | thank my mentor, Professor Christopher 1. Byrt Washington Un tems with random inputs, Eleetrical and Systems Engineering, versity, who has been a great source of inspiration for me in both teachingx PREFACE and research. I also wish to thank Professor P.C. Joshi, Indian Institute of Technology Kanpur, for assigning me the task of teaching Statistics for the M.Se. Mathematics students at IT Kanpur. I also thank my colleagues, Professors M.S. Gopinathan and CS, Padmanabha Iver at IITM-K, Thiruvananthapuram, Kerala, and Professors E.D, Jemmis and VM. Nandakumaran at Indian Institute of Science Education and Research (IISER), ‘Thirnvananthapwram, for their advice and encouragement. I thank my Publishers, PHT Learning, New Delhi, especially their editorial and production team, for their caroful processing of the manuscript T hope that the studen meeting their needs, T welcome any valuable contents of the book. faculty and other readers will find this book very useful for wexestions and constructive comments for improving the YV. Sundarapandian E-mail: sundardiiitmak.nc.inChapter 1 Probability 1.1 BRIEF HISTORY OF PROBABILITY It is well-known that the theory of probability had its origin in gambling and games of chanee, The foundation of probability theory, as a precise mathematical science, was laid in the mid-seventeenth century by two great French mathematicians Blaise Pascal (1623-1662) and Pierre de Fermat (1601 1665). In 1654, the French gambler and noble man Chavalier de Mere (1607-1648) asked many prominent mathematicians including Pascal several questions related to problems of ehance, the best known of which was the problem of points. The problems posed by de Mere paved the way for a fmuiful correspondence of letters on the games of chance between Pascal and Fermat, The famous Pascal-Fermat correspondence not only solved de Mere’s problems but laid the foundation for more general results on the games of chance also. To tackle these problems, Fermat used combinatorial analysis (finding the number of possible and that of favourable outcomes in ideal games of chance using permutations and combinations), while Pascal reasoned by recursion (an iterative process in which the result of the next case is determined by the present case). EXAMPLE LL, (Problem of Points) Two players, A and B, play a series of fair games until one person has won six games. Both A and B have wagered the same amount of money with the understanding that the winner takes all. But, suppose thatthe series of fair games is prematurely stopped for whatever reasons at whieh point A has won five games and B three games, How is the stake to he divided among the players? Solution. (Pascal and Fermat) As A bas won five games and B three games, A needs 9 win just one more game, while B needs to win three more games, Thetefore, the play willbe over after at most three further games, Its elear that there area total of 8 possible Future outcomes, and all these outcomes are equally likely. ‘The exhaustive possible outcomes of the play are AAA, AAB, ABA, ABB, BAA, BAB, BBA and BBB. Probabiity(A winning) — N2-9f outcomes in which A wins Toualno, of outwomes No. of outcomes in which Bwins | Probabilty(B winning) = —~“T atmo. of outcomes -