Mathematical Statistics

for Economics and Business

Springer
New York
Berlin
Heidelberg
Barcelona
Hong Kong
London
Milan
Paris
Singapore
Tokyo
Ron C. Mittelhammer
Mathematical
Statistics
for Economics
and Business
With 83 Illustrations
Springer
Ron C. Mittelhammer
Program in Statistics
and
Department of Agricultural Economics
Washington State University
Pullman, WA 99164-6210
USA
library of Congress Cataloging-in-Publication Data
Mittelhammer, Ron.
Mathematical statistics for economics and business / Ron
Mittelhammer.
p. cm.
Includes bibliographical references and
ISBN-13: 978-1-4612-8461-1 e-ISBN-13: 978-1-4612-3988-8
DOl: 10.1007/978-1-4612-3988-8
1. Commercial statistics. 2. Mathematical statistics. I. Title.
HF1017.M53 1995
519.5-dc20 95-37686
Printed on acid-free paper.
1996 Springer-Verlag New York Inc.
Softcover reprint of the hardcover 1 st edition 1996
All rights reserved. This work may not be translated or copied in whole or in part without the written
permission of the publisher (Springer-Verlag New York, Inc., 175 Fifth Avenue, New York, NY 10010,
USA), except for brief excerpts in connection with reviews or scholarly analysis. Use in connection with
any form of information storage and retrieval, electronic adaptation, computer software, or by similar or
dissimilar methodology now known or hereafter developed is forbidden.
The use of general descriptive names, trade names, trademarks, etc., in this publication, even if the former
are not especially identified, is not to be taken as a sign that such names, as understood by the Trade Marks
and Merchandise Marks Act, may accordingly be used freely by anyone.
Production managed by Laura Carlson; manufacturing supervised by jacqui Ashri.
Typeset by The Bartlett Press, Inc.
9 8 7 6 5 4 3 (Corrected third printing, 1999)
To my wife Linda,
and to Carl, Edith, Dolly and Ralph
Preface
This book is designed to provide beginning graduate stu-
dents and advanced undergraduates with a rigorous and accessible foundation
in the principles of probability and mathematical statistics underlying statis-
tical inference in the fields of business and economics. The book assumes no
prior knowledge of probability or statistics and effectively builds the subject
"from the ground up." Students who complete their studies of the topics in
this text will have acquired the necessary background to achieve a mature and
enduring understanding of statistical and econometric methods of inference
and will be well equipped to read and comprehend graduate-level economet-
rics texts. Additionally, this text serves as an effective bridge to more advanced
study of both mathematical statistics and econometric theory and methods.
The book will also be of interest to researchers who desire a decidedly business
and economics-oriented treatment of the subject in terms of its topics, depth,
breadth, examples, and problems.
Without the unifying foundations that come with training in probability
and mathematical statistics, students in statistics and econometrics classes too
often perceive the subject matter as a potpourri of formulae and techniques
applied to a collection of special cases. The details of the cases and their solu-
tions quickly fade for those who do not understand the reasons for using the
procedures they attempt to apply. Many institutions now recognize the need for
a more rigorous study of probability and the mathematical statistics principles
viii Preface
in order to prepare students for a higher level and a longer lasting understanding
of the statistical techniques employed in the fields of business and economics.
Furthermore, quantitative analysis in these fields has progressed to the point
where a deeper understanding of the principles of probability and statistics is
now virtually necessary for one to read and contribute successfully to quanti-
tative research in economics and business. Contemporary students themselves
know this and need little convincing from advisors that substantial statistical
training must be acquired in order to compete successfully with their peers and
to become effective researchers. Despite these observations, there are very few
rigorous books on probability and mathematical statistics foundations that are
also written with the needs of business and economics student in mind.
This book is the culmination of 15 years of teaching graduate level statis-
tics and econometrics classes for students who are beginning graduate programs
in business Iprimarily finance, marketing, accounting, and decision sciences),
economics, and agricultural economics. When I originally took on the teach-
ing assignment in this area, I cycled through a number of very good texts in
mathematical statistics searching for an appropriate exposition for beginning
graduate students. With the help of my students, I ultimately realized that the
available textbook presentations were optimizing the wrong objective func-
tions for our purposes! Some books were too elementary, other presentations
did not cover multivariate topics in sufficient detail, and proofs of important
results were omitted occasionally because they were "obvious" or "clear" or
"beyond the scope of the text." In most cases they were neither obvious nor
clear to students, and in many cases, useful and accessible proofs of the most
important results can and should be provided at this level of instruction. Suf-
ficient asymptotic theory was often lacking and/or tersely developed. At the
extreme, material was presented in a sterile mathematical context at a level
that was inaccessible to most beginning graduate students while nonetheless
leaving notable gaps in topic coverage of particular interest to business and eco-
nomics students. I then began to teach the course from lecture notes that I had
created and iteratively refined them as I interacted with scores of students who
provided me with feedback regarding what was working-and what wasn't-
with regard to topics, proofs, problems, and exposition. I am deeply indebted
to the hundreds of students who persevered through, and contributed to, the
many revisions and continual sophistication of my notes. Their influence has
had a substantial impact on the text: It is a time-tested and class-tested prod-
uct. Other students at a similar stage of development should find it honest,
accessible, and informative.
Instructors attempting to teach a rigorous course in mathematical statistics
soon learn that the typical new graduate student in economics and business is
thoroughly intelligent, but often lacks the sophisticated mathematical training
that facilitates understanding and assimilation of the mathematical concepts
involved in mathematical statistics. My experience has been that these stu-
dents can understand and become functional with sophisticated concepts in
mathematical statistics if their backgrounds are respected and the material
is presented carefully and thoroughly, using a realistic level of mathematics.
Preface ix
Furthermore, it has been my experience that most students are actually eager
to see proofs of propositions, as opposed to merely accepting statements on
faith, so long as the proofs do not insult the integrity of the nonmathemati-
ciano Additionally, students almost always remark that the understanding and
the long-term memory of a stated result is enhanced by first having worked
through a formal proof of a proposition, and then working through examples
and problems that require the result to be applied.
With the preceding observations in mind, the prerequisites for the book
include only the usual introductory college-level courses in basic calculus (in-
cluding univariate integration and differentiation, partial differentiation, and
multivariate integration of the iterated integral type) and basic matrix algebra.
The text is largely self-contained for students with this preparation. A signifi-
cant effort has been made to present proofs in ways that are accessible. Care has
been taken to choose methods and types of proofs that exercise and extend the
learning process regarding statistical results and concepts learned prior to the
introduction of the proof. A generous number of examples are presented with
a substantial amount of detail to illustrate the application of major theories,
concepts, and methods. The problems at the end of the chapters are chosen
to provide an additional perspective to the learning process. The majority of
the problems are word problems designed to challenge the reader to become
adept at what is generally the most difficult hurdle-translating descriptions
of statistical problems arising in business and economic settings into a form
that lends itself to solutions based on mathematical statistics principles. I have
also warned students through the use of asterisks (*) when a proof, concept, ex-
ample, or problem may be stretching the bounds of the prerequisites so as not
to frustrate the otherwise diligent reader, and to indicate when the help of the
instructor or additional readings may be useful.
The book is designed to be versatile. The course that inspired this book is
a 4-credit, semester-long, intensive mathematical statistics foundation course.
I do not lecture on all of the topics contained in the book in the 50 contact
hours available in the semester. The topics that I do not cover are taught in the
first half of a subsequent semester-long 3-credit course in statistics and econo-
metric methods. I have tended to treat chapters 1 through 4 in detail, and I
recommend that this material be thoroughly understood before venturing into
the statistical inference portion of the book. Thereafter, the choice of topics
is flexible. For example, the instructor can control the depth at which asymp-
totic theory is taught by her choice of whether the starred topics in chapter 5
are discussed. While random sampling, empirical distribution functions, and
sample moments should be covered in chapter 6, the instructor has leeway in
the degree of emphasis that she places on other topics in the chapter. Point
estimation and hypothesis testing topics can then be mixed and matched with
a minimal amount of back-referencing between the respective chapters.
Distinguishing features of this book include the care with which topics
are introduced, motivated, and built upon one another; use of the appropriate
level of mathematics; the generous level of detail provided in the proofs; and a
familiar business and economics context for examples and problems. This text
x Preface
Acknowledgments
is a bit longer than some of the others in the field. The additional length comes
from additional explanation, and the detail in examples, problems, and proofs
and not from a proliferation of topics which are merely surveyed rather than
fully developed. As I see it, a survey of statistical techniques is useful only
after one has the fundamental statistical background to appreciate what is
being surveyed. And this book provides the necessary background.
I am indebted to large number of people for their encouragement and comments.
Millard Hastay, now retired from the Washington State University economics
faculty, is largely responsible for my unwavering curiosity and enthusiasm for
the field of theoretical and applied statistics and econometrics. George Judge
has been a constant source of encouragement for the book project and over the
years has provided me with very valuable and selfless advice and support in
all endeavors in which our paths have crossed. I thank Jim Chalfant for giving
earlier drafts of chapters a trial run at Berkeley, and for providing me with
valuable student and instructor feedback. Thomas Severini at Northwestern
provided important and helpful critiques of content and exposition. Martin
Gilchrist at Springer-Verlag provided productive and pleasurable guidance to
the writing and revision of the text. I also acknowledge the steadfast support of
Washington State University in the pursuit of the writing of this book. Of the
multitude of past students who contributed so much to the final product and
that are too numerous to name explicitly, lowe a special measure of thanks to
Don Blayney, now of the Economic Research Service, and Brett Crow, currently
a promising Ph.D candidate in Economics at WSU, for reviewing drafts of the
text literally character by character and demanding clarification in a number
of proofs and examples. I also wish to thank many past secretaries who toiled
faithfully on the book project. In particular, I wish to thank Brenda Campbell,
who at times literally typed morning, noon, and night to bring the manuscript
to completion, and without whom completing the project would have been
infinitely more difficult. Finally, I thank my wife Linda, who proofread many
parts of the text, provided unwavering support, sustenance, and encouragement
to me throughout the project, and, despite all of the trials and tribulations,
remains my best friend.
Contents
Preface vii
1. Elements of Probability Theory 1
1.1. Introduction 1
1.2. Experiment, Sample Space, Outcome, and Event 2
1.3. Nonaxiomatic Probability Definitions 5
1.4. Axiomatic Definition of Probability 8
1.5. Some Probability Theorems 15
1.6. A Digression on Events 20
1.7. Conditional Probability 22
1.8. Independence 29
1.9. Bayes's Rule 34
Key Words, Phrases, and Symbols 36
Problems 37
2. Random Variables, Densities, and Cumulative Distribution
Functions 43
2.1. Introduction 43
2.2. Univariate Random Variables and Density Functions 44
xii Contents
Probability Space Induced by a Random Variable 45
Discrete Random Variables and Probability Density
Functions 48
Continuous Random Variables and Probability Density
Functions 51
Classes of Discrete and Continuous PDFs 55
Mixed Discrete-Continuous Random Variables 58
2.3. Univariate Cumulative Distribution Functions 60
CDF Properties 63
Duality Between CDFs and PDFs 65
2.4. Multivariate Random Variables, PDFs, and CDFs 67
Multivariate Random Variable Properties and Classes of
PDFs 69
Multivariate CDFs and Duality with PDFs 72
Multivariate Mixed Discrete-Continuous and Composite
Random Variables 76
2.5. Marginal Probability Density Functions and CDFs 77
Bivariate Case 77
N-Variate Case 81
Marginal Cumulative Distribution Functions (MCDFs) 83
2.6. Conditional Density Functions 83
Bivariate Case 84
Conditioning on Elementary Events in Continuous Cases 86
N -Variate Case 88
Conditional CDFs 89
2.7. Independence of Random Variables 90
Bivariate Case 90
N-Variate 93
Independence Between Random Vectors and Between
Functions of Random Vectors 95
2.8. Extended Example of Multivariate Concepts in the
Continuous Case 97
2.9. Events Occurring with Probability Zero 100
Key Words, Phrases, and Symbols 101
Problems 101
3. Mathematical Expectation and Moments 109
3.1. Expectation of a Random Variable 109
3.2. Expectation of a Function of Random Variables 116
Expectation Properties 121
Multivariate Extensions 122
3.3. Conditional Expectation 125
Regression Function 129
Conditional Expectation and Regression in the Multivariate
Case 130
Contents xiii
3.4. Moments of a Random Variable 132
Relationship Between Moments About the Origin and Mean 137
Existence of Moments 138
Nonmoment Measures of Probability Density Characteristics 139
3.5. Moment- and Cumulant-Generating Functions 141
Uniqueness and Inversion of MGFs 143
Cumulant-Generating Function 145
Multivariate Extensions 146
3.6. Joint Moments, Covariance, and Correlation 148
Covariance and Correlation 148
Correlation, Linear Association, and Degeneracy 152
3.7. Means and Variances of Linear Combinations of Random
Variables 156
3.8. Necessary and Sufficient Conditions for Positive
Semidefiniteness 161
Key Words, Phrases, and Symbols 162
Problems 162
4. Parametric Families of Density Functions 169
4.1. Parametric Families of Discrete Density Functions 170
Family Name: Uniform 170
Family Name: Bernoulli 171
Family Name: Binomial 172
Family Name: Multinomial 174
Family Name: Negative Binomial and Geometric 175
Family Name: Poisson 177
Family Name: Hypergeometric 183
Family Name: Multivariate Hypergeometric 185
4.2. Parametric Families of Continuous Density Functions 186
Family Name: Uniform 186
Family Name: Gamma 187
Gamma Subfamily Name: Exponential 190
Gamma Subfamily Name: Chi-Square 192
Family Name: Beta 194
4.3. The Normal Family of Densities 197
Family Name: Univariate Normal 197
Family Name: Multivariate Normal Density 202
4.4. The Exponential Class of Densities 213
Key Words, Phrases, and Symbols 215
Problems 216
5. Basic Asymptotics 221
5.1. Introduction 221
5.2. Elements of Real Analysis 222
xiv Contents
Limit of a Sequence 224
Continuous Functions 228
Convergence of Function Sequence 230
Order of Magnitude of a Sequence 231
5.3. Types of Random-Variable Convergence 233
Convergence in Distribution 234
Convergence in Probability 241
Convergence in Mean Square lor Convergence in Quadratic
Mean) 249
.. Almost-Sure Convergence lor Convergence with
Probability 1) 253
Relationships Between Convergence Modes 258
5.4. Laws of Large Numbers 258
Weak Laws of Large Numbers IWLLN) 259
Strong Laws of Large Numbers ISLLN) 263
5.5. Central Limit Theorems 268
Independent Scalar Random Variables 269
'"Dependent Random Variables 281
Multivariate Central Limit Results 282
5.6. Asymptotic Distributions of Differentiable Functions of
Asymptotically Normally Distributed Random Variables 286
Key Words, Phrases, and Symbols 290
Problems 291
6. Sampling, Sample Moments, Sampling Distributions, and
Simulation 297
6.1. Introduction 297
6.2. Random Sampling 299
Random Sampling from a Population Distribution 300
Random Sampling Without Replacement 303
Sample Generated by a Composite Experiment 305
Commonalities in Probabilistic Structure of Random
Samples 306
Statistics 307
6.3. Empirical or Sample Distribution Function 308
EDF: Scalar Case 308
EDF: Multivariate Case 313
6.4. Sample Moments and Sample Correlation 314
Scalar Case 314
Multivariate Case 320
6.5. Properties of Xn and When Random Sampling from a
Normal Distribution 328
6.6. Sampling Distributions: Deriving Probability Densities of
Functions of Random Variables 331
MGF Approach 332
Contents xv
CDF Approach 332
Equivalent Events Approach /Discrete Case) 334
Change of Variables /Continuous Case) 335
6.7. t-and F-Densities 341
t-Density 341
Family Name: t-Family 343
F-Density 345
Family Name: F-Family 346
6.8. Random Sample Simulation and the Probability Integral
Transformation 348
6.9. Order Statistics 351
Key Words, Phrases, and Symbols 355
Problems 356
7. Elements of Point Estimation Theory 363
7.1. Introduction 363
7.2. Statistical Models 365
7.3. Estimators and Estimator Properties 370
Estimators 370
Estimator Properties 371
Finite Sample Properties 373
Asymptotic Properties 383
Class of Consistent Asymptotically Normal/CAN)
Estimators and Asymptotic Properties 385
7.4. Sufficient Statistics 389
Minimal Sufficient Statistics 393
Sufficient Statistics in the Exponential Class 397
Sufficiency and the MSE Criterion 398
Complete Sufficient Statistics 400
Sufficiency, Minimality, and Completeness of Functions of
Sufficient Statistics 404
7.5. Results on MVUE Estimation 405
Cramer-Rao Lower Bound 408
Complete Sufficient Statistics and MVUEs 420
Key Words, Phrases, and Symbols 421
Problems 422
8. Point Estimation Methods 427
8.1. Introduction 427
8.2. Least Squares and the General Linear Model 428
The Classical GLM Assumptions 429
Estimator for f3 Under Classical GLM Assumptions 433
Estimator for a
2
A
Under Classical GLM Assumptions 437
Consistency of f3 439
xvi Contents
Consistency of S2 441
Asymptotic Normality of 443
Asymptotic Normality of S2 447
Summary of Estimator Properties 449
Violations of Classic GLM Assumptions 452
GLM Assumption Violations: Property Summary and
Epilogue 459
Least Squares Under Normality 460
MVUE Property of {3 and 52 462
8.3. The Method of Maximum Likelihood 464
MLE Mechanics 465
MLE Properties: Finite Sample 470
MLE Properties: Large Sample 472
MLE Invariance Principle 485
MLE Property Summary 489
8.4. The Method of Moments 490
Method of Moments Estimator 491
MOM Estimator Properties 493
Generalized Method of Moments (GMM)
Estimator 495
GMM Properties 497
Key Words, Phrases, and Symbols 502
Problems 502
9. Elements of Hypothesis-Testing Theory 509
9.1. Introduction 509
9.2. Statistical Hypotheses 510
9.3. Basic Hypothesis-Testing Concepts 514
Statistical Hypothesis Tests 514
Type I Error, Type II Error, and Ideal Statistical
Tests 516
Controlling Type I and II Errors 519
Type I/Type II Error Tradeoff 522
Test Statistics 526
Null and Alternative Hypotheses 527
9.4. Parametric Hypothesis Tests and Test Properties 527
Maintained Hypothesis 529
Power Function 530
Properties of Statistical Tests 531
P Values 535
Asymptotic Tests 538
9.5. Results on UMP Tests 539
Neyman-Pearson Approach 539
Monotone Likelihood Ratio Approach 550
Exponential Class of Densities 556
Contents xvii
*Conditioning in the Multiple Parameter Case 571
Concluding Remarks 586
9.6. Noncentral t-Distribution 586
Family Name: Noncentral t-Distribution 586
Key Words, Phrases, and Symbols 588
Problems 589
10. Hypothesis-Testing Methods 595
10.1. Introduction 595
10.2. Heuristic Approach 596
10.3. Generalized Likelihood Ratio Tests 601
Test Properties: Finite Sample 602
Test Properties: Asymptotics 608
10.4. Lagrange Multiplier Tests 616
10.5. Wald Tests 622
10.6. Tests in the GLM 625
Tests When e Is Multivariate Normal 625
Testing R{3 = r, R{3 :::: r, Or R{3 ::: r When R Is 11 x k):
T-Tests 628
Bonferroni Joint Tests of Ri{3 = ri, Ri{3 :::: ri, or Ri{3 ::: ri,
i=I, ... ,m 631
Testing When e Is Not Multivariate Normal 635
Testing R{3 = r or RI{3) = r When R Has q Rows:
Asymptotic X
2
- Tests 636
Testing RIf3J = I, RIf3J :::: I, or RIf3J ::: I When R Is a
Scalar Function: Asymptotic Normal Tests 636
10.7. Confidence Intervals and Regions 640
Defining Confidence Regions via Duality with Critical
642
Properties of Confidence Regions 647
Confidence Regions from Pivotal Quantities 649
Confidence Regions as Hypothesis Tests 654
10.8. Nonparametric Tests of Distributional Assumptions 654
Functional Forms of Probability Distributions 655
IID Assumption 663
10.9. Noncentral X2- and F-Distributions 666
Family Name: Noncentral X2-Distribution 666
Family Name: Noncentral F-Distribution 667
Key Words, Phrases, and Symbols 668
Problems 669
Appendix A. Math Review: Sets, Functions, Permutations,
Combinations, and Notation 677
A.1. Introduction 677
xviii Contents
A.2. Definitions, Axioms, Theorems, Corollaries, and Lemmas 677
A.3. Elements of Set Theory 679
Set-Defining Methods 680
Set Classifications 681
Special Sets, Set Operations, and Set Relationships 682
Rules Governing Set Operations 684
A.4. Relations, Point Functions, and Set Functions 687
Cartesian Product 688
Relation (Binary) 689
Function 691
Real-Valued Point Versus Set Functions 693
A.S. Combinations and Permutations 696
A.6. Summation, Integration and Matrix Differentiation Notation 699
Key Words, Phrases, and Symbols 701
Problems 702
Appendix B. Useful Tables 705
B.1. Cumulative Normal Distribution 706
B.2. Student's t Distribution 707
B.3. Chi-square Distribution 708
B.4. F-Distribution: 5% Points 710
B.S. F-Distribution: 1 % Points 712
Index 715