Probability and Measure

Patrick Billingsley was Professor Emeritus of Statistics and Mathematics at the University of Chicago and a world-renowned authority on probability theory before his untimely death in 2011. He was the author of Convergence of Probability Measures (Wiley), among other works. Dr. Billingsley edited the Annals of Probability for the Institute of Mathematical Statistics. He received his PhD in mathematics from Princeton University.

“Like the previous editions, this Anniversary edition will be well received by students of mathematics, statistics, economics, and a wide variety of disciplines that require a solid understanding of probability theory.”  (Int. J. Microstructure and Materials Properties, 1 February 2013)

• FOREWORD xi PREFACE xiiiPatrick Billingsley 1925–2011 xvChapter1 PROBABILITY 11. BOREL’S NORMAL NUMBER THEOREM, 1The Unit IntervalThe Weak Law of Large NumbersThe Strong Law of Large NumbersStrong Law Versus WeakLengthThe Measure Theory of Diophantine Approximation*2. PROBABILITY MEASURES, 18SpacesAssigning ProbabilitiesClasses of SetsProbability MeasuresLebesgue Measure on the Unit IntervalSequence Space*Constructing s-Fields*3. EXISTENCE AND EXTENSION, 39Construction of the ExtensionUniqueness and the p–? TheoremMonotone ClassesLebesgue Measure on the Unit IntervalCompletenessNonmeasurable SetsTwo Impossibility Theorems*4. DENUMERABLE PROBABILITIES, 53General FormulasLimit SetsIndependent EventsSubfieldsThe Borel-Cantelli LemmasThe Zero-One Law5. SIMPLE RANDOM VARIABLES, 72DefinitionConvergence of Random VariablesIndependenceExistence of Independent SequencesExpected ValueInequalities6. THE LAW OF LARGE NUMBERS, 90The Strong LawThe Weak LawBernstein's TheoremA Refinement of the Second Borel-Cantelli Lemma7. GAMBLING SYSTEMS, 98Gambler's RuinSelection SystemsGambling PoliciesBold Play*Timid Play*8. MARKOV CHAINS, 117DefinitionsHigher-Order TransitionsAn Existence TheoremTransience and PersistenceAnother Criterion for PersistenceStationary DistributionsExponential Convergence*Optimal Stopping*9. LARGE DEVIATIONS AND THE LAW OF THE ITERATED LOGARITHM, 154Moment Generating FunctionsLarge DeviationsChernoff's Theorem*The Law of the Iterated LogarithmChapter2 MEASURE 16710. GENERAL MEASURES, 167Classes of SetsConventions Involving 8MeasuresUniqueness11. OUTER MEASURE, 174Outer MeasureExtensionAn Approximation Theorem12. MEASURES IN EUCLIDEAN SPACE, 181Lebesgue MeasureRegularitySpecifying Measures on the LineSpecifying Measures in RkStrange Euclidean Sets*13. MEASURABLE FUNCTIONS AND MAPPINGS, 192Measurable MappingsMappings into RkLimits and MeasurabilityTransformations of Measures14. DISTRIBUTION FUNCTIONS, 198Distribution FunctionsExponential DistributionsWeak ConvergenceConvergence of Types*Extremal Distributions*Chapter3 INTEGRATION 21115. THE INTEGRAL, 211DefinitionNonnegative FunctionsUniqueness16. PROPERTIES OF THE INTEGRAL, 218Equalities and InequalitiesIntegration to the LimitIntegration over SetsDensitiesChange of VariableUniform IntegrabilityComplex Functions17. THE INTEGRAL WITH RESPECT TO LEBESGUE MEASURE, 234The Lebesgue Integral on the LineThe Riemann IntegralThe Fundamental Theorem of CalculusChange of VariableThe Lebesgue Integral in RkStieltjes Integrals18. PRODUCT MEASURE AND FUBINI’S THEOREM, 245Product SpacesProduct MeasureFubini's TheoremIntegration by PartsProducts of Higher Order19. THE Lp SPACES*, 256DefinitionsCompleteness and SeparabilityConjugate SpacesWeak CompactnessSome Decision TheoryThe Space L2An Estimation ProblemChapter4 RANDOM VARIABLES AND EXPECTED VALUES 27120. RANDOM VARIABLES AND DISTRIBUTIONS, 271Random Variables and VectorsSubfieldsDistributionsMultidimensional DistributionsIndependenceSequences of Random VariablesConvolutionConvergence in ProbabilityThe Glivenko-Cantelli Theorem*21. EXPECTED VALUES, 291Expected Value as IntegralExpected Values and LimitsExpected Values and DistributionsMomentsInequalitiesJoint IntegralsIndependence and Expected ValueMoment Generating Functions22. SUMS OF INDEPENDENT RANDOM VARIABLES, 300The Strong Law of Large NumbersThe Weak Law and Moment Generating FunctionsKolmogorov's Zero-One LawMaximal InequalitiesConvergence of Random SeriesRandom Taylor Series*23. THE POISSON PROCESS, 316Characterization of the Exponential DistributionThe Poisson ProcessThe Poisson ApproximationOther Characterizations of the Poisson ProcessStochastic Processes24. THE ERGODIC THEOREM*, 330Measure-Preserving TransformationsErgodicityErgodicity of RotationsProof of the Ergodic TheoremThe Continued-Fraction TransformationDiophantine ApproximationChapter5 CONVERGENCE OF DISTRIBUTIONS 34925. WEAK CONVERGENCE, 349DefinitionsUniform Distribution Modulo 1*Convergence in DistributionConvergence in ProbabilityFundamental TheoremsHelly's TheoremIntegration to the Limit26. CHARACTERISTIC FUNCTIONS, 365DefinitionMoments and DerivativesIndependenceInversion and the Uniqueness TheoremThe Continuity TheoremFourier Series*27. THE CENTRAL LIMIT THEOREM, 380Identically Distributed SummandsThe Lindeberg and Lyapounov TheoremsDependent Variables*28. INFINITELY DIVISIBLE DISTRIBUTIONS*, 394Vague ConvergenceThe Possible LimitsCharacterizing the Limit29. LIMIT THEOREMS IN Rk, 402The Basic TheoremsCharacteristic FunctionsNormal Distributions in RkThe Central Limit Theorem30. THE METHOD OF MOMENTS*, 412The Moment ProblemMoment Generating FunctionsCentral Limit Theorem by MomentsApplication to Sampling TheoryApplication to Number TheoryChapter6 DERIVATIVES AND CONDITIONAL PROBABILITY 42531. DERIVATIVES ON THE LINE*, 425The Fundamental Theorem of CalculusDerivatives of IntegralsSingular FunctionsIntegrals of DerivativesFunctions of Bounded Variation32. THE RADON–NIKODYM THEOREM, 446Additive Set FunctionsThe Hahn DecompositionAbsolute Continuity and SingularityThe Main Theorem33. CONDITIONAL PROBABILITY, 454The Discrete CaseThe General CaseProperties of Conditional ProbabilityDifficulties and CuriositiesConditional Probability Distributions34. CONDITIONAL EXPECTATION, 472DefinitionProperties of Conditional ExpectationConditional Distributions and ExpectationsSufficient Subfields*Minimum-Variance Estimation*35. MARTINGALES, 487DefinitionSubmartingalesGamblingFunctions of MartingalesStopping TimesInequalitiesConvergence TheoremsApplications: DerivativesLikelihood RatiosReversed MartingalesApplications: de Finetti's TheoremBayes EstimationA Central Limit Theorem*Chapter7 STOCHASTIC PROCESSES 51336. KOLMOGOROV'S EXISTENCE THEOREM, 513Stochastic ProcessesFinite-Dimensional DistributionsProduct SpacesKolmogorov's Existence TheoremThe Inadequacy of RTA Return to Ergodic TheoryThe Hewitt–Savage Theorem*37. BROWNIAN MOTION, 530DefinitionContinuity of PathsMeasurable ProcessesIrregularity of Brownian Motion PathsThe Strong Markov PropertyThe Reflection PrincipleSkorohod EmbeddingInvariance*38. NONDENUMERABLE PROBABILITIES, 558IntroductionDefinitionsExistence TheoremsConsequences of Separability*APPENDIX 571NOTES ON THE PROBLEMS 587BIBLIOGRAPHY 617INDEX 619