John Bather is the author of Decision Theory: An Introduction to Dynamic Programming and Sequential Decisions, published by Wiley.

"This textbook...draws on his many years of experience in teaching this topic as well as on his considerable professional expertise in the area. It is ideally suited to its stated purpose as a student text." (Short Book Reviews, Vol. 20. No. 3, December 2000) "...I was impressed with this book..." (The Statistician, Vol.51, No.2 2002) "...excellent for the audience to whom it is addressed, and it is to be hoped that the author will write a further textbook..." (Jnl of the Operational Research Society, Vol 54(10) 2003)

• Preface xi1 Introduction 11.1 Mathematical Induction 11.2 Historical Background 21.3 Dynamic Programming 51.4 The Executioner’s Tale 81.5 Summary 8Exercises 10I Deterministic Models 112 Multi-Stage Decision Problems 132.1 Maximizing Utilities 132.2 A General Model 172.3 Applications 19Exercises 253 Networks 273.1 Shortest Paths 273.2 Directed Networks 293.3 Critical Path Analysis 30Exercises 374 Further Applications 394.1 Discrete Actions 394.2 The Knapsack Problem 394.3 A Simple Replacement Model 424.4 Scheduling Problems 444.5 Johnson’s Algorithm 45Exercises 495 Convexity 515.1 Convex and Concave Functions 515.2 Allocation Problems 565.3 Concave Utility Functions 60Exercises 64II Stochastic Models 676 Markov Systems 696.1 Introduction 696.2 Stochastic Dynamic Programming 706.3 Applications 72Exercises 787 Optimal Stopping 797.1 Introduction 797.2 Stopping Times and Stopping Sets 827.3 Applications 90Exercises 948 Special Problems 978.1 Introduction 978.2 Selling an Asset 978.3 The Marriage Problem 1048.4 Prophet Inequalities 109Exercises 116III Markov Decision Processes 1199 General Theory 1219.1 Introduction 1219.2 Minimizing Discounted Expectations 1229.3 Policy Improvements 1309.4 A Machine Replacement Model 13710 Minimizing Average Costs 14510.1 Introduction 14510.2 Long-Term Average Costs 14810.3 Extension to Infinitely Many States 15310.4 Optimal Inventory Policies 15811 Statistical Decisions 16511.1 Introduction 16511.2 Testing Statistical Hypotheses 16611.3 The Sequential Probability Ratio Test 170Notes On the Exercises 177Chapter 1 177Chapter 2 177Chapter 3 178Chapter 4 179Chapter 5 179Chapter 6 180Chapter 7 181Chapter 8 183References 185Index 187

John Bather is the author of Decision Theory: An Introduction to Dynamic Programming and Sequential Decisions, published by Wiley.

"This textbook...draws on his many years of experience in teaching this topic as well as on his considerable professional expertise in the area. It is ideally suited to its stated purpose as a student text." (Short Book Reviews, Vol. 20. No. 3, December 2000) "...I was impressed with this book..." (The Statistician, Vol.51, No.2 2002) "...excellent for the audience to whom it is addressed, and it is to be hoped that the author will write a further textbook..." (Jnl of the Operational Research Society, Vol 54(10) 2003)

• Preface xi1 Introduction 11.1 Mathematical Induction 11.2 Historical Background 21.3 Dynamic Programming 51.4 The Executioner’s Tale 81.5 Summary 8Exercises 10I Deterministic Models 112 Multi-Stage Decision Problems 132.1 Maximizing Utilities 132.2 A General Model 172.3 Applications 19Exercises 253 Networks 273.1 Shortest Paths 273.2 Directed Networks 293.3 Critical Path Analysis 30Exercises 374 Further Applications 394.1 Discrete Actions 394.2 The Knapsack Problem 394.3 A Simple Replacement Model 424.4 Scheduling Problems 444.5 Johnson’s Algorithm 45Exercises 495 Convexity 515.1 Convex and Concave Functions 515.2 Allocation Problems 565.3 Concave Utility Functions 60Exercises 64II Stochastic Models 676 Markov Systems 696.1 Introduction 696.2 Stochastic Dynamic Programming 706.3 Applications 72Exercises 787 Optimal Stopping 797.1 Introduction 797.2 Stopping Times and Stopping Sets 827.3 Applications 90Exercises 948 Special Problems 978.1 Introduction 978.2 Selling an Asset 978.3 The Marriage Problem 1048.4 Prophet Inequalities 109Exercises 116III Markov Decision Processes 1199 General Theory 1219.1 Introduction 1219.2 Minimizing Discounted Expectations 1229.3 Policy Improvements 1309.4 A Machine Replacement Model 13710 Minimizing Average Costs 14510.1 Introduction 14510.2 Long-Term Average Costs 14810.3 Extension to Infinitely Many States 15310.4 Optimal Inventory Policies 15811 Statistical Decisions 16511.1 Introduction 16511.2 Testing Statistical Hypotheses 16611.3 The Sequential Probability Ratio Test 170Notes On the Exercises 177Chapter 1 177Chapter 2 177Chapter 3 178Chapter 4 179Chapter 5 179Chapter 6 180Chapter 7 181Chapter 8 183References 185Index 187

John Bather is the author of Decision Theory: An Introduction to Dynamic Programming and Sequential Decisions, published by Wiley.

"This textbook...draws on his many years of experience in teaching this topic as well as on his considerable professional expertise in the area. It is ideally suited to its stated purpose as a student text." (Short Book Reviews, Vol. 20. No. 3, December 2000) "...I was impressed with this book..." (The Statistician, Vol.51, No.2 2002) "...excellent for the audience to whom it is addressed, and it is to be hoped that the author will write a further textbook..." (Jnl of the Operational Research Society, Vol 54(10) 2003)

• Preface xi1 Introduction 11.1 Mathematical Induction 11.2 Historical Background 21.3 Dynamic Programming 51.4 The Executioner’s Tale 81.5 Summary 8Exercises 10I Deterministic Models 112 Multi-Stage Decision Problems 132.1 Maximizing Utilities 132.2 A General Model 172.3 Applications 19Exercises 253 Networks 273.1 Shortest Paths 273.2 Directed Networks 293.3 Critical Path Analysis 30Exercises 374 Further Applications 394.1 Discrete Actions 394.2 The Knapsack Problem 394.3 A Simple Replacement Model 424.4 Scheduling Problems 444.5 Johnson’s Algorithm 45Exercises 495 Convexity 515.1 Convex and Concave Functions 515.2 Allocation Problems 565.3 Concave Utility Functions 60Exercises 64II Stochastic Models 676 Markov Systems 696.1 Introduction 696.2 Stochastic Dynamic Programming 706.3 Applications 72Exercises 787 Optimal Stopping 797.1 Introduction 797.2 Stopping Times and Stopping Sets 827.3 Applications 90Exercises 948 Special Problems 978.1 Introduction 978.2 Selling an Asset 978.3 The Marriage Problem 1048.4 Prophet Inequalities 109Exercises 116III Markov Decision Processes 1199 General Theory 1219.1 Introduction 1219.2 Minimizing Discounted Expectations 1229.3 Policy Improvements 1309.4 A Machine Replacement Model 13710 Minimizing Average Costs 14510.1 Introduction 14510.2 Long-Term Average Costs 14810.3 Extension to Infinitely Many States 15310.4 Optimal Inventory Policies 15811 Statistical Decisions 16511.1 Introduction 16511.2 Testing Statistical Hypotheses 16611.3 The Sequential Probability Ratio Test 170Notes On the Exercises 177Chapter 1 177Chapter 2 177Chapter 3 178Chapter 4 179Chapter 5 179Chapter 6 180Chapter 7 181Chapter 8 183References 185Index 187