Adam B. Levy - Böcker

​​​​​​Stationarity and Convergence in Reduce-or-Retreat Minimization presents and analyzes a unifying framework for a wide variety of numerical methods in optimization. The author’s “reduce-or-retreat” framework is a conceptual method-outline that covers any method whose iterations choose between reducing the objective in some way at a trial point, or retreating to a closer set of trial points. The alignment of various derivative-based methods within the same framework encourages the construction of new methods, and inspires new theoretical developments as companions to results from across traditional divides. The text illustrates the former by developing two generalizations of classic derivative-based methods which accommodate non-smooth objectives, and the latter by analyzing these two methods in detail along with a pattern-search method and the famous Nelder-Mead method.In addition to providing a bridge for theory through the “reduce-or-retreat” framework, this monograph extends and broadens the traditional convergence analyses in several ways. Levy develops a generalized notion of approaching stationarity which applies to non-smooth objectives, and explores the roles of the descent and non-degeneracy conditions in establishing this property. The traditional analysis is broadened by considering “situational” convergence of different elements computed at each iteration of a reduce-or-retreat method. The “reduce-or-retreat” framework described in this text covers specialized minimization methods, some general methods for minimization and a direct search method, while providing convergence analysis which complements and expands existing results.​ ​

Optimization is presented in most multivariable calculus courses as an application of the gradient, and while this treatment makes sense for a calculus course, there is much more to the theory of optimization. Optimization problems are generated constantly, and the theory of optimization has grown and developed in response to the challenges presented by these problems. This textbook aims to show readers how optimization is done in practice and help them to develop an appreciation for the richness of the theory behind the practice.Exercises, problems (including modeling and computational problems), and implementations are incorporated throughout the text to help students learn by doing. Python notes are inserted strategically to help readers complete computational problems and implementations.The Basics of Practical Optimization, Second Edition is intended for undergraduates who have completed multivariable calculus, as well as anyone interested in optimization. The book is appropriate for a course that complements or replaces a standard linear programming course.

Numerical minimization of an objective function is analyzed in this book to understand solution algorithms for optimization problems. Multiset-mappings are introduced to engineer numerical minimization as a repeated application of an iteration mapping. Ideas from numerical variational analysis are extended to define and explore notions of continuity and differentiability of multiset-mappings, and prove a fixed-point theorem for iteration mappings. Concepts from dynamical systems are utilized to develop notions of basin size and basin entropy.  Simulations to estimate basins of attraction, to measure and classify basin size, and to compute basin are included to shed new light on convergence behavior in numerical minimization.Graduate students, researchers, and practitioners in optimization and mathematics who work theoretically to develop solution algorithms will find this book a useful resource.