دانلود کتاب بهینه‌سازی و یادگیری از طریق جستجوی گرادیان تصادفی

This monograph covers stochastic optimization and learning, mostly focusing on the methodology of stochastic approximations and gradient estimation techniques.We focus on continuous parameter problems and do not discuss discrete optimization. Our approach is theoretical, but emphasis is given to algorithms that implement the methods. Part I covers many important applications of stochastic approximations for optimization and learning that do not require the use of gradient estimation. Part II introduces gradient estimation for stochastic processes, which then significantly broadens the range of application of stochastic approximations, some of which are described in Part III. Stochastic approximation has been intensively covered in the past decades. For general treatments of stochastic approximation, we refer toWasan [323], Polyak and Tsypkin [250], Ljung [205], Kushner and Clark [193], Polyak [248], Bertsekas and Tsitsiklis [32, 33], Bena¨ım [24, 25], Pflug [245], Kushner and Yin [197], Meyn [216], Borkar [39], and Nemirovski et al. [226]. Gradient estimation has been included in Ho and Cao [159], Cao [58, 60], Fu [101, 102], Kroese et al. [191], Glasserman [114, 119], Pflug [245], and Rubinstein with various coauthors [267, 268, 270]. This monograph differs from the aforementioned standard references in structure, choice of topics, and pedagogical format. We address an audience of graduate students and researchers. We have carefully worked out full examples of applications and have included a set of exercises and problems that aim to help the instructors as well as the students as they go through the material. Mathematical theory, the art of modeling, and numerical algorithms complement each other when solving real problems. We follow this principle as the underlying philosophy of the book, aiming to strengthen both the readers’ theoretical and practical skills, neither of which should dominate the other. The book has three main parts: • Part I provides a thorough treatment of first-order methods in optimization with an emphasis on stochastic approximation (SA). The key results are presented and proven, and many illustrating examples are provided. Part I follows the standard approach in SA by studying recursive learning algorithms via ordinary differential equations (ODEs).We take great care in Part I to present the theory and also the numerical methods. Guidelines on how to apply the algorithms in practice are included in a final section of each chapter called “Practical Considerations,” which addresses in detail typical trade-offs one faces in practice. • Part II is a self-contained treatment of gradient estimation and summarizes the research done in this area in the last three decades. Many examples are provided, and the available theory is explained in depth. It gives a unique, exhaustive presentation of gradient estimation of expectations of functionals of stochastic processes. We build the models from the “static” case of one random variable to stochastic processes, including models with random stopping times and long-term stationary expectations (called sometimes the “infinite horizon” model in simulation methodology). At a first reading, sections indicated by an asterisk (∗) may be omitted. • Part III blends the algorithmic results of Part I with the gradient estimators provided in Part II, and presents complete SA application stories (from problem formulation, to choice of algorithm and gradient estimator, to output analysis). Moreover, applications of SA in other field of research such as statistics and machine learning are discussed, and some advanced material on infinitesimal perturbation analysis and measure-valued differentiation is provided.