دانلود کتاب بهینه‌سازی بدون مشتق – مبانی نظری، الگوریتم‌ها و کاربردها

The pursuit of solving optimization problems has long been a cornerstone in the realms of computer science and artificial intelligence. While specialized algorithms tailored to specific problems have been developed, there has also been a signifi-cant emergence of general-purpose optimization methods. These methods, including random search, simulated annealing, and evolutionary algorithms, are often crafted based on heuristic principles. Consequently, their properties are predominantly explored through empirical studies, with a comprehensive theoretical understanding still largely out of reach.
Yang Yu, the first author of this monograph, previously co-authored a book entitled Evolutionary Learning: Advances in Theories and Algorithms with Prof. Zhi-Hua Zhou and Chao Qian, which was published by Springer in 2019. In that work, the central theoretical focus was on running time analysis, which evaluates the time and sample complexity required to find an optimal solution. Subsequently, Yang devel-oped an interest in exploring alternative theoretical foundations for general-purpose optimization methods, complementing the well-established theories of machine learning. Given that machine learning is underpinned by a solid statistical frame-work, it naturally raises the question: can we establish a similarly strong theoret-ical foundation for optimization methods from a statistical perspective? Specifically, such a theory could elucidate how these optimization methods approximate optimal solutions.
Collaborating with the second author, Hong Qian, who was Yang’s Ph.D. student in 2013, a preliminary framework was developed. Initially, this framework was designed to meet certain theoretical desiderata. Surprisingly, the resulting algorithm also demonstrated competitive practical performance against some state-of-the-art methods. The third author, Yi-Qi Hu, who became Yang’s Ph.D. student in 2015, joined the effort to further refine and extend the framework and enhance the algo-rithm. This monograph provides a comprehensive overview of the authors’ research, encompassing the development of the framework, algorithm design, and practical applications.
The monograph is structured into four parts. Part I briefly introduces derivative-free optimization within the context of machine learning. Part II presents the classification-based optimization framework along with its basic algorithm. To tackle practical challenges such as sequential execution, high-dimensionality, noisy evalu-ations, and large-scale parallel execution, Part III introduces several variants of the basic algorithm. This part also includes an introduction to ZOOpt, a general opti-mization toolbox built on classification-based optimization algorithms. Part IV show-cases various applications of classification-based optimization in the field of auto-matic machine learning, including hyper-parameter selection, algorithm selection, and neural architecture search.
The authors extend their heartfelt gratitude to their families, friends, and collaborators for their unwavering support and contributions.