دانلود کتاب خطوط لوله داده با Apache Airflow، ویرایش دوم

The world of data is never dull, and much has changed since the original release of this book. Whereas large language models (LLMs) used to be a niche research topic, nowadays everyone—even our mothers—has heard of AI tools such as ChatGPT. For better or worse (depending on who you ask), this shift has led to a huge boom in companies adopting AI to optimize their processes and shift to more data-driven decision making. Besides this technological acceleration, global challenges such as geopolitical strife and climate change add even more pressure to adapt to an ever-changing environment, making high-quality data more critical than ever. In response to these developments, the landscape of data tooling has not stood still. There is more contention than ever in the space of data orchestrators, trending toward more integrated, secure, and developer-friendly platforms. Accordingly, Airflow has evolved considerably since the first edition of this book was released, adding several new features and culminating in the recent release of a new major milestone: Airflow 3. Working on this second edition, we found that we needed to make substantial changes to bring the book up to date with all the changes since Airflow 2.0, including the following: An entirely new UI Significant changes in Airflow’s scheduling logic New features such as data-aware (event-based) scheduling, new executor types, and the option to combine multiple executors Changes in deployment, shifting to managed solutions or Kubernetes New use cases, such as generative AI (GenAI)–related workloads (e.g., RAG) To incorporate all these changes, we reworked the book considerably, adding many new sections, screenshots, and chapters and restructuring the code examples to make them easier to use and enable them to run in a generic way across all chapters. We hope that these additions do justice to the incredible amount of work that the community has put into developing this new release. Altogether, this updated book aims to provide a comprehensive introduction to Airflow 3, covering everything from building simple workflows and developing custom components to designing and managing Airflow deployments. We intend to complement the many excellent blogs and other online documentation by consolidating various topics into a single, concise, easy-to-follow resource. Through our combined years of experience working with Airflow, we hope to give you a strong foundation to begin your journey with this powerful tool.