دانلود کتاب کارایی بالا با Spark

We wrote this book for data engineers, data scientists, and ML practitioners who are looking to get the most out of Spark. If you’ve been working with Spark and invested in Spark but your experience so far has been mired by memory errors and mysterious, intermittent failures, this book is for you. If you have been using Spark for some exploratory work or experimenting with it on the side but have not felt confident enough to put it into production, this book may help. If you are enthusiastic about Spark but have not seen the performance improvements from it that you expected, we hope this book can help. This book is intended for those who have some working knowledge of Spark and may be difficult to understand for those with little or no experience with Spark or distributed computing. For recommendations of more introductory literature, see “Supporting Books and Materials”. We expect this text will be most useful to those who care about optimizing repeated queries in production, rather than to those who are primarily doing exploratory work. While writing highly performant queries is perhaps more important to the data engineer, writing those queries with Spark, in contrast to other frameworks, requires a good knowledge of the data, which is usually more intuitive to the data scientist. Thus, it may be more useful to a data engineer who may be less experienced with thinking critically about the statistical nature, distribution, and layout of data when considering performance. We hope that this book will help data engineers think more critically about their data as they put pipelines into production. Similarly for data scientists we hope to provide more understanding of how Spark works so they can use their knowledge of the data for high performance queries. We want to help our readers ask questions such as “How is my data distributed?” “Is it skewed?” “What is the range of values in a column?” and “How do we expect a given value to group?” and then apply the answers to those questions to the logic of their Spark queries. However, even for data scientists using Spark mostly for exploratory purposes, this book should cultivate some important intuition about writing performant Spark queries, so that as the scale of the exploratory analysis inevitably grows, you may have a better shot of getting something to run the first time. We hope to guide data <span class=”keeptogether”> scientists</span>, even those who are already comfortable thinking about data in a distributed way, to think critically about how their programs are evaluated, empowering them to explore their data more fully and more quickly and to communicate effectively with anyone helping them put their algorithms into production. Regardless of your job title, it is likely that the amount of data with which you are working is growing quickly. Your original solutions may need to be scaled, and your old techniques for solving new problems may need to be updated. We hope this book will help you leverage Apache Spark to tackle new problems more easily and old problems more efficiently.