دانلود کتاب قراردادهای داده‌ای – توسعه محصولات – خط تولید درجه‌ای در مقیاس

If you have picked up this book, it’s likely that you’ve deeply felt the pain of managing data while lacking control of your data’s ingestion and generation. Though at one point our industry centered around well-thought-out on-prem implementations with robust data models, the rise of cloud computing and the explosion of data products within organizations, via AI, have incentivized speed to market at the cost of turning the data layer into chaos. Many data teams within this situation find themselves constantly being reactive, repeatedly fixing the next data-related fire within the company. At its core, we believe this challenge in our industry stems from the difficulty of change management between historically siloed teams of data producers and consumers. Specifically, there is a disconnect between upstream application code, which defines how data is captured within a software system, and the downstream data products that leverage this data. We argue that data contracts serve as a mechanism for aligning data producers and consumers through automation and defining expectations as code. What Are Data Contracts? Data contracts are an architecture pattern that enables an agreement between data producers and consumers that is established, updated, and enforced via an API. They’re part of a larger movement called shift left, where you use automation to enable upstream software developers to account for required enforcement pertinent to their domain—this approach was first validated within DevOps and DevSecOps. Data contracts consist of four key components: • Data assets that need protection via change management • A contract specification file that codifies expectations of data assets as versioncontrolled Code • Detection via an ability to extract, analyze, and take action on changes to metadata related to data assets under contract • Prevention by automating data contract enforcement within the developer workflow, typically during CI/CD pipelines We argue that the data industry is having its shift left moment, and that data contracts are critical for this change.