Data Pipelines with Apache Airflow, Second Edition

Inbunden, Engelska, 2026

649 kr

Skickas . Fri frakt över 249 kr.

Beskrivning

Scripts keep crashing and stakeholders demand data they can trust. Late-night alerts drain your energy and delay business decisions. Manual fixes multiply as pipelines sprawl across clouds and clusters. You need orchestration that scales without sacrificing reliability or sleep. Apache Airflow promises order, yet its power can feel overwhelming. This book makes Airflow mastery achievable, practical, and immediately rewarding. Dynamic scheduling with Dataset API: Align complex, irregular jobs to real-world data availability. Taskflow API patterns: Write cleaner Python code, reduce boilerplate, and speed team onboarding. Container-native deployments: Run pipelines on Kubernetes for elastic scaling and cost control. Comprehensive testing strategies: Catch issues before production, slash incident time, protect reputation. Production-ready best practices: Logging, security, and monitoring that keep auditors and leaders happy. Custom operator design: Extend Airflow to any system, unlocking limitless integration possibilities. Data Pipelines with Apache Airflow, Second Edition gathers five seasoned consultants into one definitive field guide. Their combined experience turns cutting-edge features into steps you can reproduce today. It is the trusted companion for every data engineer. The book starts with Airflow architecture, then walks through DAG design, testing, deployment, and operations. Updated chapters reveal Taskflow, Dataset scheduling, and Kubernetes setups, explained through real projects, not toy examples. Clear language, diagrams, and downloadable code remove guesswork. Finish the last page knowing your pipelines deploy reliably, recover gracefully, and scale effortlessly. Sleep through the night while Airflow delivers fresh, accurate data to every downstream consumer. Ideal for data engineers, DevOps, machine-learning engineers, and Python-savvy analysts ready to level-up orchestration skills.

Produktinformation

Utgivningsdatum:2026-02-12
Mått:190 x 235 x 3 mm
Vikt:910 g
Format:Inbunden
Språk:Engelska
Antal sidor:512
Upplaga:2
Förlag:Manning Publications
ISBN:9781633436374

Utforska kategorier

Databaser inom Data och IT
Ledarskap och motivation inom Ekonomi och Ledarskap
Grafik och bildbehandling inom Data och IT

Mer om författaren

Julian de Ruiter is a Data + AI engineering lead known for transforming messy pipelines into scalable platforms. With global consulting experience, Julian brings analytical rigor and clear storytelling to every page. He distills advanced engineering tactics into practical guidance that accelerates reader confidence. Ismael Cabral is a machine-learning engineer and renowned Airflow trainer trusted by companies across four continents. Drawing on cross-industry projects, Ismael delivers concise, hands-on advice readers can apply the same day. He turns complex workflow concepts into simple patterns that unlock rapid innovation. Kris Geusebroek is a data-engineering consultant and maintainer of Whirl, the open-source Airflow testing toolkit. Kris blends deep technical skill with a coach’s mindset, helping teams adopt disciplined engineering habits. He channels that expertise into repeatable methods that keep pipelines reliable and testable. Daniel van der Ende is a veteran data engineer and early Airflow contributor who has worked on diverse on-prem and cloud setups. Daniel’s pragmatic voice demystifies community-driven features and hidden pitfalls. He packages frontline lessons into actionable tips that save readers weeks of trial and error. Bas Harenslak is a staff architect at Astronomer, guiding enterprises in large-scale Airflow deployments. With a software-engineering background, Bas writes with precision, warmth, and strategic insight. He translates architectural puzzles into elegant solutions that future-proof reader infrastructure.

Recensioner i media

This is an all-encompassing guide on Airflow, from the baby steps to hardcore DevOps stuff. Pavel Filatov, Data Engineer, flatmap.blog It's the perfect introduction to Airflow. Although it seems to assume a reader with very little development experience, it still touches all the important topics and address most questions very efficiently. Gregor Zurowski, Founding Engineer, ResearchHub

Innehållsförteckning

PART 1: GETTING STARTED 1 MEET APACHE AIRFLOW 2 ANATOMY OF AN AIRFLOW DAG 3 TIME-BASED SCHEDULING IN AIRFLOW 4 ASSET-AWARE SCHEDULING 5 TEMPLATING TASKS USING THE AIRFLOW CONTEXT PART 2: BEYOND THE BASICS 6 DEFINING DEPENDENCIES BETWEEN TASKS 7 TRIGGERING WORKFLOWS WITH EXTERNAL INPUT 8 COMMUNICATING WITH EXTERNAL SYSTEMS 9 EXTENDING AIRFLOW WITH CUSTOM OPERATORS AND SENSORS 10 TESTING PART 3: AIRFLOW IN PRACTICE 11 RUNNING TASKS IN CONTAINERS 12 BEST PRACTICES 13 PROJECT: FINDING THE FASTEST WAY TO GET AROUND NYC PART 4: AIRFLOW IN PRODUCTION 14 PROJECT: KEEPING FAMILY TRADITIONS ALIVE WITH AIRFLOW AND GENERATIVE AI 15 OPERATING AIRFLOW IN PRODUCTION 16 SECURING AIRFLOW 17 AIRFLOW DEPLOYMENT OPTIONS APPENDICES APPENDIX A: RUNNING CODE SAMPLES APPENDIX B: PROMETHEUS METRIC MAPPING