Airflow Production Tips — Proper Task (Not DAG) Catchup

Airflow Production Tips — Proper Task (Not DAG) Catchup

Photo by Jackson Simmer on Unsplash

Apache Airflow has become the de facto standard for Data Orchestration. However, throughout the years and versions, it accumulated a set of nuances and bugs which can hinder its production usage.

This series of articles aims at walking Apache Airflow users through the process of overcoming these issues, the same issues I have faced.

Note: As usual, all of the code is available in my …

airflow apache-airflow dag data engineering etl orchestration production python tips