Apache Airflow in 2022 : 10 rules to make it work

Apache Airflow in 2022: 10 Rules to Make It Work

Airflow is by default very permissive and without very strict rules you are likely to fail in your pipeline stack implementation.

if you are not careful your shortcuts will cost you a lot afterwards

This over permissive approach frustrated many contributors into implementing their own opinionated alternatives such as Prefect, Dagster … And all were introduced by blog post explaining Airflow’s shortcomings.

Airflow lets you run your jobs without …

airflow apache dagster data science kubernetes programming rules work