Blog
Airflow TaskGroups: All you need to know!
Airflow TaskGroups have been introduced to make your DAG visually cleaner and easier to read. They are meant to replace SubDAGs which was the historic way of grouping your tasks. The problem with SubDAGs is that they are much more than that. They bring a lot of complexity as you
Airflow Timetable: Schedule your DAGs like never before
Airflow Timetable. This new concept introduced in Airflow 2.2 is going to change your way of scheduling your data pipelines. Or I would say, you’re finally going to have all the freedom and flexibility you ever dreamt of for scheduling your DAGs. What if you want to run your DAG
Airflow Trigger Rules: All you need to know!
By default, your tasks get executed once all the parent tasks succeed. this behaviour is what you expect in general. But what if you want something more complex? What if you would like to execute a task as soon as one of its parents succeeds? Or maybe you would like
Airflow on Kubernetes : Get started in 10 mins
Airflow on Kubernetes is quite popular isn’t it? There is a good chance that you know Kubernetes, that you even have a Kubernetes cluster and you would like to deploy and run Airflow on it. However, Kubernetes is hard. There is so many things to deal with that it can
Airflow API : The guide to get started now!
The new Airflow API is here! After waiting impatiently for years, the Airflow API is finally stable and reliable enough to be used in production! Back in the day, I’m pretty sure that you remember well the Experimental API and its lack of endpoints and documentation. In fact, as it
Airflow Sensors : What you need to know
Airflow Sensors are one of the most commonly used type of operators. Why? Because they allow you to check if a criteria is met to get completed. You need to wait for a file? check if a SQL entry exists? delay the execution of your DAG? That’s the few possibilities
