Airflow Trigger Rules: All you need to know!

By default, your tasks get executed once all the parent tasks succeed. this behaviour is what you expect in general. But what if you want something more complex? What if you would like to execute a task as soon as one of its parents succeeds? Or maybe you would like

Read More »
airflow on kubernetes

Airflow on Kubernetes : Get started in 10 mins

Airflow on Kubernetes is quite popular isn’t it? There is a good chance that you know Kubernetes, that you even have a Kubernetes cluster and you would like to deploy and run Airflow on it. However, Kubernetes is hard. There is so many things to deal with that it can

Read More »
airflow rest api

Airflow API : The guide to get started now!

The new Airflow API is here! After waiting impatiently for years, the Airflow API is finally stable and reliable enough to be used in production! Back in the day, I’m pretty sure that you remember well the Experimental API and its lack of endpoints and documentation. In fact, as it

Read More »
airflow sensors

Airflow Sensors : What you need to know

Airflow Sensors are one of the most commonly used type of operators. Why? Because they allow you to check if a criteria is met to get completed. You need to wait for a file? check if a SQL entry exists? delay the execution of your DAG? That’s the few possibilities

Read More »
airflow dag

Airflow DAG: Creating your first DAG in 5 minutes

Looking for creating your first Airflow DAG? Wondering how to process your data in Airflow? What are the steps to code your own data pipelines? You’ve come to the right place! At the end of this short tutorial, you will be able to code your first Airflow DAG! You might

Read More »

How to use the BranchPythonOperator

Looking for a way to choose one task or another? You want to execute a task based on a condition? You multiple tasks but only one should be executed if a criterion is true? You’ve come to the right place! The BranchPythonOperator does exactly what you are looking for. It’s

Read More »