airflow dag

Airflow DAG: Creating your first DAG in 5 minutes

Looking for creating your first Airflow DAG? Wondering how to process your data in Airflow? What are the steps to code your own data pipelines? You’ve come to the right place! At the end of this short tutorial, you will be able to code your first Airflow DAG! You might

Read More »

How to use the BranchPythonOperator

Looking for a way to choose one task or another? You want to execute a task based on a condition? You multiple tasks but only one should be executed if a criterion is true? You’ve come to the right place! The BranchPythonOperator does exactly what you are looking for. It’s

Read More »
airflow xcom

Airflow XCOM : The Ultimate Guide

Wondering how to share data between tasks? What are XCOMs in Apache Airflow? Well you are at the right place. In this tutorial, you are going to learn everything you need about XComs in Airflow. What are they, how they work, how can you define them, how to get them

Read More »

The PostgresOperator: All you need to know

One of the first operators I discovered with Airflow was the Postgres Operator. The Postgres Operator allows you to interact with your Postgres database. Whether you want to create a table, delete records, insert records, you will use the PostgresOperator. Nonetheless, you will quickly be faced to some questions. How

Read More »

Airflow on AWS: Set up a production-ready architecture

How to set up Apache Airflow on AWS EKS? That’s one of the top questions I received from my students. If you started with Airflow pretty recently, there is a good chance that you are still using it in local or on an EC2 instance with the local executor. That’s

Read More »

Apache Airflow Best Practices – Part 1

Since I started creating courses a year ago, I got so many messages asking me what are the best practices in Apache Airflow. As engineer, we always seek for the best ways to apply what we learn while being constantly improving ourselves. In this series of tutorial, I would like

Read More »