Last Posts
Apache Airflow monitoring with TIG: Part 1
Monitoring Apache Airflow should be your top priority when you are in production. Without monitoring you have no way to know if anything goes wrong. Imagine that Airflow stops working for any reason, since it is your orchestrator, your data
How to use templates and macros in Apache Airflow
Templates and Macros in Apache Airflow are the way to pass dynamic data to your DAGs at runtime. Let’s imagine that you would like to execute a SQL request using the execution date of your DAG? How can you do
Apache Airflow with Kubernetes Executor and MiniKube
In this tutorial, we are going to see how to use Apache Airflow with Kubernetes Executor. If you are using Airflow in production, there is a big chance that your workload fluctuates over time. Sometimes you have many taks to execute and
How to use the DockerOperator in Apache Airflow
Wondering how to use the DockerOperator in Apache Airflow to kick off a docker and run commands? Let’s discover this operator through a practical example. In a more and more containerized world, it can be very useful to know how to
Apache Airflow | How to use the PythonOperator
Wondering how can we run python code through Airflow ? The Airflow PythonOperator does exactly what you are looking for. It is a very simple but powerful operator, allowing you to execute a Python callable function from your DAG. You
Apache Airflow | With Statement and DAG
If you take a look at some DAG examples in my course “The Complete Hands-On Course to Master Apache Airflow”, you may notice the use of the “with” statement when a dag object is created. The question is why? Why it