Blog
Airflow on AWS: Set up a production-ready architecture
How to set up Apache Airflow on AWS EKS? That’s one of the top questions I received from my students. If you started with Airflow pretty recently, there is a good chance that you are still using it in local or on an EC2 instance with the local executor. That’s
Apache Airflow Best Practices – Part 1
Since I started creating courses a year ago, I got so many messages asking me what are the best practices in Apache Airflow. As engineer, we always seek for the best ways to apply what we learn while being constantly improving ourselves. In this series of tutorial, I would like
Running Apache Airflow locally on Kubernetes
Wouldn’t be convenient to be able to run Apache Airflow locally with the Kubernetes Executor on a multi-node Kubernetes cluster? That’s could be a great way to test your DAGs and understand how Airflow works in a Kubernetes environment isn’t it? Well that’s exactly what we are going to do
Apache Airflow 2.0: What to expect?
Apache Airflow 2.0 is coming with a lot of big improvements. In this post, I’m gonna explain to you what are the most important features to expect what they solve. If you are totally new to Airflow, check my introduction course right there, or if you want an in-depth approach
Variables in Apache Airflow: The Guide
Wondering how to deal with variables in Apache Airflow? Well you are at the right place. In this tutorial, you are going to learn everything you need about the variables in Airflow. What are they, how they work, how can you define them, how to get them and more. If
Apache Airflow | How to use the BashOperator
Wondering how can you execute bash commands through Airflow ? The Airflow BashOperator does exactly what you are looking for. It is a very simple but powerful operator, allowing you to execute either a bash script, a command or a set of commands from your DAGs. You may have seen
