Wondering how to use the DockerOperator in Apache Airflow to kick off a docker and run commands? Let’s discover this operator through a practical example. In a more and more containerized world, it can be very useful to know how to interact with your Docker containers through Apache Airflow. In this article, we are going to …
Wondering how can we run python code through Airflow ? The Airflow PythonOperator does exactly what you are looking for. It is a very simple but powerful operator, allowing you to execute a Python callable function from your DAG. You may have seen in my course “The Complete Hands-On Course to Master Apache Airflow” that I …
If you take a look at some DAG examples in my course “The Complete Hands-On Course to Master Apache Airflow”, you may notice the use of the “with” statement when a dag object is created. The question is why? Why it is considered as a best practice to use this Python keyword when you instantiate a …
