Blog

dockeroperator_with_spark

How to use the DockerOperator in Apache Airflow

Wondering how to use the DockerOperator in Apache Airflow to kick off a docker and run commands? Let’s discover this operator through a practical example. In a more and more containerized world, it can be very useful to know how to interact with your Docker containers through Apache Airflow. In this

Read More »
python-operator-basics-in-apache-airflow

Apache Airflow | How to use the PythonOperator

Wondering how can we run python code through Airflow ? The Airflow PythonOperator does exactly what you are looking for. It is a very simple but powerful operator, allowing you to execute a Python callable function from your DAG. You may have seen in my course “The Complete Hands-On Course

Read More »
use-of-with-keyword-in-apache-airflow

Apache Airflow | With Statement and DAG

If you take a look at some DAG examples in my course “The Complete Hands-On Course to Master Apache Airflow”, you may notice the use of the “with” statement when a dag object is created. The question is why? Why it is considered as a best practice to use this Python

Read More »
How to connect with SSH using PuTTY

Connect with SSH using PuTTY to a remote host

PuTTY is an open source SSH and telnet client developed originally by Simon Tatham for Windows and UNIX systems. The purpose of PuTTY is to provide an easy way to connect with any server running an SSH daemon in order to give you the illusion to work on the remote

Read More »