The Complete Hands-On Course to Master Apache Airflow (5)

The Complete Hands-On Introduction to Apache Airflow

Apache Airflow is an open-source  platform to programmatically author, schedule and monitor workflows. If you have many ETL(s) to manage, Airflow is a must-have.

In the Complete Hands-On Introduction to Apache Airflow, you are going to learn everything you need in order to start using Airflow and discover how powerful it is. Through theory and practical videos, you will see basic as well as advanced concepts.

Materials (required for the course)

Download the materials by clicking on the link below


Section 1: Course Introduction

  1. Important Prerequisites
  2. Course Objectives
  3. Course Structure
  4. Who I am
  5. Development Environment

Section 2: First Approach to Airflow

  1. Introduction
  2. Why use Airflow?
  3. What is Airflow?
  4. How Airflow works?
  5. [Practice] Installing Airflow
  6. [Practice] Quick Tour of Airflow UI
  7. [Practice] Quick Tour of Airflow CLI
  8. Quiz Time!
  9. Recap

Section 3: Coding Your First Data Pipeline with Airflow

  1. Introduction
  2. What is a DAG?
  3. [Practice] Time to code your first DAG
  4. What is an Operator?
  5. [Practice] Let’s use Operators
  6. Operator Relationships and Bitshift Composition
  7. [Practice] Adding dependencies
  8. How the Scheduler Works?
  9. [Practice] A Quick Play With Backfill and Catchup
  10. What is a Workflow?
  11. Quiz Time!
  12. Recap

Section 4: Databases and Executors

  1. Introduction
  2. Sequential Executor with SQLite
  3. Local Executor with PostgreSQL
  4. [Practice] Configure a DAG with Local Executor and PostgreSQL
  5. Celery Executor with PostgreSQL and RabbitMQ
  6. [Practice] Configure a DAG with Celery Executor, PostgreSQL and RabbitMQ
  7. Quiz Time!
  8. Recap

Section 5: Advanced Concepts

  1. Introduction
  2. Minimising Repetitive Patterns With SubDAGs
  3. [Practice] Minimising a DAG with SubDAGs
  4. How to Interact With External Sources Using Hooks?
  5. [Practice] Get Results From PostgreSQL Using Hooks
  6. How to Share Data Between Your Tasks With XCOMs?
  7. [Practice] Share Your First Messages Using XCOMs
  8. How to Execute Tasks According To Criteria Using Branching
  9. [Practice] Make Your First Conditional Tasks
  10. Control Your Tasks With SLAs
  11. [Practice] Defining a SLA in a DAG 
  12. Quiz Time!
  13. Recap

Section 6: Customising Your Apache Airflow Installation With Plugins

  1. Introduction
  2. Adding Functionalities to Apache Airflow
  3. [Practice] Creating a Hook to Interact With Elasticsearch
  4. [Practice] Creating a Transfer Operator PostgresqlToElasticsearch
  5. [Practice] Adding a View to Apache Airflow UI
  6. Quiz Time!
  7. Recap

Section 7: Using Apache Airflow With Docker

  1. Introduction
  2. Quick Reminder About Docker
  3. [Practice] Building And Running Your First Airflow Docker Image
  4. [Practice] Running Airflow Using Sequential Executor With Docker
  5. [Practice] Running Airflow Using Local Executor With Docker
  6. [Practice] Scaling Airflow Using Celery Executor With Docker
  7. Quiz Time!
  8. Recap