The Complete Hands-On Course to Master Apache Airflow (5)

The Complete Hands-On Course to Master Apache Airflow

Apache Airflow is an open-source  platform to programmatically author, schedule and monitor workflows. If you have many ETL(s) to manage, Airflow is a must-have.

In the complete hands-on course to master Apache Airflow, you are going to learn everything you need about Apache Airflow in order to use it in production through theory and practical video courses. Starting from very basic notions such as, what is Airflow and how it works, we will dive into advanced concepts such as, how to create plugins and make real dynamic pipelines.

Materials (required for the course)

Tell me where do you come from and enter you email to receive the link. Don’t worry, you will not get spammed. I don’t like spams either.

Curriculum

Section 1: Course Introduction

  1. Important Prerequisites
  2. Course Objectives
  3. Course Structure
  4. Who I am
  5. Development Environment

Section 2: First Approach to Airflow

  1. Introduction
  2. Why use Airflow?
  3. What is Airflow?
  4. How Airflow works?
  5. [Practice] Installing Airflow
  6. [Practice] Quick Tour of Airflow UI
  7. [Practice] Quick Tour of Airflow CLI
  8. Quiz Time!
  9. Recap

Section 3: Coding Your First Data Pipeline with Airflow

  1. Introduction
  2. What is a DAG?
  3. [Practice] Time to code your first DAG
  4. What is an Operator?
  5. [Practice] Let’s use Operators
  6. Operator Relationships and Bitshift Composition
  7. [Practice] Adding dependencies
  8. How the Scheduler Works?
  9. [Practice] A Quick Play With Backfill and Catchup
  10. What is a Workflow?
  11. Quiz Time!
  12. Recap

Section 4: Databases and Executors

  1. Introduction
  2. Sequential Executor with SQLite
  3. Local Executor with PostgreSQL
  4. [Practice] Configure a DAG with Local Executor and PostgreSQL
  5. Celery Executor with PostgreSQL and RabbitMQ
  6. [Practice] Configure a DAG with Celery Executor, PostgreSQL and RabbitMQ
  7. Quiz Time!
  8. Recap

Section 5: Advanced Concepts

  1. Introduction
  2. Minimising Repetitive Patterns With SubDAGs
  3. [Practice] Minimising a DAG with SubDAGs
  4. How to Interact With External Sources Using Hooks?
  5. [Practice] Get Results From PostgreSQL Using Hooks
  6. How to Share Data Between Your Tasks With XCOMs?
  7. [Practice] Share Your First Messages Using XCOMs
  8. How to Execute Tasks According To Criteria Using Branching
  9. [Practice] Make Your First Conditional Tasks
  10. Control Your Tasks With SLAs
  11. [Practice] Defining a SLA in a DAG 
  12. Quiz Time!
  13. Recap

Section 6: Customising Your Apache Airflow Installation With Plugins

  1. Introduction
  2. Adding Functionalities to Apache Airflow
  3. [Practice] Creating a Hook to Interact With Elasticsearch
  4. [Practice] Creating a Transfer Operator PostgresqlToElasticsearch
  5. [Practice] Adding a View to Apache Airflow UI
  6. Quiz Time!
  7. Recap

Section 7: Using Apache Airflow With Docker

  1. Introduction
  2. Quick Reminder About Docker
  3. [Practice] Building And Running Your First Airflow Docker Image
  4. [Practice] Running Airflow Using Sequential Executor With Docker
  5. [Practice] Running Airflow Using Local Executor With Docker
  6. [Practice] Scaling Airflow Using Celery Executor With Docker
  7. Quiz Time!
  8. Recap
Scroll to Top