Apache Airflow 2.0 is coming with a lot of big improvements. In this post, I’m gonna explain to you what are the most important features to expect what they solve. If you are totally new to Airflow, check my introduction course right there, or if you want an in-depth approach check my course here. Let’s get started.
Apache Airflow 2.0: The next big release
The version 2.0 brings new features to take Apache Airflow at the next level in terms of scalability, resiliency and performance.
- The scheduler won’t be the single point of failure anymore. It becomes highly available like the other components.
- The REST API is going to be stable following the OPEN API 3 specification.
- Dag serializing allows you to make your web server and soon, the scheduler as well, stateless.
- KEDA with the Celery Executor is an incredible to scale Apache Airflow and the Kubernetes Executor is finally challenged
and much more.
If you are wondering, when Apache Airflow 2.0 will come out, well, it is scheduled to the 3rd quarter 2020 but no promises. Don’t forget, it’s an open source project, the vast majority of people are working on it for free, so be patient.
Check the video below to have a complete sum up of all the features with beautiful examples 🙂
Enjoy.
