Open-source data pipeline tool for transforming and integrating data. The modern replacement for Airflow. Effortlessly integrate and synchronize data from 3rd party sources. Build real-time and batch pipelines to transform data using Python, SQL, and R. Run, monitor, and orchestrate thousands of pipelines without losing sleep. Have you met anyone who said they loved developing in Airflow? That’s why we designed an easy developer experience that you’ll enjoy. Each step in your pipeline is a standalone file containing modular code that’s reusable and testable with data validations. No more DAGs with spaghetti code. Start developing locally with a single command or launch a dev environment in your cloud using Terraform. Write code in Python, SQL, or R in the same data pipeline for ultimate flexibility.
Features
- Integrate and synchronize data from 3rd party sources
- Run, monitor, and orchestrate thousands of pipelines without losing sleep
- Build real-time and batch pipelines to transform data using Python, SQL, and R
- Start developing locally with a single command or launch a dev environment in your cloud using Terraform
- Write code in Python, SQL, or R in the same data pipeline for ultimate flexibility
- Each step in your pipeline is a standalone file containing modular code that’s reusable and testable with data validations
Project Samples
