End-to-End Data Pipeline

Build a complete, production-grade data pipeline: Airbyte ingestion → Delta Lake → dbt transformation → Metabase dashboard. Orchestrated with Airflow. The capstone for the Data Engineer track.

advancedAirbyteApache AirflowdbtDelta LakeApache SparkMetabaseDockerPostgreSQL

What you will build, step by step

  1. Architecture & Design
  2. Data Sources & Ingestion
  3. dbt Transformation Models
  4. Airflow Orchestration
  5. Metabase Dashboard & Final Deliverables