Back to Projects
Cloud Data Engineering • 2025The Vision & Architecture
Enterprise cloud analytics transforming
raw rental transactions into Gold reporting.
Production-grade cloud data engineering pipeline implementing a Medallion Architecture (Bronze to Silver to Gold) on Snowflake and dbt Core. Ingests raw Airbnb booking, listing, and host data via AWS S3 stages with automated SCD Type 2 historization snapshots and Jinja-driven incremental models.
Core Capabilities
Medallion Architecture on Snowflake
Structured Bronze (raw external stage), Silver (cleansed, standardized, typed), and Gold (dimensional Star Schema marts) isolation within Snowflake.
SCD Type 2 Historization Snapshots
Implemented dbt snapshot pipelines tracking historical state changes in Superhost status and listing prices over time without state loss.
Cost-Optimized Incremental Models
Engineered dbt incremental models processing only new or updated records (is_incremental()), slashing warehouse compute costs.
Dynamic Jinja Meta-Programming
Constructed reusable SQL generation macros and config blocks, drastically reducing boilerplate code across transformation models.
Automated Data Quality Assertions
Integrated comprehensive dbt schema tests, uniqueness assertions, not-null constraints, and referential integrity validations.
Technology Stack
Architectural Decisions
Decoupled Snowflake compute and AWS S3 storage
Independent scaling allows inexpensive long-term raw data staging in S3 while scaling warehouse compute only during transformation runs.
ELT architecture over legacy ETL
Loading raw data into Snowflake stages preserves original fidelity and enables rapid SQL-based transformation iteration in dbt.