Feb 8, 2025 · 5 min read · Project: Slowly Changing Dimensions in Snowflake Using Streams and Tasks Introduction This project implements a real-time data pipeline for continuous data ingestion and transformation into a Snowflake data warehouse. It leverages various cloud techn...
Join discussion
Feb 8, 2025 · 2 min read · This project demonstrates a serverless data lake architecture using various AWS services. The architecture is designed to ingest, process, and analyze CSV files stored in an Amazon S3 bucket. AWS Lambda functions, Glue Crawlers, Glue Jobs, CloudWatch...
Join discussion
Feb 8, 2025 · 1 min read · Project Overview This project demonstrates a real-time data pipeline for stock market data. It integrates Apache Kafka for data streaming and multiple AWS services for data storage and querying. Key Features: Real-time data ingestion using Kafka Pro...
Join discussion
Feb 8, 2025 · 1 min read · Business Overview Snowflake's Data Cloud is based on a cutting-edge data platform delivered as a service(SaaS). Snowflake provides data storage, processing, and analytic solutions that arequicker, easier to use, and more versatile than traditional op...
Join discussion
Feb 8, 2025 · 2 min read · This project implements a scalable data pipeline to extract, transform, and load real estate data from Redfin into Snowflake using AWS services. The data is later visualized in Power BI to provide insights into real estate trends. Overview The pipeli...
Join discussion
Feb 8, 2025 · 2 min read · This project demonstrates an event-driven architecture using AWS services to build a real-time data processing pipeline. The architecture is designed to process events generated in an S3 bucket and uses services like SNS, SQS, and Lambda for reliable...
Join discussion
Feb 8, 2025 · 2 min read · This project demonstrates an ETL pipeline built with Apache Airflow on an AWS EC2 instance. The pipeline pulls data from the OpenWeather API and Amazon S3, performs transformations, loads the data into an RDS PostgreSQL database, joins the datasets, ...
Join discussion
Feb 8, 2025 · 1 min read · DataFlow Insights is an automated data pipeline project designed to streamline data ingestion, processing, and visualization using AWS services. This project pushes daily data to Amazon S3, automatically crawls and catalogs it using AWS Glue, queries...
Join discussion