@anishShilpakar

Real-World Data Engineering: Healthcare Claims Pipeline Using AWS SNS, SQS, Snowpipe, and dbt

Dec 29, 2025 · 24 min read · Part 1 of 2: This post covers building the foundational data pipeline for a healthcare claims analytics project called claims-pulse. We walk through realistic data generation, uploading to S3 using bo

How I Built a Production-Grade E-Commerce Data Analytics Pipeline With AWS and Databricks

Nov 16, 2025 · 40 min read · Introduction Most data engineering tutorials hand you a clean CSV file and say, "Here, load this into a table." Real life is nothing like that. In the real world, your data source is an HTTP API that

Deploying Kubernetes on AWS EC2: Infrastructure Management Made Easy for Data Engineers

Oct 30, 2024 · 12 min read · Introduction In data engineering, efficient infrastructure management is as crucial as building robust data pipelines. As data volumes grow, scalability and resilience become essential for seamless processing and analytics. Kubernetes simplifies this...

Real-Time Data Engineering: Streaming Unstructured Data to AWS with Apache Spark

Sep 17, 2024 · 15 min read · Introduction In the era of big data, organizations generate vast amounts of unstructured data in formats such as JSON, CSV, and TXT files. Processing and analyzing this data efficiently is crucial for real-time decision-making. This project demonstra...

Amazon Q: The AI Chatbot Boosting Your Business to the Next Level

Jan 21, 2024 · 6 min read · Image: Amazon Q Overview Today, in the vast landscape of technology, one fascinating area that's capturing everyone's attention is AI, especially generative AI. You've probably heard of generative AI tools like ChatGPT, DALL·E, GitHub Copilot, Midjou...