Aareebakhizer7inareebakhizer.hashnode.dev·2d ago · 7 min readPart 3 - Stage by Stage: Building the Home Assistant ETLQuick context if you're landing here mid-series: I built a data platform on top of Home Assistant to capture and analyze smart home sensor data that would otherwise be purged after 10 days. Part 2 cov00
NCN Chandra Prakash Reddyindevopstour.hashnode.dev·Sep 5 · 11 min readFrom Data Lake to Lakehouse: Building Modern Analytics Platforms on AWS with S3 Tables & AWS GlueOkay, we will be honest – data architecture is usually one of those topics that makes your head spin. I was there at AWS Community Day Chennai on 07-Mar-2026. There were a lot of good talks on many ki00
MSManu Shuklainecorpit.hashnode.dev·Aug 21 · 14 min readAWS Glue 6.0 cut the Spark DPU rate 30% to $0.308, but only on 6.0AWS Glue 6.0 cut the Spark DPU rate 30% to $0.308, but only on 6.0 Summary. AWS Glue 6.0 became generally available on 21 August 2026 with what the announcement calls a 30% price reduction. The number00
PAPrakash Agrawalinprakashagrawal.hashnode.dev·Jul 24 · 7 min readETL Configuration with S3, Glue Studio and Athena in AWS The AWS ETL Architecture ETL stands for extract, transform and load. It is the process of taking data from a source, preparing or transforming it into a useful form, and loading it into a destination 00
SSapotaCorpinsapotacorpvn.hashnode.dev·Jul 12 · 13 min readData engineering for a regulated fintech: a 10-month AWS lake buildClient: Regional fintech operator (anonymized per agreement) Timeline: 10-month engagement (6-month build + 4-month fine-tuning & warranty) Team: 6 engineers including 2 from Sapota's data team, worki00
SSapotaCorpinsapotacorpvn.hashnode.dev·Jul 12 · 7 min readIngesting every partner's file format without rewriting code each timeA credit bureau we worked with had built its ingestion the way most platforms start: one fixed file template, loaded in batches, parsed by code written specifically for that shape. It worked right up 00
SSapotaCorpinsapotacorpvn.hashnode.dev·Jul 12 · 8 min readValidating Data Against Its Own History: Baseline Data Quality with GlueDQA row level data quality gate is good at exactly one thing: deciding whether a single value, looked at in isolation, is acceptable. Is the credit limit a non negative number? Is the new IC twelve digi00
HTHammad Tariqinthehammadtariq.hashnode.dev·Jun 23 · 3 min readAWS Glue Cost Optimization: The 8 Traps That Inflate Your BillAWS Glue Cost Optimization: The 8 Traps That Inflate Your Bill If you run AWS Glue at any real volume, you've probably had the conversation: the bill came in two or three times higher than the forecas10
Ppratikdhandeinpratikdhande.hashnode.dev·May 31 · 5 min readZero Downtime Migration with AWS DMS - Architecting End to End IoT Data PipelineI am writing this blog to share my work for Merk Sharp and Dohme (MSD) client. This is not the actual client project. This is a personal protfolio project that reflects the challenges, and money mange10
VDVishnu Dinthedatatrench.hashnode.dev·May 29 · 9 min readSpark Architecture Simply ExplainedYou have been using Spark for months -- running notebooks, submitting jobs, reading docs. But when someone asks you to explain what actually happens when a job runs, you find yourself stalling. The co00
