MSManu Shuklainecorpit.hashnode.dev·1d ago · 14 min readAWS Glue 6.0 cut the Spark DPU rate 30% to $0.308, but only on 6.0AWS Glue 6.0 cut the Spark DPU rate 30% to $0.308, but only on 6.0 Summary. AWS Glue 6.0 became generally available on 21 August 2026 with what the announcement calls a 30% price reduction. The number00
PAPrakash Agrawalinprakashagrawal.hashnode.dev·Jul 24 · 7 min readETL Configuration with S3, Glue Studio and Athena in AWS The AWS ETL Architecture ETL stands for extract, transform and load. It is the process of taking data from a source, preparing or transforming it into a useful form, and loading it into a destination 00
SSapotaCorpinsapotacorpvn.hashnode.dev·Jul 12 · 13 min readData engineering for a regulated fintech: a 10-month AWS lake buildClient: Regional fintech operator (anonymized per agreement) Timeline: 10-month engagement (6-month build + 4-month fine-tuning & warranty) Team: 6 engineers including 2 from Sapota's data team, worki00
SSapotaCorpinsapotacorpvn.hashnode.dev·Jul 12 · 7 min readIngesting every partner's file format without rewriting code each timeA credit bureau we worked with had built its ingestion the way most platforms start: one fixed file template, loaded in batches, parsed by code written specifically for that shape. It worked right up 00
SSapotaCorpinsapotacorpvn.hashnode.dev·Jul 12 · 8 min readValidating Data Against Its Own History: Baseline Data Quality with GlueDQA row level data quality gate is good at exactly one thing: deciding whether a single value, looked at in isolation, is acceptable. Is the credit limit a non negative number? Is the new IC twelve digi00
HTHammad Tariqinthehammadtariq.hashnode.dev·Jun 23 · 3 min readAWS Glue Cost Optimization: The 8 Traps That Inflate Your BillAWS Glue Cost Optimization: The 8 Traps That Inflate Your Bill If you run AWS Glue at any real volume, you've probably had the conversation: the bill came in two or three times higher than the forecas10
Ppratikdhandeinpratikdhande.hashnode.dev·May 31 · 5 min readZero Downtime Migration with AWS DMS - Architecting End to End IoT Data PipelineI am writing this blog to share my work for Merk Sharp and Dohme (MSD) client. This is not the actual client project. This is a personal protfolio project that reflects the challenges, and money mange10
VDVishnu Dinthedatatrench.hashnode.dev·May 29 · 9 min readSpark Architecture Simply ExplainedYou have been using Spark for months -- running notebooks, submitting jobs, reading docs. But when someone asks you to explain what actually happens when a job runs, you find yourself stalling. The co00
Iilshaadincodeless-sync.hashnode.dev·May 28 · 11 min readAWS Glue Alternatives: Simpler Ways to Sync API Data to RDSAWS Glue can transform terabytes of data across S3 buckets, orchestrate complex ETL workflows, and handle schema evolution at scale. It's also one of the most over-engineered ways to get your Stripe c00
ÖOÖmer Oruç ÇELİKinoorucelik.hashnode.dev·Mar 6 · 9 min readFrom 3,600 to 400 API Calls: Optimizing PySpark on AWS Glue with the Yield PatternStack: AWS Glue · PySpark · Step Functions · APIs · Power BI Welcome to my very first technical blog post, I'm Omer and I love many things in life, but for now, you will know only two of them: detail00
