ARAnkit Rajinankit-data-engineering.hashnode.dev·May 2 · 4 min readMy GCP Pipeline Was Running Fine… But Doing Nothing (Service Account Lesson)The Situation I was working on a pipeline where everything looked correct from the outside. A file was landing in Cloud Storage, the Cloud Function was triggering as expected, and logs were showing su00
ARAnkit Rajinankit-data-engineering.hashnode.dev·Apr 25 · 3 min read7 Skills That Make Data Engineering Feel Less HardWhen I started working in data engineering, nothing felt simple. Tasks that looked straightforward took longer than expected, pipelines broke without clear reasons, and debugging often felt like guess00
ARAnkit Rajinankit-data-engineering.hashnode.dev·Apr 18 · 4 min readDataflow Job Not Starting? Debugging a Job Name Collision in GCPWhile working on a GCP Dataflow pipeline for an event-driven ingestion system, I ran into an issue that was surprisingly tricky to debug. Everything looked correct from the outside. Files were landing00
ARAnkit Rajinankit-data-engineering.hashnode.dev·Apr 11 · 4 min readThe Small metadata.json File That Changed How I Work With Image DatasetsWhen working with image datasets for machine learning, it is easy to assume that the images themselves are the most important part of the pipeline. At least that is what I thought when I first started00
ARAnkit Rajinankit-data-engineering.hashnode.dev·Apr 4 · 5 min readWhen a Cloud Function Keeps Failing and the Code Isn’t the ProblemRecently I was working on a small automation using Google Cloud Functions. The function itself was simple. Whenever a file was uploaded to a Cloud Storage bucket, the function would process that file 00
