Tag feed

#dataquality

17 posts1 followers

Explore Hashnode

Alternatives

Trending tags this week

BEBlaine Elliottblog.anomalyarmor.aiApr 27 · 13 min read

How Do I Monitor Schema Changes in a Data Warehouse?

You monitor schema changes in a data warehouse by periodically querying metadata catalogs (like INFORMATION_SCHEMA), subscribing to event-driven notifications, or comparing structural hashes of your tables over time. Each method trades off between de...

0

BEBlaine Elliottblog.anomalyarmor.aiApr 20 · 19 min read

What Is Data Downtime and How Do You Measure It?

Data downtime is the total period during which data is missing, erroneous, or otherwise unfit for use. It is the data equivalent of application downtime: the window between when something breaks and when it is fully resolved. During data downtime, da...

0

BEBlaine Elliottblog.anomalyarmor.aiApr 12 · 11 min read

The 6 Dimensions of Data Quality: Definitions, Examples, and How to Monitor Each

The six dimensions of data quality are accuracy, completeness, consistency, timeliness, validity, and uniqueness. Each dimension measures a different aspect of whether data is fit for its intended use. Together they define whether a dataset can be tr...

0

BEBlaine Elliottblog.anomalyarmor.aiApr 11 · 9 min read

You Don't Need to Write Data Tests

Spend five minutes in any data engineering forum and you'll find the same confession repeated in different words: "We just eyeball row counts and pray." It shows up on Reddit, Hacker News, the dbt Community Forum, Stack Overflow. The phrasing changes...

0

BEBlaine Elliottblog.anomalyarmor.aiApr 11 · 9 min read

Data Observability vs Data Quality: What's the Difference and Do You Need Both?

Data observability and data quality get used interchangeably, but they solve different problems. Confusing them leads to buying the wrong tool, building the wrong monitors, and missing the issues that actually break things. Here's the short version: ...

0

BEBlaine Elliottblog.anomalyarmor.aiApr 11 · 7 min read

Data Quality Monitoring for Snowflake and Databricks: A Practical Guide

Last Tuesday a source team renamed order_status to status in Snowflake. No announcement. Your dbt models kept running, but every query referencing order_status silently returned NULLs. The revenue dashboard showed a 40% drop. It took four hours to tr...

0

MMandlaiammandla.hashnode.devMar 26 · 9 min read

I Built a FinTech Data Pipeline a CFO Can Actually Trust — Lessons from the Journey

So I gave myself a challenge, to build a production-grade data pipeline from scratch, using real industry tools, at zero cost, and document every stumble along the way. This is that story. The Proble

0

DADaniel Anthony Romitelli Jrcraftedbydaniel.hashnode.devMar 19 · 14 min read

I Stopped Letting Emails Poison My Extractor: The Pre-LLM Gate That Made the Rest of the Pipeline Reliable

I knew something was wrong the first time I saw a “candidate” come back with the recruiter’s phone number. Nothing was broken in the obvious places. Extraction ran. Persistence succeeded. The UI showed a clean-looking result. But the identity was wro...

0

DADaniel Anthony Romitelli Jrcraftedbydaniel.hashnode.devMar 19 · 9 min read

The Six‑Tier Enrichment Cascade: How I Stop “Helpful” Data From Overwriting True Data

I noticed something that felt impossible at first: the more enrichment I added, the more often my records got worse. Not catastrophically. Not in a way that threw errors. Just… quietly. A city would flip to something nearby. A state would “correct” i...

0

DADaniel Anthony Romitelli Jrcraftedbydaniel.hashnode.devMar 19 · 14 min read

The CRM Sync Engine I Had to Reverse‑Engineer: Two‑Step Fetches, 50‑Field Limits, and a Mapper That Refuses to Drift

I found the bug the way you always find the worst ones: not in a unit test, not in staging, but in a "why is this field blank again?" message after a real sync. The Zoho CRM API call looked correct. The request even included a fields parameter. And y...

0

#dataquality

Search Hashnode

#dataquality

Explore Hashnode

Trending tags this week

How Do I Monitor Schema Changes in a Data Warehouse?

What Is Data Downtime and How Do You Measure It?

The 6 Dimensions of Data Quality: Definitions, Examples, and How to Monitor Each

You Don't Need to Write Data Tests

Data Observability vs Data Quality: What's the Difference and Do You Need Both?

Data Quality Monitoring for Snowflake and Databricks: A Practical Guide

I Built a FinTech Data Pipeline a CFO Can Actually Trust — Lessons from the Journey

I Stopped Letting Emails Poison My Extractor: The Pre-LLM Gate That Made the Rest of the Pipeline Reliable

The Six‑Tier Enrichment Cascade: How I Stop “Helpful” Data From Overwriting True Data

The CRM Sync Engine I Had to Reverse‑Engineer: Two‑Step Fetches, 50‑Field Limits, and a Mapper That Refuses to Drift