Tag feed

#datalake

13 posts10 followers

Explore Hashnode

Alternatives

Trending tags this week

HGHarshit Gargaws-data-mesh.hashnode.devJul 8, 2025 · 14 min read

Part 5 - Network Security in Data Mesh

Recap: Where We Stand In our AWS Data Mesh series so far, we have established the foundational concepts of data mesh, set up the foundational AWS environment, and demonstrated how to ingest and catalog a single table using AWS Glue, making it availab...

0

HGHarshit Gargaws-data-mesh.hashnode.devJun 25, 2025 · 13 min read

Part 3 – Optimizing Data Mesh

Recap: Where We Stand Till now in this series, we established a secure, governed AWS data mesh platform, onboarded a real-world dataset, and enabled both publishers and consumers to collaborate seamlessly using S3, Glue, Lake Formation, and Athena. O...

0

HGHarshit Gargaws-data-mesh.hashnode.devJun 18, 2025 · 10 min read

Part 2 – Operationalizing Data Mesh

In the previous part, we laid the secure and governed foundations for our data mesh on AWS. If you missed it, check out Part 1 – Laying the Foundations for all the setup details, including KMS encryption, S3 buckets, Lake Formation, and Glue configur...

0

HGHarshit Gargaws-data-mesh.hashnode.devJun 16, 2025 · 11 min read

Part 1 - Laying the Foundations

Welcome to Part 1 of our hands-on series for building a data mesh on AWS. In this guide, we’ll set up a secure, governed data mesh on AWS. This part will focus on building central catalog components from below Architecture Diagram Prerequisites AWS...

0

MMANDARdata-organisation-options.hashnode.devJun 10, 2025 · 5 min read

Data Warehouses VS Data Lakes VS Data Lakehouses

Data capturing, processing and deriving insights from it has become crucial for businesses. To cater this need organizations are adopting sophisticated storage and processing solutions. One of the most discussed architectures for data management is t...

0

ASAhmed Shaabanahmedshaaban1999.hashnode.devMay 9, 2025 · 5 min read

My Attempt - And Failure - In Creating A Poor Man's Data Warehouse Using DuckDB

I know what some of you are probably thinking: "Oh great, another article about DuckDB." And honestly, I get it. A few months ago, I was in the same boat. It felt like DuckDB was following me around—blog posts, tweets, random mentions. It was everywh...

0

ASAhmed Shaabanahmedshaaban1999.hashnode.devApr 17, 2025 · 6 min read

Data Warehouses vs. Data Lakes: Understanding the right fit

One can easily assume that data lakes are the natural evolution of the old, clunky data warehouses. Another can claim that the difference is all about your workload; that is, if it is more about machine learning, then you would be better with a data ...

0

SNSachin Nandanwarazureguru.netFeb 26, 2025 · 5 min read

Row Level Security(RLS) and Data Lake Query Fallback

Few months ago I had penned an article on RLS(Row Level Security) in Microsoft Fabric. You can find that article here. With a practical example, this article will focus solely on how RLS affects the Data Lake behavior. There are multiple factors that...

0

SNSachin Nandanwarazureguru.netNov 4, 2024 · 12 min read

Implement Medallion architecture with SCD Type 2 in Microsoft Fabric - Part 1

This article is part one of a two-part series on implementing Medallion architecture in Microsoft Fabric. In the first part, we will see how to push data from its raw format to a Bronze layer and the fundamental steps involved in this initial transfo...

0

AMAlex Mboutchouangblog.alexwalker.appApr 28, 2024 · 4 min read

Understanding the Differences Between Database, Data Warehouse, and Data Lake

In today's digital landscape, the phrase "data is the new oil" resonates more than ever, underscoring the pivotal role that data plays in shaping our modern world. As our lives become increasingly intertwined with technology, decisions across virtual...

0

#datalake

Search Hashnode

#datalake

Explore Hashnode

Trending tags this week

Part 5 - Network Security in Data Mesh

Part 3 – Optimizing Data Mesh

Part 2 – Operationalizing Data Mesh

Part 1 - Laying the Foundations

Data Warehouses VS Data Lakes VS Data Lakehouses

My Attempt - And Failure - In Creating A Poor Man's Data Warehouse Using DuckDB

Data Warehouses vs. Data Lakes: Understanding the right fit

Row Level Security(RLS) and Data Lake Query Fallback

Implement Medallion architecture with SCD Type 2 in Microsoft Fabric - Part 1

Understanding the Differences Between Database, Data Warehouse, and Data Lake