#data-lake articles

Self-Hosted Databricks Alternative: Homelab Data Lakehouse (AI/ML Data Factory)

Self-Hosted Databricks Alternative: Homelab Data Lakehouse Part of the homelab build series. Trino, Apache Iceberg, Project Nessie, dbt, Spark, MLflow and OpenMetadata on RKE2, with highly available s

12 georgeezejiofor.com

Mugesh Krishna {M K }Sep 26 · 8 min read

Databricks LTAP: Bringing PostgreSQL OLTP and Lakehouse Analytics Together

Introduction Modern applications continuously generate transactional data. PostgreSQL is commonly used as the operational database because applications need fast and reliable INSERT, UPDATE, DELETE, a

organization-data-analysis-agent.hashnode.dev

vikas sharmaSep 22 · 4 min read

Best Data Lake Solutions for Growing Business Data

Every growing business eventually reaches a point where its data becomes too diverse to manage through isolated storage systems. Customer information, application logs, documents, media files, backups

vkdigiservices.hashnode.dev

aslangamzenur079Sep 11 · 4 min read

I Finally Understood Why We Need OLTP, OLAP, Data Lakes and Lakehouses

Lately I have been spending more time on data engineering and one question kept coming back to me Where should the data actually live At first this sounded like a simple storage decision. Then I reali

gamzenuraslan.hashnode.dev

MyData InsightsAug 7 · 7 min read

Lakehouse vs Data Lake: Governance and Security Differences Every Enterprise Should Know

Storage and scale get the attention, but governance and security are the real difference between a Data Lake and a Lakehouse. How they compare on metadata, lineage, access control and compliance. The

mydatainsights-blogs.hashnode.dev

Sergio González TéllezJul 20 · 3 min read

Why do AI systems with increasingly sophisticated models continue to produce mediocre or inconsistent results?

SEED-011 PROBLEM Why do AI systems with increasingly sophisticated models continue to produce mediocre or inconsistent results? INSIGHT A well-modeled, validated, and governed data lake often contri

evankhandev.hashnode.dev

SapotaCorpJul 12 · 13 min read

Data engineering for a regulated fintech: a 10-month AWS lake build

Client: Regional fintech operator (anonymized per agreement) Timeline: 10-month engagement (6-month build + 4-month fine-tuning & warranty) Team: 6 engineers including 2 from Sapota's data team, worki

sapotacorpvn.hashnode.dev

Braeson NyaheraJul 5 · 5 min read

Slowly Changing Dimensions(SCDs) with examples; Complete Guide

What are dimensions? Dimensions are descriptive data elements used to classify or categorize data. For example in a sales database, dimensions might include product, customer, store. Dimensions includ

braeson.hashnode.dev

Tencent Cloud -Cloud Log ServiceJun 15 · 5 min read

Deliver CLS Logs to Tencent Cloud DLC for Spark-Based Analysis

Log platforms often start with search and alerting, then grow into data processing and analytics workflows. Tencent Cloud Log Service (CLS) already supports delivery to Ckafka and COS. The source work

tencentcloud-cls.hashnode.dev

Sachin NandanwarApr 6 · 6 min read

Retrieve the hierarchical directory structure from Azure ADLS Gen2 storage

In one of my earlier article, I demonstrated how to leverage DatalakeServiceClient to create and modify files in a Lakehouse within Microsoft Fabric. Similarly, we can use DatalakeServiceClient for op

azureguru.net

Search Hashnode

Trending tags this week