Priyansh KhodiyarforDatazipdatazip.io·Jan 7, 2025OLake Architecture, How did we do it?When building OLake, our goal was simple: Fastest DB to Data LakeHouse (Apache Iceberg to start) data pipeline. Checkout GtiHub repository for OLake - https://github.com/datazip-inc/olake Over time, many of us who’ve worked with data pipelines have d...51 readsOLakeolake
Priyansh KhodiyarforDatazipdatazip.io·Oct 10, 20247 Proven Techniques for Handling Changing Data Type during Semi-Structured Data Ingestion a.k.a Polymorphic KeysPolymorphic keys in semi-structured data—where a single field can hold values of different types (and to make things worse, they keep on changing each time) —are a challenge to deal with, especially when figuring out how exactly to store them in the ...174 readsOLakeSemi-Structured-Data
