Big Data File Formats ExplainedWhich one to choose and how to use them? All you should know about it including performance tips. I'm a big fan of data warehouse (DWH) solutions with ELT-designed (Extract-Load-Transform) data pipelines. However, at some point, I faced the requireme...Mar 2, 2023 · 8 min readdatamike.hashnode.dev
Hive Partitioning LayoutLakehouse design is one of my favorites because it gives us the best of two worlds. In a data warehouse, we can manage and transform data easily. It can be a single source of truth for all data transformations we do. Whereas in a data lake, we can us...Mar 1, 2023 · 5 min readdatamike.hashnode.dev
Data Platform Architecture TypesIt is easy to get lost with the abundance of data tools available in the market right now. The Internet is filled with opinionated stories (often speculative) about which data tools to use and how to make our Data Stack modern in this particular year...Feb 20, 2023 · 9 min readdatamike.hashnode.dev
Data Platform Unit and Integration Tests ExplainedWhat is a unit or integration test? How to do this exercise and how to apply it to our data pipelines? This is a good question I asked myself at the very beginning of my data engineering career. This topic seemed a bit difficult but believe me, it pa...Feb 20, 2023 · 9 min readdatamike.hashnode.dev
How to set website social share image in pure HTMLIt is not a secret that websites like LinkedIn, Facebook and Twitter do a social preview for your website link: It depends on what you have under <social> section of your website. So if you want to customise it or change to a new one there is an eas...Jan 31, 2023 · 2 min readdatamike.hashnode.dev