datamike.hashnode.dev·Mar 2, 2023 · 8 min readBig Data File Formats ExplainedWhich one to choose and how to use them? All you should know about it including performance tips. I'm a big fan of data warehouse (DWH) solutions with ELT-designed (Extract-Load-Transform) data pipelines. However, at some point, I faced the requireme...00
datamike.hashnode.dev·Mar 1, 2023 · 5 min readHive Partitioning LayoutLakehouse design is one of my favorites because it gives us the best of two worlds. In a data warehouse, we can manage and transform data easily. It can be a single source of truth for all data transformations we do. Whereas in a data lake, we can us...00
datamike.hashnode.dev·Feb 20, 2023 · 9 min readData Platform Architecture TypesIt is easy to get lost with the abundance of data tools available in the market right now. The Internet is filled with opinionated stories (often speculative) about which data tools to use and how to make our Data Stack modern in this particular year...00
datamike.hashnode.dev·Feb 20, 2023 · 9 min readData Platform Unit and Integration Tests ExplainedWhat is a unit or integration test? How to do this exercise and how to apply it to our data pipelines? This is a good question I asked myself at the very beginning of my data engineering career. This topic seemed a bit difficult but believe me, it pa...00
datamike.hashnode.dev·Jan 31, 2023 · 2 min readHow to set website social share image in pure HTMLIt is not a secret that websites like LinkedIn, Facebook and Twitter do a social preview for your website link: It depends on what you have under <social> section of your website. So if you want to customise it or change to a new one there is an eas...00