ASAlex Schoenenwaldintechwood.hashnode.dev·Oct 19, 2022 · 5 min readFrom DynamoDB to TimestreamOnce upon a time… In 2018, we began collecting data with our sensors and persisted them in a DynamoDB. This decision was sufficient until recently. Even when Timestream was released in 2020 we thought: “Dang, Timestream is exactly made for our Use Ca...00
ASAlex Schoenenwaldintechwood.hashnode.dev·Jan 13, 2022 · 6 min readHow to collect and visualize data lineage in an AWS-based data lakeIn this article, we will go over a reference architecture for a Spark/Glue-based Data lake in AWS, we will discuss on the possiblities how to collect data lineage in this architecture and then describe how to visualize and interact with the collected...00
ASAlex Schoenenwaldintechwood.hashnode.dev·Jan 11, 2022 · 7 min readWhy metadata is key in data lakesAutomating data collection and centralization is great - but how can you keep up with context information of and interdependencies between the datasets? This is where metadata (in particualr data lineage - describing the origins and transformation hi...00
