Freda Victorlearndataengineering.hashnode.dev·Sep 17, 2024Automating a Scalable Data Pipeline with AWS Glue, Athena, and TerraformIntroduction In today’s data-driven world, companies need to process large volumes of data quickly and efficiently to derive meaningful insights. As a data engineer, you’re tasked with building robust, scalable systems that extract, transform, and lo...AWS Glue
Om Kaleomkale.dev·May 19, 2024Step-by-Step Process for Data Projects: Roles and Responsibilities ExplainedAs a newcomer🌱 to the field of data engineering, I've been learning a lot about how data teams work together to complete a project. I'd like to share some insights that might be helpful for others who are just starting out. Here's a walkthrough of t...34 readsdata-engineering
Warui Wanjiruwaruithemystery.hashnode.dev·Jul 19, 2023Unleashing the power of Data : StorageIntroduction: Welcome to the second installment of our blog series on data storage. In the previous blog post, we explored the origins of data and the importance of understanding source systems. In this article, we will dive deeper into the role of s...11 likes·30 readsData Storagedata engineer
Warui Wanjiruwaruithemystery.hashnode.dev·Jul 18, 2023Unleashing the power of Data: Choosing the right Storage SystemIntroduction Welcome back to my blog series on the data engineering lifecycle; in-depth! In our previous installment, we explored data generation and source systems, laying the foundation for the data lifecycle. In this article, we will delve into th...10 likes·117 readsData Storagedata-engineering
Warui Wanjiruwaruithemystery.hashnode.dev·Jul 6, 2023Unleashing the power of Data: Undercurrents and their Impact on Source SystemsIntroduction Welcome to another edition of our blog series on generation systems, where we delve into the fascinating world of data engineering and the critical role of source systems in this process. As data engineers, we understand that source syst...10 likes·91 readsData Generation: Source Systemsdata-engineering
