@sdw-online

Use data contracts to automate data workflows - part 2

Dec 30, 2024 · 14 min read · Preface📖 In part 1, we explained what a data contract is why we need them, and what a typical one contains In this blog, we dive into a demo to explore how they actually work so we can process data safer, faster and effectively. Goal🎯 The data...

Use data contracts to automate data workflows - part 1

Dec 24, 2024 · 12 min read · Preface 📚 I’ve actually written a post on data contracts before, so have a quick scan here if you want to see a project I created on them using Python, AWS S3 and other libraries (like Selenium and Soda). What is a contract? 🤔 Let’s first talk abou...

REST API to MySQL database using Python

Mar 3, 2024 · 13 min read · Preface💫 In the previous blog post to this, we created a data pipeline that scrapes football data from a website and loads it to an AWS S3 bucket as a CSV file, all through Python. This time, we’ll be sending HTTP requests to a REST API endpoint, fo...

1

Create a web scraping pipeline with Python using data contracts

Feb 13, 2024 · 24 min read · Preface 💫 This is a practical end-to-end data pipeline demo to show what a data project incorporating data contracts looks like. We’ll be scraping the Premier League table standings for the 2023/24 season, as of 13th February 2024 (the date this art...

VSVishnu commented

For 2024…

Dec 31, 2023 · 2 min read · 🤨 What? If your goal is to learn Python and how it applies to data engineering and analytics scenarios for 2024, my goal is to help you with that. For 2024, I’ll be prioritizing more video content to show you multiple ways to use Python (as well as ...