Building a memory-efficient ETL to convert JSON to CSV in Python with prefect
Have you ever built a simple ETL (Extract-Transform-Load) pipeline in python to convert a JSON file to CSV? Easy peasy, right????? But have you ever tried to transform a large dataset (Large here meaning a dataset in which the dataset size is larger ...
madeofajala.hashnode.dev15 min read
Mike Logaciuk
DevOps Enginner
Isn't a little bit over engineered? Apart from Prefect?
You could use just Pandas and import from CSV and export to JSON :)