Building a memory-efficient ETL to convert JSON to CSV in Python with prefect
Have you ever built a simple ETL (Extract-Transform-Load) pipeline in python to convert a JSON file to CSV? Easy peasy, right????? But have you ever tried to transform a large dataset (Large here meaning a dataset in which the dataset size is larger ...
Responses
Markdown supported · ⌘↵ to post
Mike Logaciuk
DevOps Enginner
Isn't a little bit over engineered? Apart from Prefect?
You could use just Pandas and import from CSV and export to JSON :)