Parquet batch ingestion with Python, Postgres and Docker.Introduction We'll use Python to ingest into a Postgres database the NYC TLC green taxi trip records for September 2019, which are available in a Parquet file at the following URL:(https://d37ci6vzurychx.cloudfront.net/trip-data/green_tripdata_2019-0...Jan 27, 2024 · 4 min readdataml.hashnode.dev
Ingesting and visualizing data using Python and PostgresWe'll ingest the NYC taxi zone lookup data into a postgres database through a postgres container and visualize it using python, pandas, sqlalchemy and matplotlib. This setup allows us to use the best analytical capabilities of both SQL and Pandas. Th...Jan 1, 2024 · 3 min readdataml.hashnode.dev
Modeling and Transforming data for analytics using SQL, Dbt and AWS Redshift.Introduction I'll be going through a hypothetical scenario at an event ticketing company and describe the data modeling and transformation process in the STAR format. The github repository of the project is linked below.(https://github.com/Aditya-Tiw...Nov 16, 2023 · 5 min readdataml.hashnode.dev
