Parquet batch ingestion with Python, Postgres and Docker.Introduction We'll use Python to ingest into a Postgres database the NYC TLC green taxi trip records for September 2019, which are available in a Parquet file at the following URL:(https://d37ci6vzurychx.cloudfront.net/trip-data/green_tripdata_2019-0...4 min read·dataml.hashnode.devJan 27, 2024
Ingesting and visualizing data using Python and PostgresWe'll ingest the NYC taxi zone lookup data into a postgres database through a postgres container and visualize it using python, pandas, sqlalchemy and matplotlib. This setup allows us to use the best analytical capabilities of both SQL and Pandas. Th...3 min read·dataml.hashnode.devJan 1, 2024
Modeling and Transforming data for analytics using SQL, Dbt and AWS Redshift.Introduction I'll be going through a hypothetical scenario at an event ticketing company and describe the data modeling and transformation process in the STAR format. The github repository of the project is linked below.(https://github.com/Aditya-Tiw...5 min read·dataml.hashnode.devNov 16, 2023