How to set up my Yelposhere data engineering project on GCP?What is YelpoSphere? YelpoSphere is a data pipeline that extracts, transforms, and loads data from Yelp's public dataset into a GCP environment and performs various analyses on the data. The pipeline combines GCS, BigQuery, Dataproc, Cloud Composer, ...1SApr 16, 2023 · 13 min readitsadityagupta.hashnode.dev
Storage Transfer Service in GCPIntroduction The Storage Transfer Service is a data migration service provided by Google Cloud Platform (GCP) that allows users to move large amounts of data between on-premises data centres, cloud storage solutions, and other cloud providers. The St...1SMar 12, 2023 · 4 min readitsadityagupta.hashnode.dev
Setting up a GCP Service accountIntroduction A service account in Google Cloud Platform (GCP) is a special type of account that allows applications running on GCP to interact with other GCP services, such as Google Cloud Storage, Google Cloud SQL, and Google Compute Engine. Service...Mar 9, 2023 · 3 min readitsadityagupta.hashnode.dev
Setting up the development environment on Google Virtual MachineI'm participating in this year's cohort of the Data Engineering Zoomcamp 2023. This is a community-led, free data engineering course of about 8 weeks. In this blog, I'll summarise the steps to configure a Google Virtual Machine to make it ready for t...Mar 6, 2023 · 7 min readitsadityagupta.hashnode.dev
Linkedin Database DesignIntroduction LinkedIn stands out as a professional networking platform with a user base of over 750 million professionals worldwide. It handles an enormous amount of user data, including profiles, connections, job postings, and much more. In this blo...Feb 17, 2023 · 4 min readitsadityagupta.hashnode.dev