AAaparna aravindinaparnaaravind.hashnode.dev·Dec 1, 2022 · 3 min readHello DuckDBI stumbled upon DuckDB recently when I was doing data analysis with python Pandas. The 3 steps - set up/installation, data exploration, and execution, and it was a very pretty quick implementation. In this post, we will be discussing DuckDB, and what...00
AAaparna aravindinaparnaaravind.hashnode.dev·Nov 23, 2022 · 5 min readHandling schema changes in snowflakeWith spark-snowflake connector writes We have discussed the internals of spark snowflake writes in the previous post, in this post lets observe how the schema changes such as missing column/extra columns, data type changes behaves with spark-snowfla...00
AAaparna aravindinaparnaaravind.hashnode.dev·Dec 10, 2021 · 4 min readDeequ for generating data quality reportsEnsuring data quality checks are really important in data driven projects To make sure of data correctness for correct business decisions Validate the data beforehand to avoid broken production pipelines Validate data from disperse sources(ftp, data...00
AAaparna aravindinaparnaaravind.hashnode.dev·Nov 3, 2021 · 4 min readSnowflake — AWS Lambda IntegrationIn this post I would like to walk you through on the steps for Snowflake AWS lambda integration “AWS Lambda is a serverless, event-driven compute service that lets you run code for virtually any type of application or backend service without provisio...00
AAaparna aravindinaparnaaravind.hashnode.dev·Aug 12, 2021 · 4 min readSpark-Snowflake Writes-Behind the scenesSnowflake have gained in popularity being a platform that offers data storage and its processing in cloud. And the trends also got reflected in the EDP design patterns (Enterprise Data Platform), Snowflake is getting mapped as the recommended DW laye...00
