20 essential PySpark operationsTable Of Contents: Setting Up PySpark Loading data Basic operations Column operations Row operations Aggregate functions Window functions Joins Performance Optimisation Best Practices and Tips Conclusion References As a Machine Learning...Mar 27, 2025 路 8 min readtanupriya.com
Demystifying Setup.py: The Blueprint Behind Python Packages 馃摝Ever wondered what makes 'pip install' work? Let's break down the mechanics behind setup.py 馃摝 At its core, setup.py is a Python script that defines your package's identity card. It contains essential metadata about your package - from its name and v...Jan 5, 2025 路 1 min readtanupriya.com
A Beginner's Guide to Spark: Insights from an MLEIntroduction to Spark Spark is a distributed computing system designed for large-scale data processing. Here are key reasons why Spark is suitable for ML pipelines: High Volume of Data: Many ML problems require processing large datasets, which mig...Dec 9, 2024 路 3 min readtanupriya.com
Notes from Andrej Karpathy's 1-Hour Lecture on LLMsBefore we begin, a short excerpt from the book The Defining Decade: The right time to take action is at the sweet spot鈥攚aiting too long or acting too early can lead to missed opportunities. Echoing this idea, I decided to learn about the Generative A...Oct 21, 2024 路 4 min readtanupriya.com
My Journey as a Machine Learning EngineerIntroduction One of the questions I get as a Machine Learning Engineer is whether I am required to read research papers and be aware of the latest algorithms and models. The answer is no. Don鈥檛 get me wrong, it benefits from knowing the various model...Jul 14, 2023 路 7 min readtanupriya.com
