Mehul Kansalmehulkansal.hashnode.dev·Jun 3, 2024Week 3: Spark Transformations - Navigating Schema and Data Types 🧭Introduction Welcome back, data enthusiasts! This week, we'll unravel the intricacies of schema inference and enforcement, data type handling, creating and refining dataframes, and removing duplicates. Let's get started right away! Schema inference, ...Discuss·26 readsdata-engineering
Mehul Kansalmehulkansal.hashnode.dev·May 27, 2024Week 2: Elevate your Spark Game - Exploring Higher Level APIs! 📈Introduction Hey there, data enthusiasts! This week, we will explore the power of Spark's higher level APIs - Dataframes and Spark SQL. Let's begin using Spark like a pro and unveil the enhanced data processing capabilities of this dynamic duo. Task ...Discussdata-engineering
SHREYAS GANDHIshreyas0607.hashnode.dev·Mar 25, 2024How to fetch data from APIWhat is API ? API stands for Application programming interface. API is like a messenger that allows different software applications to communicate with each other.It defines the methods and rules for how software components should interact, much like...DiscussAPIs
grippybyteblog.grippybyte.com·Feb 25, 2024Converting CSV to DataFrame in PythonThe article is maintained by the team at commabot. To convert a CSV file to a DataFrame in Python, we can use the pandas library. Here's a step-by-step guide to doing this: Install Pandas: If you haven't already installed pandas, you can do so by run...DiscussPython
vijayvijay2023.hashnode.dev·Feb 1, 2024Unleashing the Power of Data Science with Python and PandasWelcome to the dynamic realm of Data Science, where insights are extracted, patterns are uncovered, and decisions are driven by the language of data. In this blog, we embark on a journey through the fundamentals of Data Science, with a particular foc...DiscussData Science
padmanabha reddypadmanabha.hashnode.dev·Jun 27, 2023Apache Spark - Structured APIApache Spark's Structured API is a high-level programming interface that enables users to manipulate and analyze structured and semi-structured data in a distributed computing environment. It is built on top of the Spark Core engine, providing a user...Discuss·39 reads#apache-spark
Renjitha Krenjithak.hashnode.dev·May 13, 2023Demystifying Big Data Analytics with Apache Spark : Part-3When it comes to dealing with mountains of data, Apache Spark has emerged as a powerful tool for processing and analyzing large-scale datasets. But what makes Spark even more appealing to many data professionals is its integration with good old, Stru...Discuss·1 like·93 readsbig data
Pius Mutumamutuma.hashnode.dev·Apr 8, 2023Merging DataFrames in Pandas: Techniques and Best PracticesMerging multiple data sources is a common task in data analysis and processing, and the Pandas library provides a powerful set of tools for performing this operation in Python. The ability to combine data from different sources allows us to gain insi...DiscussData Science
Pius Mutumamutuma.hashnode.dev·Feb 11, 2023Mastering Subsetting in Python: Techniques for Selecting Rows and Columns in DataFramesData is everywhere, and the ability to extract relevant information from it is essential in today's world. However, with massive amounts of data comes the challenge of organizing and manipulating it effectively. That's where mastering subsetting in P...Discuss·38 reads#Subsetting #Python, #DataFrames, #Pandas, #DataFrameSlicingandIndexing,#DataAnalysis, #DataManipulation
PyProDevread.pypro.dev·Dec 11, 2022Mastering Python Dictionaries & Pandas DataFramesPython dictionary and Pandas dataframe are the most frequent data structures used in dealing with data. The Pandas DataFrame, is a standard popular data structure to work with tabular data for advanced data analysis. In this article, we will get hand...Discuss·97 readsPython
