Approaches to Remove Duplicate Rows

Create a data frame with columns Name and Age. data=[("Alice", 25), ("Bob", 30), ("Alice", 25), ("Kate", 22)] cols= ["Name", "Age"] df = spark.createDataFrame(data, cols) Approach 1:- Using dropDuplicates() #Approach 1 dedup_df=df.dropDuplicates(sub...