Delta table with change data capture(CDF)What is CDF: The Change Data Feed (CDF) feature allows Delta tables to track row-level changes between versions of a Delta table. When enabled on a Delta table, the runtime records “change events” for all the data written into the table. This include...Jan 9, 2024 · 2 min readhitek.hashnode.dev
SparkException: Job aborted due to stage failure: exceeds max allowed : spark.rpc.message.maxSizeProblem: How to fix Spark serialized task failure — a guide on how to fix spark.rcp.message.maxsize Apache Spark is one of the most popular distributed computing solutions for processing large amounts of data. However, while working with large amount...Jan 6, 2024 · 2 min readhitek.hashnode.dev