Spark DataFrames and Spark SQL: Schema, DDL, and the Catalyst Optimizer
TLDR: Catalyst is Spark's query compiler. It takes any DataFrame operation or SQL string, parses it into an abstract syntax tree, resolves column references against the catalog, applies a library of a
