PPhilindatarecipes.hashnode.dev·Jan 29, 2022 · 20 min read§2.1 Apache Spark on KubernetesWritten in London by me Back to the index In the previous article, a bilingual Spark codebase was containerised and run locally as a multi-container Docker application. Such encapsulations of workloads are a prerequisite for using orchestrators lik...00
PPhilindatarecipes.hashnode.dev·Jan 29, 2022 · 16 min read§2.2 Apache Spark on KubernetesWritten in London by me Back to the index Native Execution on Kubernetes With Spark 3.1.1, native execution support for Kubernetes has become generally available. The previous article explains how Docker Desktop's K8s bundle can be installed and ho...00
PPhilindatarecipes.hashnode.dev·Jan 29, 2022 · 15 min read§2.3 Apache Spark on KubernetesWritten in London by me Back to the index Native Execution on Kubernetes With Spark 3.1.1, native execution support for Kubernetes has become generally available. The previous article explains how minikube can be installed and how a suitable local ...01I
PPhilindatarecipes.hashnode.dev·Jul 4, 2021 · 15 min readA deep dive into Spark's OptimizerMade in London by me Back to the index First published here §9 A deep dive into the Catalyst Optimizer The Catalyst optimizer is a crucial component of Apache Spark. It optimizes structural queries – expressed in SQL, or via the DataFrame/Dataset A...00
PPhilindatarecipes.hashnode.dev·Jul 4, 2021 · 17 min readApache Spark and DockerMade in London by me Back to the index First publication by me here §1 Apache Spark and Docker One of the major recent additions to Apache Spark has been the full support for Kubernetes. When Kubernetes is chosen as cluster manager, users must subm...00
