Ilum

Ilum is the best way to manage and monitor your Apache Spark cluster on Kubernetes. With Ilum, you can easily connect to your kubernetes cluster and submit spark jobs or integrate Ilum with Apache Hadoop Yarn. Ilum also provides an easy-to-use web interface for monitoring your spark cluster and jobs.

Ilum was introduced to make managing Spark jobs easy. It provides an API and web interface to launch, monitor and control spark jobs. It was built with Kubernetes as a Spark containers manager in mind, making Spark and Kubernetes integration trivial. No more waiting for commands to be executed via the CLI or sifting through long logs to find errors.