MKMaria Karanasouintryexceptfinally.hashnode.dev·Mar 20, 2021 · 23 min readMachine Learning Interpretability — Shapley Values with PySparkInterpreting Isolation Forest’s predictions — and not only Photo by Joshua Golde on Unsplash The problem: how to interpret Isolation Forest’s predictions More specifically, how to tell which features are contributing more to the predictions. Since Is...00
MKMaria Karanasouintryexceptfinally.hashnode.dev·Jan 7, 2021 · 5 min readA Spark Attack!It is well known — or should be — that spark is not secured by default. It is right there in the docs Security in Spark is OFF by default So you should be well aware that you’ll need to put the effort to secure your cluster. And there are many th...00
MKMaria Karanasouintryexceptfinally.hashnode.dev·Nov 28, 2020 · 3 min readA Spark AttackTime to check your spark clusters Image by Author It is well known — or should be — that spark is not secured by default. It is right there in the docs Security in Spark is OFF by default So you should be well aware that you’ll need to put the effo...00
MKMaria Karanasouintryexceptfinally.hashnode.dev·Mar 24, 2020 · 5 min readIsolation Forest and PysparkLessons learned Debugging PySpark and Isolation Forest — Image by author So, after a few runs with the PySpark ml implementation of Isolation Forest presented here, I stumbled upon a couple of things and I thought I’d write about them so that you don...00
MKMaria Karanasouintryexceptfinally.hashnode.dev·Mar 21, 2020 · 3 min readPyspark — Parallel read from databaseHow to leverage spark to read in parallel from a database Spark Parallelization A usual way to read from a database, e.g. Postgres, using spark would be something like the following: user = 'postgres' password = 'secret' db_driver = 'org.postgresql.D...00
