How to run Pyspark on AWS Fargate
I'm using AWS Batch to run a few pyspark batch jobs and many times, when a job is submitted, it takes a few minutes to start the job. This delay may range from 2-15 minutes depending on the availability of EC2 machine and on the configuration provide...
kapilg.hashnode.dev3 min read