Tag feed

#apache-spark

104 posts46 followers

Anas Rabei

How can I solve this Python error : ImportError cannot import name KafkaProducer

I have a very strange problem. I wrote a spark streaming job that monitor an HDFS directory, reads the newly added files, and send the file content to Kafka. When submitting the job I got that error ImportError: cannot import name KafkaProducer Wh...

May 1, 2016FC

0

AR

Anas Rabei

Is there a way to manually allocate specific number of workers for a spark streaming job?

Suppose we have apache spark running on only one machine. It runs in standalone mode with 5 workers and 2 cores for each worker. We need to submit three spark streaming applications. How can I submit the first application and allocate two workers to...

Apr 18, 2016

#apache-spark

Search Hashnode

#apache-spark

How can I solve this Python error : ImportError cannot import name KafkaProducer

Is there a way to manually allocate specific number of workers for a spark streaming job?