Nice read.
For us though gevent or eventlet has never worked, we have tried gevent multiple times but randomly all the kubernetes of this gevent celery app just stop processing tasks and logs show some broken pipe error we tried to research a lot about it but were unsuccessful, we now run threads for IO bound tasks and performance is pretty good so far. I really did wanted to use gevent because of the huge performance benefit it provides as it uses co-operative scheduling.
Great article. (need to fix "--concurrency" spelling in some of the examples)
this is nicely explained! I was looking for such a helpful celery resource and hey I have found the gem! thank you so much for making such a dedicated celery blog!
Erick
If you only have a hammer, you see every problem as a nail
Thanks, this clears a bit of stuff up for me. Do you happen to know whether you can set the worker pool programmatically? I.e. let's say I have a Celery instance
appand create a worker withapp.worker_main(...), can I set the worker pool in the arguments?