Search posts, tags, users, and pages
You're are right, Tom.
It is a bit confusing but in the context of the gevent pool, time.sleep is the same as gevent.sleep.
When you use the gevent pool, Celery calls gevent's monkey.patch_all() which in turn patches the time module. So even though it says time.sleep, it is actually gevent.sleep.
Bjoern Stiel I think it's trickier than that. When I print time.sleep in a celery task, I see:
[2023-12-15 07:45:10,095: WARNING/MainProcess] <module 'time' (built-in)>
When I do gevent.sleep:
[2023-12-15 07:47:18,453: WARNING/MainProcess] <function sleep at 0x7fe73c14b2e0>
It wouldn't be (built-in) if it's monkey-patched.
I do see that time.sleep gets monkey-patched if I run this in the shell though:
from gevent import monkey monkey.patch_all() True import time time.sleep
<function sleep at 0x7e5d2da2fba0>
Celery must not monkey-patch time by default for its gevent pool.
hmm, I've just double checked, and this is what I'm trying:
import inspect ...
@app.task(name="sleep") def sleep(): print(inspect.getmodule(time.sleep)) time.sleep(0.5)
When I run this task through a Celery worker with the --pool=gevent option, I see this in the output:
14:11:15 worker.1 | [2023-12-15 14:11:15,316: WARNING/MainProcess] <module 'gevent.hub' from '/opt/homebrew/lib/python3.11/site-packages/gevent/hub.py'> 14:11:15 worker.1 | [2023-12-15 14:11:15,829: INFO/MainProcess] Task sleep[3728dc0f-f208-4a7a-a2ca-bbd71367bc0d] succeeded in 0.5130529159214348s: None
Which looks like time.sleep has been monkey patched.
I've pasted a gist of how I run this here: gist.github.com/bstiel/a9c1951d28b01a99579c11ad76…
Do you mind sharing yours?
Bjoern Stiel Here it is but I can't reproduce it now... I was also using requests in a different task to pull data from jsonplaceholder. I do see it patched now. Sorry to bother you.
Here's the gist: gist.github.com/thuibr/1cf030c557e728c07e2b734492…
Tom Huibregtse no worries! I can't see nothing wrong with your gist either. Let me know if you come across it again.