By Graeme
Hi
I have been looking through all the questions, and tutorials for Django and Celery for App Platform but I can’t seem to see anything that is helpful.
The Celery work seems to start, and then dies with a fatal error. I have pasted what is happening below.
If anyone has any ideas that would be most appreciated!
Kind regards,
apps@workers-799ccfc99b-qg7bq:~$ celery -A csapi worker -l info
-------------- celery@workers-799ccfc99b-qg7bq v5.0.5 (singularity)
--- ***** -----
-- ******* ---- Linux-4.4.0-x86_64-with-glibc2.27 2021-05-25 19:31:04
- *** --- * ---
- ** ---------- [config]
- ** ---------- .> app: csapi:0x2ac9d4dcb250
- ** ---------- .> transport: redis://:**@159.203.115.161:6379/1
- ** ---------- .> results: disabled://
- *** --- * --- .> concurrency: 8 (prefork)
-- ******* ---- .> task events: OFF (enable -E to monitor tasks in this worker)
--- ***** -----
-------------- [queues]
.> celery exchange=celery(direct) key=celery
[tasks]
. Cancel Expired Options
. Check Cabin for Release
. Log API Call
. Released Cabin
. csapi.celery.debug_task
. operations.tasks.resize_image
. publish.tasks.publish_element
[2021-05-25 19:31:04,464: CRITICAL/MainProcess] Unrecoverable error: OSError(38, 'Function not implemented')
Traceback (most recent call last):
File "/workspace/.heroku/python/lib/python3.9/site-packages/celery/worker/worker.py", line 203, in start
self.blueprint.start(self)
File "/workspace/.heroku/python/lib/python3.9/site-packages/celery/bootsteps.py", line 116, in start
step.start(parent)
File "/workspace/.heroku/python/lib/python3.9/site-packages/celery/bootsteps.py", line 365, in start
return self.obj.start()
File "/workspace/.heroku/python/lib/python3.9/site-packages/celery/concurrency/base.py", line 129, in start
self.on_start()
File "/workspace/.heroku/python/lib/python3.9/site-packages/celery/concurrency/prefork.py", line 107, in on_start
P = self._pool = Pool(processes=self.limit,
File "/workspace/.heroku/python/lib/python3.9/site-packages/celery/concurrency/asynpool.py", line 460, in __init__
super().__init__(processes, *args, **kwargs)
File "/workspace/.heroku/python/lib/python3.9/site-packages/billiard/pool.py", line 1046, in __init__
self._create_worker_process(i)
File "/workspace/.heroku/python/lib/python3.9/site-packages/celery/concurrency/asynpool.py", line 477, in _create_worker_process
return super()._create_worker_process(i)
File "/workspace/.heroku/python/lib/python3.9/site-packages/billiard/pool.py", line 1142, in _create_worker_process
on_ready_counter = self._ctx.Value('i')
File "/workspace/.heroku/python/lib/python3.9/site-packages/billiard/context.py", line 181, in Value
return Value(typecode_or_type, *args, lock=lock,
File "/workspace/.heroku/python/lib/python3.9/site-packages/billiard/sharedctypes.py", line 83, in Value
lock = ctx.RLock()
File "/workspace/.heroku/python/lib/python3.9/site-packages/billiard/context.py", line 110, in RLock
return RLock(ctx=self.get_context())
File "/workspace/.heroku/python/lib/python3.9/site-packages/billiard/synchronize.py", line 207, in __init__
SemLock.__init__(self, RECURSIVE_MUTEX, 1, 1, ctx=ctx)
File "/workspace/.heroku/python/lib/python3.9/site-packages/billiard/synchronize.py", line 71, in __init__
sl = self._semlock = _billiard.SemLock(
OSError: [Errno 38] Function not implemented
This textbox defaults to using Markdown to format your answer.
You can type !ref in this text area to quickly search our full set of tutorials, documentation & marketplace offerings and insert the link!
Hey there,
Thanks for that info. Looking into this further it looks like Celery uses multiprocessing pools that use a syscall that is not supported by App Platform at this time. However this is something that we are working on and will be addressing soon. Unfortunately there is no ETA that I can share.
Hi
Thanks for taking the time to respond.
Yes, I have tried it on a 2gb container as well but no luck with that either.
Is there an update on this issue? It’s very common to set up Celery task queues alongside Django apps to handle async operations.
Get paid to write technical tutorials and select a tech-focused charity to receive a matching donation.
Full documentation for every DigitalOcean product.
The Wave has everything you need to know about building a business, from raising funding to marketing your product.
Stay up to date by signing up for DigitalOcean’s Infrastructure as a Newsletter.
New accounts only. By submitting your email you agree to our Privacy Policy
Scale up as you grow — whether you're running one virtual machine or ten thousand.
Sign up and get $200 in credit for your first 60 days with DigitalOcean.*
*This promotional offer applies to new accounts only.