Celery Task Received but Not Executing: The Four Real Causes

Your Celery worker is up. It is connected to the broker. The log even shows it accepting work. And yet the task never runs. No traceback, no timeout, no error, just a line that says the task was received and then nothing, forever. You restart the worker, it drains, and then it does it again an hour later. This is one of the most maddening states in Celery, because every signal you have says the system is healthy while it quietly does no work at all.

The trap is that "received but not executing" is a symptom with at least four unrelated causes. The internet will confidently tell you to add -Ofair, or to lower your prefetch, or to switch pool types, and sometimes one of those helps and sometimes it makes no difference, because the person giving the advice was fixing a different underlying problem than the one you have. You cannot fix this by pattern-matching a flag. You have to find out what the worker is actually blocked on.

The log that lies to you

This is the line everyone pastes into a search box, and it has looked the same for years:

[2026-07-22 23:34:27,700: INFO/MainProcess] Received task: myapp.tasks.trigger_build[b248771c-6dd5-469d-bc53-eaf63c4f6b60]

What that line actually means is narrow and important: the MainProcess pulled the message off the broker into the worker's local memory. It says nothing about whether a child worker process ever picked it up to run. In a prefork worker (the default), the MainProcess is a dispatcher; the real execution happens in forked child processes. A healthy execution looks like this, with a second line from a pool process:

[2026-07-22 23:34:27,700: INFO/MainProcess] Received task: myapp.tasks.trigger_build[b248...]
[2026-07-22 23:34:27,712: INFO/ForkPoolWorker-3] myapp.tasks.trigger_build[b248...] succeeded in 0.011s

If you only ever see the MainProcess "Received" line and never a ForkPoolWorker line, the message crossed from the broker into the worker but never reached a pool process that was free to run it. That gap is the entire bug, and it has several causes.

The four causes, and how to tell them apart

Cause 1: every pool process is blocked on a shared resource

This is the most common and the most invisible. Your task, or something it imports at module load, opens a database connection. Your DB connection pool allows, say, 5 connections. Your worker runs concurrency 8. Eight child processes try to run tasks, five grab connections, three block waiting for one to free up, and the tasks that are holding connections are themselves slow or deadlocked. The pool is full, every child is parked in connection.acquire(), and the MainProcess keeps prefetching more messages it can never dispatch. Nothing logs, because the block happens before the task body runs its first line.

The give-away is that the worker is pinned but using no CPU, and restarting it fixes things briefly (it drops the connections and starts fresh) until the pool fills again.

Cause 2: one slow task holds a prefetched batch hostage

By default worker_prefetch_multiplier is 4, so each child reserves up to 4 messages. If a child is chewing on one 10-minute task, the other 3 messages it reserved are stuck behind it, marked as taken, unable to move to a free worker. From the outside, those 3 tasks are "received" and idle. This is the case the -Ofair flag and prefetch tuning actually address.

Cause 3: SQS or Redis visibility-timeout redelivery

On Redis and SQS there is no real broker-side ack; Celery emulates it with a visibility_timeout. If a task runs longer than that timeout, the broker assumes the worker died and hands the same message to another worker, while the first worker is still running it. You get duplicate execution, tasks that seem to restart, and workers that look busy on work that is being yanked out from under them. Critically, task_acks_late does not override this: the broker redelivers on the visibility timeout regardless. You must raise the timeout to exceed your longest task.

Cause 4: the worker is killed before it starts the task

Common under Kubernetes autoscaling and memory limits: the worker prefetches, then receives SIGTERM (scale-down) or SIGKILL (OOM) before the child runs the task. The message sits until the visibility timeout expires, then another worker picks it up. From the logs of the first pod, it is "received but never executed" because the pod was gone.

The fix, in order

Step 1: find out what the worker is actually blocked on

Stop guessing. Install py-spy and dump the stack of a stuck child process. This one command resolves cause 1 versus cause 2 versus a real deadlock in seconds.

pip install py-spy
# find the worker child PIDs
ps aux | grep ForkPoolWorker
# dump one that is stuck (needs privileges; add --cap-add SYS_PTRACE in Docker)
py-spy dump --pid 4123

If the top of the stack is inside psycopg2, connection.acquire, or a socket read to your database, you have cause 1: a resource block. If it is inside your task body doing real work, you have cause 2: a slow task hogging its prefetch. If the process is not even there, you have cause 4. This single step is the whole diagnosis, and it is why the guessing-with-flags approach wastes hours.

Step 2: for a resource block, size the pool to the concurrency

Your database (or HTTP, or Redis) connection pool must have room for every concurrent worker plus overhead. If concurrency is 8, do not run a DB pool of 5. Either raise the pool ceiling or lower --concurrency so they match. This is the same class of mistake as running out of Postgres connections: the fix is matching supply to demand, not blindly raising one number.

Step 3: for prefetch starvation, take one at a time

If your tasks are long and variable, stop reserving batches. Set these together, because acks-late without prefetch=1 still lets a worker grab extra:

# celeryconfig.py / settings
worker_prefetch_multiplier = 1   # reserve one message at a time
task_acks_late = True            # ack only after the task finishes, so a crash requeues it

The -Ofair flag on the worker command line (celery -A app worker -Ofair) changes distribution so a child only gets a new task when it is free, rather than round-robin regardless of state. It helps with mixed fast/slow workloads, but understand it is a scheduling tweak, not a fix for a blocked pool. If cause 1 is your problem, -Ofair changes nothing.

Step 4: for SQS/Redis, set visibility timeout above your longest task

broker_transport_options = {
    "visibility_timeout": 3600,  # seconds; must exceed your slowest task
}

If your longest task can run 20 minutes, a 30-minute visibility timeout stops the broker from redelivering it mid-run. Combine with idempotent tasks so that if a redelivery does slip through, running twice is safe.

Verification

You have fixed it when the pool-process log line appears for every received task. Watch the two together:

celery -A myapp worker --loglevel=info | grep -E "Received task|succeeded in"

Every Received task from MainProcess should be followed, within your expected latency, by a ForkPoolWorker-N ... succeeded in line for the same task ID. If you inspect active tasks, they should be moving, not stuck:

celery -A myapp inspect active     # tasks currently running in pool processes
celery -A myapp inspect reserved   # tasks prefetched but not started - should be small

A large and unchanging reserved count is the machine-readable version of the symptom. It should stay near zero once prefetch is 1 and the pool is unblocked.

What people get wrong

Cargo-culting -Ofair. It is the top answer on every thread and it only addresses cause 2. If your workers are blocked on a DB pool, -Ofair distributes the blocked work more fairly and you still do zero of it. Diagnose before you flag.

Cranking concurrency to "go faster". If tasks are stuck on a shared resource, more child processes means more contention for the same 5 connections, which makes it worse, not better. The bottleneck is the resource, and more consumers of a scarce resource is not throughput.

Assuming task_acks_late protects long tasks on Redis/SQS. It does not touch the visibility timeout. Late acks decide whether a task is requeued on worker crash; the visibility timeout decides when the broker gives up waiting. Two different clocks. You need both set correctly.

When it is still broken

If py-spy shows the child sitting in your own code with no external wait, you have a genuine deadlock or an infinite loop in the task, so read the exact line it is parked on and reproduce it in isolation. If tasks vanish entirely and never appear as reserved, suspect a routing problem: the worker may be consuming a different queue than the one you are publishing to, so check celery -A myapp inspect active_queues against your task's routing. If workers keep dying and taking prefetched work with them, treat it as a memory problem first and read the Celery worker memory leak playbook, because an OOM-killed worker produces exactly this "received but never ran" shape from the pod that died.