Celery's unresolved worker memory leak: what actually helps when nothing fixes it

Your Celery workers start lean. A day later they are heavier. A week later the box is swapping, the OOM killer is circling, and you have restarted the workers by hand for the third time. Task volume is flat. Individual tasks are not holding references. You have profiled the task code and it is clean. And still the resident memory of the worker process climbs, slowly and relentlessly, like a tide coming in.

Before you spend another weekend on this, you need to know something the top search results will not tell you plainly: this is a real, long-standing, unresolved bug in Celery itself, and there is no confirmed fix. I am not going to sell you a one-liner that "solves" it, because the maintainers have said in as many words that they cannot reliably reproduce it. What I can give you is an honest account of what the evidence actually shows, and the mitigations that keep your server standing while the leak remains unsolved.

What you actually see

There is no exception and no error string here, which is part of why it is so maddening. The only signal is the memory curve and, sometimes, a log line that keeps repeating:

# RSS of the worker climbs monotonically, unrelated to task rate:
celery worker RSS: 210 MB -> 480 MB -> 900 MB over ~48h

# Often, but not always, accompanied by broker reconnection churn:
[WARNING] consumer: Connection to broker lost, trying to re-establish...
[INFO] Connected to amqp://guest@127.0.0.1:5672//

Watch the curve directly rather than waiting for the OOM killer to tell you about it after the fact:

# Resident memory of every celery process, once a minute
watch -n 60 'ps -o pid,rss,comm -C celery --sort=-rss'

The critical thing to establish first is which process is growing, because it changes everything about what will help.

Why this is hard: it is not one leak, it is a bucket labelled "memory grows"

The 4843 issue has hundreds of comments spanning years, and reading it carefully the pattern is not one bug with one fix. It is several different memory behaviours filed under one title, because the visible symptom is identical. A mitigation that clears it up completely for one person does nothing for the next, which is exactly what you would expect if they are chasing different underlying causes that happen to look the same from the outside.

Two facts from that thread cut through most of the bad advice. First, the maintainers themselves said they know there is a problem but cannot reproduce it, which means there is no authoritative fix to point you to and anyone claiming otherwise is overstating their case. Second, several reporters are clear that the growth is in the parent worker process, not the child processes that execute tasks. That second fact quietly invalidates the most popular fix on the internet.

The mitigations, in the order that makes sense

Step 1: Establish which process is leaking

Run the ps watch above and read the comm and command lines. In a prefork worker the parent shows as MainProcess and the children as ForkPoolWorker-N. Note which resident set is climbing over hours. This one observation decides your path: children point you at Step 2, the parent points you at Step 4. Do not skip this. Applying the child fix to a parent leak is why so many people report the popular advice "did nothing".

Step 2: If children grow, cap them with max_tasks_per_child

This is the most cited mitigation and it genuinely works for leaks that accumulate in the task-executing children. It tells Celery to retire a child after it has run a set number of tasks and fork a fresh one, handing the leaked memory back to the OS.

# celeryconfig.py
worker_max_tasks_per_child = 100

# or on the command line
celery -A proj worker --max-tasks-per-child=100

Pick the number by trading recycling cost against growth: too low and you pay fork overhead constantly, too high and memory climbs far between recycles. There is also worker_max_memory_per_child, which recycles a child once it crosses a memory threshold in kilobytes, which is often a better fit because it targets the actual symptom:

worker_max_memory_per_child = 300000  # 300 MB, then the child is replaced

Step 3: Correlate with broker reconnections

Grep your logs for reconnection churn, because several reporters tie parent growth to it:

grep -c "Connection to broker lost" /var/log/celery/*.log

If that count is high and rising alongside memory, your leak may be feeding on unstable broker connectivity rather than on your tasks at all. The fix there is not in Celery, it is in the broker path: stabilise the network to RabbitMQ, raise or tune heartbeat settings so short blips do not force full reconnects, and check you are not running a broker that is itself under memory pressure and dropping connections. A leak that only grows when the broker flaps is a different animal from one that grows with task count.

Step 4: If the parent grows, use an external safety net

This is the honest answer for the parent-process case, and it is not a defeat. When the leak is in the parent, no Celery setting recycles it, so you recycle it from outside. Put the worker under a supervisor that enforces a memory limit and restarts it cleanly.

Under systemd:

# /etc/systemd/system/celery.service
[Service]
ExecStart=/opt/venv/bin/celery -A proj worker --max-tasks-per-child=100
MemoryMax=1500M
Restart=always
RestartSec=5

Under Kubernetes, a memory limit plus the default restart policy does the same job: the pod is OOM-killed and rescheduled before the node is endangered. The point is to bound the blast radius. This is the same philosophy I apply to any process that cannot be trusted to stay healthy on its own, and it pairs with actually understanding what the kernel does when memory runs out, which I covered in what really happens when your server runs out of RAM. If you are on Kubernetes and want to be sure the restarts you are seeing are memory kills and not something else, finding the real cause of an OOMKilled pod walks through confirming it.

Scheduled proactive restarts are also legitimate here. Restarting workers nightly, during a quiet window, is not admitting defeat; it is capping an unbounded process on your terms instead of the OOM killer's.

Verification: prove the ceiling holds, not that the leak is gone

You are not verifying a cure, because there may not be one. You are verifying that your ceiling works.

1. Leave the ps watch running for a full day under normal load. With Step 2 in place on a child leak, memory should saw-tooth: climb, drop on recycle, climb again, never trending to infinity.
2. For a parent leak with Step 4, confirm the supervisor actually enforces the limit. On systemd, check systemctl status celery shows restarts happening at the memory ceiling, not crashes. On Kubernetes, kubectl get pod -w should show a controlled restart with reason OOMKilled and the node memory never approaching full.
3. Watch the OOM killer's own log to confirm it is no longer firing on your worker: journalctl -k | grep -i oom should stop showing your celery PID.
4. Track the recycle rate. If children are recycling every few seconds, your threshold is too aggressive and you are burning CPU on forks; raise it.

What people get wrong

Believing there is a fix and just searching harder for it. There is not one, at least not a confirmed general one, and treating a blog post's "this solved it for me" as authoritative wastes days. The maintainers cannot reproduce it. Anyone who could would have shipped the patch. Accept the ambiguity and manage the symptom.

Applying max_tasks_per_child to a parent leak and declaring it broken. It is not broken, it is aimed at the wrong process. This is the single most common source of "I tried the top answer and nothing happened" reports. Identify the leaking process first.

Blaming your own task code without evidence. A leak in the parent process, which never executes your task functions, is strong evidence the problem is not your task holding references. Spend your profiling time confirming which process grows before you rewrite task code that may be innocent.

Switching the pool as a first move. Prefork and gevent have completely different memory models, so a switch changes the numbers, sometimes for the better. But it also changes concurrency semantics, and a blocking task on gevent behaves very differently. Do not swap your execution model to dodge a memory curve without measuring both the memory and the throughput consequences under real load.

When it is still broken

Memory climbs even with children recycling on a tight limit. Then the leak is in the parent, or in a C extension that the child recycle does not fully reclaim. Move to the external safety net in Step 4 and stop trying to make an internal setting solve an external problem.

It only leaks under a specific task. Isolate it: route that task to a dedicated queue and worker, and watch whether the leak follows it. If it does, you may have a genuine per-task leak, often in a library the task uses (image processing, PDF generation, and native database drivers are common culprits), which is a different and more tractable problem than the general 4843 bug.

You are on an old Celery. Some memory behaviours have shifted between major versions. Reproduce on the current stable release before assuming your version's behaviour matches the thread, because you may be fighting a bug that a later release changed, for better or worse.

The mindset that gets you out of this is the one that is hardest to accept: you are not going to win the argument with the leak, so stop trying to. Bound it, recycle the process that grows, alert on the curve, and get your weekends back. An unsolved upstream bug is not an excuse for an unbounded process on your server.