ImagePullBackOff in Kubernetes: The Five Real Causes and How to Tell Them Apart

You push a deploy, watch the rollout, and one pod refuses to come up. kubectl get pods shows ImagePullBackOff, then flips to ErrImagePull, then back again, forever. The container never starts. Nothing in your application code is wrong, because your application never ran.

Here is the thing almost nobody says out loud: ImagePullBackOff is not a cause, it is a category. It is the kubelet telling you "I tried to pull the image, it failed, and I am now backing off before trying again." At least five completely unrelated problems produce that exact same status. A typo in a tag looks identical to a missing registry secret, which looks identical to a Docker Hub rate limit. If you fix the wrong one, the status does not change, and you conclude Kubernetes is broken. It is not.

The entire skill here is reading one line of output that most people scroll straight past. Restarting the pod does nothing but reset the backoff timer. Deleting and recreating it does nothing either. The answer is already on your screen the moment you run kubectl describe.

The exact status and error you are seeing

The status column cycles between two values:

NAME                        READY   STATUS             RESTARTS   AGE
web-7d9f8c6b4d-2xq7p        0/1     ImagePullBackOff   0          92s
web-7d9f8c6b4d-2xq7p        0/1     ErrImagePull       0          2m10s

ErrImagePull is the pull attempt failing. ImagePullBackOff is the kubelet waiting (with exponential backoff, capped at five minutes) before trying again. They are the same underlying failure at two points in the retry cycle. The status itself carries no diagnostic value. The Events do.

Read the one line that actually tells you the cause

kubectl describe pod web-7d9f8c6b4d-2xq7p

Scroll to the bottom. You want the Events table and specifically the Failed line:

Events:
  Type     Reason     Age                 From     Message
  ----     ------     ----                ----     -------
  Normal   Pulling    2m (x4 over 3m)     kubelet  Pulling image "myco/api:v1.4.2"
  Warning  Failed     2m (x4 over 3m)     kubelet  Failed to pull image "myco/api:v1.4.2": rpc error: code = NotFound desc = failed to pull and unpack image "docker.io/myco/api:v1.4.2": failed to resolve reference "docker.io/myco/api:v1.4.2": docker.io/myco/api:v1.4.2: not found
  Warning  Failed     2m (x4 over 3m)     kubelet  Error: ErrImagePull
  Normal   BackOff    45s (x6 over 3m)    kubelet  Back-off pulling image "myco/api:v1.4.2"
  Warning  Failed     45s (x6 over 3m)    kubelet  Error: ImagePullBackOff

The load-bearing text is inside that first Failed message. Everything after desc = is the containerd runtime's actual complaint. In the example above it is not found. Yours might say something else. That string is your diagnosis.

Why this happens: the five real causes, by reason string

Each cause writes a recognisably different message. This table is the whole article compressed into one lookup.

What the Events say	Real cause	Where to fix it
not found, manifest unknown, manifest for ... not found	The image:tag does not exist. Typo, wrong tag, or the tag was overwritten/deleted in the registry.	Pod spec image: field
unauthorized, authentication required, pull access denied	Private image, and either no imagePullSecrets, or the secret exists but is not attached to the ServiceAccount the pod runs as.	ServiceAccount / pod spec secret
toomanyrequests: You have reached your pull rate limit	Docker Hub anonymous pull limit (100 pulls per 6 hours per IP as of 2025). Many nodes behind one NAT IP share the quota.	Add a Docker Hub login secret, or a pull-through cache
no such host, connection refused, i/o timeout, x509	The node cannot reach the registry at all. DNS, firewall, private registry outside the VPC, or an untrusted registry CA.	Node networking / containerd config
failed to resolve reference with a wrong hostname	Registry hostname typo, or missing registry prefix so it defaulted to docker.io.	Pod spec image: field

The fix, in numbered steps

Step 1: Confirm the image and tag actually exist

Copy the exact reference from the pod spec and try to pull it yourself from a machine that has registry access, or resolve just its manifest without downloading layers using crane:

# Full pull test
docker pull docker.io/myco/api:v1.4.2

# Lighter: resolve the manifest only (google/go-containerregistry)
crane manifest docker.io/myco/api:v1.4.2

If crane manifest or docker pull also says not found or manifest unknown, the tag genuinely is not in the registry. Common reasons: the CI build pushed v1.4.2 but your manifest asks for v1.4.2. with a trailing character, the tag was latest and got overwritten, or you are pointing at the wrong repository. Fix the image: field and redeploy. There is nothing to fix on the cluster.

Step 2: For a private registry, check the secret is attached, not just referenced

This is the one that eats hours. Having an imagePullSecrets entry in the pod spec is not the same as the ServiceAccount carrying it, and pods pull using their ServiceAccount. Check what the pod's ServiceAccount actually has:

kubectl get pod web-7d9f8c6b4d-2xq7p -o jsonpath='{.spec.serviceAccountName}{"\n"}'
kubectl get serviceaccount default -o jsonpath='{.imagePullSecrets}{"\n"}'

Either attach the secret to the ServiceAccount (so every pod using it inherits the credential) or reference it directly in the pod/Deployment spec:

# Option A: patch the ServiceAccount once
kubectl patch serviceaccount default \
  -p '{"imagePullSecrets": [{"name": "regcred"}]}'

# Option B: reference it explicitly in the Deployment pod template
spec:
  template:
    spec:
      imagePullSecrets:
        - name: regcred
      containers:
        - name: api
          image: myregistry.example.com/myco/api:v1.4.2

Create the secret itself with kubectl create secret docker-registry regcred --docker-server=... --docker-username=... --docker-password=.... A secret in the wrong namespace will not work: imagePullSecrets must live in the same namespace as the pod. If you keep copying it between namespaces by hand, read why those manual copies always drift.

Step 3: If the reason is toomanyrequests, authenticate even for public images

The Docker Hub rate limit is not an authentication failure, but the fix is authentication. An anonymous IP gets 100 pulls per six hours, and every node behind your NAT gateway spends from the same bucket, so one rolling deploy across a fleet can exhaust it. A free Docker Hub account raises the limit and, crucially, meters it per account instead of per shared IP.

kubectl create secret docker-registry dockerhub \
  --docker-server=https://index.docker.io/v1/ \
  --docker-username=YOUR_USER \
  --docker-password=YOUR_TOKEN \
  --namespace=default

Attach it exactly as in Step 2. For a busy cluster, a pull-through cache (Harbor, or your cloud provider's registry mirror) is the durable answer, because it pulls each image once and serves it locally.

Step 4: If the node cannot reach the registry, look at the node, not the pod

no such host, i/o timeout, or an x509 error means the container runtime on the node failed at the network or TLS layer. No pod-spec change will help. SSH to the node (or use a debug pod) and reproduce the pull at the containerd level:

# On the node, using containerd's CLI
sudo crictl pull myregistry.example.com/myco/api:v1.4.2

# Check containerd's registry / TLS config
sudo cat /etc/containerd/config.toml | grep -A5 registry

A private registry with a self-signed CA needs that CA trusted in containerd's config on every node. A registry inside a VPC needs the node's security group and DNS to reach it. These are node-level, and they will hit every pod that lands on that node.

Verification: proving it is actually fixed

Do not delete the pod to test the fix. The kubelet retries on its own backoff schedule, so once the cause is gone it will pull successfully within a minute or two. Watch it:

kubectl get pod web-7d9f8c6b4d-2xq7p -w

NAME                        READY   STATUS              RESTARTS   AGE
web-7d9f8c6b4d-2xq7p        0/1     ImagePullBackOff    0          4m
web-7d9f8c6b4d-2xq7p        0/1     ContainerCreating   0          5m
web-7d9f8c6b4d-2xq7p        1/1     Running             0          5m10s

Confirm the Events now show a successful pull:

kubectl describe pod web-7d9f8c6b4d-2xq7p | grep -A2 Pulled

  Normal  Pulled   20s   kubelet  Successfully pulled image "myregistry.example.com/myco/api:v1.4.2" in 3.1s

What people get wrong

Restarting or recreating the pod. This is the reflex, and it is useless. The image reference and the credentials are identical on the new pod, so it fails identically. All you did was reset the backoff timer to zero, which if anything makes Docker Hub rate limiting worse by burning more pull attempts.

Assuming unauthorized means the password is wrong. Nine times out of ten the credential is fine and the secret simply is not attached to the pod's ServiceAccount, or it lives in a different namespace. Check attachment before you rotate credentials.

Blaming the cluster for a toomanyrequests error. This one is Docker Hub, not Kubernetes. The message even says so. Teams spend an afternoon on RBAC and network policy for a problem that a free registry account fixes in two minutes. If your errors are on EKS and involve identity rather than images, that is a different problem covered in kubectl Unauthorized on EKS.

Setting imagePullPolicy: Never to skip the pull. That does not fetch the image from anywhere, it just requires the image to already be present on the node. On a fresh node it fails with ErrImageNeverPull, which is a worse dead end.

When it is still broken

• The Events are empty or stale. Run kubectl get events --field-selector involvedObject.name=<pod> --sort-by=.lastTimestamp. Events age out after an hour by default, so a long-stuck pod may have lost its original message. Delete and recreate once to get a fresh Event with the current reason.
• It pulls from your laptop but not from the node. That is Step 4 every time. Your laptop has different DNS, different credentials in ~/.docker/config.json, and a different egress IP. Reproduce with crictl pull on the node.
• Only some pods fail. Suspect a per-node problem: one node missing the registry CA, or one node's IP already rate-limited. Check which node the failing pods scheduled onto with kubectl get pod -o wide.
• Digest mismatch. If you pin by digest (@sha256:...) and the tag was re-pushed, the digest no longer resolves. Re-pin to the current digest.

The discipline that fixes this every time is boring and reliable: never guess, never restart, read the desc = string, and match it to the row in the table. The status lies by being generic; the Events tell the truth.