DevOps & Containers
47 articles in this topic.
ArgoCD Stuck on 'Waiting for Completion of Hook': Breaking the Deadlock
Your ArgoCD sync has waited on a PreSync Job for forty minutes and Terminate changes nothing. Here is why the hook never completes and the exact steps that clear it.
CrashLoopBackOff with no logs: a systematic way to find the real cause
CrashLoopBackOff is a retry policy, not a diagnosis. Four categories, two commands to tell them apart, and why three of the four are not application bugs at all.
ERR_REQUIRE_ESM on Vercel but not locally: the Next.js 16.2 monorepo bug
Your Next.js build passes and the Vercel deploy 500s with ERR_REQUIRE_ESM. Here is the missing .next/package.json that causes it, and three workarounds that hold.
Fixing Helm's "another operation is in progress" Without Nuking the Release
An interrupted helm upgrade leaves the release Secret stuck at pending-upgrade and blocks every deploy. Here is the safe unstick order, and why the label patch fails.
Helm 'has no deployed releases': recovering a broken release history
When every Helm revision is failed or pending, upgrade and rollback both refuse to run. Here is how the release Secrets work, how to repair the history safely, and how to stop it recurring.
Helm Upgrade Stuck in PENDING_UPGRADE After a Hook Timeout: The Cleanup Path
A pre-upgrade hook outran --timeout, and now Helm refuses every retry and shows two deployed revisions. Here is why the release state froze and how to clear it safely.
kubectl Unauthorized on EKS: Check the IAM Identity, Not RBAC
EKS returns Unauthorized when it cannot map your IAM principal to a Kubernetes user, so no ClusterRoleBinding will fix it. Here is the diagnosis order that actually resolves it.
Kubernetes DNS Timeouts That Come and Go: ndots, conntrack and the Five Second Stall
Intermittent i/o timeout errors on Kubernetes DNS are three different bugs wearing one shirt. How to tell ndots amplification, a kernel conntrack race and a starved CoreDNS apart.
Kubernetes Has No Cross-Namespace Secrets: Why Manual Copies Always Drift
A Pod cannot reference a Secret in another namespace and Kubernetes will not sync copies for you. Here is why, and the three mechanisms that actually hold up.
The Shared /health Endpoint That Kills Healthy Kubernetes Pods
Pointing liveness and readiness at one /health endpoint that checks your database turns a 30 second blip into a cluster-wide crash loop. Here is the correct split.
Kubernetes HPA Not Scaling: The Conditions You Are Not Reading
Your HPA reports an unknown target and never scales. kubectl describe hpa already told you why: how to read AbleToScale, ScalingActive and ScalingLimited, and fix each one.
JavaScript Heap Out of Memory on next build: Find the Real Cause
Raising --max-old-space-size hides the problem instead of fixing it. Three different build phases can exhaust the heap, and the last log line tells you which one you have.
Postgres 'Could Not Resize Shared Memory Segment' Is Not Your Disk
No space left on device from Postgres in Docker means a 64MB /dev/shm, not a full volume. Here is the mechanism, the shm-size fix, and how to size it without inviting the OOM killer.
Pod Has Unbound PersistentVolumeClaims: The Real Checklist
One Kubernetes error covers four unrelated storage failures. Here is how to tell them apart from the PVC events, and why deleting the PVC can destroy your data.
Error Acquiring the State Lock: When Terraform force-unlock Is Safe
A stale Terraform lock blocks every apply. How to read the Lock Info block, decide in 90 seconds whether force-unlock is safe, and find the resources a killed apply orphaned.
ArgoCD StatefulSet Stuck OutOfSync With ServerSideApply: Fix the Diff, Not the Manifest
ArgoCD reports permanent drift on a StatefulSet after you enable ServerSideApply. Every field in the diff is an API server default, not your change. Here is the real fix.
CI Auto-Retry Is Hiding Your Flaky Tests, Not Fixing Them
Retry-on-failure keeps CI green by throwing away the one thing you need to diagnose flakiness. Here is how to make flakes visible, quarantine them, and root-cause the real nondeterminism.
Docker Compose Postgres connection refused: localhost is not your database
Inside a container, localhost means that container. Why your app cannot reach the Postgres service, the service-name fix, and the healthcheck that stops the race condition.
Why GitHub Actions Cache Never Hits: The Wrong-File Hash and Fork-PR Traps
Your cache key looks correct but every run is a miss. The usual culprits are a hash that changes every run and forked PRs that can restore but never save.
The GitHub Actions Pwn Request: How Fork PRs Steal Your Secrets
pull_request_target plus a fork checkout hands your CI secrets to any attacker. Here is the mechanism, the tj-actions lesson, and the two-workflow fix.
x509 Certificate Signed by Unknown Authority on the ingress-nginx Admission Webhook: The Real Fix
An x509 trust error on the ingress-nginx or cert-manager admission webhook blocks every Ingress cluster-wide. The cause is a drifted caBundle. Patch it, do not delete the webhook.
kubectl 'did you specify the right host or port?': read the address first
localhost:8080 means kubectl found no kubeconfig at all. A real address means a real network failure. How to tell them apart in ten seconds and fix each one properly.
Why Kubernetes Pods and Namespaces Get Stuck Terminating (and When Force-Delete Makes It Worse)
A pod or namespace stuck in Terminating is almost never fixed by --force. Here is what the finalizers are actually waiting for, and the finalizer surgery to do only after you have checked.
OOMKilled and Nothing Else: Finding What Really Ate Your Pod
Kubernetes records OOMKilled and exit code 137, then deletes the evidence. Here is how to read cgroup counters, kernel logs and working set to find the actual cause.
Your kubectl edit Keeps Getting Reverted in GitOps: The Right Way to Hotfix
ArgoCD self-heal and Flux reconciliation undo manual kubectl edits within minutes, so the incident looks like it came back. Here is the correct emergency runbook that does not disable GitOps cluster-wide.
Kubernetes Rolling Updates Still Return Connection Refused: The preStop Fix
Your rollout shows zero unavailable replicas and still drops a burst of connection refused errors. The SIGTERM-versus-endpoint-removal race, and the preStop hook that closes it.
NEXT_PUBLIC_ Vars Are Baked at Build Time: Why One Docker Image Can't Serve Staging and Prod
Your prod container still points at the staging API because NEXT_PUBLIC_ vars are string-replaced into the client bundle during next build, not read at container start. The two fixes that actually work.
Next.js 'Found multiple lockfiles': a silent monorepo footgun, not just noise
Next.js picks your workspace root by walking up for a lockfile. A stray one in a Docker layer or template silently changes which files ship. Pin the root explicitly.
FATAL: role "postgres" does not exist in Docker: the POSTGRES_USER volume trap
The Postgres Docker image only reads POSTGRES_USER the first time it initializes an empty volume. Change it later and you get a role error that looks like a permissions bug but isn't.
Error loading ASGI app: why uvicorn cannot import your FastAPI module
Uvicorn resolves 'module:app' against sys.path, not your file's location. The real cause, the right command for flat, package, src and Docker layouts, and how to verify it.
Your Logs, Metrics, and Traces Can't Talk: Fix Correlation Before the Next Incident
During an incident you should click from a metric spike to the exact traces and logs behind it, not squint at three clocks. The fix is one shared trace ID on every signal, built in before you need it.
FastAPI Memory Leaks in Production: A Root-Cause Checklist
FastAPI workers whose memory climbs to OOM usually are not leaking, they are blocking the event loop or queuing unbounded work. A checklist to find the real cause before blaming the framework.
Why Your FastAPI Logs Vanish Under Gunicorn and Uvicorn (and the Config That Works Everywhere)
FastAPI logs work in local dev but disappear in Docker under gunicorn+uvicorn, especially below WARNING. The cause is disable_existing_loggers and uvicorn's own loggers. Here is the fix.
Kubernetes Secrets Are base64, Not Encrypted: What Actually Protects Them
A Kubernetes Secret is base64-encoded plaintext, not a vault. Here is what actually protects it: encryption at rest, tight RBAC, and an external secrets manager.
Kubernetes Won't Restart Pods When a ConfigMap Changes: The checksum/config Pattern
Kubernetes never restarts pods when a ConfigMap or Secret changes, and env-var config never updates at all. Here is why that is deliberate and the checksum/config annotation that forces a rollout.
'0/3 nodes are available: Insufficient memory' - requests, not usage
Your nodes look idle in Grafana but the scheduler says Insufficient memory. That is because it counts requests, not usage - and taints and volume affinity hide behind the same message.
Monorepo CI in GitHub Actions: Why Path Filters Break When Services Share Code
Workflow-level paths filters gate whole workflows, not jobs, and know nothing about shared code. Here is the detect-changes pattern that actually works.
Certificates for a Multi-Server Cluster: Issue Once, Distribute Everywhere
Running certbot on every node for the same domain burns your Let's Encrypt rate limit and splits your cluster's identity. Issue once with DNS-01 and distribute instead.
'No module named pydantic_core._pydantic_core' in Docker or Lambda Is an Architecture Mismatch
This error is not a missing package. pydantic-core ships a compiled Rust wheel, and installing it for the wrong CPU or glibc means the files exist but cannot load.
OOMKilled Exit Code 137: Leak, Bad Limit, or JVM Off-Heap?
Exit code 137 looks the same whether it is a leak, a limit set too low, or JVM off-heap memory. Here is how to read the memory curve and tell the three apart.
Prometheus Cardinality Explosion: Find the Label Eating Your RAM
Prometheus is OOMing and queries time out with no config change to blame. One unbounded label is minting millions of series. Here is how to find and drop it.
One Bad Request Killed Your Whole vLLM Server: AsyncEngineDeadError Explained
AsyncEngineDeadError means one request crashed vLLM's shared background loop and took every other request down with it. Here is the architecture behind it and the mitigations that actually contain it.
Alert Fatigue Is a Discipline Problem, Not a Tooling Problem
Noisy on-call is not fixed by a better tool. It is fixed by deleting alerts that map to no action, tuning thresholds to real baselines, and correlating symptoms into one incident.
Flaky CI Tests From Live Third-Party APIs: Mock the Boundary, Don't Rerun
Retrying a flaky test that calls a live payment or SMS API makes CI slower and less honest. Mock the boundary, verify the real integration on a schedule, and track flake rate.
ImagePullBackOff in Kubernetes: The Five Real Causes and How to Tell Them Apart
ImagePullBackOff is one status hiding five unrelated causes. Read the Events reason string, match it to the cause, and stop restarting the pod.
'Too Many Open Files' Won't Go Away? You Are Editing the Wrong Limit
Raising limits.conf does nothing for a systemd service. Here is where the file-descriptor limit really lives, how to set LimitNOFILE, and how to prove it reached the process.
What to Log So 3am-You Can Actually Debug It
Generic logging advice never shows the one line that ends an incident. A real failure walked through, the five fields that turn 'operation failed' into a diagnosis, and a PR checklist.