Let's Encrypt Renewal Fails Behind Cloudflare Proxy: Use DNS-01, Not http-01

Your renewal cron ran overnight and this morning the certificate is still the old one, three days from expiry. You SSH in, run Certbot by hand, and it sits on the challenge for a while before telling you the validation failed. Nothing about your web server changed. The only thing you did last week was flip Cloudflare's proxy on for that hostname, because the site was getting hammered.

That orange cloud is the whole story. When Cloudflare proxies a hostname, requests to port 80 and 443 no longer reach your origin directly. Cloudflare terminates the connection at its edge, does whatever it does, and re-proxies to you. An http-01 challenge depends on Let's Encrypt fetching a file from your origin over that exact path, and now there is a reverse proxy sitting in the middle rewriting, caching and occasionally blocking it.

The advice you will find most often is "just turn the proxy off". That works, and it also throws away the DDoS protection and CDN you turned the proxy on for in the first place. You do not have to choose. Switch the challenge type to DNS-01 and Certbot renews happily with the proxy left fully on, because DNS validation never touches port 80 at all.

Why http-01 is unreliable behind the orange cloud

The http-01 challenge is a deal between your ACME client and Let's Encrypt. Certbot writes a token file under /.well-known/acme-challenge/ on your origin, tells Let's Encrypt "go fetch it", and Let's Encrypt makes an HTTP request to http://yourdomain/.well-known/acme-challenge/<token> from its own validation servers. If the file it gets back matches, you have proven you control the host.

Now put Cloudflare's proxy in the path. That request from Let's Encrypt does not arrive at your origin. It arrives at Cloudflare's edge, and several things can go wrong before it ever reaches you:

• Always Use HTTPS turns the plain port 80 request into a 301 redirect to HTTPS at the edge. Some ACME validation paths follow it, but you have now added a hop that can fail on its own.
• Caching rules or a "Under Attack" mode can intercept, challenge or serve a stale response for the /.well-known/ path.
• A WAF or bot-fight rule sees an unfamiliar client hitting an odd path and returns a 403 or a JS challenge, which is not the token file.
• The edge terminates TLS with Cloudflare's certificate, not yours, so the request that reaches your origin is a re-proxied one and any origin-side assumption about the client IP or the raw request is now wrong.

The maddening part is that it often works. The token file is small and static, so on a good day it sails through the cache and validates. On a bad day, or after you tighten a firewall rule, it silently stops. Intermittent renewal failures behind Cloudflare are almost always this. If you are seeing a hard connection error rather than a validation mismatch, that is a different diagnosis and my write-up on Certbot connection refused on http-01 walks through the port-80 and IPv6 causes.

Why DNS-01 does not care about the proxy

DNS-01 proves control of a domain in a completely different way. Instead of fetching a file, Let's Encrypt asks you to publish a specific TXT record at _acme-challenge.yourdomain. It then does a DNS lookup for that record and checks the value. There is no HTTP request to your origin, so Cloudflare's proxy, its cache, its WAF and its redirect rules are all out of the loop. The record lives in DNS, and Cloudflare is your DNS provider anyway, which is the neat part: the same account that runs the proxy also exposes an API to write that TXT record automatically.

DNS-01 is also the only way to get a wildcard certificate from Let's Encrypt. If you have ever wanted *.yourdomain, this is the path you were going to end up on regardless.

The fix: DNS-01 with a scoped Cloudflare token

Step 1: Create a scoped API token, not the global key

In the Cloudflare dashboard go to My Profile, API Tokens, Create Token. Do not use the Global API Key; it can do anything to your account. Create a token with exactly one permission: Zone - DNS - Edit, scoped to the specific zone or zones you need certificates for.

Older guides also told you to add Zone - Zone - Read so the plugin can look up the zone ID. With a token scoped to specific zones you generally do not need it, but if issuance fails with a zone-lookup error, add Zone:Read and try again. Copy the token value once; Cloudflare will not show it to you a second time.

Step 2: Install the Cloudflare DNS plugin

sudo apt update
sudo apt install python3-certbot-dns-cloudflare

If you installed Certbot through snap, the plugin ships with it and you enable it instead:

sudo snap set certbot trust-plugin-with-root=ok
sudo snap install certbot-dns-cloudflare

Step 3: Store the token in a locked-down credentials file

Create /root/.secrets/cloudflare.ini with the token only. The API token form uses one line:

# /root/.secrets/cloudflare.ini
dns_cloudflare_api_token = your_scoped_token_here

This file is a credential. Lock it down or Certbot will warn you that it is world-readable:

sudo chmod 600 /root/.secrets/cloudflare.ini

Step 4: Issue the certificate over DNS-01

sudo certbot certonly \
  --dns-cloudflare \
  --dns-cloudflare-credentials /root/.secrets/cloudflare.ini \
  -d yourdomain.com -d www.yourdomain.com

Certbot writes the TXT record through the API, waits for it to propagate, tells Let's Encrypt to validate, and cleans the record up afterwards. You should see it report the certificate saved under /etc/letsencrypt/live/yourdomain.com/ with the expiry date. For a wildcard, use -d yourdomain.com -d '*.yourdomain.com'.

If the DNS write is racing propagation on a slow zone, add --dns-cloudflare-propagation-seconds 30 to give the record time to appear on the authoritative servers before validation.

Step 5 (acme.sh alternative)

If you run acme.sh rather than Certbot, the same idea uses the dns_cf mode. Export the token and account ID, then issue:

export CF_Token="your_scoped_token_here"
export CF_Account_ID="your_account_id"
acme.sh --issue --dns dns_cf -d yourdomain.com -d www.yourdomain.com

acme.sh stores those values in its account config after the first run, so scheduled renewals pick them up without you re-exporting anything.

Verifying the renewal path actually works

Do not wait 60 days to find out. Force a dry-run renewal, which runs the whole challenge against the Let's Encrypt staging servers without touching your rate limits or issuing a real certificate:

sudo certbot renew --dry-run

You want to see Congratulations, all simulated renewals succeeded and, crucially, the log line showing it used the dns-cloudflare authenticator rather than falling back to a standalone or webroot http-01. If a certificate was originally issued with http-01, its renewal config still says http-01, and renew keeps using it. Re-issue it once with the DNS flags above to rewrite the renewal config, or edit the authenticator line in /etc/letsencrypt/renewal/yourdomain.com.conf.

Confirm the live certificate is the freshly issued one and note the expiry:

sudo certbot certificates

While you are here, make sure a silent renewal failure would actually reach you. A cert that stops renewing is invisible until the browser errors appear; I covered building an alert for exactly that in Certbot renewal fails silently.

What people get wrong

Turning the proxy off permanently. This is the top answer in most threads and it solves the wrong problem. You enabled the orange cloud for DDoS absorption, a hidden origin IP and CDN caching. Greying it out forever to make a 90-day renewal succeed is paying a large recurring cost to avoid a one-time config change. DNS-01 keeps the proxy on.

Grey-clouding only during renewal, by hand. Temporarily switching the challenge hostname to DNS-only during the renewal window and back afterwards does work, and it is a reasonable fallback if you genuinely cannot use the API. But doing it manually means a human has to remember every 60 days, and the one time they forget is the time it expires. If you must automate it, you are already scripting against the Cloudflare API, at which point DNS-01 is less code and never exposes the origin.

Confusing Cloudflare Origin CA certificates with Let's Encrypt. Cloudflare offers its own Origin CA certificates, valid up to 15 years, that you install on your origin so Cloudflare can talk to it over TLS. They are only trusted by Cloudflare, not by browsers. They are excellent for the edge-to-origin hop and useless as a public certificate. If a visitor ever hits your origin directly, or you set SSL mode to anything other than Full, an Origin CA cert alone will throw trust errors. It is a complement to a publicly trusted cert, not a replacement, and mixing the two up is behind a lot of "why is my site untrusted" confusion.

Using the Global API Key in the credentials file. It works, and it hands root-of-account access to any process that can read that file. Use a token scoped to Zone:DNS:Edit on one zone. If it leaks, the blast radius is one DNS record type on one domain, not your entire Cloudflare account.

When it is still broken

• Validation fails with a zone-lookup error. Your token cannot see the zone. Add Zone:Read to the token's permissions, or confirm the token's zone scope actually includes the domain you are issuing for.
• The TXT record is written but validation times out. Propagation is racing you. Add --dns-cloudflare-propagation-seconds 60. If you have DNSSEC or a secondary DNS provider, the record has to appear on every authoritative server before Let's Encrypt is satisfied.
• It works by hand but the cron renewal still uses http-01. The renewal config was written when the cert was first issued. Check the authenticator line in /etc/letsencrypt/renewal/yourdomain.com.conf and re-issue with the DNS flags to rewrite it.
• You hit a rate limit while testing. Always test with --dry-run against staging. Real issuance is capped, and my note on "too many certificates already issued" covers what the limits are and how long they take to clear.

The general lesson is worth keeping: when a validation depends on a network path, and you put a proxy in that path, you own every failure mode the proxy can produce. Move the proof somewhere the proxy cannot interfere, and the whole class of problem disappears. For DNS-01, that somewhere is the DNS record itself.