The SSL Certificate Chain Explained

Almost every confusing TLS bug — "it works in my browser but fails in curl", "our partner's server rejects our API", "the cert is valid but the app won't connect" — traces back to one misunderstood concept: the SSL certificate chain. A certificate is not trusted on its own. It's trusted because it links, step by step, up to a root that the client already trusts. When that link is broken or incomplete, things fail in ways that look random until you understand the chain. This post explains the three links and shows you how to debug the most common breakage.

The three links: leaf, intermediate, root

A working chain has three kinds of certificate:

• Leaf (end-entity) certificate. This is your certificate — the one for example.com, issued for your specific domain. It's what openssl s_client shows first.
• Intermediate certificate(s). Issued by the CA's root to a working sub-CA, then used to sign your leaf. There can be more than one. CAs use intermediates so the precious root key can stay offline and so they can rotate signing keys without reissuing every customer cert.
• Root certificate. The trust anchor. Root certs are not sent over the wire — they live in the client's trust store (the OS, the browser, or the language runtime). The client already has it, or it doesn't.

Validation works bottom-up: the client receives your leaf, sees it was signed by an intermediate, sees that intermediate was signed by a root it already trusts, and the chain is complete. Trust flows down from the root; verification climbs up from the leaf.

Root CA            (in the client's trust store — never sent)
  └─ Intermediate  (your server must send this)
       └─ Leaf     (your certificate — your server sends this)

The single most important rule: your server must send the leaf and every intermediate. It must not send the root (pointless — the client has it) and must not omit the intermediate (fatal — the client can't bridge the gap).

Why "works in my browser but fails in curl"

This is the canonical incomplete-chain symptom, and the cause is subtle. When a server forgets to send the intermediate, different clients behave differently:

• Browsers often paper over it. Many browsers cache intermediates they've seen before, and some can fetch a missing intermediate on the fly via the AIA (Authority Information Access) extension in the leaf certificate. So the site loads fine for you.
• curl, openssl, Go, Java, mobile apps and most server-to-server clients do not. They validate strictly against what the server actually sent. No intermediate, no trust, hard failure.

So the site "works" for the developer testing in Chrome and "fails" for the payment provider's Go service calling your API. The certificate is fine; the chain the server sends is incomplete. The error you'll see from the strict clients is typically unable to get local issuer certificate (openssl/curl) or x509: certificate signed by unknown authority (Go).

Debugging an incomplete chain

Start by looking at exactly what the server sends:

openssl s_client -connect example.com:443 -servername example.com -showcerts </dev/null

Count the certificates in the output. You should see your leaf plus at least one intermediate. If you only see the leaf, that's your bug. To verify the chain validates from the client's perspective:

echo | openssl s_client -connect example.com:443 -servername example.com 2>/dev/null \
  | openssl x509 -noout -subject -issuer

If the leaf's issuer is an intermediate CA but -showcerts shows no such certificate being sent, the intermediate is missing. The fix is on the server: concatenate the intermediate(s) below your leaf in the certificate file your server points at (this is what "fullchain.pem" means — leaf + intermediates, in order, leaf first). Most ACME clients produce a fullchain.pem for exactly this reason; the classic mistake is configuring the server to use cert.pem (leaf only) instead.

Other chain problems worth recognising:

Chains change under you

Even a perfectly configured chain isn't a set-and-forget affair. CAs rotate intermediates — Let's Encrypt and others periodically switch the intermediate they sign with — and when your cert renews, the intermediate in your fullchain.pem can change too. Root transitions (the kind that broke older Android and OpenSSL 1.0.x devices a few years back) are the same problem at the root level. A chain that validated last quarter can quietly break on the next renewal if the new intermediate isn't deployed alongside the new leaf.

That's why chain validation belongs in your monitoring, not just your initial setup checklist. Checking expiry alone misses the entire class of bugs above — a cert can be perfectly in-date and still untrusted because the chain it's served with is broken.

Monitor the chain, not just the date

DomainOps connects to each HTTPS endpoint you add and validates the whole chain the server actually presents — not just the leaf's expiry date — so an incomplete or broken chain is caught the same way a looming expiry is, and routed to email, Slack or Pushover. See how endpoint monitoring works, or add your first endpoint and see what your servers are really sending.