TLS and certificates
Inspect and verify certificate chains with OpenSSL, issue keys and CSRs, install trust anchors, configure servers and diagnose failed TLS handshakes.
On this page
Cheatsheet#
| Task | Command |
|---|---|
| Inspect a live certificate | openssl s_client -connect host.example.com:443 -servername host.example.com </dev/null | openssl x509 -noout -text |
| Expiry dates | openssl x509 -in cert.pem -noout -dates |
| Subject and SANs | openssl x509 -in cert.pem -noout -subject -ext subjectAltName |
| Verify a chain offline | openssl verify -CAfile root.pem -untrusted intermediate.pem cert.pem |
| Does the key match the certificate | diff <(openssl pkey -in key.pem -pubout) <(openssl x509 -in cert.pem -noout -pubkey) |
| Read a CSR | openssl req -in req.csr -noout -text -verify |
| New key and CSR | openssl req -new -newkey rsa:2048 -noenc -keyout key.pem -out req.csr -config san.cnf |
| Self-signed for testing | openssl req -x509 -newkey ed25519 -noenc -days 30 -keyout key.pem -out cert.pem -subj '/CN=localhost' -addext 'subjectAltName=DNS:localhost' |
| PEM to PKCS#12 | openssl pkcs12 -export -in cert.pem -inkey key.pem -certfile chain.pem -out bundle.p12 |
| PKCS#12 to PEM | openssl pkcs12 -in bundle.p12 -noenc -out all.pem |
| Show the chain the server sends | openssl s_client -connect host.example.com:443 -servername host.example.com -showcerts </dev/null |
| Test one protocol version | openssl s_client -connect host.example.com:443 -tls1_2 </dev/null |
| SHA-256 fingerprint | openssl x509 -in cert.pem -noout -fingerprint -sha256 |
Commands target OpenSSL 3.x. -noenc replaced -nodes in 3.0; -nodes still works but is deprecated. Run openssl version first when output differs from what is shown here.
The chain of trust#
A certificate binds a public key to one or more names and is signed by an issuer. The client builds a path from the server’s leaf certificate through intermediates to a root in its own trust store. At each step it checks the signature, the validity dates, the key usage and CA constraints; for the leaf it also checks that a Subject Alternative Name matches the hostname it asked for.
Three things fail independently:
- Trust: the root is not in this client’s store (private CA, or an old store in a container image).
- Chain completeness: the server did not send the intermediates. Clients do not fetch missing intermediates reliably; some browsers do,
curl, Go and Python do not. - Name match: no SAN covers the requested hostname. Clients ignore the subject CN when SANs are present, and current browsers ignore CN entirely.
# The chain as sent, with the verification result at the end
openssl s_client -connect api.example.com:443 -servername api.example.com -showcerts </dev/null
# Leaf subject, issuer, dates and SANs
openssl s_client -connect api.example.com:443 -servername api.example.com </dev/null 2>/dev/null \
| openssl x509 -noout -subject -issuer -dates -ext subjectAltNameVerify return code: 0 (ok)Anything other than 0 (ok) on the last line of s_client output names the failure. -servername sets SNI (OpenSSL 1.1.1 and later send the host from -connect by default, but an IP in -connect sends none). Without the right SNI a server with several sites returns its default certificate, which looks like a name mismatch that real clients never see.
| Symptom | Cause |
|---|---|
unable to get local issuer certificate (code 20) | Server sent an incomplete chain, or the root is not in this trust store |
certificate has expired (code 10) | Leaf or an intermediate is past notAfter; check every certificate in the chain |
Hostname mismatch / no alternative certificate subject name matches | No SAN covers the name, or wrong SNI |
self-signed certificate in certificate chain (code 19) or self-signed certificate (code 18) | A private CA or a self-signed leaf is in use and not trusted here |
tlsv1 alert unknown ca | The peer rejected the certificate you sent, usually the client certificate in mTLS |
no shared cipher / handshake failure alert | No protocol, cipher or group in common, or a key type the offered suites cannot use |
wrong version number | The port does not speak TLS (plain HTTP, or STARTTLS needed) |
| Works in a browser, fails in curl | Missing intermediate that the browser had cached or fetched |
Reading certificates#
openssl x509 -in cert.pem -noout -text # everything
openssl x509 -in cert.pem -noout -subject -issuer -dates
openssl x509 -in cert.pem -noout -ext subjectAltName,keyUsage,extendedKeyUsage,basicConstraints
openssl x509 -in cert.pem -noout -fingerprint -sha256
openssl x509 -in cert.pem -noout -checkend 604800 # exit 1 if it expires within 7 days
openssl crl2pkcs7 -nocrl -certfile chain.pem | openssl pkcs7 -print_certs -noout # subject and issuer of each cert in a bundleThe fields that matter operationally: Not After, Subject Alternative Name, Key Usage and Extended Key Usage (a server certificate needs serverAuth), and Basic Constraints (a CA certificate has CA:TRUE, a leaf has CA:FALSE).
Public certificate lifetimes are getting shorter. Under CA/Browser Forum ballot SC-081 the maximum is 200 days from 15 March 2026, 100 days from 15 March 2027 and 47 days from 15 March 2029. Manual renewal does not scale to that; automate with ACME.
Keys and CSRs#
openssl genpkey -algorithm EC -pkeyopt ec_paramgen_curve:P-256 -out key.pem # widely supported, small, fast
openssl genpkey -algorithm RSA -pkeyopt rsa_keygen_bits:2048 -out key.pem # maximum compatibility
openssl genpkey -algorithm ed25519 -out key.pem # fine for private use; public CAs generally do not issue Ed25519 server certificates
chmod 600 key.pem
openssl req -new -key key.pem -out req.csr -config san.cnf
openssl req -in req.csr -noout -text -verify # confirm SANs before sending# san.cnf: the SAN list is what clients check
[req]
distinguished_name = dn
req_extensions = v3_req
prompt = no
[dn]
CN = api.example.com
O = Example Pty Ltd
C = AU
[v3_req]
keyUsage = critical, digitalSignature, keyEncipherment
extendedKeyUsage = serverAuth
subjectAltName = @alt
[alt]
DNS.1 = api.example.com
DNS.2 = api.internal.example.com
IP.1 = 192.0.2.10Confirm a key and certificate belong together before deploying:
diff <(openssl pkey -in key.pem -pubout) <(openssl x509 -in cert.pem -noout -pubkey) && echo matchA private CA#
For internal services, issue from one CA and distribute its root, rather than self-signing each host. Per-host self-signed certificates must each be trusted individually and cannot be replaced as a set.
# Root CA: long-lived, key kept offline
openssl req -x509 -new -newkey rsa:4096 -noenc -days 3650 -keyout ca.key -out ca.pem \
-subj '/CN=Example Internal Root CA/O=Example Pty Ltd/C=AU' \
-addext 'basicConstraints=critical,CA:TRUE,pathlen:1' \
-addext 'keyUsage=critical,keyCertSign,cRLSign'
# Issue a leaf from a CSR, applying the extensions from san.cnf
openssl x509 -req -in req.csr -CA ca.pem -CAkey ca.key -CAcreateserial -days 90 \
-extfile san.cnf -extensions v3_req -out cert.pem
openssl verify -CAfile ca.pem cert.pemcert.pem: OKWith an intermediate CA, serve the leaf followed by the intermediates and leave out the root:
cat cert.pem intermediate.pem > fullchain.pemClients already hold the root, so sending it only adds bytes to every handshake. Some strict clients reject chains in the wrong order. For a managed internal CA with short-lived certificates, see Vault or IdM.
Trust stores#
Each runtime can have its own store. Adding a CA to the operating system does not always reach the application.
| Platform | Where to add a CA | Command |
|---|---|---|
| Fedora / RHEL | /etc/pki/ca-trust/source/anchors/ | update-ca-trust (root) |
| Debian / Ubuntu | /usr/local/share/ca-certificates/*.crt (must end .crt) | update-ca-certificates (root) |
| Alpine | /usr/local/share/ca-certificates/ | update-ca-certificates (needs the ca-certificates package) |
| curl / OpenSSL | System bundle | SSL_CERT_FILE, --cacert |
Python requests | certifi bundle, not the system store | REQUESTS_CA_BUNDLE |
| Node.js | Bundled Mozilla list | NODE_EXTRA_CA_CERTS |
| Java | cacerts keystore in the JDK | keytool -importcert -cacerts -alias my-ca -file ca.pem |
| Go | System store on Linux | SSL_CERT_FILE or SSL_CERT_DIR |
In a container the store must be in the image. Copy the CA into the distribution’s anchor directory and run its update command in the image build. x509: certificate signed by unknown authority from a scratch or distroless image with no CA bundle means there is no store at all.
Formats#
| Format | Contents | Typical extension |
|---|---|---|
| PEM | Base64 between -----BEGIN ...----- lines; can hold several objects | .pem, .crt, .key |
| DER | Binary encoding of one object | .der, .cer |
| PKCS#12 | Password-protected bundle of key and certificates | .p12, .pfx |
| PKCS#8 | Standard private key container (BEGIN PRIVATE KEY) | .key |
| PKCS#1 | RSA-only legacy key format (BEGIN RSA PRIVATE KEY) | .key |
| JKS | Legacy Java keystore; Java 9 and later default to PKCS#12 | .jks |
openssl x509 -in cert.der -inform der -out cert.pem # DER to PEM
openssl x509 -in cert.pem -outform der -out cert.der # PEM to DER
openssl pkcs12 -export -in cert.pem -inkey key.pem -certfile chain.pem -out bundle.p12
openssl pkcs12 -export -legacy -in cert.pem -inkey key.pem -out old.p12 # 3DES/RC2 for old Java or Windows that cannot read the AES default
openssl pkcs12 -in bundle.p12 -noenc -out all.pem # everything, key unencrypted
openssl rsa -in key.pem -traditional -out key.pkcs1.pem # PKCS#8 to PKCS#1 for old software
keytool -importkeystore -srckeystore bundle.p12 -srcstoretype PKCS12 -destkeystore store.jks -deststoretype JKSIn OpenSSL 3.0 and later openssl rsa writes PKCS#8 unless you pass -traditional, and pkcs12 -export encrypts with AES-256 by default.
Testing a server#
openssl s_client -connect host.example.com:443 -servername host.example.com </dev/null # handshake detail and verify result
openssl s_client -connect host.example.com:443 -tls1_2 </dev/null # force TLS 1.2
openssl s_client -connect host.example.com:443 -tls1_2 -cipher 'ECDHE-RSA-AES128-GCM-SHA256' </dev/null # one TLS 1.2 suite
openssl s_client -connect host.example.com:443 -cert client.pem -key client.key </dev/null # mTLS
openssl s_client -connect host.example.com:443 -status </dev/null | grep -A5 'OCSP' # stapled OCSP response, if any
openssl s_client -connect host.example.com:25 -starttls smtp </dev/null # STARTTLS
openssl s_time -connect host.example.com:443 -new -time 10 # full handshakes per second
nmap --script ssl-enum-ciphers -p 443 host.example.com # every protocol and suite the server accepts-cipher sets TLS 1.2 and older suites only; TLS 1.3 suites are set with -ciphersuites. Look for Server Temp Key or Negotiated TLS1.3 group in the output to see the key exchange group, for example X25519MLKEM768 for the hybrid post-quantum exchange (OpenSSL 3.5 and later).
For HTTP-level checks through TLS, see HTTP and curl.
Cipher suites and security levels#
A TLS 1.2 cipher suite names four things: key exchange (ECDHE), authentication (RSA or ECDSA, decided by the certificate’s key type), bulk cipher (AES128-GCM, CHACHA20-POLY1305) and the PRF hash. TLS 1.3 fixed key exchange and authentication outside the suite, so its five suites name only the cipher and hash (TLS_AES_128_GCM_SHA256, TLS_AES_256_GCM_SHA384, TLS_CHACHA20_POLY1305_SHA256 and the two CCM variants). A server with an ECDSA certificate can never negotiate an ECDHE-RSA-* suite, which is the usual reason a hand-written ssl_ciphers list produces no shared cipher after switching key types.
openssl ciphers -v 'ECDHE+AESGCM:ECDHE+CHACHA20' # expand a cipher string into the TLS 1.2 suites it selects
openssl ciphers -v -s -tls1_3 # -s: only suites usable with the current security level and protocol
openssl ciphers -v 'DEFAULT:@SECLEVEL=2' | wc -l
openssl ciphers -stdname -v 'ECDHE-RSA-AES128-GCM-SHA256' # IANA name alongside the OpenSSL name
openssl list -tls-groups # key exchange groups this build supports (OpenSSL 3.5+)
openssl list -signature-algorithmsOpenSSL’s security level (@SECLEVEL=n, default 1 in a plain build, 2 on Fedora, RHEL and Debian) rejects weak keys and algorithms regardless of the cipher string: level 2 refuses RSA and DH under 2048 bits, SHA-1 signatures and TLS below 1.2. When s_client fails against an old appliance with dh key too small or ca md too weak, the refusal is local. Test with -cipher 'DEFAULT:@SECLEVEL=0' to prove it, then fix the appliance, because every modern client will refuse it too. The distribution-wide policy on Fedora and RHEL is update-crypto-policies --show (DEFAULT, FUTURE, LEGACY); LEGACY is the only way to talk TLS 1.0 from those systems and applies to every program.
Cipher strings compose with : (add), ! (remove permanently), - (remove but allow later re-add) and + (move to the end). ECDHE+AESGCM means suites having both properties. The keywords HIGH, MEDIUM, kRSA and aNULL still exist but are not a sensible way to write a policy; name the suites you want, as the Mozilla generator does. For nginx, Apache and HAProxy the TLS 1.3 list is set separately (ssl_conf_command Ciphersuites, SSLOpenSSLConfCmd Ciphersuites, ssl-default-bind-ciphersuites) and the defaults are fine.
Server configuration#
Follow the Mozilla server-side TLS guidelines and generate snippets with the Mozilla configuration generator. The “intermediate” profile (guidelines version 6.0) allows TLS 1.2 and 1.3, ECDHE key exchange with AES-GCM or ChaCha20-Poly1305, and no finite-field DHE. TLS 1.3 defines five cipher suites; the three general-purpose ones are enabled by default in OpenSSL, and the two CCM suites are not.
ssl_protocols TLSv1.2 TLSv1.3;
ssl_ecdh_curve X25519MLKEM768:X25519:prime256v1:secp384r1; # X25519MLKEM768 needs OpenSSL 3.5+
ssl_ciphers ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305;
ssl_prefer_server_ciphers off;
ssl_session_timeout 1d;
add_header Strict-Transport-Security "max-age=63072000" always;OCSP stapling only helps with certificates that contain an OCSP URL. Let’s Encrypt removed OCSP URLs from new certificates in May 2025 and shut down its OCSP responders in August 2025, publishing revocation through CRLs only. With those certificates ssl_stapling on does nothing, and nginx logs a warning.
Warning
HSTS is hard to undo. Browsers enforce HTTPS for the full max-age after they last saw the header, even if you remove it. Start with a short max-age and add includeSubDomains only when every subdomain serves valid HTTPS.
Mutual TLS#
In mTLS the server also requests a certificate from the client and verifies it against a CA bundle of its own choosing, which need not be the CA that issued the server’s certificate. Authorisation is then a policy on the client certificate’s subject or SANs. The server sends the list of acceptable CA names in the CertificateRequest message; a client that has no certificate matching them sends an empty certificate, and the server answers with certificate required (TLS 1.3) or handshake failure.
# Issue a client certificate from the private CA: clientAuth EKU, no serverAuth
openssl req -new -newkey ec -pkeyopt ec_paramgen_curve:P-256 -noenc -keyout client.key -out client.csr -subj '/CN=deploy-bot/O=Example Pty Ltd'
openssl x509 -req -in client.csr -CA ca.pem -CAkey ca.key -CAcreateserial -days 90 -out client.pem \
-extfile <(printf 'keyUsage=critical,digitalSignature\nextendedKeyUsage=clientAuth\nsubjectAltName=URI:spiffe://example.com/ns/my-namespace/sa/my-app')
# Which CAs the server will accept client certificates from
openssl s_client -connect api.example.com:8443 -servername api.example.com </dev/null 2>/dev/null | sed -n '/Acceptable client certificate CA names/,/---/p'
# Test as the client; -verify_return_error makes s_client exit non-zero instead of continuing
openssl s_client -connect api.example.com:8443 -servername api.example.com -cert client.pem -key client.key -CAfile ca.pem -verify_return_error </dev/null
curl --cert client.pem --key client.key --cacert ca.pem https://api.example.com:8443/whoamissl_client_certificate /etc/nginx/client-ca.pem; # CA bundle for client certificates; also decides the names advertised in CertificateRequest
ssl_verify_client on; # optional or optional_no_ca to let the application decide
ssl_verify_depth 2;
proxy_set_header X-Client-DN $ssl_client_s_dn; # $ssl_client_verify is SUCCESS, FAILED:reason or NONERevocation is the operational cost of mTLS: with 90-day client certificates a CRL still matters for a stolen laptop. nginx takes ssl_crl; Envoy and most service meshes rotate certificates hourly and treat expiry as revocation. Client keys belong in the same handling as server keys: not in a repository, not in an environment variable that ends up in a crash dump.
ACME and certbot#
ACME automates the domain-validation dance: the client proves control of each name (http-01 by serving a token on port 80, dns-01 by publishing a TXT record at _acme-challenge.<name>, tls-alpn-01 on port 443), then the CA issues. dns-01 is the only challenge that can issue wildcards and the only one that works for hosts unreachable from the internet. Let’s Encrypt rate limits are per registered domain (50 certificates a week) and per exact name set (5 duplicates a week); use the staging directory for anything experimental.
certbot certonly --webroot -w /var/www/html -d www.example.com -d example.com -m ops@example.com --agree-tos --key-type ecdsa
certbot certonly --standalone --preferred-challenges http -d api.example.com # needs port 80 free; stop the web server or use --pre-hook/--post-hook
certbot certonly --dns-cloudflare --dns-cloudflare-credentials /root/.secrets/cloudflare.ini -d '*.example.com' -d example.com
certbot --nginx -d www.example.com # obtains and edits the nginx server block
certbot certonly --server https://acme-staging-v02.api.letsencrypt.org/directory --dry-run -d test.example.com
certbot certificates # every managed lineage, expiry and paths
certbot renew --dry-run # test every renewal against staging
certbot renew --deploy-hook 'systemctl reload nginx' # runs only for lineages that actually renewed
certbot revoke --cert-name www.example.com --reason keycompromise
certbot delete --cert-name old.example.com
systemctl list-timers 'certbot*' 'snap.certbot*' # renewal timer, twice daily; certbot only renews when under 30 days remainCertificates land in /etc/letsencrypt/live/<name>/ as symlinks (fullchain.pem, privkey.pem, chain.pem, cert.pem) into archive/; point the server at live/ and never copy the files elsewhere, or renewal silently stops reaching it. Options given at first issuance are saved in /etc/letsencrypt/renewal/<name>.conf and reused by renew; a --deploy-hook supplied once is saved there too. Other ACME clients worth knowing: acme.sh (shell, many DNS providers), lego (Go, single binary), and the built-in ACME in Caddy and Traefik, which need no client at all.
cert-manager#
cert-manager runs ACME (and private CA, Vault and Venafi issuers) inside Kubernetes. A Certificate resource declares the names and target Secret; the controller creates a CertificateRequest, drives the Order and Challenge resources for ACME, writes tls.crt, tls.key and ca.crt into the Secret and renews at renewBefore. Since v1.18 the private key is rotated on every renewal by default (rotationPolicy: Always).
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
name: letsencrypt
spec:
acme:
server: https://acme-v02.api.letsencrypt.org/directory
email: ops@example.com
privateKeySecretRef:
name: letsencrypt-account-key # ACME account key, created by cert-manager
solvers:
- http01:
ingress:
ingressClassName: nginx # a temporary Ingress serves the token
- dns01:
route53:
region: ap-southeast-2 # credentials via IRSA or accessKeyID/secretAccessKeySecretRef
selector:
dnsZones: [example.com] # use dns01 for these names; needed for wildcards
---
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
name: my-app
namespace: my-namespace
spec:
secretName: my-app-tls
dnsNames: [my-app.example.com]
duration: 2160h # 90d; ignored by Let's Encrypt, which sets its own
renewBefore: 720h # 30d
privateKey:
algorithm: ECDSA
size: 256
issuerRef:
name: letsencrypt
kind: ClusterIssuerkubectl get certificate -A # READY False means look further down the chain
kubectl describe certificate my-app -n my-namespace # Events say which CertificateRequest it is waiting on
kubectl get certificaterequest,order,challenge -n my-namespace
kubectl describe challenge -n my-namespace # the reason the CA could not validate: DNS not propagated, Ingress unreachable, CAA
cmctl status certificate my-app -n my-namespace # the whole chain in one command
cmctl renew my-app -n my-namespace # force renewal now
kubectl get secret my-app-tls -n my-namespace -o jsonpath='{.data.tls\.crt}' | base64 -d | openssl x509 -noout -dates -ext subjectAltNameThe annotation cert-manager.io/cluster-issuer: letsencrypt on an Ingress or Gateway makes cert-manager create the Certificate itself; see Kubernetes and Gateway API. A Challenge stuck in pending for http01 usually means the temporary Ingress is not reachable from the internet on port 80: check the path from outside with curl http://my-app.example.com/.well-known/acme-challenge/test. For dns01, cert-manager checks propagation itself before telling the CA, so pending with DNS record for "..." not yet propagated is normal for the TTL of the zone’s SOA.
Certificate lifecycle#
Automate renewal, and alert on expiry separately from the automation. The usual failure is renewal that stopped working weeks ago with nobody watching.
# Days until a live endpoint's certificate expires
end=$(openssl s_client -connect api.example.com:443 -servername api.example.com </dev/null 2>/dev/null \
| openssl x509 -noout -enddate | cut -d= -f2)
echo $(( ( $(date -d "$end" +%s) - $(date +%s) ) / 86400 )) daysGenerate a new key at renewal instead of reusing the old one. Keep private keys out of version control and shared storage, and treat a key that has been emailed or pasted into a chat as compromised: revoke the certificate and issue a new one.
For Prometheus-based expiry alerts, the blackbox exporter exposes probe_ssl_earliest_cert_expiry; see Prometheus.
Expiry and revocation checks#
-checkend takes seconds and exits 1 when the certificate will have expired by then, which makes it the right primitive for cron and CI. Check every certificate in a chain, not only the leaf: intermediates expire too, and a server that ships an expired intermediate fails for every client even though the leaf looks fine.
openssl x509 -in cert.pem -noout -checkend $(( 30 * 86400 )) || echo 'expires within 30 days'
# Every certificate the server sends, with days remaining
openssl s_client -connect api.example.com:443 -servername api.example.com -showcerts </dev/null 2>/dev/null \
| awk '/BEGIN CERT/{c++} {print > "chain-" c ".pem"}'
for f in chain-*.pem; do printf '%s %s\n' "$(openssl x509 -in "$f" -noout -enddate | cut -d= -f2)" "$(openssl x509 -in "$f" -noout -subject)"; done; rm -f chain-*.pem
# Revocation: OCSP if the certificate has a responder URL, otherwise the CRL distribution point
openssl x509 -in cert.pem -noout -ocsp_uri
openssl ocsp -issuer intermediate.pem -cert cert.pem -url "$(openssl x509 -in cert.pem -noout -ocsp_uri)" -resp_text -noverify | head
openssl x509 -in cert.pem -noout -ext crlDistributionPoints
curl -sS "$(openssl x509 -in cert.pem -noout -ext crlDistributionPoints | grep -o 'http[^ ]*')" | openssl crl -inform der -noout -text | head
openssl verify -crl_check -CAfile chain.pem -CRLfile crl.pem cert.pem # offline: OK, or "certificate revoked"Certificate Transparency logs record every publicly trusted certificate; crt.sh and the CT log APIs let you list every certificate ever issued for a domain, which is how you find the forgotten test host that is about to expire. Query https://crt.sh/?q=%25.example.com&output=json and filter not_after.
Troubleshooting#
| Symptom | Likely cause | Check |
|---|---|---|
| Client reports unknown issuer | Missing intermediate or untrusted root | s_client -showcerts: count certificates sent; openssl verify with the chain |
| Only some clients fail | Old trust store (containers, old Java or Android), or a missing intermediate that browsers fetch | Test with curl and openssl from the failing environment |
| Name mismatch only when tested by IP | No SNI sent, so the default certificate is returned | Add -servername or use curl --resolve |
| New certificate deployed but old one served | Server not reloaded, another node behind the load balancer, or the TLS terminates at a proxy or CDN | Compare fingerprints per backend with --resolve or -connect <ip> |
key values mismatch when starting nginx | Key and certificate do not belong together, or chain order wrong (leaf must be first) | Key match diff above; check the first cert in fullchain.pem |
| Handshake fails with old clients | Client lacks TLS 1.2 or ECDHE suites | nmap --script ssl-enum-ciphers; decide whether to support that client |
mTLS unknown ca or certificate required | Client cert not sent, or issued by a CA the server does not trust | s_client -cert -key; server’s client CA bundle |
x509: certificate signed by unknown authority in a container | No CA bundle, or the private CA not added to the image | Look for /etc/ssl/certs/ca-certificates.crt or /etc/pki/tls/certs/ca-bundle.crt in the image |
| Certificate issuance refused by the CA | CAA record does not list that CA | dig +short CAA example.com; see DNS |
| Protocol check says a version is unsupported | Your OpenSSL build or security level refuses it locally, not the server | Retry with -cipher 'DEFAULT:@SECLEVEL=0' for TLS 1.0/1.1 tests only |
dh key too small or ca md too weak | Local security level 2 rejects the peer’s DH parameters or SHA-1 signature | update-crypto-policies --show; fix the peer, do not lower policy permanently |
certificate required alert | mTLS server got no client certificate; wrong CA, or the client did not send one because no acceptable CA name matched | s_client output under Acceptable client certificate CA names |
certbot Timeout during connect (likely firewall problem) | Port 80 not reachable from the internet for http-01, or the wrong server answered | curl -I http://example.com/.well-known/acme-challenge/x from outside; check CDN and IPv6 AAAA record |
certbot too many certificates already issued | Duplicate-certificate rate limit hit by a renewal loop or repeated testing | --dry-run or the staging --server for tests; wait a week or add a name to change the set |
| certbot renewed but the server still serves the old certificate | No --deploy-hook, or the server reads from a copy instead of live/ | certbot certificates paths versus the server config; add --deploy-hook 'systemctl reload nginx' |
cert-manager Certificate not READY | Issuer misconfigured, challenge failing, or the ACME account key Secret missing | cmctl status certificate; kubectl describe challenge -n <ns> |
cert-manager Challenge pending with propagation check failed | dns01 record not visible from the cluster’s resolver, split-horizon DNS | dig +short TXT _acme-challenge.example.com @1.1.1.1; set --dns01-recursive-nameservers on the controller |
Java PKIX path building failed | The JVM’s cacerts, not the OS store, lacks the CA | keytool -list -cacerts | grep -i my-ca; import with keytool -importcert -cacerts |
Python CERTIFICATE_VERIFY_FAILED but curl works | requests uses certifi, not the system bundle | REQUESTS_CA_BUNDLE=/etc/pki/tls/certs/ca-bundle.crt, or pip-system-certs |
Browser shows NET::ERR_CERT_AUTHORITY_INVALID on an internal site while openssl verify passes | Root distributed to the OS but the browser (Firefox) has its own store, or the leaf lacks serverAuth EKU | Firefox security.enterprise_roots.enabled; openssl x509 -ext extendedKeyUsage |
sslv3 alert bad certificate from the client | Client rejected the server certificate for a reason its own logs will state; often a CA constraint (pathlen) or a name constraint | openssl verify -show_chain -CAfile root.pem -untrusted intermediate.pem cert.pem |
Oneliners#
# Expiry for a list of hosts
while read -r h; do d=$(openssl s_client -connect "$h:443" -servername "$h" </dev/null 2>/dev/null | openssl x509 -noout -enddate | cut -d= -f2); printf '%-35s %s\n' "$h" "$d"; done < hosts.txt
# Fail if a certificate expires within 30 days
openssl x509 -in cert.pem -noout -checkend 2592000 || echo 'renew now'
# SANs on a live certificate
openssl s_client -connect host.example.com:443 -servername host.example.com </dev/null 2>/dev/null | openssl x509 -noout -ext subjectAltName
# Chain as served, subject and issuer only
openssl s_client -connect host.example.com:443 -servername host.example.com -showcerts </dev/null 2>/dev/null | awk '/BEGIN CERT/,/END CERT/' | openssl crl2pkcs7 -nocrl -certfile /dev/stdin | openssl pkcs7 -print_certs -noout
# Certificate stored in a Kubernetes TLS secret
kubectl get secret my-app-tls -n my-namespace -o jsonpath='{.data.tls\.crt}' | base64 -d | openssl x509 -noout -subject -dates
# Deployed certificate against the file on disk
diff <(openssl x509 -in cert.pem -noout -fingerprint -sha256) <(openssl s_client -connect host.example.com:443 -servername host.example.com </dev/null 2>/dev/null | openssl x509 -noout -fingerprint -sha256)
# Protocol versions a server accepts (local OpenSSL policy may block tls1 and tls1_1)
for p in tls1 tls1_1 tls1_2 tls1_3; do printf '%-8s ' "$p"; openssl s_client -connect host.example.com:443 -servername host.example.com -"$p" </dev/null >/dev/null 2>&1 && echo yes || echo no; done
# Throwaway certificate for local testing, in the current directory
openssl req -x509 -newkey ed25519 -noenc -days 30 -keyout localhost.key -out localhost.crt -subj '/CN=localhost' -addext 'subjectAltName=DNS:localhost,IP:127.0.0.1'
# Decode a certificate held as base64 DER in a variable
base64 -d <<< "$CERT_B64" | openssl x509 -inform der -noout -text
# Serial numbers of every certificate in a bundle
openssl storeutl -noout -text -certs bundle.pem | grep -E -A1 'Serial Number:|Subject:'
# Verify a chain and print the path that was built
openssl verify -show_chain -CAfile root.pem -untrusted intermediate.pem cert.pem
# Verify against the system store instead of a file (Fedora/RHEL and Debian paths)
openssl verify -CApath /etc/ssl/certs cert.pem
# Is this certificate a CA, and how deep may it issue
openssl x509 -in cert.pem -noout -ext basicConstraints
# Key type and size of a certificate's public key
openssl x509 -in cert.pem -noout -pubkey | openssl pkey -pubin -noout -text | head -1
# Hash of the public key, for HPKP-style pinning or to compare keys across renewals
openssl x509 -in cert.pem -noout -pubkey | openssl pkey -pubin -outform der | openssl dgst -sha256 -binary | base64
# Which key exchange group and cipher the server negotiated with TLS 1.3
openssl s_client -connect host.example.com:443 -servername host.example.com </dev/null 2>/dev/null | grep -E 'Protocol|Cipher|group'
# Does the server support session resumption (look for "Reused, TLSv1.3" in the second connection)
openssl s_client -connect host.example.com:443 -servername host.example.com -sess_out sess.pem </dev/null >/dev/null 2>&1; openssl s_client -connect host.example.com:443 -servername host.example.com -sess_in sess.pem </dev/null 2>/dev/null | grep -E '^(New|Reused)'
# Test a server through an HTTP proxy
openssl s_client -proxy proxy.example.com:3128 -connect host.example.com:443 -servername host.example.com </dev/null
# Test TLS on a non-HTTP port with STARTTLS (smtp, pop3, imap, ldap, postgres, mysql, xmpp, ftp)
openssl s_client -connect db.example.com:5432 -starttls postgres </dev/null 2>/dev/null | openssl x509 -noout -subject -dates
# Test a specific backend behind a load balancer while sending the right SNI and Host
curl -sv --resolve host.example.com:443:192.0.2.10 https://host.example.com/ -o /dev/null 2>&1 | grep -E 'subject|expire|issuer'
# Split a PEM bundle into one file per certificate
awk '/BEGIN CERT/{n++} {print > "cert-" n ".pem"}' bundle.pem
# Which certificate in a bundle is the root (issuer == subject)
for f in cert-*.pem; do [[ $(openssl x509 -in "$f" -noout -subject) == "$(openssl x509 -in "$f" -noout -issuer | sed 's/^issuer/subject/')" ]] && echo "$f is self-signed"; done
# Convert an OpenSSH public key to PEM for tooling that wants X.509-style keys
ssh-keygen -f ~/.ssh/id_ed25519.pub -e -m PKCS8
# Encrypt a private key at rest with a passphrase (prompts) and decrypt it again
openssl pkey -in key.pem -aes-256-cbc -out key.enc.pem; openssl pkey -in key.enc.pem -out key.pem
# Check a CSR's signature and print the requested SANs only
openssl req -in req.csr -noout -verify -ext subjectAltName
# Find every certificate file under /etc and print the ones expiring within 60 days
find /etc -name '*.pem' -o -name '*.crt' 2>/dev/null | while read -r f; do openssl x509 -in "$f" -noout -checkend $((60*86400)) >/dev/null 2>&1 || { openssl x509 -in "$f" -noout -subject >/dev/null 2>&1 && echo "$f"; }; done
# Local test server that speaks TLS and echoes the request, for testing clients and their trust stores
openssl s_server -accept 8443 -cert localhost.crt -key localhost.key -www
# Local test server that requires a client certificate from your CA
openssl s_server -accept 8443 -cert server.pem -key server.key -CAfile ca.pem -Verify 1 -www
# Compare the certificate two hosts serve (are both nodes behind the balancer deployed?)
diff <(openssl s_client -connect 192.0.2.10:443 -servername host.example.com </dev/null 2>/dev/null | openssl x509 -noout -serial) <(openssl s_client -connect 192.0.2.11:443 -servername host.example.com </dev/null 2>/dev/null | openssl x509 -noout -serial) && echo same
# Generate a CSR from an existing certificate, keeping subject and SANs, for renewal with a new key
openssl x509 -in cert.pem -x509toreq -signkey newkey.pem -out renew.csr -copy_extensions copyallScripts#
Expiry sweep over a list of host:port endpoints, checking every certificate in the served chain and the verify result, with a non-zero exit when anything is inside the warning window. Suitable as a cron job or a CI gate.
#!/usr/bin/env bash
# usage: tls-expiry.sh [-w DAYS] endpoints.txt (one host or host:port per line; # comments allowed)
set -euo pipefail
warn=30
while getopts ':w:' o; do case $o in w) warn=$OPTARG ;; *) echo "usage: $0 [-w days] file" >&2; exit 2 ;; esac; done
shift $((OPTIND - 1))
file=${1:?endpoints file required}
tmp=$(mktemp -d); trap 'rm -rf -- "$tmp"' EXIT
now=$(date +%s); rc=0
while IFS= read -r ep; do
[[ $ep =~ ^[[:space:]]*(#|$) ]] && continue
host=${ep%%:*}; port=${ep##*:}; [[ $port == "$host" ]] && port=443
if ! out=$(timeout 15 openssl s_client -connect "$host:$port" -servername "$host" -showcerts </dev/null 2>/dev/null); then
printf 'FAIL %-40s connect failed\n' "$ep"; rc=1; continue
fi
verify=$(grep -o 'Verify return code: .*' <<< "$out" | tail -1)
[[ $verify == 'Verify return code: 0 (ok)' ]] || { printf 'FAIL %-40s %s\n' "$ep" "$verify"; rc=1; }
awk -v d="$tmp/" '/BEGIN CERT/{n++} /BEGIN CERT/,/END CERT/{print > d n ".pem"}' <<< "$out"
for c in "$tmp"/*.pem; do
end=$(openssl x509 -in "$c" -noout -enddate | cut -d= -f2)
days=$(( ( $(date -d "$end" +%s) - now ) / 86400 ))
subj=$(openssl x509 -in "$c" -noout -subject | sed 's/^subject=//')
if (( days < warn )); then printf 'WARN %-40s %4d days %s\n' "$ep" "$days" "$subj"; rc=1
else printf 'ok %-40s %4d days %s\n' "$ep" "$days" "$subj"; fi
done
rm -f "$tmp"/*.pem
done < "$file"
exit "$rc"Issue a server certificate from the private CA described above with a single command: generates a fresh key, builds the SAN list from the arguments, signs with a serial from the CA’s counter file and writes a ready-to-serve fullchain.pem. Keeps the CA key passphrase-protected and prompts for it.
#!/usr/bin/env bash
# usage: ca-issue.sh OUTDIR NAME [NAME...] e.g. ca-issue.sh /etc/pki/my-app api.example.com 192.0.2.10
# expects CA_DIR (default ./ca) to contain ca.pem and ca.key (encrypted); writes OUTDIR/{key,cert,fullchain}.pem
set -euo pipefail
ca_dir=${CA_DIR:-./ca}; days=${DAYS:-90}
out=${1:?output directory required}; shift
(( $# )) || { echo 'at least one DNS name or IP required' >&2; exit 2; }
[[ -f $ca_dir/ca.pem && -f $ca_dir/ca.key ]] || { echo "no CA in $ca_dir" >&2; exit 1; }
umask 077; mkdir -p "$out"
san=''
for n in "$@"; do
if [[ $n =~ ^[0-9]+(\.[0-9]+){3}$ || $n == *:*:* ]]; then san+="IP:$n,"; else san+="DNS:$n,"; fi
done
san=${san%,}
cn=$1
openssl genpkey -algorithm EC -pkeyopt ec_paramgen_curve:P-256 -out "$out/key.pem"
openssl req -new -key "$out/key.pem" -subj "/CN=$cn" -out "$out/req.csr"
openssl x509 -req -in "$out/req.csr" -CA "$ca_dir/ca.pem" -CAkey "$ca_dir/ca.key" \
-CAserial "$ca_dir/serial" -CAcreateserial -days "$days" -out "$out/cert.pem" \
-extfile <(printf 'basicConstraints=CA:FALSE\nkeyUsage=critical,digitalSignature\nextendedKeyUsage=serverAuth\nsubjectAltName=%s\n' "$san")
cat "$out/cert.pem" "$ca_dir/ca.pem" > "$out/fullchain.pem" # include the root only if clients need it; drop it for public-style chains
rm -f "$out/req.csr"
chmod 644 "$out/cert.pem" "$out/fullchain.pem"
openssl verify -CAfile "$ca_dir/ca.pem" "$out/cert.pem"
openssl x509 -in "$out/cert.pem" -noout -subject -enddate -ext subjectAltNameCompare what each backend behind a load balancer serves against the certificate on disk, so a partial deployment is caught before the old certificate expires.
#!/usr/bin/env bash
# usage: tls-fleet-check.sh HOSTNAME fullchain.pem IP... e.g. tls-fleet-check.sh api.example.com /etc/pki/api/fullchain.pem 192.0.2.10 192.0.2.11
set -euo pipefail
name=${1:?hostname}; file=${2:?certificate file}; shift 2
(( $# )) || { echo 'no backend IPs given' >&2; exit 2; }
want=$(openssl x509 -in "$file" -noout -fingerprint -sha256 | cut -d= -f2)
printf 'expected %s\n' "$want"
rc=0
for ip in "$@"; do
got=$(timeout 10 openssl s_client -connect "$ip:443" -servername "$name" </dev/null 2>/dev/null | openssl x509 -noout -fingerprint -sha256 2>/dev/null | cut -d= -f2 || true)
if [[ -z $got ]]; then printf 'FAIL %-16s no certificate (connect or handshake failed)\n' "$ip"; rc=1
elif [[ $got == "$want" ]]; then printf 'ok %-16s\n' "$ip"
else printf 'DIFF %-16s %s\n' "$ip" "$got"; rc=1; fi
done
exit "$rc"Further reading#
- Mozilla server-side TLS and the configuration generator
man openssl-s_client,man openssl-x509,man openssl-req,man openssl-verify, or the OpenSSL 3 manual pages- RFC 8446, TLS 1.3 and RFC 5280, X.509 profile
- RFC 8555, ACME, the certbot user guide and Let’s Encrypt rate limits
- cert-manager documentation: issuers, ACME solvers and the
Certificateresource man openssl-ciphersfor cipher string syntax and security levels, or online