# HTTP and curl

> Use curl to test HTTP endpoints, break request time into DNS, TCP, TLS and server phases, script calls safely, and read what status codes mean.

Canonical: https://www.wiki.jodisand.me/http/
Reviewed: 2026-09-24
Related: [DNS](https://www.wiki.jodisand.me/dns/index.md), [TLS and certificates](https://www.wiki.jodisand.me/tls/index.md), [Squid](https://www.wiki.jodisand.me/squid/index.md), [Traefik](https://www.wiki.jodisand.me/traefik/index.md), [Bash](https://www.wiki.jodisand.me/bash/index.md)


## Cheatsheet

| Task | Command |
| --- | --- |
| Response headers only (sends HEAD) | `curl -I https://example.com` |
| Headers of a real GET | `curl -sS -D - -o /dev/null https://example.com` |
| Follow redirects | `curl -L https://example.com` |
| Fail with a non-zero exit on 4xx/5xx | `curl -fsS https://example.com` |
| Show request and response headers, TLS details | `curl -v https://example.com` |
| Status code only | `curl -o /dev/null -sw '%{http_code}\n' https://example.com` |
| Timing breakdown | `curl -o /dev/null -sw '@curl-format.txt' https://example.com` |
| POST JSON | `curl --json @body.json https://api.example.com/items` |
| Upload a file as multipart form | `curl -F file=@report.csv https://api.example.com/upload` |
| Bearer token | `curl -H "Authorization: Bearer $API_TOKEN" https://api.example.com` |
| Connect to a chosen IP, keep Host and SNI | `curl --resolve api.example.com:443:192.0.2.10 https://api.example.com` |
| Skip certificate verification | `curl -k https://example.com` (diagnosis only) |
| Force an HTTP version | `curl --http1.1 URL`, `curl --http2 URL`, `curl --http3 URL` |
| Through a proxy | `curl -x http://proxy.example.com:3128 https://example.com` |
| Retry transient failures | `curl --retry 3 --retry-delay 2 --retry-connrefused URL` |
| Bound the time | `curl --connect-timeout 3 --max-time 10 URL` |
| Keep cookies between calls | `curl -c jar.txt -b jar.txt URL` |

`--json` (curl 7.82.0 and later) sends the body with `Content-Type: application/json` and `Accept: application/json`. `--http3` needs a curl built with HTTP/3 support; check `curl --version` for `HTTP3` in the features line.

## A request that is failing

```sh
# Handshake, request headers (>), response headers (<), status
curl -sSv https://api.example.com/health -o /dev/null

# Where the time goes
curl -o /dev/null -sw 'code=%{http_code} dns=%{time_namelookup} conn=%{time_connect} tls=%{time_appconnect} ttfb=%{time_starttransfer} total=%{time_total}\n' https://api.example.com/health

# Bypass DNS and hit one backend, keeping the Host header and TLS SNI
curl -sv --resolve api.example.com:443:192.0.2.10 https://api.example.com/health -o /dev/null
```

| Failure | Where it happened |
| --- | --- |
| `Could not resolve host` (exit 6) | DNS; see [DNS](https://www.wiki.jodisand.me/dns/#a-name-that-will-not-resolve) |
| `Connection refused` (exit 7) | Host reached, nothing listening on that port, or a firewall sent a reset |
| `Connection timed out` (exit 28 with `--connect-timeout`) | Packets dropped: firewall, security group, routing; see [iproute2](https://www.wiki.jodisand.me/iproute2/#a-connectivity-problem) |
| `SSL certificate problem` (exit 60) | Incomplete chain, name mismatch, expiry, or CA missing from this trust store; see [TLS](https://www.wiki.jodisand.me/tls/) |
| `SSL_ERROR_SYSCALL` or `unexpected eof` during handshake | Something in the path closed the connection: a proxy, a load balancer without the certificate, or a port that does not speak TLS |
| Works with `-k` | The problem is certificate validation, not connectivity |
| Long `time_appconnect` | Slow TLS handshake: high latency (several round trips), a large chain, or a CPU-starved terminator |
| Long `time_starttransfer`, everything else short | The server is slow to produce the first byte |
| `Empty reply from server` (exit 52) | The server or a middlebox closed the connection without sending a response |

## Timing

All `time_*` write-out values are measured from the start of the transfer, so each includes the phases before it.

```sh
cat > curl-format.txt <<'EOF'
     dns  %{time_namelookup}s
 connect  %{time_connect}s
     tls  %{time_appconnect}s
 waiting  %{time_starttransfer}s
   total  %{time_total}s
    size  %{size_download} bytes
    code  %{http_code}
 version  %{http_version}
EOF

curl -o /dev/null -s -w '@curl-format.txt' https://api.example.com/health
```

```text
     dns  0.012345s
 connect  0.030112s
     tls  0.071870s
 waiting  0.180553s
   total  0.181204s
    size  15 bytes
    code  200
 version  2
```

Subtract to isolate each phase:

| Phase | Formula |
| --- | --- |
| DNS | `time_namelookup` |
| TCP handshake | `time_connect - time_namelookup` |
| TLS handshake | `time_appconnect - time_connect` (0 for plain HTTP) |
| Server processing plus one round trip | `time_starttransfer - time_appconnect` |
| Body transfer | `time_total - time_starttransfer` |

When a proxy is in use, `time_connect` is the connection to the proxy.

```sh
# Ten samples, sorted, to separate a slow server from a slow tail
for i in $(seq 10); do curl -o /dev/null -sw '%{time_total}\n' https://api.example.com/health; done | sort -n
```

## Useful flags

| Flag | Effect |
| --- | --- |
| `-f` / `--fail` | Exit 22 on HTTP 400 and above, without printing the body |
| `--fail-with-body` | Exit 22 on HTTP errors but still output the body (7.76.0 and later) |
| `-s` / `-S` | Silent; `-sS` stays silent but still prints errors |
| `-L` | Follow redirects; `--max-redirs` bounds it (default 50) |
| `-i` | Include response headers in the output |
| `-D -` | Dump response headers to stdout while the body goes elsewhere |
| `--compressed` | Ask for a compressed response and decode it |
| `-A`, `-e` | Set User-Agent and Referer |
| `--data-binary` | Send the payload exactly as given |
| `-G` | Move `-d` data into the query string and send GET |
| `--url-query` | Append a URL-encoded query parameter (7.87.0 and later) |
| `-w` | Write-out template: timings, sizes, redirect URL, status |
| `-Z` | Run several URLs in parallel |
| `--trace-ascii -` | Full dump of everything sent and received |

## Sending data

```sh
# JSON body; -H is needed because -d defaults to form encoding
curl -X POST https://api.example.com/items \
  -H 'Content-Type: application/json' \
  -d '{"name":"widget","qty":4}'

# JSON from a file with --json (sets Content-Type and Accept)
curl --json @payload.json https://api.example.com/items

# Multipart form upload
curl -F 'file=@report.csv' -F 'note=monthly' https://api.example.com/upload

# URL-encoded query string
curl -G https://api.example.com/search --data-urlencode 'q=name with spaces' --data-urlencode 'limit=10'

# PATCH with a merge patch
curl -X PATCH -H 'Content-Type: application/merge-patch+json' -d '{"qty":5}' https://api.example.com/items/1

# PUT a file
curl -T ./artifact.tgz https://uploads.example.com/path/
```

`-d` sends POST with `Content-Type: application/x-www-form-urlencoded` unless you override the header. `-d @file` strips carriage returns, newlines and null bytes from the file. `--data-binary @file` sends it byte for byte, which matters for signed payloads, YAML and NDJSON.

## TLS checks

```sh
# Negotiated protocol, cipher, certificate subject, issuer and expiry
curl -svI https://example.com 2>&1 | grep -E 'SSL connection|subject:|issuer:|expire date'

# Force a TLS version range
curl --tlsv1.3 -sI https://example.com
curl --tlsv1.2 --tls-max 1.2 -sI https://example.com

# Trust a private CA for this call
curl --cacert ca.pem https://internal.example.com

# Client certificate (mTLS)
curl --cert client.pem --key client.key https://mtls.example.com
```

`-k` shows the problem is certificate validation and nothing more. Do not leave it in a script: it accepts any certificate, including an attacker's. curl does not check revocation unless you ask with `--cert-status` (OCSP stapling). For chain repair and `openssl s_client`, see [TLS](https://www.wiki.jodisand.me/tls/#testing-a-server).

## Proxies

```sh
curl -x http://proxy.example.com:3128 https://api.example.com
export https_proxy=http://proxy.example.com:3128 no_proxy=.example.com,localhost,10.0.0.0/8
curl --proxy-user "alice:$PROXY_PASSWORD" -x http://proxy.example.com:3128 https://api.example.com
curl -x socks5h://localhost:1080 https://api.example.com   # socks5h: the proxy resolves DNS
curl -sv https://api.example.com -o /dev/null 2>&1 | grep -iE 'CONNECT|proxy'
```

curl reads lowercase `http_proxy` only; `HTTPS_PROXY` and `https_proxy` both work. `no_proxy` entries match the host itself or any name inside that domain (`example.com` matches `www.example.com`). CIDR ranges in `no_proxy` work from curl 7.86.0; other tools (Python `requests`, Go, Java) each have their own rules, so test the client you actually run. A proxy in the path shows as a `CONNECT` in verbose output for HTTPS, and often adds `Via` or `X-Cache` headers. See [Squid](https://www.wiki.jodisand.me/squid/) for the proxy side.

`--proxy-user` on the command line is visible in `ps`. For anything shared, put credentials in a `.netrc` file or a config file read with `-K`.

## Redirects

`-L` follows `Location` headers. For 301, 302 and 303 curl may switch a POST to GET on the next request; 307 and 308 keep the method and body. curl drops `Authorization` and `Cookie` headers when a redirect leaves the original origin, unless `--location-trusted` is set.

```sh
# Every hop with its status and Location
curl -sIL https://example.com | grep -iE '^HTTP/|^location:'

# Final URL after all redirects
curl -o /dev/null -sw '%{url_effective}\n' -L https://example.com
```

## Caching

A cache (browser, CDN, [Squid](https://www.wiki.jodisand.me/squid/)) stores a response according to its `Cache-Control` and reuses it while its age is below `max-age`; `Age` says how long the cache has held it. A stale copy is revalidated with a conditional request, and a `304` answer costs no body.

| Directive | Where | Effect |
| --- | --- | --- |
| `max-age=N` | Response | Fresh for N seconds from the origin's `Date` |
| `s-maxage=N` | Response | Same, for shared caches only; overrides `max-age` there |
| `no-cache` | Response | Store, but revalidate with the origin before every reuse |
| `no-store` | Response | Never write to disk or memory; for credentials and personal data |
| `private` | Response | Browser may cache, shared caches must not |
| `stale-while-revalidate=N` | Response | Serve stale for N seconds while refreshing in the background |
| `Vary: Accept-Encoding` | Response | Cache key includes that request header; `Vary: *` disables caching |
| `no-cache` | Request | Force revalidation; what a browser hard reload sends |

`no-cache` means "always check", not "do not cache"; `no-store` keeps a response out of caches. Validation pairs `ETag` with `If-None-Match` or `Last-Modified` with `If-Modified-Since`.

```sh
# See what a cache would do with this response
curl -sSI https://example.com/app.js | grep -iE '^(cache-control|etag|last-modified|age|vary|expires|x-cache):'

# Conditional GET: 304 if the ETag still matches
curl -so /dev/null -w '%{http_code}\n' -H 'If-None-Match: "5d8c72a5edda8d6a"' https://example.com/app.js

# Download again only when the ETag changed (7.68.0 and later)
curl -sS --etag-compare etag.txt --etag-save etag.txt -o app.js https://example.com/app.js

```

A CDN or proxy adds `Age` and a hit or miss header (`X-Cache`, `CF-Cache-Status`); two requests in a row with a rising `Age` prove the cache holds the object.

## CORS

CORS is enforced by browsers, not servers. A script on `https://app.example.com` calling `https://api.example.com` gets the response only if the API answers with `Access-Control-Allow-Origin` matching the page's origin (or `*`). curl ignores CORS, so "works in curl, fails in the browser" with an `Access-Control` console error is a missing header, not a connectivity problem.

Any method other than GET, HEAD or POST, a non-form `Content-Type`, or a custom header such as `Authorization` triggers a preflight: the browser sends `OPTIONS` with `Origin`, `Access-Control-Request-Method` and `Access-Control-Request-Headers`, and sends the real request only if the answer allows them.

```sh
# Reproduce a preflight; the server must return 2xx and the Access-Control-* headers
curl -sS -X OPTIONS https://api.example.com/items \
  -H 'Origin: https://app.example.com' \
  -H 'Access-Control-Request-Method: PUT' \
  -H 'Access-Control-Request-Headers: authorization,content-type' \
  -D - -o /dev/null | grep -iE '^(HTTP/|access-control-)'

```

| Response header | Meaning |
| --- | --- |
| `Access-Control-Allow-Origin` | The one origin allowed, or `*`; must echo the exact origin when credentials are used |
| `Access-Control-Allow-Methods` | Methods permitted after preflight |
| `Access-Control-Allow-Headers` | Request headers permitted after preflight |
| `Access-Control-Allow-Credentials: true` | Cookies and `Authorization` may be sent; incompatible with `*` |
| `Access-Control-Expose-Headers` | Response headers script may read beyond the safe set |
| `Access-Control-Max-Age` | Seconds the browser may cache the preflight result |

A response that varies `Access-Control-Allow-Origin` per request must send `Vary: Origin`, or a shared cache serves one origin's answer to another. Preflights carry no cookies or `Authorization`, so an endpoint that rejects unauthenticated `OPTIONS` breaks CORS.

## Authentication

| Scheme | Request header | curl |
| --- | --- | --- |
| Basic | `Authorization: Basic base64(user:pass)` | `-u "alice:$PASSWORD"` or `--basic` |
| Bearer | `Authorization: Bearer <token>` | `--oauth2-bearer "$API_TOKEN"` or `-H` |
| Digest | Challenge-response with a server nonce | `--digest -u "alice:$PASSWORD"` |
| Negotiate (Kerberos / SPNEGO) | `Authorization: Negotiate <ticket>` | `--negotiate -u :` |
| AWS Signature v4 | Signed headers and canonical request | `--aws-sigv4 aws:amz:ap-southeast-2:s3 -u "$AWS_ACCESS_KEY_ID:$AWS_SECRET_ACCESS_KEY"` |
| Mutual TLS | Client certificate in the handshake | `--cert client.pem --key client.key`; see [TLS](https://www.wiki.jodisand.me/tls/) |

Basic credentials are base64, not encrypted; never send them over plain HTTP. With `-u` and no scheme flag curl sends Basic immediately; `--anyauth` requests first and picks the strongest scheme from `WWW-Authenticate` at the cost of a round trip.

```sh
# Credentials from ~/.netrc (mode 0600, one line: machine api.example.com login alice password ...)
curl -n https://api.example.com/me                     # --netrc-file for another path

# OAuth2 client-credentials flow, then use the token
token=$(curl -fsS -u "$CLIENT_ID:$CLIENT_SECRET" -d 'grant_type=client_credentials' https://auth.example.com/oauth/token | jq -r .access_token)
curl -fsS --oauth2-bearer "$token" https://api.example.com/items
```

`-u` on the command line is visible through `ps` and lands in shell history. Prefer `-n`, a config file passed with `-K`, or `--variable %API_TOKEN --expand-header 'Authorization: Bearer {{API_TOKEN}}'` (curl 8.3.0 and later), which reads the value from the environment.

## HTTP/2 and HTTP/3

HTTP/1.1 sends one request at a time per TCP connection. HTTP/2 multiplexes streams over one TLS connection and compresses headers; HTTP/3 does the same over QUIC (UDP 443), removing transport-level head-of-line blocking. The version is chosen through ALPN (`h2`, `http/1.1`) in the TLS handshake and, for HTTP/3, by an `Alt-Svc: h3=":443"` header on an earlier response.

```sh
# Which version was negotiated
curl -so /dev/null -w '%{http_version}\n' https://example.com

# ALPN offer and answer
curl -sv https://example.com -o /dev/null 2>&1 | grep -iE 'ALPN|using HTTP'

# HTTP/2 without TLS, for a backend behind a terminating proxy (no upgrade, straight h2c)
curl --http2-prior-knowledge http://localhost:8080/health

# Try HTTP/3 and fall back, or insist on it
curl --http3 -sI https://example.com
curl --http3-only -so /dev/null -w '%{http_version}\n' https://example.com


```

Load balancers usually terminate HTTP/2 at the edge and speak HTTP/1.1 to backends, so a `421` or a header-size error may come from the edge rather than the application. HTTP/3 needs UDP 443 open on every firewall in the path; when `--http3-only` fails but `--http3` works, the fallback masked a blocked UDP path.

## Cookies

curl has no cookie engine until you turn it on with `-b` or `-c`. `-b` reads cookies (a file, or a literal `name=value` string), `-c` writes every cookie the server sets to a jar in Netscape format. Using the same file for both keeps a session across calls.

```sh
# Log in, keep the session cookie, use it
curl -sS -c jar.txt -d "username=alice&password=$PASSWORD" https://app.example.com/login -o /dev/null
curl -sS -b jar.txt https://app.example.com/dashboard

# Send a literal cookie without a file
curl -b 'session=abc123; theme=dark' https://app.example.com/


# What the server set, including attributes
curl -sSI https://app.example.com/login | grep -i '^set-cookie'
```

A `Secure` cookie is never sent over `http://`; `Domain=example.com` sends it to every subdomain, no `Domain` restricts it to the host that set it. `SameSite=Lax` (the browser default) withholds the cookie on cross-site POSTs, a common cause of "logged out after the identity provider redirect"; curl ignores it. `-b` with an empty file still enables the engine, so cookies set during a redirect chain reach later hops.

## Status codes in practice

| Code | What it usually means |
| --- | --- |
| `301` / `308` | Permanent redirect; `308` must keep the method and body |
| `302` / `307` | Temporary redirect; `307` must keep the method and body |
| `304` | Not Modified: the client's cached copy is current |
| `400` | Malformed request: check `Content-Type` and body encoding |
| `401` | Missing or invalid credentials; the `WWW-Authenticate` header says which scheme |
| `403` | Authenticated but not allowed, or blocked by a WAF or IP allowlist |
| `404` | Wrong path, or the route exists only on another virtual host |
| `405` | Right path, wrong method; `Allow` lists the valid ones |
| `408` | Server timed out waiting for the request |
| `409` | Conflict: optimistic concurrency failure, duplicate create |
| `413` / `414` | Content Too Large / URI Too Long for the server or a proxy in front |
| `421` | Misdirected Request: a reused HTTP/2 connection reached a server that does not serve this host |
| `429` | Rate limited; honour `Retry-After` |
| `499` | Not standard; nginx logs it when the client closed the connection first |
| `500` | Unhandled application error; the application logs have the detail |
| `502` | A proxy could not connect to the backend or got an invalid response |
| `503` | No healthy backend, maintenance, or load shedding; may carry `Retry-After` |
| `504` | A proxy connected, but the backend did not answer within the proxy's timeout |

A 502 and a 504 from the same proxy point at different problems. 502 is connection-level (refused, reset, bad response); 504 is a timeout, and the timeout that fired is set in the proxy's configuration, not the application's. See [RFC 9110](https://www.rfc-editor.org/rfc/rfc9110#name-status-codes) for definitions.

## Using curl in scripts

```sh
if ! body=$(curl -fsS --max-time 10 --retry 3 --retry-connrefused https://api.example.com/health); then
  printf 'health check failed\n' >&2
  exit 1
fi
```

```sh
tmp=$(mktemp -d); trap 'rm -rf "$tmp"' EXIT
code=$(curl -sS -o "$tmp/body" -D "$tmp/headers" -w '%{http_code}' --max-time 30 \
  --json @payload.json https://api.example.com/items)
case $code in
  2*)  ;;
  429) sleep "$(awk 'tolower($1) == "retry-after:" {print $2}' "$tmp/headers" | tr -d '\r')" ;;
  *)   printf 'unexpected %s: %s\n' "$code" "$(cat "$tmp/body")" >&2; exit 1 ;;
esac
```

Always set `--max-time`. Without it a server that accepts the connection and never answers hangs the job indefinitely. `--retry` retries only transient errors: timeouts and HTTP 408, 429, 500, 502, 503, 504, 522 and 524. Add `--retry-connrefused` for refused connections, or `--retry-all-errors` to retry anything. See [Bash](https://www.wiki.jodisand.me/bash/) for error handling around these calls.

Retries back off from one second and double unless `--retry-delay` fixes the interval; `--retry-max-time 60` caps the whole sequence. `-w '%{json}'` prints every write-out variable as one JSON object for logging; `%{exitcode}`, `%{errormsg}` and `%{num_retries}` record what went wrong and how many retries it took.

```sh
curl -sS -o /dev/null -w '%{json}\n' --retry 3 --max-time 10 https://api.example.com/health \
  | jq '{code: .http_code, total: .time_total, retries: .num_retries, ip: .remote_ip, err: .errormsg}'
```

## Troubleshooting

| Symptom | Likely cause | Check |
| --- | --- | --- |
| `curl -I` returns 405 or different headers from GET | Server handles HEAD differently | `curl -sS -D - -o /dev/null URL` sends a GET |
| JSON API returns 400 or 415 | `-d` sent form encoding | Use `--json` or set `Content-Type` |
| Signature or checksum mismatch on upload | `-d @file` stripped newlines | Use `--data-binary @file` |
| 401 after a redirect | `Authorization` dropped on a cross-origin redirect | `curl -v` shows the second request; use the final URL directly |
| Works in a browser, fails in curl | Missing intermediate certificate, cookies, or a WAF blocking the User-Agent | `curl -v` TLS lines; compare headers from browser dev tools |
| Works with IP, fails with hostname (or the reverse) | Virtual host or SNI routing | `--resolve` keeps name and SNI while choosing the IP |
| Intermittent 502 behind a load balancer | Backend closes idle keep-alive connections before the balancer does | Backend idle timeout must exceed the balancer's |
| Unexpected proxy use or bypass | `http_proxy`, `https_proxy`, `no_proxy` in the environment | `env \| grep -i _proxy`; `curl -v` for `CONNECT` |
| Hangs with no output | No `--max-time`; firewall dropping packets | `--connect-timeout 3 -v` to see which phase stalls |
| Compressed garbage in output | Server sent compressed body without being asked, or `--compressed` omitted | `curl -sI` for `Content-Encoding`; add `--compressed` |
| Browser console shows `blocked by CORS policy`, curl works | Missing `Access-Control-Allow-Origin`, or the preflight `OPTIONS` is rejected | Replay the `OPTIONS` request with `Origin` and `Access-Control-Request-Method` headers |
| Stale content after a deploy | Long `max-age` at a CDN or in the browser, or `Vary` missing | `curl -sSI` for `Cache-Control`, `Age`, `X-Cache`; purge or version the asset name |
| Login works, next request is unauthenticated | Cookie engine off, or cookie marked `Secure` sent over `http://` | Use `-c jar.txt -b jar.txt` on both calls; check `Set-Cookie` attributes |
| 401 with `WWW-Authenticate: Digest` | Basic sent to a Digest-only server | `--digest -u`, or `--anyauth` |
| `--http3` works, `--http3-only` fails | UDP 443 blocked somewhere in the path | Test with `--http3-only`; open UDP 443 or accept the fallback |
| `--retry` did not retry | Error is not on curl's transient list (for example 404, 401 or connection refused) | Add `--retry-connrefused` or `--retry-all-errors`; check `%{num_retries}` |

## Oneliners

```sh
# Response headers of a GET, sorted
curl -sS -D - -o /dev/null https://example.com | sort

# Poll until a service is healthy
until curl -fsS --max-time 2 http://localhost:8080/health >/dev/null; do sleep 1; done

# 50 requests, 10 at a time, count the status codes
seq 50 | xargs -P10 -I{} curl -o /dev/null -sw '%{http_code}\n' https://api.example.com/ | sort | uniq -c

# Compare JSON config between two environments (needs jq)
diff <(curl -s https://staging.example.com/api/config | jq -S .) <(curl -s https://prod.example.com/api/config | jq -S .)

# Resume a partial download
curl -C - -O https://example.com/big.iso

# Send an HMAC-signed webhook; the secret comes from the environment
body='{"event":"test"}'; sig=$(printf '%s' "$body" | openssl dgst -sha256 -hmac "$WEBHOOK_SECRET" -hex | awk '{print $NF}'); curl -X POST -H "X-Signature: sha256=$sig" -H 'Content-Type: application/json' --data-binary "$body" https://api.example.com/hook

# Negotiated HTTP version and scheme
curl -so /dev/null -w '%{http_version} %{scheme}\n' https://example.com

# Plain HTTP virtual host test against an IP
curl -H 'Host: api.example.com' http://192.0.2.10/health

# Slowest of several endpoints
for u in "${urls[@]}"; do printf '%s %s\n' "$(curl -o /dev/null -sw '%{time_total}' "$u")" "$u"; done | sort -rn | head

# Every write-out variable as JSON, for a log line
curl -sS -o /dev/null -w '%{json}\n' https://api.example.com/health

# Response headers as JSON (7.83.0 and later)
curl -sS -o /dev/null -w '%{header_json}\n' https://example.com | jq '.["cache-control"]'

# Which IP and port curl actually talked to (after DNS and redirects)
curl -so /dev/null -w '%{remote_ip}:%{remote_port} %{http_code}\n' -L https://example.com





# Talk to a service on a Unix socket (Docker, Podman, systemd-style daemons)
curl -sS --unix-socket /var/run/docker.sock http://localhost/version






# HEAD every URL in a file, 10 at a time, print code and URL
xargs -a urls.txt -P10 -I{} curl -so /dev/null -w '%{http_code} {}\n' -I {}





# Bearer token from the environment without putting it on the command line (8.3.0 and later)
curl --variable %API_TOKEN --expand-header 'Authorization: Bearer {{API_TOKEN}}' https://api.example.com/me


```

## Scripts

Sample an endpoint N times and print percentile latency per phase, so a slow DNS resolver or TLS terminator is separated from a slow application.

```sh
#!/usr/bin/env bash
# usage: http-latency.sh URL [samples]
set -euo pipefail
url=${1:?url required}; n=${2:-20}
tmp=$(mktemp); trap 'rm -f "$tmp"' EXIT
for _ in $(seq "$n"); do
  curl -sS -o /dev/null --max-time 15 \
    -w '%{time_namelookup} %{time_connect} %{time_appconnect} %{time_starttransfer} %{time_total} %{http_code}\n' \
    "$url" >>"$tmp" || true
done
printf '%-8s %8s %8s %8s\n' phase p50 p90 max
for col in 1:dns 2:tcp 3:tls 4:ttfb 5:total; do
  i=${col%%:*}; name=${col#*:}
  sort -k"$i" -n "$tmp" | awk -v i="$i" -v name="$name" \
    '{v[NR]=$i} END {printf "%-8s %8.3f %8.3f %8.3f\n", name, v[int(NR*0.5)+1], v[int(NR*0.9)+1], v[NR]}'
done
printf 'codes: '; awk '{print $6}' "$tmp" | sort | uniq -c | tr '\n' ' '; echo
```

Check a list of URLs and report any whose status, latency or certificate expiry crosses a threshold, for a cron job that emails or pages.

```sh
#!/usr/bin/env bash
# usage: http-check.sh urls.txt   (one URL per line, # comments allowed)
set -euo pipefail
list=${1:?url list required}; max_ms=${MAX_MS:-2000}; min_days=${MIN_CERT_DAYS:-14}
rc=0
while IFS= read -r url; do
  [[ -z $url || $url == \#* ]] && continue
  out=$(curl -sS -o /dev/null --max-time 10 -w '%{http_code} %{time_total} %{exitcode} %{errormsg}' "$url" || true)
  read -r code total exit_code msg <<<"$out"
  ms=$(awk -v t="$total" 'BEGIN {printf "%d", t * 1000}')
  if [[ $exit_code != 0 ]]; then printf 'FAIL %s curl(%s): %s\n' "$url" "$exit_code" "$msg"; rc=1; continue; fi
  [[ $code == 2* || $code == 3* ]] || { printf 'FAIL %s status %s\n' "$url" "$code"; rc=1; }
  (( ms <= max_ms )) || { printf 'SLOW %s %sms\n' "$url" "$ms"; rc=1; }
  if [[ $url == https://* ]]; then
    host=${url#https://}; host=${host%%/*}; host=${host%%:*}
    exp=$(echo | openssl s_client -servername "$host" -connect "$host:443" 2>/dev/null | openssl x509 -noout -enddate | cut -d= -f2)
    days=$(( ($(date -d "$exp" +%s) - $(date +%s)) / 86400 ))
    (( days >= min_days )) || { printf 'CERT %s expires in %d days\n' "$url" "$days"; rc=1; }
  fi
done <"$list"
exit "$rc"
```


## Further reading

- [curl man page](https://curl.se/docs/manpage.html) and [Everything curl](https://everything.curl.dev/)
- [RFC 9110, HTTP semantics](https://www.rfc-editor.org/rfc/rfc9110)
- [RFC 9111, HTTP caching](https://www.rfc-editor.org/rfc/rfc9111)
- [RFC 6265, HTTP cookies](https://www.rfc-editor.org/rfc/rfc6265)
- [Fetch standard, CORS protocol](https://fetch.spec.whatwg.org/#http-cors-protocol)