tcpdump and Wireshark
Capture the right packets with BPF filters, read tcpdump output, capture in containers and over SSH, script with tshark, and diagnose retransmissions, MTU, TLS and DNS faults in Wireshark.
On this page
Cheatsheet#
| Task | Command |
|---|---|
| Interfaces tcpdump can see | tcpdump -D |
| Traffic to or from a host, no name lookups | tcpdump -ni eth0 host 192.0.2.10 |
| One port, verbose | tcpdump -ni eth0 -v port 443 |
| Everything except SSH (your own session) | tcpdump -ni eth0 not port 22 |
| Only TCP handshakes | tcpdump -ni eth0 'tcp[tcpflags] & (tcp-syn|tcp-ack) == tcp-syn' |
| Only RST packets | tcpdump -ni eth0 'tcp[tcpflags] & tcp-rst != 0' |
| DNS queries and answers | tcpdump -ni eth0 -vv port 53 |
| HTTP request lines in plain text | tcpdump -ni eth0 -A -s0 'tcp port 80 and (tcp[((tcp[12:1] & 0xf0) >> 2):4] = 0x47455420)' |
| Write to a file for Wireshark | tcpdump -ni eth0 -s0 -w capture.pcap port 8443 |
| Rotate files: 100 MB each, keep 10 | tcpdump -ni eth0 -w cap-%Y%m%d-%H%M%S.pcap -C 100 -W 10 -s0 |
| Stop after 1000 packets | tcpdump -ni eth0 -c 1000 -w capture.pcap |
| Read a file with a filter | tcpdump -nr capture.pcap 'host 192.0.2.10 and port 443' |
| Timestamps as ISO with microseconds | tcpdump -ni eth0 -tttt |
| Delta since the previous packet | tcpdump -ni eth0 -ttt |
| Capture remotely, view locally | ssh host 'tcpdump -ni eth0 -s0 -U -w - not port 22' | wireshark -k -i - |
| Capture in a container’s namespace | nsenter -t "$(podman inspect -f '{{.State.Pid}}' my-app)" -n tcpdump -ni eth0 |
| Count packets per conversation | tshark -r capture.pcap -q -z conv,tcp |
| Fields as CSV | tshark -r capture.pcap -T fields -e ip.src -e ip.dst -e tcp.dstport -E separator=, |
| Summary of a capture | capinfos capture.pcap |
| Slice a big capture by time | editcap -A '2026-09-24 14:00:00' -B '2026-09-24 14:05:00' big.pcap slice.pcap |
| Merge captures | mergecap -w all.pcap a.pcap b.pcap |
Commands assume tcpdump 4.99 with libpcap 1.10 and Wireshark 4.4 (tshark, capinfos, editcap, mergecap come from the wireshark-cli package on Fedora and RHEL). Capturing needs root or CAP_NET_RAW and CAP_NET_ADMIN. References: the tcpdump man page, pcap-filter(7) and the Wireshark User’s Guide.
How capture works#
libpcap opens a raw socket (AF_PACKET on Linux) on an interface and receives a copy of every frame the kernel sends or receives there, after the receive path’s checksum and offload handling and before or after the firewall depending on direction. A capture filter is compiled to a BPF program and attached to the socket, so packets that fail it are dropped in the kernel and never copied to user space. That is why a tight capture filter is the difference between a usable capture and a dropped-packet count in the thousands.
Three consequences. Outgoing packets are captured before segmentation offload, so a “packet” in the capture may be 64 KiB when the wire carried forty 1500-byte frames; disable with ethtool -K eth0 tso off gso off gro off when sizes matter. Outgoing checksums appear wrong when checksum offload is on, because the NIC fills them in later; Wireshark flags them unless checksum validation is turned off. Traffic that never reaches this host’s stack (switched traffic between two other hosts, or VM to VM on a bridge you are not on) does not appear; capture on the bridge, a mirror port, or inside the right namespace.
tcpdump -D # interfaces: eth0, any, lo, docker0, virbr0, plus nflog and usbmon pseudo-devices
tcpdump -i any -n # every interface; Linux cooked capture, no Ethernet headers
tcpdump -i eth0 -n -c 20 # 20 packets then stop
tcpdump -i eth0 -n -q # quiet: one short line per packet
tcpdump -i eth0 -n -v; tcpdump -i eth0 -n -vvv # more protocol detail: TTL, ID, checksums, DNS records
tcpdump -i eth0 -n -e # link-layer headers: MAC addresses and VLAN tags
tcpdump -i eth0 -n -s 0 # snaplen: full packets; 262144 is the default since 4.0 so -s0 is habit, but -s 96 saves space when headers are all you need
tcpdump -i eth0 -n -p # no promiscuous mode: only frames addressed to this host
tcpdump -i eth0 -n -Q in # direction: in, out or inout (Linux)
tcpdump -i eth0 -n --immediate-mode # deliver each packet as it arrives instead of buffering
tcpdump -i eth0 -n -U -w - | tee capture.pcap | tcpdump -nr - # -U: flush per packet when writing
tcpdump -i eth0 -n -B 65536 # 64 MiB kernel buffer, for busy links-n stops reverse DNS lookups, which are slow and generate traffic that lands in the capture. -nn also leaves port numbers numeric. Use them always.
BPF capture filter syntax#
The filter language is documented in pcap-filter(7). A filter is a boolean expression of primitives joined with and, or, not and parentheses. Quote the whole thing in single quotes so the shell leaves the parentheses and brackets alone.
Primitives are a type (host, net, port, portrange), a direction (src, dst, src or dst, src and dst) and a protocol (ether, ip, ip6, arp, tcp, udp, icmp, icmp6). Missing parts default: host means src or dst host, port means tcp or udp port.
tcpdump -ni eth0 host 192.0.2.10 # to or from
tcpdump -ni eth0 src host 192.0.2.10 # from only
tcpdump -ni eth0 dst 192.0.2.10 # "host" may be omitted for an address
tcpdump -ni eth0 host www.example.com # resolved once at start; multi-address names match all
tcpdump -ni eth0 net 192.0.2.0/24 # CIDR
tcpdump -ni eth0 net 192.0.2.0 mask 255.255.255.0
tcpdump -ni eth0 src net 10.0.0.0/8 and dst net not 10.0.0.0/8 # leaving the private range
tcpdump -ni eth0 port 443 # tcp or udp, src or dst
tcpdump -ni eth0 tcp dst port 443
tcpdump -ni eth0 portrange 8000-8999
tcpdump -ni eth0 udp # protocol only
tcpdump -ni eth0 icmp or icmp6
tcpdump -ni eth0 arp
tcpdump -ni eth0 ip6 # IPv6 only
tcpdump -ni eth0 ip proto 47 # GRE by number; also: ip proto ospf, vrrp
tcpdump -ni eth0 ether host aa:bb:cc:dd:ee:ff # MAC
tcpdump -ni eth0 ether broadcast or ether multicast
tcpdump -ni eth0 vlan 100 # tagged with VLAN 100; primitives after "vlan" apply to the inner packet
tcpdump -ni eth0 'vlan and ip' # any tagged IPv4
tcpdump -ni eth0 less 64; tcpdump -ni eth0 greater 1400 # length
tcpdump -ni eth0 'host 192.0.2.10 and (port 80 or port 443)'
tcpdump -ni eth0 'host 192.0.2.10 and not port 22'
tcpdump -ni eth0 'not (port 22 or port 53 or arp)' # the usual noise filter
tcpdump -ni eth0 'tcp and not host 192.0.2.1'
tcpdump -ni eth0 'src 192.0.2.10 and dst port 5432'inbound and outbound are also primitives on Linux. Where a name and a keyword clash (host gateway), quote the name or use an address.
Header byte matching#
proto[offset:size] reads bytes from a protocol header, which is how to match TCP flags, ICMP types, DNS opcodes or payload bytes. Named constants exist for TCP flags (tcp-syn, tcp-ack, tcp-fin, tcp-rst, tcp-push, tcp-urg) and ICMP types (icmp-echo, icmp-echoreply, icmp-unreach, icmp-timxceed).
tcpdump -ni eth0 'tcp[tcpflags] & tcp-syn != 0' # any SYN (including SYN-ACK)
tcpdump -ni eth0 'tcp[tcpflags] & (tcp-syn|tcp-ack) == tcp-syn' # SYN without ACK: new connection attempts
tcpdump -ni eth0 'tcp[tcpflags] & (tcp-syn|tcp-ack) == (tcp-syn|tcp-ack)' # SYN-ACK: servers answering
tcpdump -ni eth0 'tcp[tcpflags] & tcp-rst != 0' # resets: refused connections, aborted sessions
tcpdump -ni eth0 'tcp[tcpflags] & tcp-fin != 0' # closes
tcpdump -ni eth0 'tcp[tcpflags] == tcp-syn' # exactly SYN, no other flag
tcpdump -ni eth0 'tcp[13] & 2 != 0' # same as tcp-syn, by byte offset (13) and bit (2)
tcpdump -ni eth0 'icmp[icmptype] == icmp-echo or icmp[icmptype] == icmp-echoreply' # ping only
tcpdump -ni eth0 'icmp[icmptype] == icmp-unreach' # destination unreachable, including fragmentation needed
tcpdump -ni eth0 'icmp[0] == 3 and icmp[1] == 4' # fragmentation needed and DF set (PMTUD)
tcpdump -ni eth0 'icmp[icmptype] == icmp-timxceed' # traceroute responses
tcpdump -ni eth0 'ip[6] & 0x40 != 0' # DF bit set
tcpdump -ni eth0 'ip[6:2] & 0x1fff != 0' # fragments other than the first
tcpdump -ni eth0 'ip[8] < 5' # TTL under 5: traceroute probes or loops
tcpdump -ni eth0 'ip[2:2] > 1400' # IPv4 total length over 1400
tcpdump -ni eth0 'udp[8] & 0x80 == 0 and port 53' # DNS queries only (QR bit clear)
tcpdump -ni eth0 'udp[8] & 0x80 != 0 and udp[11] & 0x0f == 3 and port 53' # DNS responses with NXDOMAIN (rcode 3)
tcpdump -ni eth0 'tcp port 80 and (tcp[((tcp[12:1] & 0xf0) >> 2):4] = 0x47455420)' # payload starts with "GET "
tcpdump -ni eth0 'tcp port 80 and (tcp[((tcp[12:1] & 0xf0) >> 2):4] = 0x504f5354)' # "POST"
tcpdump -ni eth0 'tcp port 443 and (tcp[((tcp[12:1] & 0xf0) >> 2)] = 0x16) and (tcp[((tcp[12:1] & 0xf0) >> 2)+5] = 0x01)' # TLS ClientHello
tcpdump -ni eth0 'tcp[((tcp[12:1] & 0xf0) >> 2):4] = 0x48545450' # "HTTP": responses
tcpdump -ni eth0 'ether[0] & 1 = 0 and ip[16] >= 224' # unicast frame carrying a multicast destinationtcp[12:1] & 0xf0) >> 2 computes the TCP header length in bytes (data offset in the top nibble of byte 12, times 4), so tcp[that:4] is the first four payload bytes. The tcp[...] and udp[...] forms only work on IPv4; for IPv6 the transport header offset is not fixed, and libpcap 1.10 rejects them with ip6 upper-layer protocol is not supported. Capture ip6 and tcp and filter in Wireshark.
Reading the output#
A TCP line, with -n and -tttt:
2026-09-24 14:03:12.481923 IP 192.0.2.10.51234 > 198.51.100.5.443: Flags [S], seq 3812345678, win 64240, options [mss 1460,sackOK,TS val 1 ecr 0,nop,wscale 7], length 0
2026-09-24 14:03:12.503117 IP 198.51.100.5.443 > 192.0.2.10.51234: Flags [S.], seq 987654321, ack 3812345679, win 65160, options [mss 1460,sackOK,TS val 2 ecr 1,nop,wscale 7], length 0
2026-09-24 14:03:12.503150 IP 192.0.2.10.51234 > 198.51.100.5.443: Flags [.], ack 1, win 502, options [nop,nop,TS val 3 ecr 2], length 0
2026-09-24 14:03:12.503900 IP 192.0.2.10.51234 > 198.51.100.5.443: Flags [P.], seq 1:518, ack 1, win 502, length 517Flags: S SYN, . ACK, F FIN, R RST, P PSH, U URG, W CWR, E ECE. [S.] is SYN-ACK. After the handshake tcpdump prints sequence numbers relative to the first (seq 1:518 is bytes 1 to 517 of this direction) unless -S asks for absolute ones. win is the advertised receive window before scaling; multiply by 2^wscale from the SYN. length is payload bytes. mss 1460 on both SYNs is a normal Ethernet path; a smaller value shows a tunnel or PPPoE somewhere. A SYN answered by [R.] is a closed port; a SYN answered by nothing and retransmitted after 1, 2, 4 seconds is a filtered port or a routing problem.
UDP, DNS and ICMP with -v:
14:03:12.600000 IP 192.0.2.10.40000 > 192.0.2.53.53: 12345+ A? www.example.com. (33)
14:03:12.612000 IP 192.0.2.53.53 > 192.0.2.10.40000: 12345 2/0/0 A 93.184.216.34, A 93.184.216.35 (65)
14:03:13.000000 IP 192.0.2.53.53 > 192.0.2.10.40001: 12346 NXDomain 0/1/0 (110)
14:03:13.100000 IP 192.0.2.53.53 > 192.0.2.10.40002: 12347 ServFail 0/0/0 (33)
14:03:14.000000 IP 192.0.2.10 > 198.51.100.5: ICMP echo request, id 100, seq 1, length 64
14:03:14.020000 IP 198.51.100.5 > 192.0.2.10: ICMP echo reply, id 100, seq 1, length 64
14:03:15.000000 IP 192.0.2.1 > 192.0.2.10: ICMP 198.51.100.5 unreachable - need to frag (mtu 1400), length 36
14:03:16.000000 IP 192.0.2.1 > 192.0.2.10: ICMP 198.51.100.5 tcp port 8443 unreachable, length 36In DNS lines 12345+ is the query ID with + for recursion desired, A? the question, and 2/0/0 the counts of answer, authority and additional records. -vv prints the TTLs. NXDomain, ServFail and Refused name the rcode. A query with no answer within a couple of seconds followed by the same ID from a new source port is the resolver retrying; see DNS.
Timestamps: default is time of day; -tttt adds the date; -ttt prints the delta from the previous packet; -tt prints epoch seconds; -t drops them. --time-stamp-precision=nano on capable NICs. Payload: -A prints it as ASCII, -X as hex and ASCII, -x hex only; -XX and -AA include the link header.
At exit tcpdump prints N packets captured, N packets received by filter, N packets dropped by kernel. A non-zero dropped count means the buffer overflowed: tighten the filter, raise -B, write to a file instead of the terminal, or reduce -s.
Writing and rotating captures#
Write raw packets with -w and read them back with -r. Files are pcap by default; Wireshark reads them directly. tcpdump prints nothing while writing unless you add -v (a packet count) or read the file through a pipe.
tcpdump -ni eth0 -s0 -w capture.pcap 'host 192.0.2.10 and port 443' # until Ctrl-C
tcpdump -ni eth0 -s0 -w capture.pcap -c 10000 port 5432 # stop at 10000 packets
tcpdump -ni eth0 -s0 -w capture.pcap -G 300 -W 1 port 53 # stop after 300 seconds
tcpdump -ni eth0 -s0 -w 'cap-%Y%m%d-%H%M%S.pcap' -G 3600 # new file every hour, strftime name
tcpdump -ni eth0 -s0 -w cap.pcap -C 100 -W 10 # 100 MB files, cap.pcap0..9, oldest overwritten: a ring buffer
tcpdump -ni eth0 -s0 -w 'cap-%H%M.pcap' -G 600 -W 6 -z gzip # 10-minute files, 6 kept, compressed after rotation
tcpdump -ni eth0 -s0 -w cap.pcap -Z tcpdump # drop privileges to this user after opening the socket (default on Fedora)
tcpdump -ni eth0 -s 128 -w headers.pcap # headers only: small files, no payload
tcpdump -nr capture.pcap # read
tcpdump -nr capture.pcap -tttt 'tcp[tcpflags] & tcp-rst != 0' # filter while reading; same syntax
tcpdump -nr capture.pcap -w subset.pcap 'host 192.0.2.10' # extract a subset
tcpdump -nr capture.pcap -c 1 -tttt; tcpdump -nr capture.pcap -tttt | tail -1 # first and last timestamps-C counts in millions of bytes and appends a number to the name; -G rotates on time and needs strftime escapes in the name (otherwise it overwrites); -W with -C is a ring, with -G it is a total file limit. -z cmd runs cmd file after each rotation. When -Z drops privileges the output directory must be writable by that user; Permission denied on -w with a root shell is the usual sign. The systemd unit approach for long captures is a Type=simple service running tcpdump with -C/-W, so a night-long intermittent fault is captured without filling the disk; see systemd.
Capturing in containers, VMs and remotely#
A container has its own network namespace and usually no tcpdump. Capture from the host inside the container’s namespace, or on the host-side veth or bridge.
pid=$(podman inspect -f '{{.State.Pid}}' my-app) # docker inspect works the same
nsenter -t "$pid" -n tcpdump -ni eth0 -s0 -w /var/tmp/my-app.pcap port 8080 # host's tcpdump, container's namespace; file lands on the host
nsenter -t "$pid" -n ip -br addr # interfaces as the container sees them
tcpdump -ni podman0 host 10.88.0.5 # or on the bridge, by container IP
tcpdump -ni "$(ip -o link | awk -F': ' '/veth/ {print $2; exit}')" # a specific veth on the host
ip netns exec my-ns tcpdump -ni eth0 # a named namespace (ip netns add)
podman run --rm -it --net container:my-app --cap-add NET_RAW --cap-add NET_ADMIN docker.io/nicolaka/netshoot tcpdump -ni eth0 # a sidecar sharing the namespaceRootless Podman uses pasta or slirp4netns; the container’s eth0 is a user-mode device and the traffic reaches the host as normal traffic from the host’s own IP, so capture on the host interface by port. For Kubernetes, kubectl debug -it my-pod --image=nicolaka/netshoot --target=app gives a shell in the pod’s namespaces; on the node, the same nsenter approach works with the PID from crictl inspect. See Kubernetes.
VMs on libvirt or Proxmox: capture on the tap or bridge on the host (tcpdump -ni vnet3, tcpdump -ni vmbr0 host 192.0.2.30). Traffic between two VMs on the same bridge is seen on the bridge interface only if it is not offloaded to the bridge’s forwarding path; the tap devices always see it.
Remote capture over SSH, streaming into local Wireshark. -U flushes each packet, -w - writes to stdout, and the filter excludes the SSH session itself so the stream does not feed back.
ssh app.example.com 'sudo tcpdump -ni eth0 -s0 -U -w - not port 22' | wireshark -k -i -
ssh app.example.com 'sudo tcpdump -ni eth0 -s0 -U -w - not port 22' > remote.pcap # to a file instead
ssh app.example.com 'sudo tcpdump -ni eth0 -s0 -U -w - not port 22' | tshark -r - -Y 'tcp.analysis.retransmission'
ssh -J bastion.example.com db.example.com 'sudo tcpdump -ni eth0 -s0 -U -w - port 5432' | wireshark -k -i - # through a jump hostWireshark also has sshdump (an extcap interface under Capture > Options) that does the same with a GUI form, and ciscodump for IOS. sudo must not prompt for a password on the remote side, or the prompt lands in the pcap stream; use NOPASSWD for tcpdump only, or a tcpdump binary with CAP_NET_RAW.
tshark#
tshark is Wireshark without the GUI: the same dissectors, the same display filters (-Y), plus field extraction and statistics that make it scriptable. Capture filters use -f and the BPF syntax; display filters use -Y and the Wireshark syntax, and can be applied while capturing or reading.
tshark -i eth0 -f 'port 443' -c 100 # capture with a BPF filter
tshark -r capture.pcap # one summary line per packet
tshark -r capture.pcap -Y 'dns.flags.rcode != 0' # display filter on a file
tshark -r capture.pcap -Y 'tcp.analysis.retransmission' | wc -l # count retransmissions
tshark -r capture.pcap -V -Y 'frame.number == 42' # full dissection of one packet
tshark -r capture.pcap -T fields -e frame.time -e ip.src -e ip.dst -e tcp.dstport -e tcp.len -E header=y -E separator=, -E quote=d # CSV
tshark -r capture.pcap -T fields -e dns.qry.name -Y 'dns.flags.response == 0' | sort | uniq -c | sort -rn | head # most queried names
tshark -r capture.pcap -T fields -e tls.handshake.extensions_server_name -Y 'tls.handshake.type == 1' | sort | uniq -c | sort -rn # SNI per ClientHello
tshark -r capture.pcap -T fields -e http.host -e http.request.uri -Y http.request
tshark -r capture.pcap -T json -Y 'frame.number == 1' | jq '.[0]._source.layers.tcp' # JSON per packet
tshark -r capture.pcap -T ek > packets.ndjson # Elasticsearch bulk format
tshark -r capture.pcap -q -z conv,tcp # conversations: bytes and packets each way, duration
tshark -r capture.pcap -q -z conv,ip
tshark -r capture.pcap -q -z endpoints,ip # per-host totals
tshark -r capture.pcap -q -z io,phs # protocol hierarchy
tshark -r capture.pcap -q -z io,stat,10 # packets and bytes per 10-second interval
tshark -r capture.pcap -q -z 'io,stat,1,tcp.analysis.retransmission' # retransmissions per second
tshark -r capture.pcap -q -z dns,tree # DNS query types, rcodes, response times
tshark -r capture.pcap -q -z http,tree; tshark -r capture.pcap -q -z http_req,tree
tshark -r capture.pcap -q -z expert # every expert-info note, warning and error
tshark -r capture.pcap -q -z follow,tcp,ascii,0 # follow TCP stream 0 as text
tshark -r capture.pcap -q -z follow,tcp,ascii,192.0.2.10:51234,198.51.100.5:443
tshark -r capture.pcap -T fields -e tcp.stream -e tcp.analysis.ack_rtt -Y 'tcp.analysis.ack_rtt' | awk '{s[$1] += $2; n[$1]++} END {for (k in s) printf "stream %s mean rtt %.1f ms\n", k, 1000 * s[k] / n[k]}'
tshark -r capture.pcap -d tcp.port==8443,tls # decode a non-standard port as TLS
tshark -r capture.pcap -o 'tls.keylog_file:/var/tmp/sslkeys.log' -Y http2 # decrypt TLS with a key log (see below)
tshark -r capture.pcap -w subset.pcap -Y 'tcp.stream == 7' # write the packets matching a display filter
tshark -G fields | grep -i '^F.*tls.handshake' | cut -f3 | head # list field names
tshark -r capture.pcap -n # -n: no name resolution (also -N for selective)Statistics (-z) need -q to suppress the per-packet lines. -E occurrence=f takes the first value when a field repeats; -E aggregator=, joins them. -2 performs a second pass, needed for filters that depend on later packets (tcp.analysis.* on reassembly, http.response_for.uri).
capinfos, editcap and mergecap handle files:
capinfos capture.pcap # size, packet count, duration, start and end, average rate
capinfos -a -e -c capture.pcap # just start, end and count
editcap -c 100000 big.pcap part.pcap # split into files of 100000 packets
editcap -i 60 big.pcap minute.pcap # split every 60 seconds
editcap -A '2026-09-24 14:00:00' -B '2026-09-24 14:05:00' big.pcap window.pcap
editcap -s 128 big.pcap headers.pcap # truncate payloads (anonymises content, keeps headers)
editcap -d capture.pcap dedup.pcap # remove duplicates (mirror ports)
editcap -F pcap capture.pcapng capture.pcap # convert format
mergecap -w merged.pcap client.pcap server.pcap # by timestampWireshark#
Wireshark dissects every protocol it knows and annotates TCP with analysis flags (retransmission, duplicate ACK, zero window, out of order) that tcpdump does not compute. The workflow is: open the file, apply a display filter, find one bad packet, right-click Conversation Filter > TCP to see only that connection, then Follow > TCP Stream or the Statistics menus.
Display filters are a different language from BPF: fields are protocol.field, comparisons are ==, !=, >, contains, matches (regex), in {}, and booleans are &&, ||, !. Field names autocomplete in the filter bar and appear in the status bar when you select a field in the packet detail pane.
| Filter | Shows |
|---|---|
ip.addr == 192.0.2.10 | Either direction |
ip.src == 192.0.2.10 && tcp.dstport == 443 | One direction, one port |
ip.addr == 192.0.2.0/24 && !(tcp.port == 22) | A subnet, minus SSH |
tcp.port in {80 443 8443} | Set membership |
tcp.stream == 7 | One connection (the number is in the TCP header detail) |
tcp.flags.syn == 1 && tcp.flags.ack == 0 | Connection attempts |
tcp.flags.reset == 1 | Resets |
tcp.analysis.flags | Anything Wireshark thinks is wrong with TCP |
tcp.analysis.retransmission | Retransmissions (also fast_retransmission, spurious_retransmission) |
tcp.analysis.duplicate_ack | Receiver saying “I am missing something” |
tcp.analysis.zero_window | Receiver’s buffer is full: an application not reading |
tcp.analysis.window_full | Sender has filled the receiver’s window |
tcp.analysis.out_of_order | Reordering, often multiple paths or bonding |
tcp.analysis.ack_rtt > 0.2 | ACKs that took over 200 ms |
tcp.time_delta > 1 | Over a second since the previous packet in this stream (enable “Calculate conversation timestamps” in TCP preferences) |
tcp.len > 0 && tcp.flags.push == 1 | Application data segments |
tcp.window_size == 0 | Zero window advertisements |
frame.len > 1514 | Larger than a standard Ethernet frame: offload or jumbo |
ip.flags.df == 1 && ip.len > 1400 | Big DF packets, the ones that hit MTU problems |
icmp.type == 3 && icmp.code == 4 | Fragmentation needed |
| `icmp | |
dns.flags.response == 0 | Queries |
dns.flags.rcode != 0 | Errors: 2 SERVFAIL, 3 NXDOMAIN, 5 REFUSED |
dns.time > 0.5 | Answers that took over 500 ms |
dns.qry.name contains "example" | Substring on the name |
dns.qry.type == 28 | AAAA queries |
dns.flags.truncated == 1 | Truncated answers forcing TCP retry |
tls.handshake.type == 1 | ClientHello |
tls.handshake.type == 2 | ServerHello |
tls.handshake.extensions_server_name == "www.example.com" | SNI |
tls.alert_message.desc == 48 | Alert: unknown CA (40 handshake failure, 42 bad certificate, 46 certificate unknown, 70 protocol version, 112 unrecognised name) |
tls.record.version == 0x0303 && tls.handshake.type == 2 | Server chose TLS 1.2 (TLS 1.3 also says 0x0303 here; check supported_versions) |
http.request.method == "POST" | HTTP requests |
http.response.code >= 500 | Server errors |
http.time > 2 | Responses over 2 seconds after the request |
http.host matches "^api\\." | Regex |
http2 | HTTP/2 frames (needs decrypted TLS) |
arp.duplicate-address-detected | Two MACs claiming one IP |
frame contains "password" | Bytes anywhere in the frame |
_ws.expert.severity >= warning | Expert-info warnings and errors |
| `!(arp |
Useful menus: Statistics > Conversations (sort by bytes to find the elephant), Statistics > Protocol Hierarchy, Statistics > I/O Graph (plot tcp.analysis.retransmission against all packets), Statistics > TCP Stream Graphs > Time Sequence (Stevens) for a picture of stalls, Analyze > Expert Information for the list of everything flagged, Analyze > Follow > TCP/UDP/TLS/HTTP Stream for the payload as text with each direction coloured, View > Time Display Format > Seconds Since Previous Displayed Packet for gap hunting. Edit > Preferences > Protocols > TCP: “Analyze TCP sequence numbers”, “Relative sequence numbers”, “Calculate conversation timestamps” on; “Validate the TCP checksum” off (offload makes outgoing checksums look wrong).
Colouring: black background with red text is a TCP problem; dark grey is a plain TCP segment; light purple TCP, light blue UDP, light green HTTP; a black line with red is an RST or a checksum error.
Diagnosing common faults#
Retransmissions and slow transfers#
Filter tcp.analysis.retransmission || tcp.analysis.fast_retransmission. Isolated retransmissions are normal on any WAN path; a stream with more than 1 to 2 percent has loss. Duplicate ACKs from the receiver before each retransmission point at loss on the path toward the receiver; a retransmission with no duplicate ACKs is a retransmission timeout (RTO), usually the whole tail of a burst lost, or the ACK path failing. Spurious retransmissions mean the ACK arrived late rather than data being lost: latency, or a middlebox delaying ACKs. Compare tcp.analysis.ack_rtt across the connection to see where latency sits.
Zero window (tcp.analysis.zero_window) is not a network problem. The receiving application is not reading its socket fast enough: a blocked thread, a slow disk, a GC pause. tcp.analysis.window_full on the sender side with an otherwise healthy path says the receive buffer (net.ipv4.tcp_rmem, or the application’s SO_RCVBUF) is too small for the bandwidth-delay product. Time Sequence (Stevens) graph shows both as flat plateaus.
tshark -r capture.pcap -q -z 'io,stat,5,tcp.analysis.retransmission,tcp.analysis.duplicate_ack,tcp.analysis.zero_window'
tshark -r capture.pcap -Y 'tcp.analysis.retransmission' -T fields -e tcp.stream | sort | uniq -c | sort -rn | head # worst streams
tshark -r capture.pcap -Y 'tcp.stream == 7' -T fields -e frame.time_relative -e tcp.seq -e tcp.len -e tcp.analysis.flags # one stream, annotatedMTU and path MTU discovery#
Symptoms: small requests work, large responses hang; SSH connects but a big ls output freezes; TLS handshake stalls after ClientHello when the server certificate is large; things work over one path and not over a VPN or tunnel. The capture shows a large segment with DF set leaving, no ACK arriving, the same segment retransmitted at growing intervals, and no ICMP type 3 code 4 coming back because a firewall dropped it. When ICMP does arrive (icmp.type == 3 && icmp.code == 4) the sender should immediately resend with a smaller size; if it keeps sending large segments, PMTU discovery is disabled or the ICMP is not reaching the stack (nftables dropping ICMP, or a NAT that cannot map it).
tcpdump -ni eth0 'icmp[icmptype] == icmp-unreach and icmp[1] == 4' # PMTUD messages arriving
tcpdump -ni eth0 -v 'ip[6] & 0x40 != 0 and ip[2:2] > 1400 and tcp' # big DF packets going out; -v prints the total length
ping -M do -s 1472 198.51.100.5 # 1472 + 28 = 1500; lower until it passes to find the path MTU
tracepath -n 198.51.100.5 # reports pmtu along the path
ip route get 198.51.100.5 # cached mtu in the route entry after PMTUD
ip link set dev tun0 mtu 1400 # fix at the tunnel; or clamp MSS in nftables: tcp flags syn tcp option maxseg size set rt mtuLook at the MSS in the SYNs: 1460 means both ends assume a 1500-byte path. A tunnel or PPPoE in between needs MSS clamping (nft add rule inet filter forward tcp flags syn tcp option maxseg size set rt mtu) so the endpoints never send more than the path carries. See firewalld, nftables and iptables for where that rule goes.
TLS handshakes#
A TLS 1.3 handshake in the clear is ClientHello (type 1, carrying SNI, ALPN, supported versions, key shares), ServerHello (type 2), then encrypted records. TLS 1.2 also shows Certificate, Server Key Exchange and Server Hello Done in the clear. An alert (tls.alert_message) ends a failed handshake; the description says what failed on the side that sent it.
| Observation | Meaning |
|---|---|
| ClientHello, then RST from the server | Server has nothing on that port speaking TLS, or SNI-based routing rejected the name |
| ClientHello, then nothing, retransmitted ClientHello | Path or firewall drops it after the handshake starts (often MTU when the hello is large) |
ClientHello, then alert 40 handshake_failure | No common cipher suite, protocol version or curve |
ClientHello, then alert 70 protocol_version | Client offers only versions the server disabled (or the reverse) |
ClientHello, then alert 112 unrecognized_name | SNI does not match a configured virtual host |
ServerHello + Certificate, then client alert 48 unknown_ca | Client does not trust the chain: missing intermediate, private CA not installed |
ServerHello + Certificate, then client alert 42 bad_certificate or 46 certificate_unknown | Name mismatch, expired, or revoked according to the client |
| Handshake completes, application data, then RST | Application-level failure after TLS, not a TLS problem |
Encrypted Alert after handshake in TLS 1.3 | Alerts are encrypted post-handshake; decrypt with a key log or read the application’s logs |
tshark -r capture.pcap -Y 'tls.handshake.type == 1' -T fields -e ip.src -e tls.handshake.extensions_server_name -e tls.handshake.extensions.supported_version # who asked for what
tshark -r capture.pcap -Y 'tls.alert_message' -T fields -e ip.src -e ip.dst -e tls.alert_message.desc
tshark -r capture.pcap -Y 'tls.handshake.type == 11' -T fields -e x509sat.printableString -e x509af.notAfter # certificates sent (TLS 1.2 only in the clear)
openssl s_client -connect www.example.com:443 -servername www.example.com </dev/null 2>&1 | head -30 # compare with the live server; see the TLS pageDecrypting TLS in Wireshark needs the session keys, never the server’s private key alone for modern (ECDHE) cipher suites. Browsers, curl, Go (tls.Config.KeyLogWriter), Python (ssl.SSLContext.keylog_filename) and many others write a key log when SSLKEYLOGFILE=/var/tmp/sslkeys.log is set in their environment. Point Wireshark at it under Preferences > Protocols > TLS > (Pre)-Master-Secret log filename, or tshark -o tls.keylog_file:/var/tmp/sslkeys.log. The file contains everything needed to read those sessions; treat it as a secret and delete it afterwards. See TLS for certificate-side checks.
DNS problems#
Capture on the client with port 53 and on the resolver if you run it. Look at the ratio of queries to responses, the rcode, and dns.time.
| Observation | Meaning |
|---|---|
| Queries, no responses, client retries with a new ID after 5 s | Resolver unreachable or filtered; check the destination address matches /etc/resolv.conf or systemd-resolved’s upstream |
NXDomain for my-app.my-namespace.svc.cluster.local.example.com. | Search-domain expansion; ndots behaviour, see DNS |
ServFail | Resolver could not complete recursion: upstream down, DNSSEC validation failure, or a broken zone |
Refused | Resolver’s ACL rejects this client |
Answers with dns.flags.truncated == 1, then the same query over TCP | Response over 512 bytes (or the EDNS buffer); if TCP 53 is blocked the lookup fails |
| AAAA query answered, A query never answered | IPv6-only breakage on the resolver path, or the reverse; clients wait for both |
dns.time of seconds | Resolver recursing slowly; run dig +trace from the resolver |
| Responses from an address you did not query | A transparent DNS interceptor on the path |
| Same query every few milliseconds | Application without caching or a negative-TTL of 0; fix the client or add a local cache |
tcpdump -ni eth0 -vv 'port 53 and host 192.0.2.53'
tshark -r capture.pcap -q -z dns,tree # per rcode, per qtype, response-time stats
tshark -r capture.pcap -Y 'dns.flags.response == 1 && dns.time > 0.5' -T fields -e dns.qry.name -e dns.time -e ip.src
tshark -2 -r capture.pcap -Y 'dns.flags.response == 0 && !dns.response_in' -T fields -e dns.qry.name | sort | uniq -c | sort -rn | headOneliners#
# Who is talking to this host right now, by packets per source, 10-second sample
timeout 10 tcpdump -ni eth0 -q 2>/dev/null | awk '{print $3}' | cut -d. -f1-4 | sort | uniq -c | sort -rn | head
# Top talkers by bytes sent, from a capture
tshark -r capture.pcap -T fields -e ip.src -e frame.len | awk '{b[$1] += $2} END {for (k in b) print b[k], k}' | sort -rn | head -20
# New connection attempts per destination port on this host
tcpdump -ni eth0 -q 'tcp[tcpflags] & (tcp-syn|tcp-ack) == tcp-syn' 2>/dev/null | awk '{split($5, d, "."); print d[5]}' | sort | uniq -c | sort -rn
# Connections refused by this host (RST-ACK sent in reply to SYN)
tcpdump -ni eth0 -Q out 'tcp[tcpflags] & (tcp-rst|tcp-ack) == (tcp-rst|tcp-ack)'
# Which process owns the socket in a capture line (port from tcpdump, then ss)
ss -tnp 'sport = :51234'
# Confirm a packet reaches the host before blaming the firewall (see the firewall page)
tcpdump -ni eth0 -c 5 'tcp dst port 8443 and tcp[tcpflags] & tcp-syn != 0'
# Confirm the reply leaves the host
tcpdump -ni eth0 -Q out -c 5 'tcp src port 8443'
# All DNS queries as they happen, one line each
tcpdump -ni eth0 -l port 53 2>/dev/null | grep -oP 'A+\? \K\S+' | uniq
# DNS response codes as a running tally
tcpdump -ni eth0 -l -q 'udp src port 53' 2>/dev/null | grep -oE 'NXDomain|ServFail|Refused' | uniq -c
# HTTP request lines from plain-text traffic
tcpdump -ni eth0 -l -A -s0 'tcp port 80' 2>/dev/null | grep -E '^(GET|POST|PUT|DELETE|HEAD|PATCH) |^Host: '
# SNI of every TLS connection leaving this host
tshark -i eth0 -f 'tcp dst port 443' -Y 'tls.handshake.type == 1' -T fields -e ip.dst -e tls.handshake.extensions_server_name
# Which TLS versions clients negotiate to your server
tshark -r capture.pcap -Y 'tls.handshake.type == 2' -T fields -e tls.handshake.extensions.supported_version -e tls.handshake.version | sort | uniq -c
# ARP: who has an IP (duplicate address hunting)
tcpdump -ni eth0 -e 'arp and arp[6:2] == 2 and arp[24:4] == 0xc0000210' # replies for 192.0.2.16 (hex of the address)
# DHCP conversations
tcpdump -ni eth0 -v 'udp port 67 or udp port 68'
# VRRP, HSRP, OSPF and other things a router is announcing
tcpdump -ni eth0 -v 'ip proto 112 or udp port 1985 or ip proto 89'
# VLAN tags present on a trunk port
tcpdump -ni eth0 -e -c 100 vlan 2>/dev/null | grep -oE 'vlan [0-9]+' | sort | uniq -c
# Packets with a low TTL (loop or traceroute)
tcpdump -ni eth0 -v 'ip[8] < 3'
# Fragmented IP packets (a sign of a broken MTU or DNS over UDP with large answers)
tcpdump -ni eth0 'ip[6:2] & 0x3fff != 0'
# Capture for 60 seconds into a file, then summarise
timeout 60 tcpdump -ni eth0 -s0 -w /var/tmp/cap.pcap 'not port 22'; capinfos /var/tmp/cap.pcap; tshark -r /var/tmp/cap.pcap -q -z io,phs
# Retransmission rate of a capture as a percentage
t=$(tshark -r capture.pcap -Y tcp | wc -l); r=$(tshark -r capture.pcap -Y tcp.analysis.retransmission | wc -l); awk -v t="$t" -v r="$r" 'BEGIN {printf "%d/%d = %.2f%%\n", r, t, t ? 100 * r / t : 0}'
# Longest TCP streams by duration, with bytes
tshark -r capture.pcap -T fields -e tcp.stream -e frame.time_relative -e tcp.len | awk '{if (!($1 in s)) s[$1] = $2; e[$1] = $2; b[$1] += $3} END {for (k in s) printf "%.1fs %d bytes stream %s\n", e[k] - s[k], b[k], k}' | sort -rn | head
# Handshake time (SYN to ACK) per connection
tshark -r capture.pcap -Y 'tcp.flags.syn == 1 && tcp.flags.ack == 1' -T fields -e ip.src -e tcp.time_delta | sort -k2,2nr | head
# HTTP response times over 1 s, with URL
tshark -2 -r capture.pcap -Y 'http.time > 1' -T fields -e http.time -e http.response_for.uri | sort -rn | head
# Extract files transferred over HTTP into a directory
tshark -r capture.pcap --export-objects http,/var/tmp/http-objects -q
# Strip payloads before sharing a capture
editcap -s 96 capture.pcap headers-only.pcap
# Anonymise addresses while keeping structure (tcprewrite is in tcpreplay)
tcprewrite --seed=42 --infile=capture.pcap --outfile=anon.pcap
# Replay a capture onto an interface (tcpreplay; sends packets)
tcpreplay -i eth1 --mbps=10 capture.pcap
# Watch kernel drop counters while capturing to know if the capture itself is lossy
watch -n1 'cat /proc/net/softnet_stat | awk "{d += strtonum(\"0x\" \$2)} END {print \"softnet drops:\", d}"'
# Check whether offloads are inflating packet sizes in captures
ethtool -k eth0 | grep -E 'segmentation-offload|receive-offload'
# Run tcpdump without root by granting the binary capabilities (persistent until the package updates)
setcap cap_net_raw,cap_net_admin=eip "$(command -v tcpdump)"Scripts#
Ring-buffer capture as a systemd service for intermittent faults: keeps the last N files on disk, runs as the tcpdump user, and is stopped with systemctl stop when the fault has been reproduced. Install the unit, then systemctl start capture@eth0.
# /etc/systemd/system/capture@.service
[Unit]
Description=Ring-buffer packet capture on %i
After=network-online.target
[Service]
Type=simple
ExecStartPre=/usr/bin/mkdir -p /var/tmp/capture
ExecStartPre=/usr/bin/chown tcpdump:tcpdump /var/tmp/capture
ExecStart=/usr/sbin/tcpdump -ni %i -s0 -Z tcpdump -w /var/tmp/capture/%i.pcap -C 200 -W 20 not port 22
Restart=on-failure
Nice=10
IOSchedulingClass=idle
[Install]
WantedBy=multi-user.targetHealth check for a TCP service from the packet level: sends one connection attempt while capturing, then reports whether the SYN left, whether a SYN-ACK or RST came back, and the handshake time. Useful when curl just says “timed out” and you need to know which side is silent.
#!/usr/bin/env bash
# usage: syn-check.sh HOST PORT [IFACE]
set -euo pipefail
host=${1:?host} port=${2:?port} iface=${3:-$(ip -o route get "$1" | grep -oP 'dev \K\S+')}
pcap=$(mktemp --suffix=.pcap); trap 'rm -f -- "$pcap"' EXIT
tcpdump -ni "$iface" -s 96 -w "$pcap" "host $host and tcp port $port" 2>/dev/null &
tp=$!; sleep 0.5
timeout 5 bash -c "exec 3<>/dev/tcp/$host/$port" 2>/dev/null && result=connected || result="failed (rc=$?)"
sleep 0.5; kill "$tp"; wait "$tp" 2>/dev/null || true
syn=$(tcpdump -nr "$pcap" "dst host $host and tcp[tcpflags] & (tcp-syn|tcp-ack) == tcp-syn" 2>/dev/null | wc -l)
synack=$(tcpdump -nr "$pcap" "src host $host and tcp[tcpflags] & (tcp-syn|tcp-ack) == (tcp-syn|tcp-ack)" 2>/dev/null | wc -l)
rst=$(tcpdump -nr "$pcap" "src host $host and tcp[tcpflags] & tcp-rst != 0" 2>/dev/null | wc -l)
hs=$(tshark -r "$pcap" -Y 'tcp.flags.syn == 1 && tcp.flags.ack == 1' -T fields -e tcp.time_delta 2>/dev/null | head -1)
printf '%s:%s via %s: %s\n' "$host" "$port" "$iface" "$result"
printf ' SYN sent: %s SYN-ACK received: %s RST received: %s handshake: %s s\n' "$syn" "$synack" "$rst" "${hs:-n/a}"
case "$syn:$synack:$rst" in
0:*) echo ' no SYN left this host: local routing, firewall output chain, or wrong interface' ;;
*:0:0) echo ' SYN left, nothing came back: remote or path filtering, or the reply is routed elsewhere' ;;
*:0:[1-9]*) echo ' RST received: port closed on the remote host, or a firewall rejecting with tcp-reset' ;;
esacCapture summariser for a directory of rotated files: prints per file the time span, packet count, retransmission count and the top talkers, so the file containing the fault can be found without opening each in Wireshark.
#!/usr/bin/env bash
# usage: pcap-summary.sh /var/tmp/capture
set -euo pipefail
dir=${1:?directory of pcap files}
for f in "$dir"/*.pcap*; do
[[ -f $f ]] || continue
span=$(capinfos -a -e -T -m -r "$f" 2>/dev/null | awk -F'\t' 'NR == 1 {print $2, "->", $3}')
pkts=$(capinfos -c -T -m -r "$f" 2>/dev/null | awk -F'\t' 'NR == 1 {print $2}')
retx=$(tshark -r "$f" -Y 'tcp.analysis.retransmission' 2>/dev/null | wc -l)
rst=$(tshark -r "$f" -Y 'tcp.flags.reset == 1' 2>/dev/null | wc -l)
top=$(tshark -r "$f" -q -z endpoints,ip 2>/dev/null | awk 'NR > 5 && NF >= 3 {print $1, $3}' | sort -k2,2nr | head -3 | tr '\n' ';')
printf '%s\n %s packets=%s retx=%s rst=%s\n top: %s\n' "$(basename "$f")" "$span" "$pkts" "$retx" "$rst" "$top"
doneTroubleshooting#
| Symptom | Cause | Fix |
|---|---|---|
tcpdump: eth0: You don't have permission to capture on that device | Not root, no CAP_NET_RAW | sudo, or setcap cap_net_raw,cap_net_admin=eip $(command -v tcpdump) |
-w fails with Permission denied as root | tcpdump dropped to the tcpdump user (-Z) and the directory is not writable by it | Write to /var/tmp, chown tcpdump the directory, or -Z root |
| Nothing captured but traffic exists | Wrong interface, or the filter uses a name that resolved to another address, or traffic is in another namespace | tcpdump -D, ip -br addr, -i any, nsenter -n into the container |
packets dropped by kernel is large | Terminal output too slow or buffer too small | -w file, tighter filter, -B 65536, -s 128 |
| Captured packets are 64 KiB | Segmentation offload on the capture host | ethtool -K eth0 tso off gso off gro off (affects performance; revert afterwards) |
Wireshark shows [Checksum incorrect] on every outgoing packet | Checksum offload; the NIC fills it in after capture | Disable checksum validation in TCP and IP preferences |
Filter error syntax error in filter expression | Shell ate the parentheses or &, or a keyword used as a hostname | Single-quote the whole filter; use addresses |
ip6 upper-layer protocol is not supported by proto[x] | tcp[...] byte offsets need IPv4 | Capture ip6 and tcp and filter in Wireshark or tshark |
tcpdump -w - over SSH shows nothing until Ctrl-C | Output buffered | Add -U, and --immediate-mode for low rates |
| SSH remote capture pcap is corrupt | sudo password prompt or a shell banner went into the stream | NOPASSWD for tcpdump, ssh -T, and a shell with no login output |
Wireshark says Malformed Packet on a known-good protocol | Truncated snaplen, non-standard port not decoded, or a middlebox | -s0, Decode As, check frame.cap_len < frame.len |
| Only one direction of traffic appears | Asymmetric routing, or capturing on a bridge where the other direction is hardware-switched | Capture on the tap or the host, or on both ends and mergecap |
| Wireshark cannot open a 5 GB file | Memory; Wireshark loads the whole file | editcap -c or -A/-B to slice, or tshark for statistics |
| No TLS decryption despite a key log | Key log written after the session, wrong file, or TLS 1.3 with a client that does not write CLIENT_TRAFFIC_SECRET_0 lines | Check the file has entries for the session’s client random; restart the client with SSLKEYLOGFILE set before it connects |
tshark: -z ... is not a valid statistic | Wrong name or missing -q prerequisites | tshark -z help |
| Time on the two captures does not line up | Clock skew between hosts | editcap -t <offset> on one file before mergecap; sync with chrony |
Many TCP Spurious Retransmission in Wireshark | ACKs delayed or the capture point is behind a proxy that ACKs early | Look at tcp.analysis.ack_rtt and where the capture was taken |
Further reading#
- tcpdump(1): every option, the output format for each protocol and the exit status.
- pcap-filter(7): the complete capture filter grammar and the header byte expression syntax.
- Wireshark User’s Guide: capturing, display filters, statistics, follow streams and preferences.
- Wireshark display filter reference: every field name by protocol.
- tshark(1), editcap(1), mergecap(1), capinfos(1): the command-line tools and the
-zstatistics list. - Wireshark wiki: TLS: key log decryption, supported cipher suites and preferences.