Packet Tracing with tcpdump, wireshark and tshark
Tracing could really be broken into multiple phases: capture, visualization and analysis. Often these are combined.
Here is an example showing how to do a HTTP packet capture with tcpdump and view it with wireshark or tshark, the command-line version of wireshark.
tcpdump -n -s0 -w /tmp/dump.pcap port 80
Press Cntl-C when done.
libpcap, the underlying software used by tcpdump, wireshark, etc has something called bpf. Filters can be combined with and, or to create complex, powerful filters to include data of interest or exclude noise.
host 192.168.1.1 port 80
Using a '.pcap' extension is a good idea since it's a registered and recognized for files stored in the Libpcap format.
Always be cognizant of the snaplen (-s#). It can be set to 0 for capturing all bits of a packet, but often something shorter like 256 is a better choice to minimize storage needs.
Wireshark: Load a pcap using the GUI...
Tshark: the text version "tshark"
tshark -r /tmp/dump.pcap
tcpdump -n -r /tmp/dump.pcap
These can be used within Wireshark and Tshark to limit the types of packets displayed, which is very helpful in the #Analysis phase.
(Need some examples)
I find having Bytes in Flight column helpful.
To add it, Edit your columns, add a "Custom" column named BiF and set the field name to "tcp.analysis.bytes_in_flight"
Let's say you want to view the HTTP headers in the capture from above? Amazingly easy to do once you know how. In wireshark, you can click one of the packets you know is part of the HTTP transaction, then right-click and choose Follow TCP Stream. (HTTP rides on TCP)
Tracing with SSLdump
ssldump is like tcpdump except you can decode SSL/TLS traffic... provide that you have access to the private key.
Here is a usage example...
/usr/local/sbin/ssldump -d -k /etc/ssl/private/ssl.key host 192.168.0.16
See the output from this example.