Welcome back to Linux Networking Mastery!
Up to this point we have covered:
- Part 1 – network stack basics and inspection tools
- Part 2 – interface and IP configuration (temporary + persistent via Netplan, nmcli, systemd-networkd)
- Part 3 – routing tables, static/policy routing, namespaces, simple router setup
- Part 4 – name resolution, systemd-resolved, per-link/global DNS, troubleshooting
- Part 5 – firewalls with nftables, firewalld, ufw, stateful rules
- Part 6 – services (hardened SSH, Nginx basics, NFS/Samba shares, DHCP with dnsmasq)
Now we shift focus from configuration to visibility and debugging.
A production Linux system rarely works perfectly on the first try. You'll need tools to see what's happening in real time, capture problematic packets, measure performance, identify bottlenecks, and correlate events across logs.
This post introduces the most useful modern monitoring and troubleshooting utilities, shows typical workflows, and ties them back to issues you might encounter from earlier parts (DNS failures, firewall blocks, routing loops, slow services, etc.).
1. Socket and Connection Statistics – ss
ss (socket statistics) is the modern replacement for netstat. It's faster and shows more detail.
Most common usages:
# All listening sockets
ss -ltnp
# All established TCP connections with process name
ss -tnp
# Connections to/from a specific port
ss -tn 'sport = :80 or dport = :80'
# UDP sockets
ss -unap
# Summary statistics
ss -s
# Filter by state (very useful for troubleshooting)
ss -tn state established
ss -tn state syn-sent # stuck handshakes
Compare with old netstat (still installed on many systems but slower):
netstat -tulnpe # legacy equivalent
2. Packet Capture – tcpdump
tcpdump remains the gold standard CLI packet analyzer.
Basic examples:
# Capture everything on interface (quick look)
sudo tcpdump -i enp0s3 -n
# Capture HTTP traffic (port 80 or 443)
sudo tcpdump -i any -n tcp port 80 or tcp port 443
# Capture DNS queries/responses
sudo tcpdump -i any -n udp port 53 or tcp port 53
# Show ASCII payload (-A) for debugging HTTP
sudo tcpdump -i any -A -s0 'tcp port 80 and (((ip[2:2] - ((ip[0]&0xf)<<2)) - ((tcp[12]&0xf0)>>2)) != 0)'
# Save to file for later analysis
sudo tcpdump -i enp0s3 -w capture.pcap 'host 8.8.8.8'
Then read later:
tcpdump -nnr capture.pcap
Pro tip: Combine with Wireshark (GUI) by saving .pcap files and opening them on your workstation.
3. Performance & Bandwidth Testing – iperf3
Install: sudo apt install iperf3 or sudo dnf install iperf3
Server mode (on remote machine):
iperf3 -s
Client mode (measure throughput):
iperf3 -c server-ip -t 30 -P 4 # 30s, 4 parallel streams
iperf3 -c server-ip -u -b 0 # UDP unlimited bandwidth test
iperf3 -c server-ip --bidir # bidirectional test
Useful for diagnosing slow links, MTU issues, firewall rate limiting, etc.
4. Real-time Bandwidth & Connection Monitoring
iftop – top-like interface for bandwidth per connection
sudo iftop -i enp0s3nload – simple per-interface graphs
sudo nload enp0s3bmon – another lightweight bandwidth monitor
sudo bmon
5. Log Analysis for Network Events
Most network-related messages appear in journalctl or /var/log/syslog / /var/log/messages.
Key queries:
# Kernel network messages (interface up/down, DHCP, etc.)
journalctl -k -u NetworkManager
journalctl -k | grep -i "enp0s3"
# SSH login attempts / failures
journalctl -u ssh
# Firewall drops (nftables logging – if enabled)
journalctl | grep -i "nft\|drop"
# systemd-resolved DNS issues
journalctl -u systemd-resolved
# dnsmasq DHCP logs
journalctl -u dnsmasq
Typical Troubleshooting Workflow
Example scenario: "Clients can't reach the web server on port 443"
Check listening socket
ss -ltn '( sport = :443 )'Check firewall
sudo nft list ruleset | grep 443
orfirewall-cmd --list-all/ufw statusCheck routing & ARP
ip route get 192.168.100.55
arp -nCapture traffic
sudo tcpdump -i enp0s3 -nn 'host 192.168.100.55 and port 443'Test connectivity layers
ping 192.168.100.55→curl -v https://192.168.100.55→curl -v https://example.comCheck logs for denials / errors
Hands-On Exercises
- Use
ssto find which process is listening on port 80/443 and all established connections to it. - Run
tcpdumpwhile youcurla website → identify the DNS query and TLS handshake. - Set up
iperf3server on one machine, client on another → measure throughput and look for packet loss. - Intentionally break something (e.g., block port 22 in firewall), then use the workflow above to diagnose and fix.
Tip: Install these tools in your lab VM now — they'll be invaluable for the container networking lab in Part 10.
What's Next?
In Part 8 we explore advanced link-layer and tunneling features: interface bonding for redundancy/load-balancing, VLAN tagging (802.1Q), software bridges, and modern VPN solutions with a focus on WireGuard (now the clear favorite in 2026).