Linux Command Quick Reference and Survival Guide

A production alert fires, and many people panic when they SSH in and find themselves staring at a dark terminal. Clicking through an IDE is easy enough, but during a real incident, you discover that writing code and troubleshooting are two different skills. The former requires your brain; the latter also requires commands at your fingertips.

This article is not a translation of man pages. It is a Linux survival kit for developers: common commands plus a troubleshooting methodology. By the end, you will be able to find problems on a server without panicking.

1. Why Developers Must Know Some Linux Commands

There are three reasons, in order of importance.

First, server troubleshooting. Containers have made everyone more accustomed to entering environments with docker exec, but underneath, it is still Linux. Where are the logs? Which process owns the port? Is memory exhausted? Who is burning CPU? The answers are all on the command line. If you do not know the commands, you can only ask an SRE for help, and the SRE may not be available immediately.

Second, efficiency. Suppose you need to find every ERROR between 2:00 and 3:00 p.m. yesterday in a 5 GB log file. Opening it in an IDE can take 30 seconds, while a single grep command returns the results in two. Linux commands become extremely powerful when combined. Once you are proficient with them, there is no going back.

Third, interviews and career progression. Interviews for mid-level and senior roles in system design, SRE, platform engineering, and backend development all cover Linux. The question is not "Can you recite every option?" but "How would you troubleshoot a production CPU spike?" If you stumble through an answer like that, you will probably be passed over.

The commands below are therefore worth studying carefully, not merely bookmarking and forgetting.

2. Files and Text: find / grep / Pipes and Redirection

1. find: Find Files

# Recursively find all .log files under the current directory
find . -name "*.log"

# Find logs modified more than 7 days ago (useful for cleanup)
find /var/log -name "*.log" -mtime +7

# Find files larger than 100 MB (commonly used when a disk is full)
find / -type f -size +100M 2>/dev/null

# Delete files immediately after finding them
find /tmp -name "*.tmp" -delete

2>/dev/null discards permission errors and is essential when running find across the entire filesystem. -mtime +7 means "modified more than seven days ago," while -mmin -5 means "modified within the last five minutes," which is useful when investigating "who just changed this file?"

2. grep: Find Content

# Search for a keyword in a file
grep "ERROR" app.log

# Recursively search every file under a directory and show filenames
grep -rn "timeout" src/

# Ignore case and print only the matching portion
grep -ioE "https?://[^ ]+" access.log

# Invert the match and show line numbers
grep -vn "INFO" app.log

-r searches recursively, -n shows line numbers, -i ignores case, -v inverts the match, -E enables extended regular expressions, and -o prints only the matching portion. Memorize these six options and you know 80% of grep.

3. Pipes | and Redirection > >>

This is the real source of the Linux command line's power: each command is a small tool, and pipes connect them into a pipeline.

# Find the top 10 most frequent IP addresses in a log
grep -oE "[0-9.]+" access.log | sort | uniq -c | sort -rn | head

# Save command output to a file (overwrite)
ls -la > files.txt

# Append instead of overwriting
echo "done" >> deploy.log

# Save standard error as well (2>&1)
npm run build > build.log 2>&1

Remember: whenever you encounter a complex task, first ask, "Can I split this into several commands and chain them together?" That is ten times faster than writing a script.

3. Processes and Resources: top / htop / ps / kill / lsof

1. top and htop: View Overall Resource Usage

top                # Monitor CPU/memory in real time; press P to sort by CPU, M by memory
htop               # An upgraded top with colors and mouse support; once installed, there is no going back

When you open top, check three things first: load average in the upper-right corner, whose three values are the average loads over 1, 5, and 15 minutes and indicate overload when they exceed the number of CPU cores; idle on the CPU line, which shows how much CPU remains available; and free on the memory line.

Press 1 to expand the view for every CPU core, H to show individual threads, and c to display full command lines. These are the three keys I use most often.

2. ps: View a Process Snapshot

ps aux | grep node          # Classic combination for viewing all node processes
ps -ef --forest             # Show the parent-child process tree
ps -eo pid,ppid,%cpu,%mem,cmd --sort=-%cpu | head

To investigate "which process is consuming CPU," ps -eo ... --sort=-%cpu | head handles it in one line.

3. kill: Send Signals

kill 12345          # Send SIGTERM by default for a graceful exit
kill -9 12345       # Send SIGKILL to force termination; the process cannot clean up, so use cautiously
killall node        # Kill every node process by name
pkill -f "npm run"  # Kill processes by matching the full command line

-9 is a last resort. Try kill first to give the process an opportunity to clean up, and use -9 only if that fails.

4. lsof: Find Who Has a File or Port Open

lsof -i :8080           # Find who is listening on port 8080, invaluable for diagnosing port conflicts
lsof -i                 # Show all network connections
lsof +D /var/log        # Find who is writing to the /var/log directory
lsof -p 12345           # Show every file opened by process 12345

When the "Address already in use" error appears, lsof -i :PORT immediately identifies the culprit. If lsof is unavailable, ss -tlnp | grep PORT provides the same information.

4. Disk and Memory: df / du / free

df -h               # Show usage for every filesystem; -h makes sizes human-readable
df -i               # Show inode usage; if the disk is not full but cannot accept files, its inodes are probably exhausted

du -sh /var/log     # Show the total size of a directory
du -h --max-depth=1 /var | sort -h      # Drill down one level at a time to find what consumes the most space

free -h              # Show memory; focus on available rather than free

Several common pitfalls:

• When a disk is full, check df -h first to identify the affected filesystem. Then use du -h --max-depth=1 | sort -h to drill down one level at a time and find large files.
• With free, look at available, not free. Linux uses idle memory for disk caching (buff/cache), which it releases automatically when needed. A low free value therefore does not necessarily mean memory is tight.
• Inode exhaustion is a hidden bug: the disk still has space, but creating a file returns "No space left." If df -i shows that all inodes are used, a directory has usually accumulated millions of small files.

5. Networking: ss / netstat / curl / ping / tcpdump

1. Ports and Connections

ss -tlnp             # The current standard for viewing listening ports; faster than netstat
ss -tan state established    # Show established connections
netstat -tlnp        # The established older tool; use it when ss is unavailable

ss replaces netstat and is included by default in modern distributions. -t means TCP, -u UDP, -l listening sockets, -n disables hostname resolution for speed, and -p shows processes.

2. curl: HTTP Requests

curl -i https://api.example.com/health        # Include response headers
curl -X POST -d '{"k":"v"}' -H "Content-Type: application/json" URL
curl -w "%{http_code} %{time_total}s\n" -o /dev/null -s URL    # Show only the status code and elapsed time
curl --resolve api.example.com:443:1.2.3.4 https://...         # Bypass DNS and connect directly to a specified IP

The -w custom output format is invaluable for diagnosing slow endpoints. time_namelookup, time_connect, and time_starttransfer respectively report DNS lookup time, connection time, and time to first byte, immediately revealing which stage is slow.

3. Connectivity

ping example.com              # Test ICMP connectivity; some firewalls block ICMP, so a failed ping does not prove the host is unreachable
traceroute example.com        # Show every routing hop to locate where the network path stalls
mtr example.com               # An enhanced traceroute that runs continuously and shows packet-loss rates

4. tcpdump: Capture Packets (The Ultimate Tool)

sudo tcpdump -i any port 8080 -nn -c 100     # Capture 100 packets on port 8080
sudo tcpdump -i eth0 host 1.2.3.4 -w out.pcap   # Capture traffic for a specific host and write it to a file for inspection in Wireshark

Packet capture is the last resort. When HTTP reveals nothing, an SSL handshake fails, or protocol behavior is strange, capture the traffic and inspect it in Wireshark. Combine this with strace -p PID to inspect system calls, and there are almost no obscure problems you cannot diagnose.

6. The Text-Processing Trio: grep / sed / awk, Essential for Log Troubleshooting

These three commands are the heart of Linux text processing. Their individual power is modest, but combined, they rival writing code.

grep (Find)

As discussed above, its core purpose is to filter lines containing a pattern.

sed (Edit)

# Replace every occurrence of foo with bar in a file (in place)
sed -i 's/foo/bar/g' file.txt

# Show only lines 10 through 20
sed -n '10,20p' app.log

# Delete empty lines
sed -i '/^$/d' file.txt

-i modifies the file in place and is dangerous. Before using it, run the command without -i to confirm that its output is correct.

awk (Process Columns)

# If the second column of access.log is the IP, count occurrences of each IP
awk '{print $2}' access.log | sort | uniq -c | sort -rn | head

# Sum the third column, such as response sizes, to calculate total traffic
awk '{sum+=$3} END {print sum}' access.log

# Add a condition: include only requests with status codes >= 500
awk '$9 >= 500 {print $7}' access.log | sort | uniq -c | sort -rn

awk splits each line into whitespace-delimited fields ($1, $2, ...), and it can also use conditions and an END block for aggregation. It is irreplaceable for log analysis.

Example combining the trio: find the endpoint with the most 500 errors:

awk '$9 == 500 {print $7}' access.log | sort | uniq -c | sort -rn | head -10

This one line replaces an entire Python script.

7. Incident Troubleshooting Workflow: Resources → Processes → Logs → Network

This is the most important section. Memorizing commands is useless if you do not know what to check first. Here is the four-step method I use:

Step 1: Check resources. Start with top or htop to inspect CPU, memory, and load. Then run df -h to check disk usage. If these three numbers are normal, you can eliminate more than half of the likely problems because full disks, exhausted memory, and saturated CPUs are the three most common categories. It is also useful to run uptime for the load and system uptime.

Step 2: Check processes. If resource usage is abnormal, drill down to individual processes. Use ps aux --sort=-%cpu | head to find CPU-intensive processes and ps aux --sort=-%mem | head to find memory-intensive ones. Run lsof -p PID to see what a process is doing. If you suspect a port conflict, use lsof -i :PORT or ss -tlnp to identify it.

Step 3: Check logs. Once you find the process, inspect what it is reporting. For application logs, use tail -f app.log for real-time output or less +F app.log when you also need to scroll backward. For system services, use journalctl:

journalctl -u nginx -f              # Follow nginx logs in real time
journalctl -u myapp --since "1 hour ago"
journalctl -p err                   # Show only error-level messages

Use dmesg -T to inspect kernel logs. OOM killer terminations and disk errors appear there, and beginners commonly overlook them during long investigations.

Step 4: Check the network. Consider the network only after the first three steps reveal no problems. Use ss -tan to inspect connection states; large numbers of TIME_WAIT or CLOSE_WAIT connections are common symptoms. Use curl -w to measure endpoint latency, ping or mtr to inspect the path, and finally tcpdump to capture packets.

After these four steps, you can locate 90% of incidents. The remaining 10% usually require deeper profiling with perf, strace, or flame graphs, which is a subject for another article.

One piece of contrary advice: do not begin by running tail -f on a log. If you dive into a sea of logs before checking resources, you will lose your bearings among ten thousand lines of ERROR messages. First determine what category of problem you have: resources, processes, or networking. Then inspect the relevant logs. It is much more efficient.

Summary

• Commands are a developer's responsibility, not just an SRE's. Independently troubleshooting a server over SSH is a foundational skill for mid-level and senior engineers. In an interview, a fluent answer to "How would you troubleshoot a production CPU spike?" separates strong candidates from the rest.
• Pipes are the soul of Linux. A combination such as grep | sort | uniq -c | sort -rn | head is ten times faster than writing a Python script. When facing a complex task, first ask, "Can I chain commands together?" rather than opening an IDE.
• The text-processing trio of grep / sed / awk is essential for log troubleshooting. awk in particular is irreplaceable for column processing, conditions, and aggregation.
• Remember three numbers when checking resources: load average, CPU idle, and disk usage. For memory, inspect available, not free; use df -i to detect inode exhaustion.
• Use the four-step incident troubleshooting workflow: resources → processes → logs → network. Start with top / df, continue with ps / lsof, then use journalctl / tail, and finish with ss / curl / tcpdump. The order matters.

One final sentence: Linux commands are not for earning a certification; they are for identifying a problem within five minutes when an alert fires at 3:00 a.m., instead of being completely lost. Once these few dozen commands become second nature, you will discover that so-called "production intuition" and "engineering instincts" are really muscle memory built from combining them.