How to Diagnose Problems Quickly on Linux Servers
-
Upload
richard-cunningham -
Category
Technology
-
view
364 -
download
2
Transcript of How to Diagnose Problems Quickly on Linux Servers
How to Diagnose Problems Quickly on Linux Servers
Richard Cunningham (@rythie)
Not working
Running Slow
Two Types of Problems
Basics
ps – List of processes (use ps -ef)
top – Similar to ps but shows memory usage and is interactive
dmesg – Messages directly from the kernel
lsof – list open file and network ports
/var/log/ - Where most/all of the logs live
du – you may be out of disk space!
Understanding 'top'
top - 17:59:20 up 272 days, 22:17, 1 user, load average: 0.27, 0.33, 0.29Tasks: 142 total, 1 running, 141 sleeping, 0 stopped, 0 zombieCpu(s): 1.4%us, 0.5%sy, 0.3%ni, 95.3%id, 1.9%wa, 0.0%hi, 0.1%si, 0.5%stMem: 2051088k total, 1986640k used, 64448k free, 48156k buffersSwap: 4194300k total, 129028k used, 4065272k free, 810744k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 1156 mysql 20 0 793m 467m 5060 S 2.0 23.4 16960:15 /usr/local/mysql-5.1 4130 richie 20 0 10916 940 668 R 2.0 0.0 0:00.01 top -cbn1 17311 backend 20 0 115m 40m 6240 S 2.0 2.0 12:54.17 /usr/local/bin/php 1 root 20 0 10408 508 480 S 0.0 0.0 6:32.75 init [3] 2 root 20 0 0 0 0 S 0.0 0.0 0:00.00 [kthreadd] 3 root 20 0 0 0 0 S 0.0 0.0 5:56.83 [ksoftirqd/0]
/var/log
ls -ltr /var/log – see most recently modified
tail -f /var/log/message – watch for new lines at the end of a log file
grep – most of those logs have too much irrelevant stuff in them, grep helps
Slowness
iostat – I/O device statistics
free – how much free memory you have
sar – historical logs of performance
Understanding 'free'
$ free -m total used free shared buffers cachedMem: 3019 2908 111 0 225 1187-/+ buffers/cache: 1495 1524Swap: 0 0 0
What are System Calls?
Compiled Program (including libraries)
Kernel
open(), read(), write()...
Hardware
011100101101010...
strace – typical output
open("/etc/configfile.conf", O_RDONLY) = 3
read(3, "#this is the config file for...”, 4096) = 32
write(1, "hello", 5) = 5
strace - invocation
strace -p <process id>
strace <program name>
Wireshark
Wireshark
Wireshark
Wireshark
Usetcpdump -s0 -w filename
To capture network traffic to a file for later analysis in wireshark (i.e. copy it to you local
machine)
Wireshark available for OSX, Windows and Linux and is Open Source
The End
Richard Cunningham(@rythie)