Fixing slow Linux servers

Q I just got a new dual Intel Xeon Linux machine running Red Hat Linux 3.0ES, MySQL 3.23.58,PHP 4.3.2 about two months ago. I use it to process and manipulate data, but it seems very sluggish most of the time. Here is a recent top output. I have a feeling that the problem lies in the extremely high iowait, but I'm not a server pro so I could use some help diagnosing my problem.

10:03:09 up 12 days, 20:07, 1 user,
load average: 4.87, 3.76, 3.03
73 processes: 72 sleeping, 1 running,
0 zombie, 0 stopped
CPU states: cpu user nice system irq
softirq iowait idle
total 25.6% 0.0% 6.8% 0.0% 1.6%
340.4% 24.4%
cpu00 20.7% 0.0% 1.9% 0.0% 1.9%
65.3% 9.9%
cpu01 1.9% 0.0% 1.9% 0.0% 0.0%
93.0% 2.9%
cpu02 1.9% 0.0% 2.9% 0.0% 0.0%
91.1% 3.9%
cpu03 0.9% 0.0% 0.0% 0.0% 0.0%
91.0% 7.9%
Mem: 1027996k av, 1012136k used,
15860k free, 0k shrd, 35760k buff
700900k actv, 132820k in_d,
14184k in_c
Swap: 1052248k av, 125840k used,
926408k free 739172k cached
PID USER PRI NI SIZE RSS SHARE
STAT %CPU %MEM TIME CPU
COMMAND
21366 ishop2 16 0 6016 3544 576
S 22.7 0.3 130:01 2 php
31523 root 24 0 2336 1992 1   196 D
1.9 0.1 0:10 3 sendmail
2974 mysql 15 0 78792 61M 1084 S
0.9 6.1 4665m 0 mysqld
31488 ishop2 15 0 4276 3276 772
S 0.9 0.3 8:49 1 php
753 ishop2 23 0 1244 1244 912 R
0.9 0.1 0:00 0 top
1 root 15 0 512 472 452 S 0.0 0.0
0:12 3 init
2 root RT 0 0 0 0 SW 0.0 0.0 0:00
0 migration/0
3 root RT 0 0 0 0 SW 0.0 0.0 0:00
1 migration/1
4 root RT 0 0 0 0 SW 0.0 0.0 0:00
2 migration/2
5 root RT 0 0 0 0 SW 0.0 0.0 0:00
3 migration/3
6 root 15 0 0 0 0 SW 0.0 0.0 0:00
0 keventd
7 root 34 19 0 0 0 SWN 0.0 0.0
0:00 0 ksoftirqd/0

I only have a single 73GB SCSI drive in the server now. My feeling is that the disk just can't handle the requests for info. Would a RAID solution improve my situation? During this top output the server is only running two of my processing scripts and the iowait is through the roof. I could get the exact same numbers if I were running seven or eight scripts instead of two. Any thoughts or suggestions would be appreciated.

A Without getting on your box and looking closer using vmstat and iostat, I would recommend disabling hyptherthreading by adding the "noht" option to the grub.conf kernel line:

default=0
timeout=10
splashimage=(hd0,0)/grub/splash.
xpm.gz
title Red Hat Enterprise Linux
(2.4.21-4.EL)
root (hd0,0)
kernel /vmlinuz-2.4.21-4.EL ro
root=/dev/hda2 noht
initrd /initrd-2.4.21-4.EL.img

Save and reboot. Test it again and see how it's working. If this makes a difference you may want to also turn hyperthreading off in the BIOS. If the I/O problem is hard disk related, then moving to RAID may help. Again this all depends what type of I/O is causing the bottleneck. A RAID 0 is very fast on both reading and writing, but offers no redundancy, in fact you are more likely to lose data as you have multiple points of failure instead of just one. RAID 1 offers the best read speeds, but you're limited to using two physical disks - not great if you need lots of space, and with a whole disk wasted not the most efficient price-wise either. RAID 5 is very common and is easy as well as cost effective to implement. It offers very good read speeds, but writes are a little slow.

Back to the list