Q I have a server that has been acting strangely lately. I sometimes need to press Enter twice at the end of each line when logged in over SSH from home, but this never happens locally. Also, some of the system commands, like lsmod, are giving me segmentation faults. Actually, it's only lsmod. I think my system might be compromised. I have no problem reloading from CD because this is just a test environment, but what can I do to confirm that I've been hacked? Also, if this was my live system, what could I do to recover from this?
A Unfortunately, there are many types of system compromises around today. From the information you've given me, it's difficult to tell what state your system is in. In a distribution that comes with precompiled binaries, system files such us lsmod definitely shouldn't be segfaulting. This could be put down to bad hardware but you would probably see more commands causing these problems if that was the case. Let's assume the worst but be sure to rule hardware out. If you find that the server has been compromised, the best thing to do is re-install your operating system. Even if you're extremely skilled at routing out the attacker, you can never be absolutely sure that you've got every single backdoor secured. If re-installation isn't an option then knowing exactly what has been done should help you get your system back to a usable state. If you have access to your bandwidth stats, now would be a good time to check them out. Of the compromised servers we see, most of them are used to launch further attacks, send spam or carry out other illicit activities. If you see any sudden increase in traffic, you should get a rough idea of when an attacker gained access. This should enable you to narrow your search down somewhat.
From the clues the bandwidth charts may have given you, go through your log files. Check /var/log/ messages for any strange ssh activity. Also, /var/log/maillog may show lots of mail leaving your server. Apache's logs can give you a clue if Apache was used to compromise the server, so look for lines containing wget, cmd, ftpget or cat. It could be that one of your pages allows remote execution of commands. If you get a status of 200 to any of the above commands, they successfully ran the command. dmesg may show if somebody has tried to put a network card into promiscuous mode or if any strange kernel modules have been loaded. You could also look at lastlog to see if there are any users you weren't expecting to log in that did so. If you use one of the RPM-based distributions you could do an RPM verify (rpm -Va). This will show you any file that differs from the installed RPM package. Any binary files should get your attention here. There are several toolkits you can use to check for rootkits. Two of my personal favourites are chkrootkit (www.chkrootkit.org) and rkhunter (http://rootkit.nl).
It's worth opening /etc/passwd to look for non-root users who have a UID of 0. While you're there, check if there are any user accounts you don't recognise. You may be lucky enough to find a 'hax0r' or 'r00t', although it could also be a service name that's slightly misspelled. Open the .bash_history file for any users that have logged in to look for any suspicious commands. The last thing I'm going to cover is processes. Tools like netstat, top and ps will all show you if there are any unusual programs running. It's worth noting that these are often the first files an attacker will overwrite, often with a version that will cover his tracks. Make sure that top's CPU and memory usage are in line with the processes it shows. Check netstat for sshd (or other processes) running on an unusual port number. Be especially cautions of the high ports (above 1,024) because these don't require root privileges to open. This is a very broad topic and my discussion is by no means intended to be definitive - entire volumes have been written on this subject and nothing will beat good, solid research here.