Q I'm very aware that the amount of memory on my server could be a bottleneck. For a start, the server seems to be using swap space all the time. But I find it hard to work out just how much memory I actually need on the system to make it run efficiently. I could just buy all the RAM I can afford, but it seems there ought to be some way to better determine where the sweet spot for memory is.
A Your question, while apparently simple, really requires a lot of Linux understanding to answer. In the first place, don't be too concerned about the swap space usage. A default Linux system will practically always use swap space. Excessive use can be a problem, though. For example, you may have the latest in multi-core processors on your box, but it is the utilisation that matters. For very data-heavy processes, unless your physical RAM is fast and copious enough, the server will spend most of its time thrashing the data around in and out of swap space, and not very much time actually processing any of it.
There are a number of Linux tools that can help you determine what is actually going on with your system: top and uptime are quite useful. As well as other information, uptime displays a triplet of numbers that shows the load average on your box for the last one, five and 15 minutes. What does the 'load' number mean. Well, it is a magic number that shows the amount of work the box is doing. Higher numbers mean a lot of work, lower numbers, not so much. Actually, it represents an exponentially damped moving average of the total queue length for the CPU. But it is easier to think of it as a magic number, and you can't take this number on its own and turn it into something useful.
Your box may be very busy, but coping very well with the load. It's only if the number crawls up and grows that your box may be experiencing trouble. Running top will show the running processes and their CPU and memory utilisation. But as I have said before, high CPU utilisation isn't necessarily bad, and low utilisation isn't always good. The latter may indicate that the data is spending too long getting to and from the process, so low utilisation with a corresponding high load value is a bad sign that your I/O isn't fast enough (buy some WD Raptors and a good controller), or you don't have enough physical RAM. By looking at a combination of top, uptime and free (which displays memory usage) you should be able to determine which is the case for you. If you can't wait for a busy time on the box to test it, you can always create some activity of your own.
For example, Apache comes with a benchmarking tool, ApacheBench (the actual binary name is ab), which can simulate high demand on the server for you. Also, as a final tip, it is useful to check through the running processes on the box. Although Linux is quite good about running services only when they are needed, there is still some mileage to be had from killing off errant processes - print daemons, sound servers, X... even the HAL daemon isn't likely to be needed, but often runs by default. If you really want to make the best use of memory, you should try recompiling binaries and taking a good look at the features you need and want. You can make surprising reductions in the memory usage of things like MySQL, Apache, PHP et al. There is a lot more useful information online, and I would recommend the article on memory analysis by Lubos Lunak at http://ktown.kde.org/~seli/memory.