Otdfbt

I have a server with 2 cores and 2GB RAM which is struggling with memory utilisation.

When I run free -m I regularly get under 50MB free - right now it shows 378MB free with only 1 user actually on the site:

 total used free shared buffers cached
Mem: 1869 1772 96 5 34 247
-/+ buffers/cache: 1491 378
Swap: 1023 34 989

When I run ps-aux I see lots of httpd requests which started yesterday and are still open.

apache 5789 0.0 3.8 1038176 72904 ? S 03:18 0:27 /usr/sbin/httpd
apache 5790 0.0 3.3 1029400 64216 ? S 03:18 0:26 /usr/sbin/httpd
apache 5792 0.1 3.4 1031124 65876 ? S 03:18 0:35 /usr/sbin/httpd
apache 28042 0.1 3.3 1029648 64868 ? S 04:53 0:28 /usr/sbin/httpd
apache 28043 0.0 3.8 1038412 73420 ? S 04:53 0:23 /usr/sbin/httpd
apache 31728 0.1 3.8 1038516 73008 ? S 07:53 0:14 /usr/sbin/httpd
root 31912 0.0 0.7 968160 14860 ? Ss 2014 0:03 /usr/sbin/httpd
apache 31914 0.0 0.0 246808 1364 ? S 2014 0:01 /usr/sbin/httpd
apache 31915 0.1 3.7 1038644 71700 ? S 2014 1:02 /usr/sbin/httpd
apache 31916 0.1 3.7 1038480 71844 ? S 2014 0:56 /usr/sbin/httpd
apache 31917 0.0 3.1 1026120 60044 ? S 2014 0:52 /usr/sbin/httpd
apache 31920 0.1 4.1 1045972 79940 ? S 2014 0:59 /usr/sbin/httpd
apache 31921 0.0 3.8 1039180 73040 ? S 2014 0:50 /usr/sbin/httpd
apache 31923 0.1 3.3 1029684 63908 ? S 2014 0:59 /usr/sbin/httpd
apache 31924 0.1 3.2 1027808 61632 ? S 2014 0:53 /usr/sbin/httpd
apache 31927 0.1 3.2 1027844 61664 ? S 2014 0:54 /usr/sbin/httpd
apache 31929 0.1 3.8 1039660 73528 ? S 2014 0:59 /usr/sbin/httpd
apache 31931 0.1 3.2 1028592 62492 ? S 2014 0:58 /usr/sbin/httpd
apache 31933 0.0 3.2 1029036 62876 ? S 2014 0:52 /usr/sbin/httpd
apache 31934 0.1 3.3 1029568 63232 ? S 2014 0:54 /usr/sbin/httpd
apache 32022 0.0 3.8 1038928 73104 ? S 2014 0:41 /usr/sbin/httpd

Output of top sorted by memory usage is:

31920 apache 20 0 1021m 78m 9064 S 0.0 4.2 0:59.48 httpd
31929 apache 20 0 1015m 71m 8428 S 0.0 3.8 1:02.44 httpd
28043 apache 20 0 1014m 71m 9176 S 0.0 3.8 0:26.59 httpd
32022 apache 20 0 1014m 71m 8796 S 0.0 3.8 0:44.04 httpd
31921 apache 20 0 1014m 71m 8408 S 0.0 3.8 0:51.95 httpd
31728 apache 20 0 1014m 71m 8768 S 0.0 3.8 0:15.00 httpd
 5789 apache 20 0 1013m 71m 8716 S 0.0 3.8 0:28.57 httpd
 591 apache 20 0 1013m 71m 8756 S 0.0 3.8 0:42.92 httpd
31916 apache 20 0 1014m 70m 8368 S 0.0 3.8 0:59.94 httpd
31915 apache 20 0 1014m 70m 8384 S 0.0 3.7 1:04.74 httpd
 5792 apache 20 0 1006m 64m 8744 S 0.0 3.4 0:37.37 httpd
 5790 apache 20 0 1006m 63m 8800 S 0.0 3.4 0:28.23 httpd
28042 apache 20 0 1005m 63m 9176 S 0.0 3.4 0:29.50 httpd
 584 apache 20 0 1006m 62m 8456 S 0.0 3.4 0:37.14 httpd
 586 apache 20 0 1006m 62m 8412 S 0.0 3.4 0:44.48 httpd
 588 apache 20 0 1006m 62m 8424 S 0.0 3.4 0:43.11 httpd
 587 apache 20 0 1006m 62m 8420 S 0.0 3.4 0:47.20 httpd
31923 apache 20 0 1005m 62m 8796 S 0.0 3.3 1:01.21 httpd
31933 apache 20 0 1005m 62m 8376 S 0.0 3.3 0:54.89 httpd
31927 apache 20 0 1004m 60m 8392 S 0.0 3.2 0:58.68 httpd
31924 apache 20 0 1003m 60m 8412 S 0.0 3.2 0:53.68 httpd
31934 apache 20 0 1003m 60m 8408 S 0.0 3.2 0:56.56 httpd
31917 apache 20 0 1003m 60m 8404 S 0.0 3.2 0:54.97 httpd
 1871 root 20 0 157m 17m 1968 S 0.0 0.9 0:53.10 lfd
 5597 apache 20 0 952m 16m 5212 S 0.0 0.9 0:01.02 httpd
31912 root 20 0 945m 14m 13m S 0.0 0.8 0:03.37 httpd
 5854 root 20 0 98364 4676 3672 S 0.0 0.2 0:00.09 sshd
 6557 postfix 20 0 82264 4420 3292 S 0.0 0.2 0:00.04 cleanup
 6560 postfix 20 0 82168 4316 3324 S 0.0 0.2 0:00.03 smtp
 4411 postfix 20 0 81964 4048 3096 S 0.0 0.2 0:00.04 pickup
 5856 root 20 0 105m 1944 1528 S 0.0 0.1 0:00.02 bash

In my httpd.conf I have:

KeepAlive On
MaxKeepAliveRequests 100
KeepAliveTimeout 2


<IfModule prefork.c>
StartServers 4
MinSpareServers 12
MaxSpareServers 24
ServerLimit 50
MaxClients 50
MaxRequestsPerChild 3000
</IfModule>

From the above I assume this means that KeepAlive is on but that it will only keep any session alive for 2 seconds - if that is correct, why do I still have lots of processes running from yesterday and how can I remediate against this?

keepalive and KeepAliveTimeout only controls how long a session to a client is kept alive, but if the session is closed, that doesn't mean at all that the process gets killed, as any process in your config is allowed to handle up to 3000 requests (MaxRequestsPerChild).

Your system is configured to keep up to 24 idle Apache processes around (12 are started immediately after you start Apache) (Options MinSpareServers and MaxSpareServers).

However, an idle Apache process usually doesn't use all that much RAM on its own, most is shared with all other processes. You need to find out what's really eating up the RAM and why.

To confirm that Apache is taking all your memory, you can use this snipped:

function memperuser() 
ps aux 

Then type memperuser.

Another useful way to see the shared and private memory break-down is to use ps_mem.py

That will show you how much shared and private memory Apache is consuming.

You should also take a look through all of the LoadModule entries in your httpd.conf and determine if you need all of them. Usually the biggest culprit is mod_php. If you don't need PHP, then simply disable that module. It might get loaded in its own config under httpd.d/

If you are not using PHP, you can also reduce the memory footprint of Apache by using mpm_event or mpm_worker instead of mpm_prefork. The Apache website has some really good documentation on how to use the different MPM modules.

You can get even further reduction of memory footprint by using the latest APR apr, apr-util and apr-iconv and Apache 2.4 branch. How you go about compiling and/or installing that is highly dependent on your setup, so I will defer to your specific OS forums.

All of these options really depend on how you are using Apache.