Otdfbt

I'm running a django site with nginx through uwsgi. The problem is uwsgi process take a lot of CPU time when traffic become heavy. The same configuration works fine on test server and siege/ab simulate heavy concurrent traffic.

Here is one strace log of uwsgi process. http://dl.dropbox.com/u/43017476/strace.log

And some sysctl:

fs.file-max = 128000
net.core.rmem_max = 16777216
net.core.wmem_max = 16777216
net.ipv4.tcp_rmem = 4096 87380 16777216
net.ipv4.tcp_wmem = 4096 65536 16777216
net.core.netdev_max_backlog = 4096
net.ipv4.tcp_max_syn_backlog = 4096
net.core.somaxconn = 250000
net.ipv4.tcp_keepalive_time = 300

nginx conf is pretty same as other tutorials from Google. I have been trying to figure it out but no luck. However I found a lot of this pieces:

22:12:02.932276 read(8, "722", 16384) = 11
22:12:02.932504 poll([fd=8, events=POLLIN], 1, 0) = 0 (Timeout)
22:12:02.932547 write(8, "173SET NAMES utf8", 19) = 19
22:12:02.932643 read(8, "712", 16384) = 11
22:12:02.933237 poll([fd=8, events=POLLIN], 1, 0) = 0 (Timeout)
22:12:02.933279 write(8, "213set autocommit=0", 21) = 21
22:12:02.933330 read(8, "71", 16384) = 11

Whereas file descriptor 8 is a mysql socket. But Mysql server seems working pretty fine. I can see each "read" system call consumes most CPU time but don't know what's going wrong here. The same database server has been serving former PHP site without problem.

Any ideas appreciated.

strace likely will not show you what is consuming CPU time as it shows system calls. The CPU time that a process is shown to consume is time that is spent executing code in the application itself. The read/write/poll you are showing is very likely normal MySQL communication, or is the application polling the mysql FD (non-blocking) to look for new traffic.

I don't really have a solution for you as to how to find the cause of your high CPU, however I figured I would point you away from spending too much time looking for the answer in strace output :)