Have you ever tried to get Apache to handle 10,000 concurrent connections? For example, you have a very busy website and you enable keepalive on your web server. Then you set the timeout to something high like 300 seconds for ridiculously slow clients (sounds crazy but I think that’s Apache’s default). All of a sudden when you run netstat it tells you that you have thousands of clients with established connections to your machine.

Apache can’t handle 10,000 connections efficiently because it uses a one-thread-per-connection model (or if you’re using prefork then one process per connection).

If you want to allow your clients to use keepalive on your very busy website you need to use a server that uses an event notification model. That means that you have a single thread or process that manages thousands of sockets or connections. The sockets don’t block the execution of the thread but instead sit quietly until something happens and then have a way of notifying the thread that something happened and it better come take a look.

Most of us use Linux these days – of course there are the BSD die hards but whatever. The linux kernel 2.6 introduced something called epoll that is an event notification system for applications that want to manage lots of file descriptors without blocking execution and be notified when something changes.

Both lighttpd and nginx are two very fast web servers that use epoll and a non-blocking event notification model to manage thousands of connections with a single thread and just a few megs of ram (ram consumption is the real reason you can’t use apache for high concurrency). You can also spawn more than one thread on both servers if you’d like to have them use more than one processor or cpu core.

I used to use lighttpd 1.4.x but it’s configuration really sucks because it’s so inflexible. I love nginx’s configuration because it’s very intuitive and very flexible. It also has some very cool modules including an experimental embedded perl module. The performance I’m getting out of it is nothing short of spectacular. I run 8 worker processes and each process consumes about 7 megs of RAM with a few modules loaded.

So my config looks like:

request ==> nginx_with_keepalive –> apache/appserver_nokeepalive

If you’d like to read more about server models for handling huge numbers of clients, check out Dan Kegel’s page on the so called c10k problem where he documents a few other event models for servers and has a history lesson on event driven IO.

Also, if you’re planning on running a high traffic server with high concurrency you should probably optimize your IP stack – here are a few suggestions I made a while back on how to do that.