Here’s how you run a highly concurrent website on a shoe-string budget:

The first thing you’ll do is get a Linode server because they have the fastest CPU and disk.

Install Apache with your web application running under mod_php, mod_perl or some other persistence engine for your language. Then you get famous and start getting emails about people not being able to access your website.

You increase the number of Apache threads or processes (depending on which Apache MPM you’re using) until you can’t anymore because you only have 360MB of memory in your server.

Then you’ll lower the KeepaliveTimeout and eventually disable Keepalive so that more users can access your website without tying up your Apache processes. Your users will slow down a little because they now have to re-establish a new connection for every piece of your website they want to fetch, but you’ll be able to serve more of them.

But as you scale up you will get a few more emails about your server being down. Even though  you’ve disabled keepalive it still takes time for each Apache child to send data to users, especially if they’re on slow connections or connections with high latency. Here’s what you do next:

Install Nginx on your new Linode box and get it to listen on Port 80. Then reconfigure Apache so that it listens on another port – say port 81 – and can only be accessed from the local machine. Configure Nginx as a reverse proxy to Apache listening on port 81 so that it sits in front of Apache like so:

YourVisitor <—–> Nginx:Port80 <—–> Apache:Port81

Enable Keepalive on Nginx and set the Keepalive timeout as high as you’d like. Disable Keepalive on Apache – this is just-in-case because Nginx’s proxy engine doesn’t support Keepalive to the back-end servers anyway.

The 10 or so Apache children you’re running will be getting requests from a client (Nginx) that is running locally. Because there is zero latency and a huge amount of bandwidth (it’s a loopback request), the only time Apache takes to handle the request is the amount of CPU time it actually takes to handle the request. Apache children are no longer tied up with clients on slow connections. So each request is handled in a few microseconds, freeing up each child to do a hell of a lot more work.

Nginx will occupy about 5 to 10 Megs of Memory. You’ll see thousands of users concurrently connected to it. If you have Munin loaded on your server check out the netstat graph. Bitchin isn’t it? You’ll also notice that Nginx uses very little CPU – almost nothing in fact. That’s because Nginx is designed using a single threaded model where one thread handles a huge number of connections. It can do this with little CPU usage because it uses a feature in the Linux kernel called epoll().

Footnotes:

Lack of time forced me to leave out all explanations on how to install and configure Nginx (I’m assuming you know Apache already) – but the Nginx Wiki is excellent, even if the Russain translation is a little rough.

I’ve also purposely left out all references to solving disk bottlenecks (as I’ve left out a discussion about browser caching) because there has been a lot written about this and depending on what app or app-server you’re running, there are some very standard ways to solve IO problems already. e.g. Memcached, the InnoDB cache for MySQL, PHP’s Alternative PHP Cache, perstence engines that keep your compiled code in memory, etc..etc..

This technique works to speed up any back-end application server that uses a one-thread-per-connection model. It doesn’t matter if it’s Ruby via FastCGI, Mod_Perl on Apache or some crappy little Bash script spitting out data on a socket.

This is a very standard config for most high traffic websites today. It’s how they are able to leave keepalive enabled and handle a huge number of concurrent users with a relatively small app server cluster.  Lighttpd and Nginx are the two most popular free FSM/epoll web servers out there and Nginx is the fastest growing, best designed (IMHO) and the one I use to serve 400 requests per second on a small Apache cluster. It’s also what guys like Wordpress.com use.

There’s a great show-down video at the Virtual Goods Summit where Mike confronts Offerpal CEO Anu Shukla about this and she delivers a rather colorful response. Video below.

Buried in the comments on TC are a few experienced words from HotOrNot founder James Hong. Love the hotel pay-per-view analogy:

We ran offers like this back in 2005 for a very short period of time at HOTorNOT, that is until we realized what was going on. In a nutshell, the offers that monetize the best are the ones that scam/trick users. Sure we had netflix ads show up, and clearly those do convert to some degree, but i’m pretty sure most of the money ended up getting our users hooked into auto-recurring SMS subscriptions for horoscopes and stuff. When I hear people defending their directory of deals by saying Netflix is in there, i am reminded of how hotel pay-per-view has non-pornographic movies. Sure it gives them good cover, but we all know where the money is made.

In the end, we decided to turn the offers off. Quite frankly, the offers made us feel dirty, and pretty much on the same level as spammers. For us, the money just wasn’t worth it. On top of that, we relied on our goodwill with users and focused on growing by having a product and company that our users liked. Our sense was that using scammy offers would make good money in the short run, but would destroy our userbase in the end. Perhaps apps on facebook don’t feel this pressure because facebook is so huge, and there are always new people to burn.

I’d like to point out that there are some game companies out there who are holding out on using offers to monetize their users. Personally, that makes me 10 times more likely to pull my credit card out for them.

PS. I don’t think the concept of letting people fulfill offers to get credits is structurally a bad one. I for one would like to see the offer networks work together to create some set of public agreement on what types of practices are banned from their network, and perhaps they can evan have some sort of certification logo. These practices will only stop when companies are not competitively crippled by NOT doing them. In effect, we need a nuclear non-proliferation treaty among the offer networks.

For soap opera fans here’s the war of words at the Virtual Goods Summit between Mike and Anu. [YouTube is currently down - so check back in a few if it doesn't show up.]

With surprisingly similar space-time coordinates.

Our love of Facebook is duly recanted.

We’re no longer Zuckerberg’s subordinates.

…and I’ve always hated that friend detail feature. </end rant>

A while ago Digg passed Slashdot in traffic. (I’ve written about this before) An article covering the phenomenon got Dugg and thousands of Digg users clicked the link to Alexa and installed the Alexa toolbar. Notice the weird spike where the graphs meet. That skewed the Alexa results even further in Digg’s favor.

So now Slashdot looks even worse to journalists – most of whom are writing about Digg and calling Rob for background. Which is why Rob can’t help asking you to pretty please install the Alexa toolbar to make slashdot look good to journalists again.