Server Downtime == Police Baricades and Angry World Series Fans

Paciolan is managing ticket sales for the Colorado Rockies. Their servers were hit with over 1500 requests per second and it took down not only the Rockies ticket sales infrastructure, but all Paciolans other customers too.

They claim to have been hit by a DDoS attack, but that’s something that’s hard to prove or disprove when you have corporate firewalls and AOL firewalls sending many requests from a single IP – it looks just like a DDoS attack but it actually isn’t.

Is 1500 requests per second a lot? No. Feedjit (my site) peaks at 140 requests per second and it does it with just two servers – and the data it’s serving is dynamic.

So a cluster of 10 to 30 servers should easily handle the load they’ve described – especially if all it’s doing is queueing visitors and only letting a handful through, which is what Paciolan’s ticketing software does.

The result? Police are erecting barricades around Coors Field. Here’s a quote from cNet:

“…many fans are apparently converging near Coors Field in hopes that the team will sell tickets in person through the box office; so many in fact that the police have closed streets around the ballpark and are erecting barricades, the paper reported.”

Ticketmaster is trying to buy Paciolan – the deal is currently under government review. Ticketmaster runs Mod_Perl (and so does Feedjit) and some very smart people who know a lot about scalability (and who I used to work with) work for Ticketmaster. So hopefully the deal will go through and mod_perl will come to the rescue.

btw, I’m doing a short talk in 2 days on how to scale your web servers fast based on my experience scaling Feedjit.

Wikipedia, Vulcans, Suss-Mahn and cerebral entropy

In case you were looking for it, here’s a list of fictional martial arts courtesy of wikipedia…

http://en.wikipedia.org/wiki/List_of_fictional_martial_arts

How did I arrive there? I was researching Vulcans.

There’s a martial art called Suss-Mahn used by T’Pol in one of the enterprise episodes (which I just watched for $1.99 courtesy of the awesome Unbox) and it’s named after producer Mike Sussman.

South Africa are the new Rugby world champs!!!!!!!!!!

It was a tougher game that I expected – in fact it was a bit of a nail biter. But we won and Bryan Habana has been named player of the year by the Rugby Board. 🙂

It was cool hanging out with other South Africans at St Andrews pub in Green Lake watching the game (The Kiwi and Kangaroo was full) but I was bummed when someone told me Lucky Dube has been killed in a car jacking. One day someone’s going to fix crime in South Africa. Lets just hope that day comes soon.

Is Google.com down for anyone else?

…or is it just me? Seems to have gone down for a few minutes at a time around 11pm to midnight for the last few evenings. It’s not tied to browser or workstation, but may be tied to our public IP address. All other websites work including news.google.com and blogsearch.google.com – just www.google.com seems to be having probs.

Panama Esmeralda

I picked up a bag of Panama Esmeralda coffee at Peets yesterday. It’s expensive, but this morning I made a cup in my french press using my usual method and it’s the best cup of coffee I’ve ever tasted. Usually I don’t buy into the whole ‘berry notes with a hint of whatever’ analysis. But the citrus in this coffee is incredible.

Friday rocks!!

…especially when you’ve been quoted on the front page of a major newspaper. 🙂 (The Seattle PI)

Congrats to Brian Dorsey and his team who built and launched Tagmindr in 6 hours. Brian and I have chatted via email several times this week and we still haven’t met. Hopefully this week at Seattle Tech Startups meetup.

Mark.

ps: Here’s the original blog entry that launched Feedjit. Here’s John Cook’s article online.