Otis Taylor – Blues God

I watched Shooter last night. Besides a few badly done slow-motion-macho-manly-walking scenes, it’s a pretty good flick. The titles at the end had this awesome blues song that I had to Google. It’s called ‘Nasty Letter’ by Otis Taylor. The man has an interesting history. He left music in the 70’s to become an antique dealer and then picked it up again in the 90’s. I bought his album ‘Truth is not Fiction’ on iTunes and it’s full of incredible gritty meaty blues.

If you’re looking for music to sip a peat and seaweed 10 year old Laphroaig single malt whisky to, then this is it.

FEEDJIT New version released

I’ve released a new version of FEEDJIT with the following improvements:

  • A real-time traffic feed updated every second on the home page
  • We’ve upgraded to a 64 bit platform that is handling a lot of traffic. Watch the real-time feed to see some of the traffic we process. However, we are currently only at 2% capacity, so if you have a high traffic blog, BRING IT!! We can handle well over 1000 hits per second.
  • We now show landing pages in your real-time traffic feed so you can see where your users arrive on your site in real-time.

“Man vs Wild” vs Les Stroud’s Survivorman

Les Stroud Rocks. There’s no question about it. Survivorman is one of the best shows ever made. Les Stroud really went into the wild with his own camera gear and filmed himself for a week without any food or water – repeatedly.

The reason it didn’t make it big is because it wasn’t fake enough.

Which is why a bullshit show called ‘Man vs Wild’ is now bigger than survivorman. James Hong (hotornot.com founder) has a little video on his blog exposing “Man vs Wild”.

The last time I saw Les, he was on Larry King chatting about a new nature movie someone else is launching. Someone needs to give the guy the credit he’s due and put some serious money behind him to do another show – and this time put it on a real network – like the Vs Channel.

Scaling from 0 to 40 hits per second in 3 days

The thing about running a widget business is that you serve as many web server requests as all your users websites, combined. And if one of your users get’s Dugg or Slashdotted, you get Slashdotted too.

After I launched FEEDJIT on Thursday (5 days ago) the traffic started picking up Friday and by Saturday morning my server was groaning under the strain. Some of the highest traffic blogs were Japanese (there are more Japanese bloggers than English) and by mid-morning the Japanese were going to sleep, so that gave me a welcome reprieve.

The first thing I did was reduce Apache’s KeepAlive timeout to 2 seconds. KeepAlive’s let clients hang on to a connection which someone else could be using. If a client uses keepalive properly then it can give you a nice performance boost, but set the timeout low so slow clients don’t waste server resources.

Then I added HTML caching for the widget serving routine using Perl’s Cache::FileCache. This gave me a huge speed increase but the stats on our widgets were 1 minute delayed – and that sucked.

By Saturday night I’d rolled out the new caching code and the server was a lot faster, but I knew it wouldn’t work long term and non-realtime stats for FEEDJIT was not an option.

By Sunday I was getting 40 hits per second and rising and the server was groaning again. I had to make some fundamental changes to the way the app was architected. The old mod_perl2, MySQL and Apache2 combination wasn’t going to cut it.

So I basically redesigned the data storage routines from the ground up. I moved from mysql to a home grown data access method.
I can’t tell you how gratifying it was when I rolled out the new code last night and watched the server load average drop from 2.5 to 0.3 (unix load where 1.0 = 100%) and hold there as our traffic continued to rise.

We have several high traffic blogs now and our busiest blogs generate around 1.5 widget loads (pageviews) per second. I’m confident that if for some reason TechCrunch added us tomorrow, we’d easily handle the traffic without breaking a sweat.

FEEDJIT notice to bloggers

I posted this a few minutes ago on FEEDJIT. Reposting here:

Sunday, August 19, 6pm PST: A message to our webmasters: Due to massive demand for FEEDJIT widgets and some extremely high traffic blogs using our service, we’ve had to delay the statistics reported by the map widget and the traffic widget by 1 minute. This is to help our servers cope with the increased load. We are working as quickly as we can to get back to real-time reporting and should have a fix within 24 hours. We are currently upgrading our servers to handle the additional traffic and improving the performance of FEEDJIT. Thanks for your patience.

Get a real-time map widget showing where in the World your blog visitors are

Bobby from Church of the Flying Spaghetti Monster came up with an idea for another FEEDJIT widget after we launched the first one on Thursday. He suggested a map showing where your users are that’s updated in real-time. That was the day before yesterday, so I launched it today. Spaghetti Monster is also our first user today (top right of the page).

It’s in the sidebar on the right of this page and you can get it for your blog here.

How to create and launch a startup in 10.5 hours

I just launched FEEDJIT. It took me about 10.5 hours (4pm until 2:30am) from the first time my hand touched the keyboard until I fixed the last bug and went live. I got a question on the Seattle Tech Startup list about how I spent my 10.5 hours. So here’s a brief summary:

  • I drew a mockup in Fireworks. It started getting complicated with user registration and so on. So I basically binned it and just wrote the software, but the mockup gave me an idea of the most basic value prop. So I made a decision to go out the door with the very very basics and see if it’s something users actually want. (1 hour)
  • I designed the database schema in SQL commands using a text editor. I mentioned this CompSci quote to a friend yesterday: “Get your data structures correct first, and the rest of the program will write itself.”. Doing the data structures in the form of a schema forced me into making all the hard decisions of what features I’m keeping and what I’m not going to have time to implement. (1 hour)
  • I wrote the functional app. I find that if I do graphical or UI work early on it can become very time consuming as I try to get just the right dropshadow on some element. So I just dove in and wrote the Javascript and server components. The app doesn’t require any registration so I could just write the widget and the server code to store and deliver the stats that are displayed. (roughly 6 hours)
  • Then the last thing I did was create the home page (the only page on the site). After cranking out code for 6 hours I was too tired to faff with dropshadows and so on. So it became purely functional. (roughly 2 hours)

I already have a server set up at serverbeach where I host this. It’s on a 10 megabit backbone connection but doesn’t cost me much. So I basically added a virtualhosts section to the httpd.conf file and copied the source code into the proper directories. I then compressed the javascript and used my SQL text files and the mysql client to dump the schema into the database. I brought it online and couldn’t figure out why everything was appearing to be in Denver, Colorado. Then I realized I’d hard-coded my own IP address into the Geo location routines. Shows you how you can screw up when you’re rushed. I fixed that and it worked perfectly.

I hope there’s some value in that. I think the smartest thing I did was to drop everything except the features that would test whether this is a product my target market would actually find useful. That remains to be seen of course, but I’m hopeful. We’re adding quite a few new blogs per hour now.