November 9th, 2011

Indexing server crash

While I was out catching up with a few people in NYC last night, our main indexing server for knowabout.it decided to completely die…after spending a few hours trying to revive it, I finally gave in and pronounced it.

Of course, it’s the one server that had some code on that was not in version control, or apparently backed up anywhere…so now I’m left with rebuilding the server from scratch (done as EC2 makes this pretty simple) and working from the last version of code I did have backed up (and trying to re-apply all the updates and changes I did since that backup).

The good news is that it’s forced me to think about stuff I’ve had running for almost a year now…and I’ve identified some areas where I can make some serious upgrades and updates to improve the system as whole…the bad news is that while I implement these new ideas and put in all the fixes, the knowabout.it service is down.

These are the just the type of issues you have when you are bootstrapping a project, trying to keep costs as low as humanly possible, and operating with just one or two founders who are over-committed to doing everything to start.

So the pressure is on to get everything fixed up (I expect to have us back and better than ever within the next day)…but this is also the advantage to being a beta project and to not (yet) being a funded company or charging people for anything…being down is not the end of the world and (most) people understand and accept the situation.

Believe it or not, this is the fun part of a startup and these are the moments and the challenges I actually enjoy.