Back when the family was on vacation, Charlie
sent me this simple idea for improving fubnub.com:
Music I’ve been listening to (by recency & frequent artists)
- Take all the songs I’ve been listening to since the last publish… rank em by players per artist, then by recency.
So, if I listened to Orgy five times, then I listened to ten random songs, you’d get the benefit of seeing that I went on an Orgy kick recently.
When I first read it I thought to myself, “seems simple enough” and I quickly thought of three or four approaches to tackle the problem (when I got home).
Having solved the problem in my head, all that was left to do was actually sit down and implement my solution right? Well you would think so, but in reality that is not the case.
You see implementation is what separates you from the crowd. It’s where and how all the really ‘smart’ people you know got to be so damn smart! Because things never are as simple as you think they are going to be and never go as you initially planned.
For this specific problem, my initial thought was to simply start storing the raw feeds…then give users a simple sub-set (or mini language if you will) to do some cool aggregation things (like grouping and sorting of data based on feed tags).
Also I figured, if for no other reason than I wanted to play with it anyway, I would store these raw feeds using Amazon’s S3 service.
So step one was to figure out how to use S3 with Perl…easy enough as there happens to be a CPAN module for S3 (of course getting the S3 module and all the required libraries installed in all my development and test environments was a small challenge in itself, but one that was eventually tackled).
So now I know where I want to store data, and I know how to actually get the data stored there (and read it back out when I want)…next up was 'what’ to actually store there…
My initial plan here was to set up an automated task that will grab a user’s feed every hour (or X period) and store anything 'new’ into a master feed. This way, when it’s time to apply a template (and my mini-language) all I have to do is read the whole master feed file…
It all seems like a simple enough solution right?
However, this solution requires the merging of two XML documents…and while it seems like a simple problem on the surface, it’s actually VERY involved.
In fact, it’s taken a few days of research - working down a promising path, building tests and examples, before hitting a dead end and realizing there’s probably a 'better’ or at least 'easier’ way.
On the plus side, this is the stuff that makes 'programming’ both challenging and fun to me. Attempting to solve real problems.
But I also think it’s the area that frustrates many developers…once you start down a path towards a solution it’s often hard to step back and think that maybe there’s a better way (which is why you see so many patch-worked systems out there).
It’s also easy to get frustrated when you are 'throwing out’ hours upon hours of work (sometimes days upon days)…but I *hope* other developers realize that it’s this very work, this very experience, that you are building your skill set with.
I can’t tell you the number of times I’ve spent hours upon hours on problems that went nowhere at the time…only to have a new problem a few days (or weeks or months) pop up that was easily handled thanks to my 'throw out’ knowledge and experience.
And that’s all stuff you only acquire by going through the process of actually implementing or at least trying to implement your solutions…so don’t just *think* you’ve solved your problems…actually solve them (and just see how much more you learn in the process)!