A blog from the mind of falicon

Gawk.it - making conversations searchable.

Thu, 24 May 2012 14:45:32 -0400

I’m the cub master for my kids school cub scout group, an assistant coach for one son’s baseball team, and an assistant coach for the other son’s track & field team. So this past weekend I was busy with a camping trip for the scouts Sat. night, a baseball game Sunday, and a track meet Sunday (the typical fun filled weekend for me these days).

Anyway - because of my schedule, I couldn’t make it to the Techcrunch hackathon this past weekend. But just because I couldn’t physically be there doesn’t mean that I wanted to miss out on the fun…

So I took a few hours this past weekend (mostly at night after the kids were sleeping) and threw together my own little hack which is now live and available to experience at - gawk.it

Since I don’t get the chance to get up on stage and introduce gawk.it to the world (like I did a few years back for appsigot) — I’ll just give the quick pitch here instead:

What gawk.it is today

Gawk.it started as a tiny experiment.

I wanted to see what would happen if I built a dead-simple way to search the content of avc.com that included both the blog posts AND the Disqus comments.

I wondered what kind of information is really in this data set? Would it be something that I actually used? If so, how often would I actually use it? What other ideas and projects might it spawn? Is there a larger idea that it might lead to?

I didn’t know, but I figured why not hack something quick together and try to find out? So that’s what I did.

Right now your gawk.it experience should provide a simple, yet powerful, way to search the blog posts and comments found on avc.com and cdixon.org (after I got this stuff working for avc, the first thing I wanted to do was expand the data set [and what better source than my currently 2nd favorite blog?]. Expanding the data set also helped me to streamline the process so I could continue to expand the data set down the road [i.e. I’ll be releasing features soon to allow others to add their blogs to the master index]).

Admittedly, only offering up search across these two blogs (and their comments) is a bit limiting at the moment…but the good news, these two blogs actually represent an extremely diverse and active group that generates lots of interesting discussions every single day. So I think even with this limited first version, you’ll find some interesting content and conversations when you gawk.it.

All that being said, I’m happy to say, the gawk.it you see today is really just the starting point (turns out the answers to the initial questions I listed above are VERY interesting and flush with opportunity).

So here’s a little bit on my thinking on what gawk.it will be in the near future.

Ever learn something from listening in on someone else’s conversation?

Bringing that ability into an online tool is the core idea behind gawk.it.

If you take a look around the internet today, you’ll notice that there are actually billions of conversations happening.

Some of these conversations have sat idle for years, just waiting for a random person to come along and breath new life into them. Some are brand new with a frenzy of action being added every second.

If you want to get involved in any of these conversations, or just mine them for the amazing history/knowledge the best ones contain, how do you find them?

Google, Bing, and DuckDuckGo are great search engines (and there are others too) but they treat every set of data the same way.

They’re not focused on conversations, or getting you into the right conversations, they’re really just focused on getting you to quality web content and facts (sometimes this content turns out to be conversational, but the chances of that are random at best).

Add to that fact that many of today’s most popular commenting systems (like Disqus) are implemented via Javascript and most popular search engines don’t actually index data loaded via Javascript (yet)…and you quickly realize that today’s “search” just doesn’t work for finding or getting involved in good conversations.

gawk.it is going to fix this problem.

…or at least that’s where I’m heading with the project at the moment.

So why not jump over to gawk.it and give it a try? (and don’t forget to give it a go on your mobile web browser as well!)

Then let me know what you think and what ideas you’ve got in the comments below!

Today's random idea: Disqus search

Mon, 14 May 2012 17:29:20 -0400

First a disclosure: I believe there are a few people already working on this idea (actually I know for a fact that at least one company already has a working prototype in this realm ‘in stealth’). Still, I don’t believe anyone is purely focused on it as a stand-alone service or with the specific details I would like to see, so I thought I would dump out my random thoughts for the idea anyway.

First a few important things I believe to be true:

1. Tons of interesting conversations, often even more interesting than the original content that sparked the conversation, are happening within Disqus across the internet.

2. Most search engines (especially Google and Bing) are not currently indexing Disqus comments in a ‘useful or meaningful’ way.

3. There is a clear, and successful, revenue model to implement around quality search.

So, what’s the actual idea?

Simple to say, slightly hard to do. Build a search engine that indexes, in a quality way, all the content flowing through Disqus in as near-realtime as possible.

You’ll notice the subtle, but key, thing in the idea is the ‘in a quality way’…what does that mean? That’s the tricky part I think.

To me, it would mean do some auto-tagging and contextual grouping (ie. analysis) between the content within a comment, the content the comment was associated with, the content within the thread as a whole, and then of course the relationship/meta-data associated with the commenters themselves.

That is, you need to develop a ‘conversation rank’ algorithm.

As you can imagine, this quickly becomes a massive data set problem (with many interesting variables)…so it’s really not a trivial thing to get correct (and there are quite possibly many ‘quality’ ways to approach/solve this).

And this is why I also believe that, for it to work well, it probably has to be it’s own standalone service (and not a feature worked into another, larger, service — though a service with a large enough scale might be a ‘good enough’ solution for now since nothing else exists).

Doesn’t Disqus already do this? If not, shouldn’t they be the ones to implement it?

Disqus does actually have a search feature built in…but it’s not at all like what I’m talking about here.

(As an aside I suspect that no one really uses the version of search they currently offer up — in fact I urge you to go play with it and you’ll see why).

So while I agree that they *should* be the ones to do it, I don’t think Disqus will actually build something like what I’m talking about/envisioning any time soon.

Why?

The reason is, I believe they are mostly focused on the blogger and helping the blogger to engage with the audience around the blogger’s content.

Beyond that primary focus, they care deeply about the experience for the commenter but from the view of making it easier/better to engage with the blogger (and others around that bloggers’ content).

So I don’t believe they are currently that focused on the data set across the Disqus universe, and the amazing stuff that *could* be done with it.

This leaves a huge opportunity for the rest of us if we so choose to act.

Side note: I do suspect that a successul implementation of what I’m talking about here, even at a small scale, would be a high acquisition target for Disqus. If nothing else, it would force them into thinking about, and acting on the bigger picture, about what people really want from the Disqus data set (ala the Summize deal for Twitter back in the day).

The other advantage that I see around this being a standalone service is that you could include many other data sets around conversations that are not currently being indexed (in any quality way).

So anyway - if I were to be thinking about building a new search engine, this is the angle I personally would be the most interested in working right now (and the one that I think is currently being the most underserved)…

What do you think?

picking a revenue model

Thu, 10 May 2012 11:16:36 -0400

Step back from your project and clear your mind for a second…now without thinking about what you are already doing, or could be doing, think about this:

How would you like to make money? What would you like to be paid for?

Once you’ve got the answer for that, dig back into your project and start making a plan to connect the two.

Explore how the things you are doing now can or will eventually lead you to the answer you gave above.

Did you cop out and say “advertising”? Then it’s all about eyeballs and clicks…what are doing to ramp that up? How long is it going to take to get to the scale you need there? How much is it going to cost (yes, the dirty secret to making money via advertising is that you generally have to buy your way into it).

Did you say “subscriptions”? Then what experience do you need to provide to make it compelling enough? How long is it going to take to get that experience perfected? How much is it going to cost to acquire a user (yes, the dirty secret to making money via subscriptions is that, like mail order, you buy your customers and profit from the lifetime value they bring you — so you better be planning an experience that keeps them around over the long haul if you intend to profit)

Did you say something else? Then you’re heading down a road less travelled these days…fret with questions and unknowns, but also lots of excitement and potential rewards…and I would love to hear about it and your journey (in the comments).

Some random thoughts on the social web...

Wed, 09 May 2012 21:44:23 -0400

I’ve been meaning to flesh out some thoughts on the ‘social web’, but haven’t found the time to sit down and actually do it. So instead of the nice, clean, thoughtful post I wanted to do, I’m just going to do the rough cut version and dump the basic notes I’ve got floating around in my head…

Social is really all about showing that you care and that you are paying attention to what someone has to say. Therefore, software should do what it can to also ‘pay attention’ to what users are saying/doing…it’s probably the best way to build an online brand and offer up an unbeatable experience for users (something worth talking about and sharing).

Looking at a few services/features that I think are good examples to help prove this point:

1. bit.ly - killer feature is analytics…basically it’s a service to help you measure how many people are paying attention to you.

2. chartbeat - killer feature is real-time analytics…basically it’s a service to help you measure how many people are paying attention to your website/content right now.

3. Facebook likes - easy way to say “I am paying attention to you and appreciate your thoughts”

4. Instagram/Pintrest - I see you and am paying attention to what you are pointing out.

5. Comments/discussions (ie. Disqus and the like) - I’m paying attention…and here are my thoughts/reactions to it.

…there are of course many more examples that I could go into…but I think, if you’re paying attention (*har* *har*), you see my point. The social web is really all about paying attention and letting the people you are paying attention to, that you are in fact paying attention.

If this is true, then I think the ‘next big thing’ on the internet is likely to be something that helps make ‘paying attention’ or ‘showing that I’m paying attention’ easier…

What do you think?

The biggest mistake I made with knowabout.it?

Mon, 07 May 2012 23:23:00 -0400

I’ve been thinking about this question a lot for the last six months or so (which is how long it took to take my gut feeling that we needed to shut it down and actually shut it down).

While I can honestly say I have no regrets with the decisions we made or the efforts we put in (the positives of the experience and the work FAR outweigh any negatives)…I do have a handful of things I can look back on with clarity now and classify as legitimate ‘mistakes’.

The biggest of which, I think was reacting to what people were telling us in the early days instead of understanding/accepting what they really meant.

In our early pitch efforts, most found our idea and initial product interesting but were unsure where the market was going or who was going to ‘win’…they encouraged us to keep in touch, to keep evolving, and perhaps focus more on design/UI/UX so the system was ‘easier to use’.

We got enough of this common reaction that we fooled ourselves into believing that design really mattered in what we were doing.

If we just had a better design/UI/UX, people would understand the product better, they would use the product more, they would tell the world about our efforts.

But the truth was, our design/UI/UX…as horrible as it really was…wasn’t the real problem.

The problem was that our story was not strong enough.

We weren’t connecting with a problem people really understood or felt yet.

We actually knew this in our guts, but blindly refused to accept that people wouldn’t simply ‘wake up’ to the problem once they saw/experienced our product.

And truth be told, many who actually did experience our product after our awesome (and expensive) redesign did ‘wake up’ to the problem.

But having to educate people about a problem is a VERY expensive and slow way to have to grow a product/service because there is no natural curiosity for it.

There is no one out there looking for a ‘solution’ to a problem that they don’t realize they actually have yet.

The frog simply doesn’t realize the water is starting to boil so he has no need to be rescued. He is not going to look for help, he is not going to shout or scream, and he certainly isn’t going to tell the other frogs in the pot about these crazy people offering a ‘ride to safety’.

It actually gets worse though.

The reality is, we even struggled ourselves to clearly explain why our service changed the world today (we could easily explain why it changed the world at scale…but guess what, every service changes the world at scale).

From a technical view, we knew we had some amazing tech. and we were pulling off some amazing stuff for insanely cheap (cheap being relative of course — by the end we were indexing/processing close to 10 million links a day and making near-real-time recommendations for about 6,000 users all for about $700 a month!).

As users ourselves, we also knew that our service felt great and ‘really worked’…but we constantly glossed over the fact that we felt that way primarily because we were SO AWARE that the problem existed.

We simply didn’t want to accept that it was going to take much longer than we could wait for the public to wake up to the real problem (though we are still 100% sure that the public will eventually wake up to it at some point).

So the lesson is this:

If you can’t get people to identify with the problem you solve, and how you fix it for them better than anyone else in about 140 characters…you have not hit on the proper problem/solution/story yet.

And until you do, nothing else matters.

Design, UI, UX, any number of awesome features or amazing technology all are basically wasted efforts (on the business level) if you aren’t hitting on, and solving, a real pain point people are already aware of, actually experiencing, and (at least indirectly) looking to have fixed.

Just think about the services you actually use every single day.

You understand their story pretty easily don’t you? Not only that, you can (and probably do) share it with people in about 140 characters or less don’t you?

How many of those do you tout because of the Design/UI/UX? (Actually I would guess, that if you’re being truthful, most you tout them “in spite of” these very things).

So from here on out it’s all about the story and the experience I’m going to provide people.

If I can’t explain that clear and simple, then it’s not ready to be thought of as a ‘business’…yet.

Luckily, 99% of the stuff I build has nothing to do with trying to be a ‘business’…yet. ;-)

A few cool, new, turfd features.

Fri, 04 May 2012 23:15:00 -0400

I continue to hack away on turfd.com as I find extra time here and there. Tonight was one of those times, and so I’ve just released a handful of small updates. I’m pretty excited about this latest batch and just wanted to share a few details with you:

1. Improved sharing and social features.

If you create a turfd account, you can now associate your Twitter and Facebook accounts to it…if you do that, you’ll be able to share links to any stories you find on turfd out to these networks quickly and easily.

Additionally, every story you share or comment on will be listed under your public turfd profile (where others can reply and engage with you around the stories you find most interesting).

Your turfd profile page is just turfd.com plus your username (for example mine is http://turfd.com/falicon) and is a great way to engage with your friends around the sports stories you find most interesting. I’ll be spending more time improving the profile experience (and discovery of other turfd users) in the near future.

2. Daily email of recent league scores.

You can now have turfd send you an email early every morning that contains the scores from the MLB, NBA, NFL, and NHL of the previous day.

I’ve had this feature running for just myself for a few days (as I tested it out) and I have to say I absolutely love this (it’s always the simple little ideas that often provide users with the most joy). It’s such a great way to catch up on the scores/action around all the leagues I care about in one simple email (and it’s generally waiting for me in my inbox when I wake up).

To take advantage of this new alerting service, just go to your settings page on turfd.com. I’ll also be working on adding in scores for other leagues and sports before too long.

3. Exclude specific sources or terms from custom feeds.

While I’ve been busy adding as many new sources as I can, I have started to hear some feedback (and notice myself) that not all the sources are created equal.

Some users love a given source while others hate it.

Rather than try to control what sources get in or don’t beyond the general classification of ‘provide professional sports information’, I’ve decided to give the power to exclude directly to users.

So now, if you don’t like a specific source or you find that your custom feeds are sometimes cluttered up with false positives…you can just list the source, or the specific terms that are causing false positives, and your custom feeds (and the related alerts) will no longer include those matches.

4. Streamlined mobile experience.

Check turfd.com via your mobile device and you should find an optimized-for-mobile-experience.

It’s not quite as full-featured as the web version just yet (and you can switch to the standard web view at any time if you really want)…but search and custom feeds are front and center so hopefully the core of what you would want is already there and a pleasure to experience. I’ll of course be spending quite a bit of time improving everything from here going forward as well.

But wait there’s more!

The above list are just a few of the latest updates I’ve released for turfd…I’ve got a massive list of others I hope to get to soon. In the meantime, please take a second to hop out and experience it all for yourself…and PLEASE make sure to tell all your friends to check out turfd.com as well!

Twitter's updated discovery efforts

Tue, 01 May 2012 22:28:00 -0400

Twitter announced an update to their Discover tab today.

As you can imagine, this is something I have a lot of opinions and thoughts around. I’m not going to put them all out here right now (mostly because I don’t feel like writing that much right now — or boring you to death), but I do want to at least take a minute to share some of my quick reactions/thoughts:

1. It’s a good start, but it’s not properly described (for what it actually does) and so it will ultimately be ignored or under-appreciated by the masses.

What I mean is that they are saying things like ‘discover information that matters to you without having to follow additional accounts’ and ‘surface content that is even more personalized and meaningful to you’…but the reality is their definition of personalized is tightly tied to who you follow (and really nothing else)…and the ‘magic’ behind this iteration is really all about popularity (ie. they are basically picking what to show you based on friends-of-friends counts, follow counts, and share counts).

Honestly it’s not enough.

At best, it will be hit or miss on finding stuff ‘that matters to me’ and it absolutely isn’t ‘personalized’.

To be personalized you’ve got to actually get to know and understand me first (hint: start by paying attention to what I say, do, and engage with…not just who I randomly choose to follow or who the people I follow, follow).

Once you’ve got a reasonable idea of who I am, and what topics I might actually be interested in, then *maybe* you’ve got a chance to help me discover content (and people) that matters to me. Until then, you’ve got a gamblers chance at finding me good content…and even less of a chance at keeping my attention long enough for you to ‘figure it out’.

2. Discovery should have more depth.

Twitter has got a ton of amazing assets they can throw at this problem…the two biggest being an amazing staff of talented/smart people AND gazillions of actual data points (not to mention a nice chunk of change in the coffers to help keep focus on the problem).

The problem is, everyone including Twitter is looking for the ‘easy’ way to solve the discovery grail…but the big wins in this space are behind the hard work.

Can you imagine how amazing it would be if Twitter actually learned to properly classify it’s users (and then eventually every single tweet)? Even a generic level of classification could go quite a long way in improving the ‘discovery’ space.

Once you’ve got classification, you can start to do grouping…and once you start to do grouping, you can start to help me dig deeper into the topics that I actually do discover and care about (ie. one-off articles here and there about randomly interesting things, is not discovery…it’s sampling).

Oh and by the way, true classification is only the starting line for ‘good discovery’ (it speaks nothing of the details involved in the listening phase).

Since we are a ways off from this actually happening on any ‘good’ level, I am willing to even accept a simple ‘hack’ to start…use Twitter trends, hash tags, or even click data to determine what’s likely related to a given article.

Bottom line, if and when you do hit upon something personally interesting to me, help me go deeper (ie. build engagement)…but what you are doing right now is just making your noisy service even noisier to me.

3. No matter what Twitter ultimately offers up in the ‘Discover’ realm, it will be a closed world solution…and this means ultimately, it can’t be truly personalized.

On any given social network, or service in general for that matter, I am but a shell of my true and whole self.

If you, somehow, merge them all into one big glob of data and look at it as a whole…you *might* have a shot at properly classifying me. And only by looking at the larger collection of data will you actually be able to determine what I have (and have not) already seen or been exposed to (properly dealing with the ‘seen it’ situation is a massive key to good discovery everyone just sort of skips over at the moment).

4. Ultimately I don’t think the Discover tab aligns with what users currently want or expect from Twitter.

Ask yourself why Twitter cares about discovery at all? What does Twitter really want to be? Do they even know yet?

I love and use Twitter all the time. It’s a noisy and chaotic place (much like the internet as a whole)…but that’s part of what and why I love it. I don’t need or want Twitter to filter the noise for me (just like I don’t want Google to skew or personalize my search results).

I want them to focus on letting more noise flow through the system and making it easier and easier for me to put my own noise into the system. Please be a publishing and communications platform…not a destination and not a media company.

5. I think discovery is an important, and missing, part of today’s internet

As such, I think it’s got to be it’s own stand alone service to actually work. I don’t think tacking it on as a feature within a given service (no matter how awesome that service, it’s staff, and it’s data set are) is going to do justice to the problem.

Every site/service has search (and in many cases had it years before Google)…none of it compares to how Google solves the problem.

Eventually I envision every site/service having discovery features (in fact, many already do)…but none of them are going to be ‘good’ solutions for what users really need or want when it comes to true discovery.

Note: Mathew Ingram* of GigaOm also posted a few thoughts on the Twitter Discover tab updates that I think are probably common-felt and worth reading.

* The same guy that for some unknown reason refused to ever acknowledge knowabout.it existed as an option in this space (though he did actually have an account at one time)…not that I’m bitter about that slight or anything. ;-)

Some of my 'big' problems with the current state of the internet

Mon, 30 Apr 2012 12:48:04 -0400

1. There’s still no great tool to help me categorize and organize everything that I’ve got access to. Though I now have access to it all, I can’t actually keep up with everything everyone is saying and doing. There isn’t even a decent tool to quickly summarize/review everything when I’ve got time/interest to catch up.

2. I’ve got to be proactive. It’s all out there and available, but I’ve got to go look for it. I’ve got to be the one initiating the search. People say “news finds them” but the reality is, it only finds them when they are actively waiting for it (ie. they are constantly checking and engaging with their email, google, twitter, facebook, etc.).

3. Engagement isn’t really happening at scale or moving forward. The ‘real’ communities on the internet are only occurring at a very few select locations…and they are generally around specific bits of content. The best options at the moment tend to be a weak conversation with a stranger in public (ala disqus), a weak conversation with a friend from the real world (ala facebook), or a quick one-liner with someone who’s opinion you value (ala twitter).

Actually, as great and interesting as the internet is, if you take a second to look around you’ll find lots more ‘problems’ than just the ones I list above…but these three are at the top of my personal list right now.

What’s on your list?

Today's random idea: alphalist.me

Fri, 27 Apr 2012 20:55:37 -0400

One of the many domains I own is alhpalist.me - I got it about a year ago, with the intention of building out a very basic email list service.

The idea is that it’s a way for users to identify themselves as ‘early adopters’ who would like to be alerted of new/interesting projects that they can sign up for (before the general public).

And of course, the advantage to any project that would use the service is the ability to gain all-important initial alpha users (and jump start the uphill battle to gaining traction).

…I still really like this idea (and would love to use it both as an early adopter user, and as a developer of various projects)…so someday I plan to take some time and try to implement it (though I think building up the initial user list will be a challenge that I won’t just be able to crank out in a day).

Anyway - what do you think of this idea? If it existed, would you sign up? If you develop apps or services, would you want to use it to gain alpha users?

The godfather of social media.

Thu, 26 Apr 2012 20:50:37 -0400

The year is 1991. Television is the common thread of most Americans. Shows like Saturday Night Live are experienced, in real time, by the majority of the viewing public.

It’s on one of these SNL episodes, on February 9th, 1991 to be specific, that a character played by Al Franken was introduced to the world. The character’s name was Stuart Smalley, and in hindsight, I think it’s clear that he was in fact the godfather of social networks.

For those of you that are too young, or just don’t remember, Stuart Smalley was the star of a self-help skit called “Daily Affirmation With Stuart Smalley” where he would often look into a mirror and state “I’m good enough, I’m smart enough, and Doggone it, people like me!”.

…and when you think about it, every time you post a status update or share something on the ‘social web’, you are basically saying the same thing.

Some rambling on Push notifications.

Wed, 25 Apr 2012 21:15:00 -0400

Push notifications should be the digital equivelant of the candy rack at the check out counter. Hitting me up with stuff I don’t really need or want, at just the moment when, and in just the right way so that, I might actually buy something.

The candy rack works I think, because of it’s location and timing. Put it in any location that doesn’t have forced traffic with forced down time so that it’s noticed, and it will barely move any product.

So how do you do that with push notifications?

I think the secret comes from paying attention. Don’t push stuff to me every chance you get, instead pay attention to what I’m doing and when I’m doing it…when you notice I’m deviating from that norm. (my norm.) by all means ‘push’ something to me to get me back on track (and then pay attention to see if my ‘norm.’ changes).

Do that, and you’ll get my “WOW”…and probably more importantly, you’ll earn MY attention…anything else and you’ll just annoy me.

some additional thoughts on hiring.

Tue, 24 Apr 2012 21:55:54 -0400

Just catching up on a few blogs and I ran across the latest nugget* from Nate Westheimer where he talks a little bit about hiring and what he’s currently focusing on as they build the team up at picturelife.

We are just starting to build out the team a bit more at PubGears** as well, so this topic is very much on my mind these days.

At this specific moment in time, we are looking for an awesome “ad ops” person and in fact interviewed our first candidate just the other day.

The very first question I asked was “when you aren’t working, what do you do for fun?”.

I asked this because I think in the early stages of a company, and especially while the whole team is VERY small (we are only a team of 4 right now), the personality and the passion a person brings into the company is the single most important factor.

That’s not to say that they don’t need skills…but by the time we’ve asked you in for an interview, I feel pretty safe assuming you’ve got enough skills (or can fake it well enough on paper and through an initial interview) that quizzing you is mostly a waste of everyone’s time. And really, I know anyone that has the right amount of passion and interest can learn whatever they need to make a real contribution to our efforts.

So - trying to get at the person’s passion, finding out what drives them, what would make them want to get up in the morning and be excited for the challenges they are going to face that day…and how well that fits with our existing group (both in what gaps it fills as well as what strengths it re-enforces)…these are the things we can’t get wrong in bringing in #5 to our group (or #6-#25*** really)…and so these are the things I’m also quite focused on and thinking about a lot these days.

When it comes to hiring, what do you focus on? What assumptions do you make? How much emotion do you bring to the process?

* Read Nate’s whole insightful post at http://innonate.com/2012/4/24/thoughts-on-hiring

** Yes, I realize I haven’t mentioned PubGears here yet, but now that we’ve officially closed down knowabout.it, I can finally start to talk about PubGears (the startup I joined back in December to build out the tech and help the company continue to scale/grow). So expect to hear a lot more about PubGears around here going forward!

*** BTW - if you’re a developer (or know one) looking to get involved in some really fun (and slightly crazy) stuff with me, I want to chat with you asap! We’ll be looking more aggressively to fill the role in the coming weeks, but if you’re ready now, so are we. ;-)

Today's random idea: mobile video review app.

Sat, 21 Apr 2012 21:00:51 -0400

Purpose: to get quick, human, feedback/reviews/opinions/suggestions on any product, brand, or service.

Basic Features:

1. Record feature: app gives user simple feature to record 30 seconds to 1 minute of video (with audio), add some quick and simple tags, and then release it to the network (basically move it into the app cloud where it awaits searches/views/votes). The instructions around using this feature should frame the users intentions around reviewing products/services in the real world (eg. just saw The Hunger Games movie? Post a 30 second reaction/review using this app, tag it as Hunger Games, and move along).

2. Watch feature: app gives user simple search feature that is used to search tags on submitted posts. Users can scroll through the results and watch the ones that are of most interest to them (they can then vote the ones they watch as helpful/not — a-la amazon reviews). The instructions around using this feature should frame the users intentions around getting advice at point of purchase/decision (eg. At the shake shack and not sure what to get? Search for ‘shake shack order’ and see what people are suggesting). Bonus Feature: Allow users to narrow the search results to videos of just their friends (use existing social networks or have them build out one specific to this app — could base it on videos from the people they find most interesting/helpful within the app).

The disclaimer:

I like this idea, but it’s not something I’m going to build (or even put much additional thought into). Like many of my random ideas I get, it came to me while thinking over some other random thoughts about mobile, social, and the real world.

Most times, when I get these ideas I think about them for a bit, maybe jot some notes down about the idea in one of many notebooks I’ve got around my house…think maybe someday I’ll work on that, and then basically forget about it.

But I’ve decided rather than just let these random ideas die in notebook around my house…I might as well let them die out here on my blog instead. So I’m going to make more of an effort to jot them down and share them here going forward.

Who knows, maybe someone, somewhere can get inspired by one or two and do something with it?

So borrow the good ones (free of charge), ignore the bad, and let’s talk in the comments about everything in between! ;-)

danlewis: The news is behind the times. Look at the bottom...

Wed, 18 Apr 2012 17:58:04 -0400

danlewis:

The news is behind the times.

Look at the bottom graph of the top picture. Relative to Twitter, there’s been virtually no news pickup of Tumblr. It’s a flat blue line. On the other hand, search volume shows that Tumblr’s been a “thing,” relative to Twitter, for almost two years. The news is two years behind.

The second picture is Tumblr v. Instagram. Outside of the sale, Tumblr’s been crushing Instagram on search volume — steeper slope and everything. And yet, until the sale, both “news” graphs were relatively flat. Instagram spiked — but only after the sale.

The third one show something different: Tumblr v. Pinterest. Similar slopes, with Pinterest perhaps a bit steeper but with less staying power, it seems. The news noticed it… which underscores how invisible Tumblr is to them. But why’d they notice Pinterest? I have a few theories, none of which are flattering to the news industry.

Fascinating. And amazing.

…the problem is that Tumblr itself is not something new (blogs have been around for a long time - Tumblr just made it a more pleasant experience to own/operate one)…additionally, using Tumblr empowers each user to build/re-enforce their own unique brand (and only slightly - if at all - that of Tumblr)…using services like Twitter and Pintrest encourage users to build the brand of Twitter and Pintrest (and only slightly that of the user themselves)

My basic understanding of OOP

Thu, 12 Apr 2012 09:22:00 -0400

I keep the number of mailing lists I monitor pretty small, and generally when I add one, I remove another. Lately one that I’ve got on my list and have been paying attention to is the PUG-IP (Python User Group In Princeton)…and lately there has been a small thread going on about what is object oriented programming (OOP) and how can a beginner understand and actually use it.

I find this a pretty interesting question as I remember struggling with this many moons ago as well (when I was first digging into Java actually)…and even more interesting to me is that, even after all these years of ‘using’ OOP, most of the gurus on the list agree that it’s a complex topic to explain to beginners.

I agree, and I think it’s because of two main reasons:

1. OOP involves a heavy does of theory…and truly understanding theory generally involves a heavy does of experience. That is, you’ve just got to use it, and you’ve got to break it, to understand it. That makes it very hard to learn.

2. The dirty little secret is that I’ve found in practice most OOP that people write is really just functional programming in disguise, and the talented functional programmer can have a pretty full and happy career without ever really understanding (or fully using) OOP. That is, OOP involves a lot of hype.

In fact I will freely admit that my own understanding and use of OOP is pretty basic (so please do take everything I say around the topic with a grain of salt).

Luckily being that this is the internet, and my own blog, I don’t actually have to be good at something to try and explain it to others…so without further ado, here’s my own quick and dirty explanation of OOP for beginners.

OOP is a style of programming in which you attempt to model your code after the way we attempt to explain the real world.

The 3 steps in OOP

1. Define an object

Classes are really just definitions of an object. In plain English it would go something like this:

“Do you know what a glyphon is? No? It’s a round thing that bounces.”

In code we do the same thing when we define a class. We name the class, and we set up functions (or methods) that explain the details of the class. If you look at the group of methods within a given class as a whole, you get a sense of the definition of a class (ie. what a object of this class can do).

2. Create an object

I think this is a step that many beginners don’t really get at first, but once you get the idea that classes are just definitions of what an object is I think it starts to make sense that defining something doesn’t actually mean the ‘something’ exists. You’ve got to create an instance of that thing before you can actually use it.

In the real world it’s the difference between talking about something and actually having that something in your hands or in the physical world. In code, it’s the bit where you generally see a statement calling the new or init functions like:

my_glyphon = Glyphon()

3. Use an object

So now that you’ve defined what a Glyphon is, and you’ve actually got one in your hands, you can finally start to ‘do’ stuff with it…and that’s where your actual ‘program’ does stuff.

In the real world it’s the part where you bounce the glyphon on the floor or throw the glyphon to your friend. In code, it’s the part where you implement your program’s specific business logic.

This is another spot where I think many beginners get depressed. It sure seems like a lot of work to have to define an object, then create the object, just so you can FINALLY start to use the object…and the sad truth is that, yes, in many cases it *is* a lot of work. And that’s also why in many cases, especially in small one-off programs (or poorly designed large programs), OOP doesn’t actually make a lot of sense.

So why use OOP at all? Well, there actually are a few advantages OOP can give you (when done correctly and used in the proper situations). Let’s take a quick look at what some of those advantages are.

The advantages to OOP (in theory)

1. Abstraction and Encapsulation.

OOP allows you to define objects. Once you define something, you can use that definition in as many places as you want, as often as you want, for as much as it makes sense for your programs.

The true power here lies in the fact that ‘you’ don’t have to be the one that actually defines an object to be able to use it in your own programs. In fact, you don’t even have to really understand that much about the details of an object to be able to use it.

In the real world, you don’t have to know how a computer works to be able to use one. You just need to know a keyboard and a mouse help you control stuff you see on a screen. In code, you just need to know some basics about a class (what methods you can call, what parameters are needed, and what those methods will return) and you can use it.

2. Inheritance

Most fans of OOP will tout inheritance as the true reason for loving OOP…and honestly I think it *is* pretty awesome (at least in theory). In basic terms, all that inheritance means is that you can use the definition of one class to help define another class.

In the real world we use the definition of one thing to explain another all the time…”Do you know what a gazbot is? No? It’s like a glyphon but it doesn’t bounce”.

In code, there are many language specific differences here, but the basic concept is the same…without having to duplicate (much) code we can define a class as ‘like’ another class, but with a specific set of differences (the methods we define in our new class define those differences).

So in our example, when we define the gazbot class, we can say give us everything from the glyphon class, but we will give a new definition for the bounce method (which in our case we will just say, it does nothing as gazbots don’t bounce)

This turns out to be really useful as it can drastically cut down on the amount of code needed in large projects and (when done well) can make the purpose of each object and program very clear and simple.

3. Polymorphism

Of the three advantages often listed with OOP, this is by far the most complex to ‘understand’ and I also think the least actually used (or at least, the least used well). But this only makes sense, because generally if it’s hard to say, it’s probably hard to understand and use too.

Anyway - on a very basic level, all that polymorphism means is that you can define something in different ways depending on the properties it’s given.

This is complex in the real world too, but it does exist, especially in something as complex as the English language. There are many words that have multiple meanings in the English language, and it’s only through context that we determine which definition to apply.

In code, it’s actually a little easier than trying to understand English. You simply have methods of the same name, that accept different parameters…then when you call the method, the parameters you pass in determine which definition will actually be applied.

So that’s it in a nutshell.

Again, I’ve only touched on the very basics of what OOP is and how it’s used (and I’m not a guru by any means). Each of the things I mentioned above is really worthy of a much more involved discussion and can probably only be truly understood through real experience. So I encourage you to explore the web for more/better details, and even more importantly I encourage you to just start playing with OOP code. The more you do, the more you’ll start to ‘get it’.

In the meantime, I do hope this has helped at least one or two others out there…and if you’ve got corrections, updates, or questions around any of this please do drop me a note in the comments below!

Conversationlist is back!

Tue, 21 Feb 2012 23:15:39 -0500

OK I gave in to the massive outcry (and when I say massive, I really just mean the three or four people that pinged me with quick questions about the closing of conversationlist)…Conversationlist is back.

As I mentioned before, the primary reason I shut it down in the first place was that it’s a complete side project…and it had been costing me about $70/month in server fees.

The cost of the server was basically that high because I hadn’t taken any time to optimize the code or the list building process…instead, as the user base grew, I was just upgrading the size of the server I hosted it on (the lazy mans way to scale).

That approach worked fine for quite awhile because it meant that I didn’t have to put anything but a little more money into the project month after month…for about two years.

That is until one day when I sat down to actually review my server costs for my various projects…and I realized that I was spending $70 a month for a dedicated server for conversationlist (a simple, for-fun, side project), and only a few times more than that for ALL the servers that power knowabout.it (an attempt at a real, world-changing-potential, company).

It was clear I needed to either take the time to retool the service, or just kill it…and well, killing it is much quicker and easier than retooling a bunch of old code right?

So I just made a knee-jerk reaction and killed it without much thought.

But the truth is, I’ve always liked the small success that conversationlist has found…and I do think it’s a neat little project with some interesting, yet-to-be-culled, potential.

So this past weekend, when I wasn’t involved in Pack 59’s Pinewood Derby or doing something on that other awesome project I haven’t told you about yet, I spent much of my free time rebuilding the Conversationlist service from scratch.

The orig. version of conversationlist was a simple PHP project that used the Twitter REST API, PostgreSQL, and cronjobs to do everything.

It was really just one big synchronous task that would go account by account, pulling the last 200 tweets the user generated, doing some basic regex code to determine mentions, and then more Twitter REST API calls to get user information…eventually it would remove the members off a list that needed to drop off and then add the ones that needed to be put on.

This worked well enough for the first five-to-six thousand users, but somewhere around that point it really started to fall apart…so a year or so ago (I actually forget exactly when), I did take the time to implement a few minor updates to help scale a bit better.

At that time, I replaced the PostgreSQL datastore with a Mongo instance (in case you didn’t know, I’ve been a HUGE fan of Mongo for the past year or two)…and I updated the cronjobs so that instead of direct PHP scripts, I had Perl scripts do some of the heavier lifting and a bit of threading so that I could process more lists in the same (or less) amount of time.

But the entire web experience was still pure PHP…and ultimately each list was still being built by a large number of Twitter REST API calls (the real bottleneck in much of the process).

Again, this worked well enough…and as long as I had a powerful enough server, it could even scale without too much trouble (but as I’ve mentioned, had lots of potential to get expensive and glitchy too).

So it was finally time for a complete re-thinking of my approach…

Since the time that Whitney came up with the idea for conversationlist (you do know he’s the real brains behind this whole thing right?), Twitter has made massive updates to the things a developer can do…and I also like to think that I’ve made massive strides in how I approach and prepare for scaling successful services as well.

So this time around, the web site is pure Python (using tornado, tweepy, and pymongo)…and, like I do with knowabout.it, I’m taking advantage of Twitter site streams (I customized a version of Perl’s AnyEvent library that I use for all my Twitter Site Streams stuff — just reused that for this update).

Though I will see nice improvements from switching from PHP (behind an Apache2 server) to Python (behind an Nginx server), Site Streams is the real key to the whole thing.

With Site Streams, I can get just the mentions (as they come in, in real time) and each includes all the details I need for adding them to lists (ie. the user’s twitter id, not just their screen name)…so I can keep a running record, in real-time, of what a given conversationlist needs to look like without any extra or time-consuming REST calls.

Sweet.

The end result also means that I can now host the entire service on a reserved M1.Small EC2 instance (that I already was paying for)…and while it’s certainly not fast by any means on that tiny hardware, it is back alive, with the ability for me to introduce some cool new features (if I find time to implement them)…and all for around $20/month right now.

Not a bad price for rebuilding about 10,000 Twitter lists a day (in case you’re wondering why that number doesn’t match numbers I’ve reported in the past…in the process of the re-tooling, I trimmed out the spam and dead accounts which dropped the overall user count from about 14,500 to just over 10,000 active…but we’ll see where that number goes over the next couple of months).

If you take the time to jump over to conversationlist.com, hopefully you’ll notice I also took the time to redesign the web site (using Twitter Bootstrap of course!).

If/when I get the chance to implement a few of the little ‘extras’ I now have in mind thanks to all this work (and now that I actually spent more than 5 minutes thinking about the project again)…I’ll see if I can’t generate enough income from it all to at least warrant an upgrade to an M1.Large instance (the service already could benefit greatly from the server upgrade, but I’m still trying to save on cost and already had this m1.small reserved instance laying around).

So anyway…it’s back…hopefully better…and getting ready to go to the next level…so stay tuned!

How's that whoclicked.it project going?

Tue, 14 Feb 2012 17:26:00 -0500

Since releasing an early beta version of my whoclicked.it project about a month ago, and gumroad getting a bit of press, I’ve had a few people randomly ask me about how it’s working out.

So I thought I would share some quick additional details.

First, the project is really just built around a personal desire I have had for a long time to know who is clicking on certain links I share. It’s not meant to be used all the time as a replacement for any other short url services, or even meant to be used for every link you share.

It’s simply meant to be used when you have a bit of content you want to be able to engage with people around…and for that rare use-case, on a personal level, I find that whoclicked.it has been great for me.

That being said, I get a lot of feedback that the ‘log in required’ page is really horrible (and I agree)…and many people have said that page alone scares them off. So I’ve definitely got to improve that page (though I have zero budget for this project at the moment). And until I do, I suspect that the service itself will probably only serve my tiny needs and not much else (I don’t think anyone else knows about, or uses whoclicked.it yet anyway — though I do think the gumroad funding helps validate my concept at least a little bit).

Anyway, the service definitely is serving my own needs. As you can hopefully see from the screen shot above, it causes a lot of bounces and drops (I expected that when designing the service in the first place), but some REAL people actually are clicking through on the links I share…and when they do, I get to know who they are and have a chance to reach out and engage with them around that content.

So it may not be a world-changing service (yet) or getting a lot of hype or attention (most of my projects never do)…but it’s def. doing what I hoped it would for me on a personal level…and I think there’s a lot of untapped potential still to be fleshed out there (so I plan to continue to evolve it over time).

p.s. A nice bonus feature I added in is that you not only get to see who clicks on your stuff, you can now see what bots are also attempting to follow your shares — and when/how often. If you’ve got time, I would love to have you check it out at http://whoclicked.it

Hey, are you a sports fan?

Tue, 07 Feb 2012 13:22:00 -0500

As of late I’ve been head’s down in a handful of really interesting things (more details to come on just what I’m digging into soon I promise)…and while it’s tons-o-fun to be building something new and exciting from scratch, it also generally means I lose almost all touch with the outside world.

I don’t pay attention to much news anyway, and I get enough of the interesting general tidbits from social media via my daily knowabout.it email, but the one area of my life that I do feel like I’m really missing out is often in the world of sports.

So, in my extremely limited free time, I started to build turfd (http://turfd.com) which basically aggregates sports stories from around the web (the idea is heavily influenced by what techmeme does with tech news).

I knew there was a ton of ‘professional’ content around sports getting published every day around the web…but I had no idea just how much there really was! (I’ve only plugged in about a dozen ‘professional’ sources to turfd and already it’s logging about 1,500 stories a day!)

That’s a lot of sports to keep up with (and I have a list of another 50+ professional sources I plan to add over the next few weeks).

So as it evolves I’ll def. be using a lot of the tricks and knowledge I’ve learned form developing knowabout.it over the past year to help you discover the most personally relevant sports news…and I’ve got plans to implement a number of alerting and filter options soon as well (prob. only for subscribers though — sorry gotta pay the server bills on these side projects some how).

For now though - in my busy day where I only have a couple of minutes a day to catch up on the world of sports - I am finding turfd a perfect solution and I’m catching tons more interesting stories around sports than I ever have before.

So if you’re into sports - why not check out what’s already available at turfd? Then let me know what you think, and especially let me know how you would like to see me evolve it to fit your needs too!

The end of conversationlist...for now.

Sun, 15 Jan 2012 14:31:00 -0500

Today I’ve finally decided to shut down the current version of conversationlist.com.

It’s been a fun project to build and maintain over the past couple of years, but interest seems to have plateaued over the past six months and we’ve never bothered to try and turn the project into a profit center (honestly, I’ve never tried to make any money with it at all - instead I just keept it running because it was a fun little project that had got some good attention).

Still, it’s at a level now that costs me about $70 a month in server resources (it rebuilds about 15,000 twitter lists every day)…and honestly I feel like Twitter lists (at least as far as I’m involved/interested in them) have run their route as well.

So I’ve decided it’s best to commit that $70 towards other projects that I’m currently excited about.

As with all the projects I do, I’ve got backups of the code and underlying data sets…so there is still a chance I might do something more with the domain in the near future (I’ve actually got a lot of interest and ideas around the general ‘conversation’ theme - so don’t be surprised to see it emerge as something new before long)…but for now it’s going dark.

If you’ve got questions, concerns, or comments around this decision please feel free to leave a comment below…just because conversationlist.com is going away, doesn’t mean I am ;-)

How would you like to know who clicks on the links you share?

Fri, 13 Jan 2012 14:06:00 -0500

Today I’m making the first public mention of a new little service I’ve built called whoclicked.it. A simple service designed to tell you who actually clicks on the links you share around the web.

It’s an idea that’s been on my mind for a couple of years now, but was primarily driven by two things:

1. Prior to building whoclicked.it, I used bit.ly for almost all my link sharing on Twitter. With bit.ly, I’ve seen on average that I generate about 15 clicks on anything I share, no matter what time, or what, I share.

On one hand it’s a nice little boost to the ego to know I’ve driven some traffic, but on the other hand it drives me NUTS to see that number each time and have no idea what to do with it. I also always wonder how many (if any) of those clicks are real people vs. twitter bots that just follow every link shared on twitter (I suspect a large portion of those 15 clicks I generally log are actually from bots). I needed to fix this problem.

2. Within knowabout.it internally we track what links people click on from within our emails and system (using the halfbite.com system I built). I built that service to help test and improve our recommendation system and algorithms…but since first building it, I’ve been completely obsessed/hooked on looking at who clicks what links. It’s very addictive and so I’m betting that many ‘normals’ would love to know who actually clicks on the links they share as well. I wanted to share that power and knowledge.

Beyond those two primary motivating factors, and after I started playing with the prototype a bit, I really got hooked on some of the additional things this service opens the opportunities for…specifically:

1. Restricted links. Because people have to log in before getting redirected to the proper long url, you can now control who actually gets to be bounced and who doesn’t. So fan clubs, private groups, and other exclusive services can now publicly share links that only certain people can actually use. Granted, it’s a pretty niche need, but still one that I think is pretty unique and cool…so I’m excited to see if people actually use it and what their feedback on it is.

2. Personal engagement. Again because you know the set of people that actually clicked through a link you shared, you now have an opportunity to have conversations with just that set of people on just that set of data. You can have these conversations outside of the content itself so that the conversation is essentially private for just your group of ‘clickers’ (actually I can put in all sorts of levels of privacy options around this engagement if users want — clearly I haven’t completely thought this part through yet).

As I slowly put the MVP for this together nights and weekends of the past month, I’ve also been explaining the concept to a few influential and opinionated friends that always make me think about things in new and interesting ways.

One of the more interesting conversations around the project I had was with Arnold. As I introduced the idea to him, he stopped me and asked me a very awesome question. “Why should I care?”…I thought it was a great way to think about any new project from the user’s point of view…so here’s what my response (more or less) was:

Why should I care?

Because you can’t engage with people, or connect on any personal level, if you don’t know who they are!

My core thesis behind this project is that 99% of the time generating an anonymous click is basically generating a worthless click.

I believe the only time that an anonymous click is valuable is when you are trying to generate volume traffic to content (which you generally only do for the content you generate yourself).

I also believe most (or at least many) link shares are not actually from the content creators themselves, and so they don’t really care about the volume of traffic they generate from the links they share.

So these beliefs caused me to wonder, if most people don’t care about the volume of traffic they generate from sharing links then why do they share links? What do they really care about when they share a link? What do they want to happen?

The answer, at least for me on a personal level, is that I share links to content I find interesting. Links to things that I want my friends - and like-minded people - to know about. Things that *I* would want the people I follow to point out to me. I share links to things that *I* want to discuss further with people.

I think that last point is the kicker - “things that *I* want to discuss further with people.”

In the world before whoclicked.it, the best I could hope for was a user or two that clicked on a link I shared taking the time to actually engage with me via the network I placed the share.

The trouble is, that leaves all the opportunity for engagement in the hands of the clicker.

Since I have no idea who is actually clicking on the links I share (or even if it’s real people clicking)…I have no way to be proactive and engage those people. Instead I’m left shouting questions and opinions to the crowd, hoping someone takes the time to shout back (loud enough that I can hear them).

But now whoclicked.it opens up the ‘opportunity for engagement’ for the link sharer.

You can find out when real people click on your links, and then you can reach out to those real people to engage with them on a personal level around the shared topic interest.

Sure you might get lower click thru counts on each of your shares, but the ones you do get will be of MUCH higher value because you’ll have the opportunity to engage with the set of users that are actually paying attention to you and have at least a little interest in a shared topic!

So…stop losing out on all those opportunities for personal engagement…check out http://whoclicked.it and then let me know what you think!