List of items I'd like to develop in the next few weeks to make monitoring/analyzing data fun.

1. Charting indexed pages count from google.com for site:foo.com using rapheljs
2. Crawling multiple sites and visualizing data using the tool presented in the last post. Should have the ability to merge multiple data sets and represent as different colors.
3. Integrate page ranking for the crawled data
   
4. Intelligently suggest pages to promote based on analysis of competitor data sets
   

So you or your company has a website. You optimized it for SEO so you get all the organic traffic from google, bing, yahoo, etc... There are plenty of webmaster tools out there that will give you page rankings for specific pages. However, if you want to see for yourself how interlinked the pages in your domain or in your competitors domain are, you will have to poke around with some code. I have been using nutch for all the crawling and indexing purposes. It scales fairly well with hadoop. Bixo is another open-source tool, however I never got the hang of it and nutch provides just what I wanted - given a seed page, crawl through the server providing a list of inLinks for all URLs. This data can further be used (w/ some cleaning) w/ TreeViz, a great visualization tool for tree structures.. I will post some screenshots later today.

I was talking w/ my boss about the potential of data that is injected in twitter, search engines and other social networking websites daily. There is not a consistent API out there that could aggregate such data, map it to a valid location and present that for outside world to use. Trazzler does it in some form, but I can't find an API exposed for use by outsiders and seems to be limited to twitter data only. For instance, Superbowl 2010 will be all over the news and on twitter starting Dec 09/Jan 10. But what does that mean?? Absolutely nothing! Unless somehow you can map that to Miami, FL, USA. Same goes with Oktoberfest (Munich, Germany?). A cool project would be create a database that would update real-time with top search queries, tweets, etc. with a uniform location associated with it along with the ability for user interference to correct any outliers.

I recently bought a Panasonic home theater system which has built-in ipod dock. Now here is an idea: I don't have a ipod touch or iphone (I am ancient and carry an ipod mini from 2005), but it would be cool if I could dock my future ipod touch in the docking station, start an app - and let it listen for a streaming channel on the home network. This way, if you have any laptop, you would be able to stream music from itunes or youtube to this network channel and get music on your surround sound system from any room in the house. Obviously, this requires some more research as I have never really written an ipod app or even seen their SDK, so I'll have to digg up some more details.. More to come later!

I've noticed recently that Selenium IDE has recently become real popular in companies with agile teams to automate their testing. Capture/playback testing is great to show-off your testing skills with a faster ROI, but it is definitely not productive for large scale testing (multiple projects, features, etc..). If you have a product that requires significant amount of testing, there is no way you can record and playback every scenario and then maintain them for future use. There will come a time, where you would need to programmatically interfere to make development of test cases against the acceptance criteria easier. A recent post from infoq about testing:

In the late afternoon there was a discussion around the role of record and playback tools such as the Selenium IDE. Lisa Crispin thinks that capture/playback tools “can be a great way to help learn a new tool, and also can be helpful in debugging test scripts or figuring out the right statements to use in a particular test. However, people shouldn’t get bogged down in only using capture/playback”. Jason Huggins, Selenium developer, explained that he is troubled by the general use of the Selenium IDE (really just a record/playback tool). It was originally intended as a “little 'trainer' airplane that jet pilots train on first. The pilots can learn a lot from the trainer, but eventually they have to move up to a real jet.”

Don't get me wrong, w/ my limited experience of selenium, it has canned out to be a pretty good deal for testing, but one has to brush up on their scripting skills --- ruby, python or maybe even java to assist w/ selenium so as they can take full advantage of the tool and automate tests on a broader scale. Integrate that w/ a continuous integration system like hudson and you'll have yourself an environment self-sufficient for agile development.

Just my $0.03

Just got these - Coby CVE92 Isolation Stereo Earphones, and I have to tell you I am very impressed by these $4.99 (+ $3.99 shipping) headphones. Sound isolation is pretty good and the stereo quality and bass is incredible for the price. The only thing I worry about is that the soft-rubber on top might fall off in time, will just have to wait for that day :P

All in all, excellent purchase!

I got no words for this... just amazing!

I noticed today a couple of websites that have started to put a footer rail to display ads and more instead of making the page clunky by putting ads and other social networking items in the top/left/right/center. I kindda find the footer to be more cleaner for some reason an elegant approach to monetize on your sites.

After having played with blogger (or blogspot) for a few weeks now, I wanted to pen down some of the important features that this blogging system is missing.

1. Lack of support for embedding videos (youtube, metacafe, etc..)
   
2. Unable to view page in full-screen when editing a post
3. Unable to post pictures where you want in your blog post; this kind of sucks - I was able to upload pictures but it posted them to the top of the post every time. I had to drag it down to the bottom if I wanted to position them there. Also, there did not seem an easy way to edit the location/url/caption/size of the image after it was published.
4. Lack of built-in templates -- there are tons out there, but they should be easily accessible to install through the Settings tab though

That's my wish list for now. Given its a free software, I don't think the tool is bad; but it would be nice to have these features implemented as they seem to be fairly common in use by folks who would like to blog on a daily basis.

Today I went to Navy Pier to watch Cirque Shangai - Bright Spirit. Two words: Bloody Fantastic!

I got our tix at goldstar for $11.25 + some service charge the day before. We were row F on the left (around 6th row from the stage) so the view was amazing. The show was filled with non-stop action. The stunts were mind-blowing and it would easily put any circus to shame. This was my first Cirque * show and I was expecting some jaw dropping experience and I think I definitely got some today.

We watched the show in the afternoon; it would have been nice to watch it at night with the lights and costumes but I don't think it really matters that much unless you are making a date night out of it. Definitely worth checking it out if you haven't been to one ever and the price is very reasonable for the show these folks put up.

+1

Enjoy!

I continue to be amazed by thisiswhyyourefat.com and the content they get/put on their site everyday.

yum!

Twitter - the dominant social micro blogging tool gained great popularity in early 2009. It has become a tool that millions use now and has even become a source of daily news for some. More so, it has become a replacement for email and I feel like its heading to become another stream for spamming people. Over the past few months, I have noticed that small companies have bots that will follow users who mention a need for a product that they might support/sell. I have had users/companies/bots add me that I have not even heard of. Big corporations like Amazon, Orbitz, etc. have deals that they tweet about every few minutes/hours. Following just a couple of these big twitter accounts floods our twitterbox with junk. The good thing (and the only difference from spam email) is that you can control what accounts you can subscribe to. Instead of SEND TO ALL mechanism, we now have a SUBSCRIBE ideology. Although, it seems to me twitter is heading to the junkyard w/ the loads of information that people are putting on there. It has become a major source of advertisement, which wasn't its initial intention (or was it?). Am sure someone has thought of putting personals on twitter and somehow making some money (looking for a good night, tonight? :P). Third-party apps are coming w/ the only intention of using the twitter API in a innovative fashion, e.g. capturing your life in data.

All in all, twitter is great for blogging your thoughts under 140 words. Lets now see how many of these 140 word blogs it takes before one puts twitter in the spam box.

I recently purchased a Sharp LC52E77UN and gotta tell you, the TV looks pretty darn awesome. Firstly I would recommend the 52" size to anyone if you have the space. Now onto the picture quality - I was a bit disappointed w/ the quality of some of the SD channels (abc, fox, etc..), however others like food and discovery channel were definitey above par - I blame it on the DirectTV subscription I get in this condo building. I am waiting to get my Mini-DVI to HDMI cable I ordered from amazon to see the difference in quality/sharpness of true 1080p when I hook my mac in. I checked out the HD version of the TV's older models in best buy and it just looks plain awesome. Not sure if I am going to subscribe to HD for now as I don't watch that much regular TV to pay $260/yr. Instead, I have been debating on getting roku (for netflix instant watch + amazon VOD) for $120 shipped w/ cables and a home theater system which would probably cost me together the same amount as HD subscription for a year -- I probably have to wait for labor day sale or something though.

I'll post pictures and more details about the TV in the upcoming weeks as I get to play around with it more.

UPDATE 08/21/2009

Got my Mini-DVI to HDMI cable few days ago. The macbook to TV hookup is awesome and the picture is phenomenal - crisp and clear. I also followed the calibration settings found at avsforum and a bit of my personal tweaking and the video quality increased drastically for both SD channels and regular DVD's

UPDATE 08/31/2009

Adding some pics...

I started watching lost on netflix instant watch this past weekend. First of all, I don't understand how people can watch this show on a weekly basis and with ads. Anywho, my first reaction after the first 5-6 episodes was:

damn. this show is addicting as hell. i need to watch the next episode RIGHT NOW!

Last night I finished the 14th episode of season 1 and my reaction was a bit different

wth! will this crap ever get over? seems like these people on the island just like to go in circles to create a never ending plot.

Said that, I will probably watch the rest of season 1 and maybe even season 2 to see if it actually gets somewhere. Although, I can't imagine how they got this going for 5 seasons so far.

I decided to go to Ikea in Bolingbrook this past weekend and to my joys I turned the radio on to listen to some current junk. I heard the song hotel room service by pitbull probably about half a dozen times. Now, I got nothing against pitbull; but this song has to be the biggest piece of shit ever played. Just for kicks, lets go through some of the lyrics.

Forget about your boyfriend and meet me at the hotel room, you can bring your girlfriends and meet me at the hotel room.

really?

let me tell you what we gon' do.
2 + 2, i'm gonna undress you.

WTF?

after party in hotel lobby,
then we off to the room like voom,

voom voom zoom?

Another victory for creating shit songs. Nothin' but non-sense!