A fair evaluation process

Came across a post on results of Android developer challenge on TechCrunch site. They had announced a challenge for the developers to submit the application ideas for the android platform. While announcing the top 50 ideas/applications, the authors have also documented the process they used for the evaluation.

I must say this is one of the most elaborate evaluation process I have ever seen documented. The author goes to the details of how extreme/exception cases were handled and what care they had to take to minimize the impact of exclusions/exceptions. To me the biggest task in any evaluation, when it is by multiple persons, is how to normalize the scores. It is likely that a particular judge likes the application because he/she has some personal interest in that application, while some other judge simply dislikes it for some other reasons. Even if it is expected that judges do a fair job during scoring, one can't forget the human element. And the process explicitly tries to take care of this aspect well.

One thing that is missing in this post is how the scores were given and whether there were any criteria/guidelines provided to the judges. Except that, a fantastic post...

References:
  1. Android - It is a first complete, open and free mobile platform by "The Open Handset Alliance"
  2. Android Developer Challenge - To build great mobile application on the Android platform. The prize money, in total, is $10 million.
  3. TechCrunch - A weblog dedicated to profile and review the new internet products and companies. A cool site to track latest things in the internet world

Comments

Post a Comment

Popular posts from this blog

Subscription Services

Image
As the internet penetration increased over the last 2 decades, it also gave rise to digital subscriptions. The internet users slowly started developing trust in internet-based payments and that too on recurring payments. It probably started with the tech community using the recurring payment systems and slowly spreading that faith to non-tech-savvy users. There was also a phase when I started using the credit card for online transactions but wasn't comfortable storing the card details with the merchant. I used to enter the details every single time. (That is how I still remember the CC number that was in use at that time) In the late 2000's I was introduced to Ecom website development and since then it has been part of my professional as well as personal life. Over the years, I learned about the  PCI-DSS standard , and the extent to which the Ecom companies make an effort to protect consumer data. There have been instances of a data breach across the globe, but alon...
Read more

Movie Review: Chhota Bheem and The Throne of Bali

Image
Set in the beautiful islands of Bali, this movie is meant to entertain the kids. But unfortunately adults may not find it that interesting.  As usual, when Bheem along with his gang and King of Dholakpur reaches Bali, they find that Bali is in danger. Apparently, Bali's Queen is King of Dholakpur's sister and were invited for crowning ceremony of the prince Arjun. Upon arrival, Bheem and the team found that everything is devastated in Bali and there's no trace of Royal family including Arjun. After much of a search, Bheem finds the Rajguru (Royal Priest) who had helped the prince escape the wrath of the invader - Rangda.  The story line is pretty much predictable and there are no surprises till the end. Bheem needs to be the ONLY savior and it turns out to be that way. (This actually is a problem for children mentality, but I'll save the thoughts for some other time.) Overall animation is similar to the serial and there didn't seem to be any attempt...
Read more

Telemarketing

Last week Mobile rings .... Me: Hello --: Hello, this is XXX calling from XXX bank. Sir, you hold XXX bank credit card, right? Me: Yes. --: Sir, XXX bank is offerring you a ready cash of Rs.4,65,000/- Me: Actually this amount is too small for me. --: How much amount you are looking for, Sir? Me: Something like 4 crores.... --: (silence) --: 4 Crores ???? --: Is any bank offering you that much amount? Me: No! --: Then? Me: If you can offer me 4 lakhs just like that some other bank will give me 4 crores as well :) --: Ok Sir. (disconnect) :) Today Mobile rings ... Me: Hello --: Hello, this is XXX calling from XXX bank. Sir, you hold XXX bank credit card, right? Me: Yes. --: Sir, XXX bank is offerring you a ready cash of Rs.4,65,000/- Me: Last week only I told you folks that this is very small amount. I am looking for amount like 4 Crores... --: ccccrore ?????? (hung up) Nice entertainment right?
Read more