A fair evaluation process

Came across a post on results of Android developer challenge on TechCrunch site. They had announced a challenge for the developers to submit the application ideas for the android platform. While announcing the top 50 ideas/applications, the authors have also documented the process they used for the evaluation.

I must say this is one of the most elaborate evaluation process I have ever seen documented. The author goes to the details of how extreme/exception cases were handled and what care they had to take to minimize the impact of exclusions/exceptions. To me the biggest task in any evaluation, when it is by multiple persons, is how to normalize the scores. It is likely that a particular judge likes the application because he/she has some personal interest in that application, while some other judge simply dislikes it for some other reasons. Even if it is expected that judges do a fair job during scoring, one can't forget the human element. And the process explicitly tries to take care of this aspect well.

One thing that is missing in this post is how the scores were given and whether there were any criteria/guidelines provided to the judges. Except that, a fantastic post...

References:
  1. Android - It is a first complete, open and free mobile platform by "The Open Handset Alliance"
  2. Android Developer Challenge - To build great mobile application on the Android platform. The prize money, in total, is $10 million.
  3. TechCrunch - A weblog dedicated to profile and review the new internet products and companies. A cool site to track latest things in the internet world

Comments

Popular posts from this blog

Why is a country referred as "She"?

Subscription Services

Online Advertising - wastage of advertiser's money?